Download the Book (PDF): Introduction Every interactive web application is a machine for answering one question over and over: something changed, so what on the screen must change with it? A user types a character into a search box. A websocket delivers a new chat message. A background request finishes and an order moves from "pending" to "shipped." Each event alters some piece of data, and the application has to find every part of the interface that depends on that data, recompute it, and update the page, ideally without touching anything that did not depend on it. State management is the name we give to the strategies for doing this. The libraries are just packaged opinions about how to do it. For most of the last decade, frontend developers argued about state management as if it were a choice of brand. You used Redux, or you used MobX, or you rebelled against both and put everything in React Context. Conference talks had titles like "You Might Not Need Redux," and job listings asked for "Redux experience" as though it were a language. That framing was always a little misleading, and it has become actively unhelpful. The modern React application does not have one state problem. It has at least five, and they want different tools. Why the old arguments stopped making sense Three shifts changed the ground underneath the debate. The first was the recognition that most of what applications were stuffing into global stores was not really application state at all. It was a cache of data that lived on a server. A list of products, the current user's profile, the comments on a post: none of these belong to the browser. The browser holds a copy, and the copy goes stale. Treating that copy as if it were authoritative client state, by dispatching FETCH_PRODUCTS_SUCCESS actions into a reducer, forced teams to reinvent caching, deduplication, retry, and invalidation by hand, usually badly. Libraries such as TanStack Query (originally React Query) and Redux Toolkit's own RTK Query made the point concrete: server state is a cache, and a cache has its own rules. Once teams moved server data out of their global stores, many discovered that what remained was small. The second shift was the return of fine-grained reactivity under a new name. Signals are not new as an idea. Knockout had observables in 2010, and spreadsheets have propagated changes through dependency graphs since the late 1970s. But between roughly 2021 and 2024, Solid, Preact, Angular, Vue, and Svelte all converged on the same primitive: a value container that knows which computations read it and notifies exactly those computations when it changes. A TC39 proposal to add signals to JavaScript itself followed. Signals challenge a premise that React developers had absorbed so deeply they had stopped noticing it: that the unit of update is the component, and that the way to respond to a change is to re-run a function and compare the result. The third shift came from React itself. React 18 introduced concurrent rendering and, with it, a hook called useSyncExternalStore that every serious store library now builds on. React 19 added use(), Actions, and useOptimistic, which take over several jobs that used to require a store. And in October 2025 the React team shipped React Compiler 1.0, a build-time tool that inserts memoization automatically. Much of the hand-tuning that state libraries existed to make easier, such as stable selectors, memoized callbacks, and avoiding re-renders of untouched subtrees, is now partly done by a compiler. The result is that "which state library should we use?" has become the wrong first question. The right first question is "what kinds of state does this application have, and how should a change in each kind reach the code that depends on it?" The argument of this book This book makes one central claim: state management is a problem of change propagation, and the most consequential decision is not which library to adopt but how to classify each piece of state so that it is handled by the propagation model that fits it. Immutable state trees and signals are two coherent answers to the propagation question. Redux and Zustand answer it by producing a new snapshot of the world and letting subscribers compare old with new. Signals answer it by tracking dependencies at read time and pushing notifications only along the edges of a graph. Neither answer is universally better. Each has a cost model, and the cost model matters more than the API. Taking that claim seriously leads to several practical conclusions that the chapters develop in detail: • Server data should almost never live in a hand-written global store. A dedicated cache with explicit staleness rules will be smaller, faster, and more correct. • Most re-render problems in React are not caused by the choice of state library. They are caused by subscribing to too much, at too high a level, and they are fixed by narrowing subscriptions. • Signals deliver their full benefit only in a renderer designed around them. Inside React they are useful, but they fight the framework's model in ways that are worth understanding before adopting them. • The arrival of the React Compiler and React 19's async primitives shrinks the territory where a client store earns its keep, but it does not eliminate it. Genuinely shared, long-lived, client-owned state such as an editor document, a multi-step workflow, or a real-time collaborative canvas still benefits from a deliberate store. Who this is for and how it is organized The reader I have in mind is a working developer or technical lead who has built React applications, has probably used at least one state library, and wants a clear mental model of why the options differ and when each is the right call. I assume familiarity with JavaScript and with React's basic hooks. I do not assume you have read the Redux source code or followed TC39 meetings. Every specialized term is defined when it first appears. The book proceeds from foundations to decisions. Chapter 1 builds a taxonomy of state, because nothing else makes sense without it. Chapter 2 explains how React decides what to re-render, which is the cost model every library either works with or works around. Chapters 3 and 4 examine the immutable-snapshot family: Redux, in its modern Redux Toolkit form, and Zustand, its minimalist descendant. Chapters 5 and 6 turn to signals, first as a general reactive model and then specifically as something you might use inside React. Chapter 7 is a practical treatment of re-render optimization, including what the React Compiler does and does not change. Chapters 8 and 9 deal with asynchronous data: dedicated server-state caches first, then the async primitives in React 19 and Server Components. Chapter 10 pulls the material into a decision framework and discusses migration between approaches in real codebases. Code appears throughout, but sparingly. The snippets are short and meant to make a mechanism visible, not to serve as copy-and-paste templates. Where I cite a version number, a performance figure, or the status of a proposal, it reflects the state of the ecosystem as of late 2026. At that point Redux Toolkit is in its 2.x line, Zustand is at version 5, TanStack Query for React is at version 5, React is in the 19.x series with React Compiler 1.0 stable, and the TC39 Signals proposal sits at Stage 1. These facts will drift. The underlying trade-offs will not drift nearly as fast, and they are what the book is really about. One more orientation before we start. It is tempting to read a book like this looking for the winner. There isn't one, and the evidence for that is in production: large, well-run applications today commonly combine a server cache, a small client store, URL state, form state managed by a form library, and component-local state, with each piece chosen deliberately. Teams that struggle are rarely the ones who picked the "wrong" library. They are the ones who put every kind of state in the same place and then spent years fighting the consequences. The goal here is to give you the vocabulary and the cost models to avoid that. Chapter 1: What State Is, and Why It Hurts The word "state" is used so loosely in frontend work that it has nearly stopped meaning anything. Developers say "put it in state" when they mean a useState hook, "global state" when they mean a Redux store, and "the state of the app" when they mean everything from the logged-in user to whether a dropdown is open. Before comparing tools, we need a sharper definition, and then a way of sorting state into kinds that behave differently. Here is a working definition. State is any value that can change over the lifetime of the application and whose current value affects what the user sees or what the application does next. A constant is not state. A value computed purely from other values is not state either; it is derived data, and treating derived data as state is one of the most common sources of bugs. State is the minimal set of facts from which everything else can be computed. That definition already tells us something useful. If you can compute it, do not store it. If a shopping cart holds items with prices, the cart total is not state. Storing the total alongside the items creates two sources of truth that can disagree, and eventually they will. The discipline of keeping state minimal and deriving everything else is older than any library. Redux's documentation calls it "keeping state minimal," Vue calls derived values "computed," and signals libraries build the same idea into their core with computed signals. Every approach in this book agrees on this point, even when they disagree on almost everything else. Five kinds of state The trouble starts when we notice that the facts an application holds are not all the same kind of fact. They differ in who owns them, how long they live, who else needs them, and what happens when they go out of date. Sorting state along these lines produces a taxonomy that has become close to standard among experienced React developers, even if the exact labels vary. Server state is data whose source of truth lives somewhere else, usually a database behind an API. The browser holds a copy. The copy can be stale the instant it arrives, other users can change the original, and the browser does not get told. Fetching it is asynchronous and can fail. Server state is typically the largest category by volume in a business application: lists of records, details of an entity, search results, user profiles. URL state is state encoded in the address bar: the current route, query parameters such as ?page=3&sort=price, and sometimes the hash. It has a special property that no other kind has. It survives a reload, it can be bookmarked, and it can be pasted into a message and opened by someone else. Anything a user would reasonably expect to share or return to belongs here. A surprising number of applications store the active filter or the selected tab in a global store and then wonder why the back button does not work. Form state is the in-progress, not-yet-submitted input a user is editing. It has its own lifecycle: initial values, dirty tracking, validation errors, touched fields, submission status. It is usually short-lived and local to one part of the screen, but it can be intricate. Dedicated form libraries exist because this lifecycle is fiddly to get right by hand. Local UI state is ephemeral state that matters to one component or a small subtree: whether a menu is open, which accordion panel is expanded, the current value of an uncontrolled input, hover and focus. It has no meaning outside the component, and nobody would miss it after a reload. Shared client state is state that the client genuinely owns, that lives longer than a single interaction, and that several distant parts of the interface need. Examples include the contents of a document in an editor, a multi-step wizard's accumulated answers before submission, a user's selected theme, the layout of panels in a dashboard, a local queue of offline actions, or the shapes on a collaborative whiteboard before they sync. This is the category that state libraries were really built for. In most applications, it is smaller than people assume. Table 1 sets out how these categories differ on the dimensions that matter most for choosing a mechanism. Table 1. Five kinds of state and their defining traits. Kind Source of truth Typical lifetime Main hazard Natural home Server Remote API or database Until stale Staleness, races Server-state cache URL Address bar Navigation session Back button breaks Router Form Component until submit One editing session Validation drift Form library or local Local UI One component One interaction Over-lifting useState Shared client Client store Session or longer Over-subscribing Dedicated store The point of the table is not that each row must use a particular library. It is that each row has a different dominant failure mode, and a mechanism suited to one row is often poorly suited to another. Server state needs staleness and retry logic that a client store does not provide. Local UI state does not need the ceremony of actions and reducers. URL state needs to be serializable and must respect browser navigation. Shared client state needs careful subscriptions so that a change in one field does not re-render everything that touches the store. How applications end up in pain It is worth describing the most common failure in detail, because almost every team that has used React for a few years has lived through some version of it. The application starts small. A developer adds Redux, or later Zustand, to hold the current user. Then the product list arrives from the API, and because the store is already there, the list goes in too, with loading and error flags beside it. Then the product details page needs data, and another slice appears. Each slice grows its own isLoading, error, lastFetched, and some ad hoc logic for when to refetch. Someone notices that navigating back to the list page triggers a fresh fetch and a loading spinner, so they add a check: if the data is less than a minute old, skip the fetch. Someone else notices that two components mounting at once fire two identical requests, so they add an in-flight flag. Then a bug report arrives: a user edited a product, returned to the list, and saw the old name. So mutations start dispatching actions that manually patch the list slice. Meanwhile, the filter selection on the list page lives in the store too, which means it persists across navigation in ways the designers did not intend, and it is not in the URL, so links to filtered views do not work. At this point the store has become a home-made, under-tested cache layered with UI flags and routing state. Every component connects to it. Re-renders cascade because components select large objects. New developers find it intimidating, and the team concludes that the state library is the problem. Often they migrate to a different library and, within a year, rebuild the same structure in the new tool. The underlying error was not the library. It was failing to distinguish kinds of state. The product list was server state and wanted a cache with deduplication, staleness, and invalidation. The filter was URL state. The edit form was form state. Only the current user's session, and perhaps a few preferences, were shared client state, and even the user profile arguably belonged in the server cache. Derived data and the single source of truth Two principles cut across all five categories and deserve explicit statement because they are violated constantly. The first is the single source of truth: each fact should be stored in exactly one place. Copying a value from one store to another, or from props into local state, creates two places that must be kept in sync. React's documentation has long warned against "mirroring props in state" for this reason. A common example is copying a server response into a client store "so that we can edit it." The moment the copy exists, there are two versions of the record, and every edit, refetch, and invalidation has to reason about which one wins. The second is derive, don't store. Anything computable from state should be computed on read, with memoization if the computation is expensive. A filtered list is derived from the full list plus the filter. A count of unread messages is derived from the messages. A boolean like canSubmit is derived from the form values and validation results. Libraries differ in how they make derivation cheap. Redux uses memoized selector functions, typically built with the Reselect library that Redux Toolkit re-exports as createSelector. Signals libraries provide computed signals that cache their result and recompute only when an input changes. React offers useMemo, and with the React Compiler, often memoizes derivations automatically. The mechanisms vary, but the principle is identical. Derivation also clarifies what "normalized" state means, a term that comes up constantly in Redux discussions. Normalization, borrowed from relational databases, means storing each entity once, keyed by ID, and referring to it elsewhere by ID rather than by embedding copies. If a blog post appears in a feed, a search result, and a detail view, a normalized store holds one copy of the post and three lists of IDs. Edit the post and every view sees the change. Redux Toolkit provides createEntityAdapter to manage this shape. Normalization is a technique for enforcing single source of truth inside a store, and it matters most when the same entity appears in many places. Time, identity, and the propagation question There is one more dimension worth naming, because it is the one that separates the major families of tools: how a change reaches the code that depends on it. Consider a simple dependency chain. A store holds a list of todos and a filter. A derived value holds the visible todos. A component renders the count of visible todos; another renders the list. When the user toggles a todo, something must notice that the visible list might have changed, recompute it, and update both components. There are two broad strategies. In the snapshot strategy, each change produces a new version of the state. The old version is left untouched; this is what "immutable" means in practice. Subscribers are notified that something changed, and each subscriber decides whether it cares by comparing the part it reads in the old version with the part it reads in the new one. Because the state is immutable, comparison can be done cheaply by reference: if the object is the same object, nothing inside it changed. Redux and Zustand use this strategy. Its virtues are predictability, easy debugging because every version can be kept and replayed, and a natural fit with React, which itself compares props and state by reference. Its cost is that every subscriber is asked on every change, even if the answer is usually "not me," and that selecting the right slice without accidentally creating new references takes care. In the dependency-tracking strategy, the system records, at the moment a computation reads a value, that the computation depends on that value. When the value changes, the system already knows exactly which computations to update and notifies only them. Signals use this strategy, as did MobX and Knockout before them. Its virtue is precision: no subscriber is asked about a change it did not read. Its costs are a more complex runtime, the need for the dependency graph to be correct, which can be subtle with conditional reads and asynchronous code, and an awkward fit with frameworks like React that re-run whole component functions. Much of the rest of this book is an exploration of these two strategies: how they are implemented, where each is cheap and where each is expensive, and how they interact with React's own rendering model. But neither strategy matters much if the state being propagated is in the wrong category to begin with. A perfectly tuned signals graph holding stale server data is still holding stale server data. Classify first. Then choose the mechanism. A practical inventory Before any architectural decision, it helps to inventory an application's state concretely. Take a real screen, say the main dashboard of a project-management tool, and list every changing value on it. For each, ask four questions. 1. Where is the source of truth? If the answer is "the server," it is server state, regardless of how often the client changes it. 2. Should it survive a reload or be shareable by link? If yes, it belongs in the URL. 3. Is it being edited and not yet committed? If yes, it is form state. 4. Does anything outside this component or its immediate children need it? If no, it is local UI state. Whatever is left after those four questions is shared client state. On a typical dashboard, the inventory might produce a dozen kinds of server data (projects, tasks, users, comments, notifications), three or four URL parameters (selected project, view mode, filters), a couple of forms (quick-add task, comment box), many bits of local UI state (open menus, hover cards, collapsed sections), and perhaps three pieces of genuinely shared client state: the current theme, the layout of resizable panels, and a drag-and-drop operation in progress. That last list is what a client store needs to hold. It is a far smaller and more tractable problem than "the state of the app." The exercise is mundane, and it is the single most effective thing a team can do to simplify a struggling frontend. The chapters that follow assume it has been done. When we discuss Redux, Zustand, and signals, we are discussing tools for shared client state, plus, in Chapters 8 and 9, the separate tools that handle server state. The question of how React decides what to re-render when any of this changes comes first. Chapter 2: How React Decides What to Re-render Every state library for React is, underneath its API, a strategy for working within React's rendering model. Some work with it closely; others work around it. You cannot evaluate any of them without a precise picture of what React actually does when state changes. This chapter builds that picture: what a render is, what triggers one, how far it spreads, why Context behaves the way it does, how external stores plug in, and where the React Compiler now fits. What a render actually is In React, "rendering" means calling your component function. That is all. When React renders a component, it runs the function, gets back a tree of elements (plain JavaScript objects describing what should be on screen), and compares that tree with the one produced last time. This comparison is called reconciliation. Only after reconciliation, in a separate commit phase, does React touch the real DOM, and then only for the parts that differ. This separation is the source of a widespread confusion. A component that "re-renders" has not necessarily caused any DOM work. If the function returns the same output, React's commit phase does nothing for it. The cost of an unnecessary render is the cost of running the function and diffing its output, plus the cost of rendering its children, which, as we will see, is usually the real problem. React renders a component for one of a small number of reasons: • Its own state changed, through a useState setter or a useReducer dispatch. • Its parent rendered. • A Context it reads changed value. • An external store it subscribes to, through useSyncExternalStore, reported a changed snapshot. The second reason is the one that surprises people. By default, when a component renders, React renders all of its children, recursively, whether or not their props changed. React does not compare props before rendering a child unless the child is wrapped in React.memo or the compiler has memoized the child element. This design is deliberate. Comparing props has a cost, most components are cheap, and rendering is supposed to be a pure function of props and state, so re-running it is always safe. But it means that a state change near the top of a tree causes the entire subtree below it to re-run. Consider an application that stores a search query in state at the root so that a header input and a results list can both see it. Every keystroke sets state at the root. The root renders, and so does every descendant: the sidebar, the footer, the settings panel, a chart nobody is looking at. If any of those are expensive, typing becomes sluggish. Nothing about this involves a state library. It is the default behavior of React, and it is the root cause of most of the re-render problems that state libraries are then blamed for or credited with solving. Bailouts, memoization, and referential identity React has several ways to skip work, collectively called bailouts. If a state setter is called with a value identical to the current one, compared with Object.is, React may skip rendering that component. This is why immutable updates matter so much in React. If you mutate an array in place and pass the same array back to the setter, React sees an identical reference and may skip the render even though the contents changed. If you create a new array, React sees a new reference and renders. React.memo wraps a component so that React compares its new props with its previous props, shallowly, key by key with Object.is, and skips the render if they are all the same. This is effective only if the props are referentially stable. If the parent creates a new object or a new arrow function on every render and passes it down, React.memo sees a different reference each time and renders anyway. This is why useMemo and useCallback exist: to keep object and function identities stable across renders so that memoized children can bail out. Before the React Compiler, getting this right by hand was a significant source of complexity and bugs. Developers either sprinkled useMemo and useCallback everywhere, at some cost in readability and memory, or used them sparingly and missed the cases that mattered. Dependency arrays that omitted a value produced stale closures; arrays that included an unstable value made the memoization useless. The key insight to carry forward is that React's skip-work machinery is built on reference equality. That is why immutable data structures and React fit together so naturally, and why the snapshot family of state libraries, Redux and Zustand, feels idiomatic in React. A new reference means "something changed"; the same reference means "nothing changed." Every optimization depends on references changing exactly when data changes and not otherwise. Where state lives: position, keys, and reset One more piece of React's model matters for state management, and it is frequently misunderstood. React state created with useState or useReducer does not live inside the component function. It lives in React's internal tree, attached to a position: a particular component type at a particular place among its siblings. As long as the same type of component renders at the same position, React preserves its state across renders. If the type changes, or the component disappears and reappears, the state is destroyed and recreated. This has two practical consequences. The first is that keys control identity. Giving a component a key tells React to treat it as a different instance whenever the key changes. A profile editor rendered as gets fresh internal state, including form fields, whenever userId changes, with no effect needed to reset anything. Without the key, switching from one user to another keeps the previous user's half-edited fields, a classic bug. Many effects that "reset state when a prop changes" can be replaced by a key. The second is that the state of a component is lost when its position in the tree changes. Conditionally rendering a component in two different branches of a JSX expression, or wrapping it in a different parent element in one branch, can silently reset it. This is one reason teams are tempted to move state into a global store: a store survives any tree restructuring. That instinct is sometimes right, for state that genuinely must outlive a component, but often it is a workaround for a component tree whose structure shifts unnecessarily. React 19.2's component offers a third option for some cases: it can hide part of the tree while preserving its state, so that a tab panel the user navigates away from keeps its scroll position and inputs when they return, without lifting that state anywhere. Context: a broadcast, not a subscription React Context is often described as a state management tool. It is more accurately a dependency-injection mechanism: a way to make a value available to a subtree without passing it through every intermediate component's props. How it propagates changes is the part that matters here. When a Context provider receives a new value, compared by reference, React re-renders every component in the subtree that reads that Context with useContext or use. There is no selector. A component that reads a Context holding { user, theme, locale } and only uses theme will still re-render when user changes, because the whole value object changed identity. This is the single most important limitation to understand about Context as a state mechanism. It is a broadcast: every consumer hears every change. For values that rarely change, such as a theme, a locale, or an authenticated user that is set once at login, this is perfectly fine and Context is the right tool. For values that change often and are read by many components, such as the contents of a form or a frequently updated cursor position, it produces exactly the over-rendering that people then try to fix with a state library. There are standard mitigations. You can split one large Context into several small ones so that consumers subscribe only to what they read. You can separate the state value from the updater functions into two contexts, so that components that only dispatch do not re-render when the state changes. You can memoize the provider's value so it does not change identity on unrelated parent renders. These techniques work, but they are manual and they do not scale gracefully. A store with selectors is essentially a way to give Context-like availability with per-consumer subscriptions. A common and costly bug illustrates the problem: function AppProvider({ children }) { const [user, setUser] = useState(null); const [cart, setCart] = useState([]); // New object every render: every consumer re-renders // whenever AppProvider renders for any reason. return ( {children} ); } Every time a cart item is added, every component reading AppContext re-renders, including components that only care about user. Wrapping the value in useMemo prevents re-renders when AppProvider renders for unrelated reasons, but it does not fix the broadcast problem: a cart change still notifies the user consumers. Splitting into UserContext and CartContext fixes that. External stores and useSyncExternalStore Redux, Zustand, and most signals integrations keep their state outside React. React does not know about it until something tells it. For years, libraries did this by subscribing in a useEffect, holding a copy of the selected value in useState, and calling the setter when the store changed. This worked tolerably until React 18 introduced concurrent rendering. Concurrent rendering lets React pause a render partway through, handle something more urgent such as a keystroke, and resume or restart later. That creates a new hazard called tearing. If a render of a large tree is paused, and during the pause an external store changes, then components rendered before the pause saw the old value and components rendered after it see the new value. The screen shows an inconsistent mix of two states. React's own state is protected against this because React controls when it updates, but an external store is not. React 18 added useSyncExternalStore to solve this. A library provides two functions: subscribe, which registers a callback to be called when the store changes, and getSnapshot, which returns the current value. React calls getSnapshot during render, checks it again before committing, and if the value changed in between, re-renders synchronously so that every component sees the same snapshot. The cost is that updates from external stores cannot take advantage of some concurrent features; they are always treated as synchronous, urgent updates. const value = useSyncExternalStore( store.subscribe, // (callback) => unsubscribe () => selector(store.getState()), ); React-Redux since version 8 and Zustand in all recent versions build on this hook. It imposes a crucial contract: getSnapshot must return the same reference when nothing has changed. If it returns a new object or array every time, React concludes the store has changed on every check and re-renders in a loop. This is the root of the "selector returning a new reference causes infinite loops" warning in Zustand v5's migration guide, and of React-Redux's development-mode warnings about selectors that return new references. It is the same reference-equality rule from earlier in this chapter, enforced more strictly. The practical consequence for anyone using a store is that selectors should return either primitives, which compare by value, or references that already exist in the state, which stay stable until that part of the state changes. A selector like state => state.todos.filter(t => !t.done) creates a new array on every call and must be memoized or replaced with a shallow-equality comparison. Chapters 3 and 4 show how Redux and Zustand each handle this. Where the React Compiler fits React Compiler 1.0, released in October 2025, changes the economics of everything above. It is a build-time tool, distributed as a Babel plugin, that analyzes component and hook code and automatically inserts memoization. The output caches JSX elements, derived values, and callbacks, recomputing them only when their inputs change. The effect is roughly what you would get if a meticulous developer had wrapped every appropriate value in useMemo, every callback in useCallback, and every child element in something like React.memo, with correct dependency lists. The compiler addresses the most common re-render cause directly. If a parent re-renders but the props it passes to a child have not changed, the compiled code reuses the previously created child element, and React skips that child. The cascading re-render of an untouched subtree largely disappears. The React team reported that at Meta's Quest Store, initial loads and cross-page navigations improved by up to 12 percent and certain interactions became more than 2.5 times faster, with neutral memory usage. Results in other codebases will vary, but the direction is clear. The compiler requires that components follow the Rules of React: render must be pure, props and state must not be mutated, hooks must be called unconditionally. Code that breaks these rules is skipped rather than miscompiled, and the compiler-powered lint rules now ship in eslint-plugin-react-hooks to flag violations. It supports React 17 and later, with a small runtime package needed for 17 and 18. What the compiler does not do is equally important for state management. It does not change the fact that a component reading a Context re-renders when that Context's value changes. It does not change the fact that a store subscription with a poorly written selector will fire too often. It does not make an expensive computation inside a component cheap; it only avoids repeating it when inputs are unchanged. And it cannot reach into an external store to make its notifications more precise. The compiler reduces the cost of a render that happens; the state library still decides whether the render happens. Chapter 7 returns to this boundary in detail. The cost model, summarized Pulling these threads together gives a cost model that applies to every library in this book. A state change in React costs, roughly, the number of components that render as a result multiplied by the average cost of rendering each, plus the cost of whatever DOM changes are actually committed. Libraries and techniques reduce cost by attacking one of these terms. Colocation and narrow subscriptions reduce the number of components notified. Memoization, manual or compiled, reduces how far renders cascade from the notified components to their children. Moving expensive work out of render, or virtualizing long lists, reduces the per-component cost. Fine-grained reactivity, which we will meet in Chapter 5, attacks the first two terms at once by updating individual DOM nodes without rendering components at all, at least in renderers built for it. A useful way to think about the snapshot family is that it reduces the number of components notified, through selectors, and relies on React's reference-equality machinery to keep everything else cheap. A useful way to think about signals is that they try to make "number of components rendered" close to zero for most updates by bypassing the component as the unit of work. Whether that is possible inside React, and at what cost, is a question we can answer only after understanding both families in detail. Chapter 3: Redux and the Immutable State Tree Redux is the most influential state library in the history of frontend development, and also one of the most misremembered. Many developers formed their opinion of it between 2016 and 2019, when using it meant writing action type constants, action creator functions, switch-statement reducers, and hand-rolled immutable updates spread across several files for every feature. That version of Redux deserved much of the criticism it received for boilerplate. It is also not the Redux that its maintainers have recommended for years. This chapter explains the core model, which has barely changed, and the modern Redux Toolkit API, which has changed nearly everything about how the model is used. The model: one tree, pure updates, explicit events Redux was created by Dan Abramov and Andrew Clark in 2015, drawing on Facebook's Flux architecture and on the Elm language's model of updates as pure functions. Its design rests on three principles that the documentation has stated consistently since. First, the global state of the application is stored in a single object tree inside a single store. Not one store per feature, not state scattered across components: one tree. This makes the whole state inspectable and serializable at any moment. Second, state is read-only. The only way to change it is to dispatch an action, a plain object describing what happened, such as { type: 'todos/added', payload: { id: 7, text: 'Buy milk' } }. Components cannot reach in and modify the tree. Third, changes are made with pure functions called reducers. A reducer takes the current state and an action and returns the next state, without side effects and without mutating the input. (state, action) => newState. The store calls the root reducer for every dispatched action and replaces its state with the result. These three rules produce a set of properties that remain Redux's main selling points. Because every change goes through a dispatched action, there is a complete log of what happened. Because reducers are pure and state is immutable, any past state can be kept and restored, which enables the Redux DevTools' time-travel debugging: you can step backward and forward through actions and watch the UI change. Because state is a plain serializable object, it can be persisted, sent to a server for crash reports, or used to reproduce a bug exactly. And because actions describe events rather than commands, multiple reducers can respond to the same action independently. A user/loggedOut action can clear the cart slice, the preferences slice, and the notifications slice without the component that dispatched it knowing about any of them. That last point is the essence of Redux's architecture and is often missed. The documentation encourages "modeling actions as events, not setters." An action named cart/setItems is a setter disguised as an event; an action named checkout/completed is an event to which many parts of the state can react. Applications that use Redux as a glorified global setState pay its costs without collecting its benefits. Redux Toolkit: the same model, a fraction of the code Redux Toolkit, abbreviated RTK, began in 2018 as "Redux Starter Kit" and reached 1.0 in 2019. It is the official, recommended way to write Redux, and the maintainers have described hand-written legacy Redux as deprecated in practice; the core createStore function is marked as deprecated in its type definitions to push people toward configureStore. Redux Toolkit 2.0, released at the end of 2023 alongside Redux core 5.0 and React-Redux 9.0, modernized packaging and types and removed long-deprecated options. As of late 2026 the toolkit is in its 2.x series, with regular minor releases adding features such as infinite queries to RTK Query and upgrades to the Immer library it depends on. The central API is createSlice, which generates action creators and a reducer from a single object: import { createSlice, configureStore, PayloadAction } from '@reduxjs/toolkit'; type Todo = { id: string; text: string; done: boolean }; const todosSlice = createSlice({ name: 'todos', initialState: [] as Todo[], reducers: { added(state, action: PayloadAction) { state.push(action.payload); // looks like mutation }, toggled(state, action: PayloadAction) { const t = state.find(t => t.id === action.payload); if (t) t.done = !t.done; }, }, }); export const { added, toggled } = todosSlice.actions; export const store = configureStore({ reducer: { todos: todosSlice.reducer } }); The reducers appear to mutate state, which would violate Redux's central rule. They do not, because createSlice runs them through Immer. Immer hands the reducer a draft, a proxy object that records every mutation attempted on it. When the reducer finishes, Immer produces a new immutable state that applies those changes using structural sharing: unchanged parts of the tree are reused by reference, and only the objects along the path to a change are copied. Toggling one todo in a list of a thousand creates a new array and a new object for that one todo; the other 999 todo objects are the same references as before. Structural sharing is what makes immutable state practical. Without it, every update would need to deep-copy the tree, which would be prohibitively slow. With it, an update costs roughly in proportion to the depth of the change, and every unchanged subtree keeps its identity. That identity is precisely what React and selectors rely on to skip work. Immer does carry a runtime cost compared with hand-written spread updates, and RTK's recent releases have included Immer upgrades aimed at reducing it, but for the vast majority of applications the cost is small relative to rendering. configureStore sets up the store with sensible defaults: the Redux DevTools extension connection, the thunk middleware for async logic, and, in development, middleware that throws an error if you accidentally mutate state outside a reducer and logs an error if you put a non-serializable value such as a class instance, a Promise, or a Date object into the store or into an action. Those development checks catch real bugs early and are a good reason not to assemble a store by hand. Reading state: selectors and memoization Components read from the store with React-Redux's useSelector hook, passing a selector, a function from the whole state to the piece the component needs. const count = useSelector((state: RootState) => state.todos.length); On every dispatched action, React-Redux runs every mounted component's selector against the new state and compares the result with the previous result using strict reference equality. If they are the same, the component does not re-render. This is how Redux scales: the store notifies all subscribers of every change, but each subscriber filters the notification down to what it reads, and only those whose selected value changed render. The cost model follows directly. Every action runs every selector, so selectors must be cheap. And a selector must return a stable reference when its inputs have not changed, or the component will re-render on every action. A selector that computes a filtered array on each call fails this test: // Re-renders on EVERY action, even unrelated ones: const open = useSelector((s: RootState) => s.todos.filter(t => !t.done)); The fix is a memoized selector created with createSelector, which Redux Toolkit re-exports from the Reselect library. A memoized selector takes one or more input selectors and a result function. It reruns the result function only when an input selector returns a different reference; otherwise it returns its cached result. import { createSelector } from '@reduxjs/toolkit'; const selectTodos = (s: RootState) => s.todos; const selectOpenTodos = createSelector([selectTodos], todos => todos.filter(t => !t.done), ); Because Immer preserves the todos array's identity when an unrelated slice changes, selectOpenTodos returns its cached array, and the component skips rendering. Reselect 5, shipped with RTK 2.0, switched its default memoization to a strategy based on weak maps that can hold more than one cached result, addressing a long-standing limitation where a selector shared between components with different arguments would constantly recompute. The general lesson is worth stating plainly, because it will recur with Zustand. In a snapshot-based store, the selector is the subscription. A component subscribes to exactly what its selector returns. Coarse selectors that return large objects cause components to re-render whenever anything in those objects changes. Fine selectors that return primitives or stable references make re-renders precise. Most performance problems attributed to Redux are, on inspection, selector problems. Side effects: thunks, listeners, and where async logic goes Reducers must be pure, so anything with a side effect, such as an API call, a timer, a log message, or a write to local storage, has to live elsewhere. Redux handles this with middleware, functions that wrap dispatch and can intercept actions before they reach the reducers. The default middleware is redux-thunk, which allows dispatching a function instead of an action object. The function receives dispatch and getState and can do whatever it likes, including awaiting promises and dispatching further actions. Redux Toolkit's createAsyncThunk wraps the common pattern of dispatching pending, fulfilled, and rejected actions around a promise. For reactive side effects, where something should happen in response to an action or a state change, RTK provides the listener middleware, created with createListenerMiddleware. It lets you register effects that run when a specific action is dispatched or when a predicate over the state becomes true. It covers most use cases that once required the more complex redux-saga or redux-observable libraries, with a much smaller conceptual footprint. A typical use is persisting a slice to local storage whenever it changes, or triggering an analytics event when a checkout completes. For data fetching, the modern answer is RTK Query, a server-state cache built on top of Redux and included in the toolkit. It deserves its own treatment and receives one in Chapter 8. The short version is that writing thunks to fetch server data and storing the results in hand-written slices is now considered an anti-pattern by the Redux maintainers themselves. Normalized entities and event modeling in practice Two Redux Toolkit features show how the model handles realistic data, and both repay a closer look. The first is createEntityAdapter, which implements the normalized shape described in Chapter 1. An adapter manages state of the form { ids: [...], entities: { [id]: item } } and provides prebuilt reducer functions such as addOne, upsertMany, updateOne, and removeOne, along with memoized selectors such as selectAll, selectById, and selectIds. An optional sortComparer keeps the ids array ordered. const commentsAdapter = createEntityAdapter({ sortComparer: (a, b) => a.createdAt.localeCompare(b.createdAt), }); const commentsSlice = createSlice({ name: 'comments', initialState: commentsAdapter.getInitialState(), reducers: { commentAdded: commentsAdapter.addOne, commentEdited: commentsAdapter.updateOne, }, }); export const { selectById: selectCommentById, selectIds: selectCommentIds } = commentsAdapter.getSelectors((s: RootState) => s.comments); The normalized shape pays off in rendering. A comment list component can select only selectCommentIds, which returns the same array reference until a comment is added, removed, or reordered. Each list item selects its own comment with selectCommentById(state, id). Editing the text of one comment produces a new object for that comment and a new entities map, but the ids array is untouched, so the list does not re-render, and only the one item whose comment changed renders. This is the snapshot model operating at its best: structural sharing plus narrow selectors gives precise updates without any dependency tracking. The second feature is less a function than a discipline: treating actions as events and letting several slices respond. Redux Toolkit supports this with the extraReducers field of createSlice, which lets a slice handle actions defined elsewhere. Consider a project-management application in which deleting a project must remove the project, remove its tasks, clear it from the "recently viewed" list, and reset the selection if it was selected. The setter-style approach dispatches four actions from the component that handles the delete button: projects/removed, tasks/removedForProject, recent/removed, and selection/cleared. The component now knows about four slices, and any new slice that holds project references must be remembered and added to the list. The event-style approach dispatches one action, projectDeleted(id), defined in the projects slice. The tasks, recent, and selection slices each declare in their extraReducers how they respond to projectDeleted. The component knows about one event. A new slice that holds project references adds its own handler without changing any existing code. All four updates happen in a single dispatch, so subscribers are notified once and the UI never renders an intermediate state in which the project is gone but its tasks remain. That last property is more important than it looks. With four separate dispatches, React-Redux batches notifications inside React event handlers, but outside them, for instance after an await in a thunk, each dispatch could in principle trigger its own round of selector runs and renders, and a component could briefly see tasks whose project no longer exists. A single event handled by several reducers makes the update atomic by construction. This is the kind of guarantee that Redux's structure provides and that lighter stores leave to developer discipline. Where Redux is strong and where it costs Redux's strengths are clearest in applications with a large amount of interrelated client state that changes in response to many kinds of events: document editors, design tools, complex workflow applications, trading interfaces, and anything that benefits from undo, replay, or deterministic reproduction of bugs. The single serializable tree and event-style actions give a degree of predictability and debuggability that no other approach in this book matches. Large teams also value its conventions. When everyone writes slices the same way and all changes flow through dispatched actions, a new developer can find where any state change originates by searching for an action type. The costs are real, though smaller than in the past. There is still more structure than a minimal store requires: slices, a configured store, a Provider, typed hooks. Every action runs every selector, which can become measurable in applications with thousands of mounted connected components and high-frequency updates such as pointer tracking; at that scale, specialized techniques or a different mechanism for the hot path are warranted. The indirection of actions and reducers is overkill for state that is simple and local. And the single store makes code splitting slightly more involved, though RTK 2.0's combineSlices with lazy injection addresses this directly. A note on TypeScript belongs here too, because it changed the experience of Redux as much as Redux Toolkit did. Early Redux with TypeScript required declaring action type unions by hand and threading them through every reducer and connected component. Modern Redux Toolkit infers almost everything: createSlice infers action creators and their payload types from the reducer signatures, configureStore infers the root state type, and the documented pattern is to derive RootState and AppDispatch types from the store once and create pre-typed hooks with useSelector.withTypes<RootState>() and useDispatch.withTypes<AppDispatch>(), an API added in React-Redux 9.1. RTK Query goes further, generating fully typed hooks per endpoint. In practice, a TypeScript codebase using modern Redux Toolkit has few explicit type annotations in its state code, and refactors such as renaming a field or changing a payload surface every affected call site at compile time. For large teams, this is one of the strongest practical arguments for Redux's structure: the same constraints that make every change go through a declared action also give the type checker a complete map of how state can change. A fair summary is that Redux trades a modest amount of ceremony for strong guarantees about how state changes. If you need those guarantees, modern Redux Toolkit provides them at a reasonable price. If you do not, a lighter store gives up the guarantees in exchange for simplicity. That trade is exactly what Zustand offers, and the next chapter examines it. Chapter 4: Zustand and the Minimal Store Zustand, German for "state," is a small state library from the Poimandres open-source collective, the group behind react-three-fiber, Jotai, and Valtio. It first appeared in 2019, and over the following years it became one of the most widely used state libraries in the React ecosystem, with weekly npm downloads in the same range as Redux Toolkit's. Its appeal is easy to state: it keeps the snapshot model that makes Redux predictable and React-friendly, and it throws away nearly all of the ceremony. No Provider, no actions, no reducers, no slices unless you want them. A store is a hook. This chapter looks at how Zustand works, what changed in version 5, the selector discipline it shares with Redux, how to structure larger stores, and the specific traps that catch teams adopting it. A store in a dozen lines Here is a complete Zustand store and two components that use it: import { create } from 'zustand'; type BearState = { bears: number; fish: number; addBear: () => void; eatFish: (n: number) => void; }; export const useBearStore = create<BearState>()(set => ({ bears: 0, fish: 10, addBear: () => set(s => ({ bears: s.bears + 1 })), eatFish: n => set(s => ({ fish: Math.max(0, s.fish - n) })), })); function BearCount() { const bears = useBearStore(s => s.bears); return <span>{bears} bears</span>; } function Controls() { const addBear = useBearStore(s => s.addBear); return <button onClick={addBear}>Add bear</button>; } create takes a function that receives set (and optionally get and the store API) and returns the initial state, including the functions that update it. It returns a React hook. Calling the hook with a selector subscribes the component to that slice of the store. set merges its argument into the current state, shallowly, at the top level, producing a new state object. The merge is one level deep only. Updating a nested object requires spreading it yourself, or using the Immer middleware described later, exactly as in plain React state. Like Redux, Zustand relies on immutable updates: the new state is a new object, unchanged top-level fields keep their references, and subscribers compare by reference. Under the hood, a Zustand store is a tiny vanilla JavaScript object with getState, setState, subscribe, and getInitialState methods, independent of React. The hook connects that store to React through useSyncExternalStore, the mechanism described in Chapter 2. The vanilla store can be used outside React entirely, which is convenient for reading or updating state from non-component code such as a websocket handler, an analytics module, or a test. // Outside any component: useBearStore.getState().addBear(); const unsub = useBearStore.subscribe(s => console.log(s.bears)); This is a real ergonomic advantage over Context-based solutions and even over React-Redux, where reaching the store from outside React requires importing the store instance and using dispatch and getState explicitly. In Zustand, the store is the module, and anything that can import it can use it. What version 5 changed Zustand 5.0 was released in October 2024. It was deliberately a small release in terms of features and a cleanup in terms of compatibility. The migration guide lists the breaking changes: default exports were removed in favor of named imports; features deprecated in version 4 were dropped; the minimum supported React version became 18 and the minimum TypeScript version 4.5; support for UMD, SystemJS, and ES5 builds was removed; and the typing of setState with the replace flag became stricter, so replacing state requires a complete state object. The change that matters most for day-to-day code concerns equality functions and selector stability. In version 4, create returned a hook that accepted an optional second argument, a custom equality function, so you could write useStore(s => [s.a, s.b], shallow). In version 5, the default create no longer accepts a custom equality function. It uses React's useSyncExternalStore directly, with strict reference comparison. If you need custom equality, you use createWithEqualityFn from zustand/traditional, which requires the use-sync-external-store package as a peer dependency. The consequence is the warning that appears prominently in the migration guide: if a selector returns a new reference on every call, it may cause an infinite loop. A selector like s => ({ bears: s.bears, fish: s.fish }) creates a new object on every invocation. React calls getSnapshot, gets a new object, concludes the store changed, renders, calls getSnapshot again, gets another new object, and so on until React aborts with a "Maximum update depth exceeded" error. In version 4 the same code merely caused extra re-renders; in version 5 it fails loudly. That is arguably an improvement, since the old behavior was a silent performance bug, but it catches teams upgrading code that was sloppy about selectors. The idiomatic fix is the useShallow hook: import { useShallow } from 'zustand/react/shallow'; const { bears, fish } = useBearStore( useShallow(s => ({ bears: s.bears, fish: s.fish })), ); useShallow wraps the selector so that it remembers its previous result and returns that same reference if the new result is shallowly equal, meaning each top-level key holds an Object.is-identical value. React then sees a stable snapshot and renders only when bears or fish actually changes. Alternatively, select each value with a separate hook call: const bears = useBearStore(s => s.bears) and const fish = useBearStore(s => s.fish). Primitive selectors are always stable, and multiple hook calls are cheap. The selector is still the subscription Everything said about selectors in the Redux chapter applies to Zustand with equal force. When the store changes, Zustand runs every subscribed component's selector and compares the result to the previous one. Components whose result is unchanged do not render. The most common Zustand performance mistake is calling the hook with no selector at all: const { bears, addBear } = useBearStore(); // subscribes to EVERYTHING Without a selector, the hook returns the entire state object, which is a new reference after every set. The component re-renders on every change to any field in the store. In a small store that is harmless. In a store that holds, say, both a rarely changing user profile and a rapidly changing drag position, every component that destructures the profile from an unselected hook will re-render dozens of times per second during a drag. The fix is always the same: select narrowly. A related subtlety concerns derived data. A selector that computes something, such as s => s.items.filter(i => i.visible), returns a new array each time and will trigger the infinite-loop failure in version 5. The options are to wrap it in useShallow (effective when the result is an array whose elements are stable references, since shallow comparison checks each element), to memoize the derivation outside the store, or to compute it in the component from a stable selected value, letting useMemo or the React Compiler cache the result. A clean pattern is to select the raw data narrowly and derive in the component: const items = useStore(s => s.items); const visible = useMemo(() => items.filter(i => i.visible), [items]); The selector returns a stable reference (the items array only changes identity when items change), and the derivation reruns only when needed. With the React Compiler enabled, the explicit useMemo can usually be omitted. Transient updates and subscribing outside render One Zustand technique deserves separate mention because it addresses the same high-frequency problem that signals are often proposed for, without leaving the snapshot model. Suppose a store holds the current playback position of a video, updated by the player roughly 60 times per second. A progress bar must reflect it. Selecting the position with the hook would re-render the progress bar component on every update, which is perhaps acceptable for one small component but wasteful if the component is complex or if several such values exist. Zustand's documentation describes transient updates: subscribe to the store directly, outside React's render cycle, and write the value to the DOM through a ref. function ProgressBar() { const barRef = useRef<HTMLDivElement>(null); useEffect(() => usePlayerStore.subscribe(s => { if (barRef.current) barRef.current.style.width = `${s.progress * 100}%`; }), []); return <div className="track"><div ref={barRef} className="bar" /></div>; } The component renders once. Afterward, each store update runs the subscription callback, which sets a single style property on a single element. No component renders, no virtual DOM diff runs, and the update cost is independent of the component tree, which is close to what a signal bound directly to the DOM achieves. The subscribe call returns an unsubscribe function, which the effect returns as its cleanup. The technique has the obvious costs of imperative DOM manipulation: the rendering logic for the bar lives in an effect rather than in JSX, and server-rendered output will not reflect the initial value unless the component also sets it during render. It should be reserved for genuinely hot values. But it shows that the snapshot model is not inherently incapable of fine-grained updates; it simply does not provide them by default, and reaching for them requires stepping outside React deliberately. Structuring larger stores Zustand imposes no structure, which is liberating for small stores and a hazard for large ones. Two patterns have emerged. The first is multiple small stores: one store per domain, such as useEditorStore, useLayoutStore, and usePreferencesStore. This is the simplest approach and works well when the domains are genuinely independent. It gives each store a small surface and makes ownership obvious. Its weakness is coordinating changes across stores. If logging out must reset three stores, something has to call all three reset functions, and there is no equivalent of Redux's single action handled by many reducers. The second is the slices pattern, documented in Zustand's own guides, in which a single store is composed from several functions, each contributing a portion of the state and its actions. Each slice creator receives set and get for the whole store, so slices can read and update one another. This gives a single store with clear internal organization, closer to Redux's structure but without the action and reducer machinery. TypeScript typing of slices requires some care and is the part of Zustand's documentation that most teams end up rereading. A useful rule of thumb is to start with separate stores and merge them into a sliced store only when cross-domain coordination becomes a recurring need. Middleware Zustand's middleware is a set of higher-order functions that wrap the store creator. Three see the most use. devtools connects the store to the Redux DevTools browser extension. Each set call appears as an entry, and you can name actions by passing a third argument to set. Time-travel works, within limits: because Zustand updates are not described by serializable action objects, the log is a sequence of state snapshots rather than a replayable event history. persist saves the store, or a chosen subset via its partialize option, to localStorage, sessionStorage, or a custom storage engine, and rehydrates it on load. It supports versioning and migration functions for when the persisted shape changes. Persisted state is a frequent source of subtle bugs in server-rendered applications, because the server cannot see localStorage and the first client render may differ from the server's HTML. The library's documentation covers hydration handling, and in practice teams often defer reading persisted values until after the client has mounted. immer lets set accept a mutating function in the style of Redux Toolkit's reducers, which is valuable for deeply nested state. Middleware composes by nesting, and the order matters: devtools generally goes outermost so that it observes the final set calls produced by the inner layers. Server rendering and the module-singleton trap Because a Zustand store created with create at module scope is a singleton, it is shared by everything that imports the module. In the browser, where there is one user per page, that is exactly what you want. On a server that renders pages for many users in the same Node.js process, as Next.js and similar frameworks do, a module-level store is shared across requests. If request A writes the current user into the store during rendering, request B might render with A's user. This is a genuine data-leak risk, not a hypothetical one. The documented solution for server-rendered applications is to create the store per request and provide it through React Context. Zustand exposes createStore from zustand/vanilla and a useStore hook that accepts a store instance and a selector. You create the store inside a provider component, typically in a useRef or useState initializer so that it is created once per component instance, put it in Context, and read it with a small custom hook. This sacrifices the "no Provider" convenience, but only in exchange for correctness. Redux faces the same issue and solves it the same way, with a store created per request; its Provider was always required. The same pattern is useful outside server rendering whenever you need several independent instances of a store, for example one per open document tab or one per instance of a complex reusable component. Context scopes the store to a subtree; the store itself still provides selector-based subscriptions, avoiding the broadcast problem of putting the state directly in Context. Actions inside or outside the store A small design decision in Zustand stores has outsized effects on how they age: whether update functions live inside the store's state or outside it. The examples so far define actions such as addBear inside the object passed to create, alongside the data. That is the style the README uses, and it has the advantage that everything about the store is in one place. Its cost is subtle. The state object now mixes data and functions, so persist must be told not to serialize the functions, devtools snapshots include them, and a component that selects "everything" to get at an action subscribes to all the data as well. The alternative, documented in Zustand's guides, keeps only data in the store and defines actions as plain module-level functions that call useStore.setState: export const usePanelStore = create(() => ({ left: 280, right: 320, collapsed: false })); export const toggleCollapsed = () => usePanelStore.setState(s => ({ collapsed: !s.collapsed })); export const resize = (side: 'left' | 'right', px: number) => usePanelStore.setState({ [side]: Math.max(160, px) }); Components import toggleCollapsed directly and never select it from the store, so calling an action never creates a subscription. The functions are trivially callable from non-React code, tree-shakable, and easy to test. The trade-off is that the store's API is spread across a module rather than visible in one object. Both styles are legitimate; the second tends to scale better as stores grow, and it reads closer to the Redux idea of separating what the state is from the events that change it. Zustand compared with Redux Zustand and Redux share a propagation model. Both hold an immutable snapshot, both notify every subscriber on every change, both rely on selectors and reference equality to filter those notifications, and both integrate with React through useSyncExternalStore. Their performance characteristics for comparable usage are therefore similar, and selector quality dominates both. They differ in what they enforce. Redux enforces that every change is an event described by a serializable action and handled by a pure reducer. That enforcement buys replayable history, event-driven decoupling between features, and conventions that hold across large teams. Zustand enforces almost nothing: updates are arbitrary functions that call set, and state can hold anything. That buys brevity, easy access from outside React, and a gentle learning curve. Neither is "lighter" in any way that matters at runtime; both libraries are small. The weight is in the conventions. A reasonable heuristic: if the value of a state change log and event-style decoupling is high, as in an editor, a complex workflow, or an application where reproducing user sessions matters, Redux Toolkit earns its structure. If the shared client state is modest and the team values directness, Zustand is usually the better fit. And if the application's shared client state is very small, as it often turns out to be once server state has been moved into a dedicated cache, plain React state and a couple of narrowly scoped Contexts may be all that is needed. Both libraries, however, accept the same fundamental bargain: the unit of change is a new snapshot, and the unit of update is the React component. The next chapter examines a model that rejects both of those assumptions. Chapter 5: Signals and Fine-Grained Reactivity Signals are the most significant idea to enter mainstream frontend development since React's virtual DOM, and also one of the oldest. The core mechanism, a value that knows who depends on it and tells them when it changes, powers spreadsheets, was the basis of Knockout's observables in 2010, and underlies MobX and Vue's reactivity system. What changed in the early 2020s was convergence. Solid made signals its foundation. Preact released a signals library in 2022. Angular rebuilt its change detection around signals. Vue's ref and computed were recognized as the same primitive under different names. Svelte 5, released in 2024, replaced its compiler-driven reactivity with "runes" built on signals. And in 2024 a group of framework authors brought a proposal to TC39, the committee that standardizes JavaScript, to make signals part of the language. This chapter explains how signals work in general, independent of any framework, so that the comparison with snapshot-based stores is precise. The following chapter deals specifically with signals inside React. The three primitives Nearly every signals implementation offers three building blocks, though the names vary. A signal (also called state, atom, observable, or ref) is a container for a value that can be read and written. Reading it returns the current value. Writing it updates the value and notifies dependents. A computed (also called memo, derived, or selector) is a value defined by a function that reads other signals or computeds. It caches its result and recomputes only when one of the values it read has changed. An effect (also called reaction, autorun, or watcher) is a function that reads signals and performs a side effect, such as updating the DOM, logging, or making a network request. It reruns when anything it read changes. In Preact Signals, the code looks like this: import { signal, computed, effect } from '@preact/signals-core'; const items = signal([{ name: 'Milk', price: 3 }, { name: 'Eggs', price: 5 }]); const discount = signal(0); const total = computed(() => items.value.reduce((sum, i) => sum + i.price, 0) * (1 - discount.value), ); effect(() => { document.title = `Cart: $${total.value.toFixed(2)}`; }); discount.value = 0.1; // effect reruns: title shows $7.20 The striking thing is what is absent. There is no subscription call, no selector, no dependency array. The effect never states that it depends on total, and total never states that it depends on items and discount. The dependencies are discovered automatically. Automatic dependency tracking The mechanism behind this is simple to describe and subtle to implement well. The runtime maintains a notion of the "currently running computation." When a computed or effect begins executing, it registers itself as current. Whenever a signal's value is read, the signal checks who is current and records that computation as a dependent. When the computation finishes, it has a precise list of every signal it read during that run. When a signal is written, it walks its list of dependents and marks them as needing an update. Those dependents in turn mark their own dependents. Effects that are marked get scheduled to rerun. Because tracking happens on every run, dependencies are dynamic. Consider a computed like () => showDetails.value ? details.value : summary.value. When showDetails is false, it reads showDetails and summary, and those are its dependencies. Changing details does nothing, because nothing currently reads it. When showDetails becomes true, the computed reruns, reads details instead, and its dependency list updates. This precision is something that snapshot-based selectors cannot easily match: a Redux selector runs on every action regardless of which branch it would take. The flip side is that only reads that happen synchronously during the computation are tracked. A read inside a setTimeout callback or after an await happens after the computation has finished and is no longer current, so it is not tracked. This is the most common source of confusing signals bugs, and frameworks provide various utilities to explicitly read without tracking, usually called untracked or peek, for the opposite case. Push, pull, and glitch-freedom A naive reactive system pushes changes eagerly: when a signal changes, every dependent recomputes immediately. This produces two well-known problems. The first is wasted work. If a computed is not currently being read by anything visible, recomputing it is pointless. The second is glitches. Imagine a signal a, a computed b = a * 2, and a computed c = a + b. When a changes from 1 to 2, an eager system might recompute c before b has been updated, momentarily producing c = 2 + 2 = 4 instead of the correct 2 + 4 = 6, and then recompute again once b updates. Any effect that runs on the intermediate value sees an inconsistent state that never logically existed. This "diamond dependency" problem is the reactive equivalent of the tearing that useSyncExternalStore prevents in React. Modern signals libraries solve both problems with a push-pull algorithm. When a signal changes, the system pushes only a cheap notification through the graph, marking dependents as possibly stale without recomputing anything. Then, when an effect runs or someone reads a computed, the system pulls: the computed checks whether any of its sources actually changed, recomputing them first if needed, in dependency order. Each computed recomputes at most once per change, and only if something reads it. Effects run only after the graph is consistent, so they never observe a glitch. Libraries add further refinements. Most computeds compare their new result with the old one and, if equal, do not notify their own dependents, so a change can be stopped partway through the graph. Writes can be batched, so that several signal updates in a row trigger dependents only once at the end. Preact Signals, for instance, provides a batch function, and its effects automatically batch updates made inside them. Tracing one update through the graph It helps to trace a single change through a small graph step by step, since the push-pull behavior is easier to see in motion than in description. Take the shopping cart from earlier. There are two signals, items and discount; one computed, total, which reads both; and one effect, which reads total and writes the document title. Add a second computed, itemCount = computed(() => items.value.length), read by a second effect that updates a badge. When the program starts, both effects run once. The title effect reads total, which has never been computed, so total runs, reading items and discount; those two signals record total as a dependent, and total records the title effect as its dependent. The badge effect reads itemCount, which runs and reads items; items now has two dependents, total and itemCount. Now the user applies a discount: discount.value = 0.1. The push phase begins. discount marks its only dependent, total, as possibly stale, and total marks its dependent, the title effect, as needing to run. Nothing is recomputed yet. itemCount and the badge effect are untouched, because discount has no edge leading to them. The pull phase follows. The scheduler runs the title effect, which reads total. total sees that it is marked stale, checks its sources, finds that discount has a new version, and recomputes. Its new value differs from the old one, so the effect receives the new total and updates the title. The badge effect never runs. The work done is one computed evaluation and one effect. Next, the user adds an item: items.value = [...items.value, newItem]. items marks both total and itemCount as stale, and each marks its effect. In the pull phase, each effect pulls its computed, each computed recomputes once, and each effect runs once. If the program had instead changed an item's price without changing the count, itemCount would recompute, find that its result was unchanged, and stop there; the badge effect would be marked but, on checking its source, would find nothing new and skip running. Compare a snapshot store holding the same data. Applying the discount produces a new state object; every subscriber's selector runs, including the badge's, which returns the same count and so does not cause a render. The badge did no rendering in either model, but in the snapshot model it was asked; in the signals model it was never contacted. For two subscribers the difference is meaningless. For two thousand, on every keystroke, it is the whole argument. Why this matters for rendering Now connect this to the cost model from Chapter 2. A state change in React costs roughly the number of components rendered times their per-component cost. Selectors reduce the number notified; memoization reduces the cascade. But the minimum unit of work is still a component function rerunning and a virtual DOM tree being diffed. A signals-based renderer can go below that minimum. In Solid, for example, a component function runs exactly once, when the component is created. The JSX inside it is compiled into code that creates real DOM nodes and sets up an effect for each dynamic expression. When a signal changes, only the effects that read it rerun, and each effect updates a single DOM text node or attribute directly. There is no virtual DOM, no diff, and no component re-render. If a signal holding a username changes, the work done is roughly one text node update, regardless of how large or deeply nested the surrounding component tree is. This is what "fine-grained reactivity" means: the unit of update is a single expression bound to a single piece of the DOM, not a component. It makes performance largely independent of component structure. You can put state at the top of the tree without causing a cascade, because nothing cascades. Ryan Carniato, Solid's creator, and others have demonstrated that this approach performs very well on standard rendering benchmarks such as the community-maintained js-framework-benchmark. The cost is that the programming model changes. Component functions are setup code rather than render functions. Destructuring props reads them once and loses reactivity, so Solid requires props to be accessed as properties, not destructured. Conditional rendering and list rendering use dedicated components such as <Show> and <For> rather than ordinary JavaScript expressions, because a plain if statement in a function that runs once would never re-evaluate. Developers moving from React frequently write code that looks correct and is silently non-reactive. The convergence across frameworks Seeing how different frameworks adopted signals clarifies what is common and what is incidental. Solid uses createSignal, which returns a getter function and a setter function: const [count, setCount] = createSignal(0), read as count(). Its whole rendering model is built on signals, as described above. Preact Signals exposes signals as objects with a .value property. In Preact itself, a signal passed directly into JSX, as in <p>{count}</p> rather than <p>{count.value}</p>, is bound to the text node, and updates bypass the component's re-render entirely. When .value is read inside a component body, the component is subscribed and re-renders like a normal Preact component. Preact Signals also ships a package for React, which Chapter 6 examines. Angular introduced signals in version 16 as a developer preview and has progressively made them the center of the framework, with signal, computed, and effect, signal-based inputs, and a path toward "zoneless" change detection that no longer relies on the zone.js library to detect changes. Angular's motivation was to replace a coarse, tree-wide change detection pass with targeted updates. Vue has had this model since Vue 3's Composition API in 2020: ref is a signal, computed is a computed, and watchEffect is an effect. Vue still uses a virtual DOM, so signals determine which components re-render, and then components diff their templates. That is a middle ground: component-level granularity driven by precise dependency tracking rather than parent re-renders. Svelte 5 introduced runes: $state, $derived, and $effect. These are compiler annotations that the Svelte compiler turns into signal-based code, giving fine-grained updates while letting developers write variables that look like plain JavaScript. The surface syntax ranges from getter functions to .value properties to compiler magic, but the core is identical: tracked reads, cached derivations, scheduled effects, push-pull propagation. The TC39 Signals proposal That convergence led directly to an attempt at standardization. The JavaScript Signals proposal was presented to TC39 in 2024 by champions including Daniel Ehrenberg and Rob Eisenberg, the original authors, with contributors from the teams behind Angular, Vue, Solid, Preact, Svelte, Ember, MobX, Qwik, and others. It reached Stage 1 in April 2024, which in TC39's process means the committee agrees the problem is worth exploring, not that it has accepted a solution. As of late 2026 the proposal remains at Stage 1. The proposal is deliberately low-level. It defines Signal.State for writable signals, Signal.Computed for lazily evaluated cached derivations, and, in a Signal.subtle namespace, a Watcher class that frameworks can use to build their own effect scheduling. It does not include a built-in effect function, because when and how effects run is a policy decision that differs between frameworks: Solid, Angular, and Vue schedule effects differently, and the proposal's authors chose to standardize the graph, not the scheduler. The stated goals are interoperability, so that a signal from one library could be read inside a computed from another and dependencies would track correctly, and performance, since a native implementation could be faster and use less memory than a JavaScript one. A polyfill exists. The champions have said they intend to advance carefully, building production-quality polyfills and integrating with multiple frameworks before seeking Stage 2, and have themselves estimated that native browser availability is at least several years away. Teams should treat the proposal as evidence of where the ecosystem's center of gravity is heading, not as something to build on today. What signals cost Fine-grained reactivity is not free. Every signal stores a list of its dependents, and every computed stores its sources, so memory use grows with the number of reactive edges. Dependency tracking adds overhead to every read inside a tracked context. For a large list where each item is wrapped in signals, setup can be more expensive than in a virtual DOM approach, even if updates are cheaper. The model also has failure modes that snapshot-based stores do not. Because there is no single state snapshot, there is no built-in history to time-travel through and no single serializable object to persist or inspect; libraries can add these, but they are not inherent. Dependency graphs can become hard to reason about: an effect that writes a signal that another effect reads can create cycles or cascades that are difficult to trace. And because reactivity is implicit, it can be unclear from reading code which values are reactive and which are plain, especially when signals are passed through functions or stored in objects. The snapshot model asks every subscriber about every change and pays for that with redundant selector runs and renders. The dependency-tracking model asks only the relevant subscribers but pays with runtime bookkeeping, a more complex mental model, and less inherent inspectability. In a renderer designed around signals, the balance tips strongly toward signals for update performance. Whether it tips the same way inside React, which was designed around the snapshot model, is the subject of the next chapter. Chapter 6: Signals Inside React If signals are so efficient, why doesn't React use them? The question comes up constantly, and the React team has answered it in various forms over the years. The short version is that React's design treats UI as a function of state that can be re-run at any time, and that this property, not raw update speed, is what React optimizes for. Signals can be used in React applications, and there are good libraries for doing so, but they work against some of React's assumptions in ways that matter. This chapter examines how signals integrate with React, where they help, where they cause friction, and what the React Compiler changes about the comparison. React's model and why signals sit awkwardly in it React's core idea is that a component is a function from props and state to a description of UI. Calling it again with the same inputs must produce equivalent output. React exploits that purity relentlessly. Concurrent rendering may call a component, throw the result away, and call it again later. Strict Mode in development calls components twice to catch impure code. Suspense may start rendering a tree, pause when data is missing, and restart. Transitions let React render a new UI in the background while the old one stays interactive, and discard it if the user moves on. All of these depend on renders being disposable. Signals introduce a different idea: a mutable value that lives outside the render cycle and changes on its own schedule. When a component reads count.value during render, it is reading a mutable cell, not an immutable snapshot. If the signal changes partway through a concurrent render, different components may see different values, which is exactly the tearing problem from Chapter 2. React's answer to external mutable state is useSyncExternalStore, and any correct signals integration for React must route through it or something equivalent, which forces signal updates back into React's synchronous, component-level rendering. At that point, much of what made signals fast in Solid, the ability to update a text node without rendering a component, is lost or must be achieved by bypassing React. There is also a philosophical objection that React team members have voiced. In React, data flows down explicitly through props, and the question "why did this component render?" has a clear answer: its state changed, its parent rendered, or a context or store it reads changed. With implicit dependency tracking, a component's dependencies are whatever it happened to read during its last render, which is harder to see in the code. React's direction has been to keep the programming model explicit and to recover performance with a compiler, rather than to make reactivity implicit at runtime. None of this means signals cannot be useful in React. It means their benefits inside React are smaller and their costs larger than in a framework built for them. Preact Signals for React The most prominent signals library for React is @preact/signals-react, maintained by the Preact team. It provides the same signal, computed, and effect primitives as the core library, plus integration with React components. The integration has gone through significant changes. Early versions patched React's internals at runtime to make components automatically subscribe to any signals they read. That approach was fragile, broke with React updates, and was incompatible with some React features. Current versions take a more conservative approach and offer two ways to make a component reactive. The recommended way is a Babel plugin, @preact/signals-react-transform. It rewrites component code at build time so that each component that reads signals is wrapped in tracking logic. After the transform, reading count.value inside a component subscribes that component to count, and changes to count re-render it. The alternative, for projects that cannot add a Babel plugin, is the useSignals hook from @preact/signals-react/runtime, called at the top of any component that reads signals. It does explicitly what the transform does automatically. import { signal, computed } from '@preact/signals-react'; import { useSignals } from '@preact/signals-react/runtime'; const count = signal(0); const doubled = computed(() => count.value * 2); function Counter() { useSignals(); // not needed if the Babel transform is configured return ( <button onClick={() => count.value++}> {count.value} doubled is {doubled.value} </button> ); } There is also an optimization that recovers some fine-grained behavior. If you pass the signal itself as a JSX child, writing {count} rather than {count.value}, the library renders a small internal component that subscribes to the signal and updates only the text it produces. The parent component does not re-render when count changes. This is the closest React gets to Solid-style updates without leaving React's rendering model. The library documents several limitations. Signals cannot be passed as DOM attributes in the React integration, so <input value={sig} /> does not bind; you must pass sig.value and accept a component re-render. Signals read inside render props may not be tracked correctly, because tracking is tied to the component that calls the render function. Object getters and setters can hide signal reads from the static analysis the transform depends on. Configuration options exist to broaden the transform's tracking to catch these cases, at some cost. Where signals help in React Despite the friction, signals have real advantages in React for particular kinds of state. The first is high-frequency updates that should touch only a small part of the UI. A live price ticker, a timer, a progress indicator, or a cursor coordinate display changes many times per second. With a snapshot store, each change runs every subscriber's selector and renders every component whose selection changed. With signals rendered directly as JSX text, a change updates only the text node, with no component render at all. For a dashboard with many independently updating values, the difference can be substantial. The second is derived state with complex dependency graphs. Computed signals cache automatically and recompute only when their actual dependencies change, including dependencies that vary by branch. Equivalent memoization with Reselect requires carefully composing input selectors, and conditional dependencies are awkward to express. The third is state that must be shared outside React. Like a Zustand store, a signal is a plain JavaScript object that any module can import, read, and write. Unlike a Zustand store, it carries its own subscription mechanism for non-React code through effect, with automatic dependency tracking. Where signals hurt in React The costs fall into three groups. Integration fragility. Reactivity depends on a build-time transform or on calling a hook in every component that reads signals. Forget the hook in one component, or use a pattern the transform cannot analyze, and that component silently stops updating. These bugs are hard to diagnose because the code looks correct. The history of runtime patching in earlier versions of the React integration also gives some teams pause about long-term stability, since the library's integration depends on React internals and conventions that the React team does not treat as a public contract. Mismatch with React's concurrent features. Signal writes are synchronous and immediate. React's transitions, which let a state update render in the background at lower priority, are built around React state. A signal update cannot be marked as a transition in the same way, so the fine control React offers over update priority is partly lost for signal-driven state. Two mental models in one codebase. A React application using signals has components whose re-renders are driven by props and React state, components whose re-renders are driven by tracked signal reads, and JSX expressions that update without any re-render. Developers must hold all three models in mind. This is a significant onboarding and maintenance cost, and it is often the deciding factor for teams that have experimented with signals and backed away. The comparison in one place Table 2 summarizes how the three approaches behave inside a React application. It describes typical usage, not edge cases, and assumes each tool is used correctly. Table 2. Redux, Zustand, and Preact Signals inside React. Dimension Redux Toolkit Zustand Preact Signals Change model Dispatched actions Direct set calls Direct .value writes Who is notified All subscribers All subscribers Only tracked readers Filtering Selectors, === Selectors, === or shallow Automatic tracking Unit of update Component Component Component or text node Debugging Action log, time travel DevTools snapshots Limited by default Main hazard Unstable selectors Unstable selectors Lost tracking The table makes the central trade visible. The snapshot libraries do redundant work, asking every subscriber about every change, but the work is explicit and the failure mode, an unstable selector, produces visible over-rendering or, in Zustand 5, a loud error. Signals do less work, but the failure mode, a read that was not tracked, produces a silently stale UI. What the React Compiler changes When signals libraries for React first gained attention in 2022, part of their pitch was that they eliminated the need for useMemo, useCallback, and React.memo. Because signal-driven updates did not cascade through the component tree, developers did not have to memoize children to prevent unnecessary renders. That was a real advantage over hand-optimized React at the time. React Compiler 1.0 narrows that advantage considerably. With the compiler enabled, a parent re-render no longer cascades to children whose props have not changed, because the compiler memoizes JSX elements automatically. The cascade problem that signals sidestepped is now largely handled at build time, for all state, without changing the programming model. What remains distinctive about signals in React is the ability to skip even the notified component's render by binding a signal directly to a text node, and the automatic dependency tracking of computeds. For most applications, that remaining advantage matters only on genuinely hot paths. A reasonable strategy, if you are drawn to signals, is to use them surgically: a signal for a high-frequency value rendered in a few places, alongside conventional state management for everything else. Using signals as the application's primary state model inside React means accepting the integration costs above across the whole codebase in exchange for performance gains that, after the compiler, are often modest. A surgical example: a live order book To see what surgical use looks like, consider a trading interface that displays an order book: two columns of price levels, perhaps 25 on each side, each showing a price and a quantity, with quantities changing many times per second as orders arrive over a websocket. Around the order book sit a chart, an order entry form, account balances, and a list of open orders, all built with ordinary React state and a Zustand store. A first implementation might hold the order book in the Zustand store and have each price-level row select its own quantity. That is a reasonable design. Each websocket message produces a new store state, every row's selector runs, and only the rows whose quantity changed re-render. With 50 rows and dozens of messages per second, the work is perhaps a few hundred selector calls and a handful of row renders per message. On a desktop machine, this is likely fine. On a low-end laptop running several other heavy tabs, profiling might show that the combination of store notifications across the whole application, since every subscriber in the store runs its selector on every message, and row renders adds up to a meaningful share of each frame. A surgical alternative keeps everything except the order book exactly as it was, and moves the 50 quantities into signals, one per price level. The websocket handler writes levelSignals[i].value = qty directly. Each row renders its quantity as {levelSignals[i]}, passing the signal itself into JSX rather than reading .value. Now an incoming message updates the text of exactly the changed quantities and renders no React components at all. The rest of the application's store subscribers never hear about order book updates because the order book is no longer in the store. The trade-offs of this change are contained. Only the order book module needs to understand signals. The integration's limitations, such as the inability to bind signals to DOM attributes, matter only if the rows need, for instance, a background color driven by the quantity; that case would need either .value and a row re-render, or a small effect that sets the style through a ref. The rest of the team can continue working with conventional React patterns. The broader lesson is that signals in React are most defensible when they are confined to a small, clearly bounded region of the application with an unusual update profile, and when profiling has shown that the conventional approach is actually too slow there. Under those conditions, they provide something the snapshot model cannot: updates whose cost is independent of the component tree. A note on other atomic and proxy-based libraries Signals are not the only alternative to snapshot stores in React, and two related families deserve mention because they occupy the middle ground. Atomic libraries such as Jotai and Recoil let you define small independent pieces of state, called atoms, and derived atoms that compute from them. Components subscribe to individual atoms. Dependency tracking between derived atoms is automatic, like computeds in signals. But updates flow through React's rendering model, using React-friendly subscriptions, so the unit of update remains the component. Jotai, also from the Poimandres collective, is actively maintained and popular. Recoil, Meta's experimental atomic library, was archived in early 2025 and should not be chosen for new projects. Proxy-based libraries such as MobX and Valtio wrap ordinary objects in JavaScript Proxies so that reads are tracked and writes trigger updates. You mutate state directly, as in state.count++, and components that read count re-render. MobX has been used in large production applications for about a decade and is the most mature dependency-tracking option for React. It has the same strengths and the same integration questions as signals: precise updates, automatic derivation, and a mental model different from React's default. The existence of these options reinforces the point of the chapter. Fine-grained, dependency-tracked state is entirely usable in React and has been for years. It has never become the default, not because it does not work, but because React's model rewards explicit data flow and snapshot semantics, and the ecosystem has invested in making those cheap rather than replacing them. With the compiler, that investment has paid off further. The next chapter turns from models to practice: how to find and fix re-render problems in a real React application, whatever state approach it uses. Chapter 7: Optimizing Re-renders in Practice Discussions of state management often treat re-renders as inherently bad, as though the goal were to minimize them. That is the wrong goal. React is designed to render often, and most renders are cheap: a component function that returns a handful of elements runs in microseconds. The goal is to keep the interface responsive, which means keeping the work done in response to each interaction below the threshold users notice. A useful benchmark is the browser's frame budget. At 60 frames per second, each frame has about 16.7 milliseconds, and the browser needs part of that for layout, paint, and its own housekeeping. Google's Interaction to Next Paint metric, a Core Web Vital since March 2024, classifies interactions as "good" when the time from input to the next paint is 200 milliseconds or less at the 75th percentile. If typing in a text field triggers 40 milliseconds of rendering per keystroke, users will feel it. If it triggers 2 milliseconds, nobody will, even if a hundred components re-rendered. This chapter is about finding the renders that matter and eliminating them with the least invasive technique available. It works through a measurement process, a hierarchy of fixes, a realistic example, and the boundary between what the React Compiler handles and what still requires design. Measure before changing anything The first rule of re-render optimization is the first rule of all performance work: measure. Intuition about where time goes in a React application is unreliable, and optimizations applied blindly add complexity without benefit. The primary tool is the React DevTools Profiler, a tab in the React Developer Tools browser extension. You start recording, perform the slow interaction, and stop. The Profiler shows each commit React made, how long it took, and which components rendered in it, with a flame graph colored by render duration. Enabling the setting that records why each component rendered shows whether a component rendered because its props changed, its state changed, its parent rendered, or a hook changed. This answers the critical question directly. If a component rendered only because its parent did, and its props were unchanged, the fix is memoization or restructuring. If it rendered because a store subscription fired, the fix is a narrower selector. The DevTools also include a "highlight updates when components render" option that flashes an outline around each component as it renders. It is crude but immediately revealing: type into a search box and watch the whole page flash, and you know the state lives too high. The browser's own performance panel complements the Profiler. React's render time is only part of an interaction. Layout thrashing, expensive style recalculation, large DOM updates, and long-running event handlers show up there and not in React's tools. React 19.2 added custom tracks to the Chrome DevTools performance panel, showing React's scheduler priorities and component render work alongside the browser's timeline. In production, the <Profiler> component can wrap parts of the tree and report render timings to an analytics endpoint. Combined with real-user monitoring of Interaction to Next Paint, this shows which interactions are slow for real users on real devices, which is often different from what developers see on fast laptops. A hierarchy of fixes Once the profiler has identified a costly render, the fixes fall into a rough order, from the least invasive and most broadly beneficial to the most specialized. It pays to try them in this order. 1. Move state down (colocation). If state lives in a component high in the tree but is only used by a small subtree, move it into that subtree. A search query used only by a search panel belongs in the search panel, not in the page root. When it changes, only the panel and its children render. This single technique fixes a large fraction of real-world re-render problems and makes code clearer at the same time. 2. Lift content up (composition). When a component that owns frequently changing state must wrap expensive children, pass the children in as props instead of rendering them directly. A component that tracks mouse position and renders {children} will re-render on every mouse move, but the children elements were created by its parent and have the same identity each time, so React skips them. This is a pure restructuring with no memoization APIs, and it works with or without the compiler. // Before: ExpensiveTree re-renders on every mouse move. function Tracker() { const [pos, setPos] = useState({ x: 0, y: 0 }); return <div onMouseMove={e => setPos({ x: e.clientX, y: e.clientY })}> <Cursor pos={pos} /><ExpensiveTree /> </div>; } // After: ExpensiveTree is created by the parent and passed in. function Tracker({ children }) { const [pos, setPos] = useState({ x: 0, y: 0 }); return <div onMouseMove={e => setPos({ x: e.clientX, y: e.clientY })}> <Cursor pos={pos} />{children} </div>; } // <Tracker><ExpensiveTree /></Tracker> 3. Narrow subscriptions. For store-connected components, select the smallest value the component needs. Replace useStore() with useStore(s => s.count). Replace a selector that returns an object with several primitive selectors, or wrap it in useShallow or a memoized createSelector. For Context, split large contexts into smaller ones by update frequency, and separate state from setters. 4. Memoize. Wrap expensive components in React.memo, stabilize props with useMemo and useCallback, or, preferably, enable the React Compiler and let it do this automatically. 5. Make each render cheaper. Move expensive computations out of render or memoize them. Virtualize long lists so that only visible rows are rendered, using a library such as TanStack Virtual or react-window. A table with 5,000 rows renders 5,000 row components on every update; a virtualized one renders the 30 or so that fit on screen. 6. Defer non-urgent work. Use useTransition or useDeferredValue to let React render an expensive update at lower priority, keeping the input responsive. A search box can update its own value urgently while the results list renders from a deferred copy of the query. The input stays at full speed; the results catch up a few frames later and are discarded if the user types again first. 7. Bypass React for the hot path. For values that update dozens of times per second, such as animation progress or a live cursor, update the DOM directly through a ref, use a signal bound directly to a text node, or use an animation library that drives styles outside the render cycle. This is the most invasive technique and should be reserved for genuinely hot paths. A worked example: the slow spreadsheet Consider a realistic case. A team builds an internal tool that displays an editable grid of 200 rows by 20 columns, so 4,000 cells. The grid's data lives in a Zustand store as an array of row objects. Users report that typing into a cell is laggy, with a visible delay before characters appear. Profiling one keystroke shows a 90-millisecond commit in which every one of the 4,000 cells rendered. The "why did this render" information shows the Grid component rendered because a hook changed, and every Cell rendered because its parent rendered. The code looks like this: function Grid() { const rows = useGridStore(s => s.rows); const updateCell = useGridStore(s => s.updateCell); return rows.map((row, r) => ( <Row key={row.id} row={row} r={r} onEdit={(c, v) => updateCell(r, c, v)} /> )); } Three problems compound. First, Grid selects the whole rows array, so any cell edit gives it a new array and it re-renders. That is acceptable on its own. Second, it passes a fresh arrow function as onEdit to every row on every render, so even a memoized Row would see changed props. Third, Row passes its row object down to each Cell, and nothing is memoized. The update itself is written correctly with immutable semantics: editing one cell produces a new rows array, a new object for the edited row, and the same references for the other 199 rows. That is the key observation. The data already carries the information needed to skip 199 rows; the components just are not using it. The first fix is to enable the React Compiler. Recompiled, Grid still renders on each edit, but the compiler caches the callback per row and memoizes each Row element based on its props. For the 199 unchanged rows, row is the same reference and the callback is cached, so React reuses the previous elements and skips them. Within the edited row, the same logic skips 19 of 20 cells. The commit drops from 4,000 cell renders to roughly one row and one cell, and profiling shows the keystroke commit falling to a few milliseconds. Without the compiler, the same result requires wrapping Row and Cell in React.memo and restructuring the callback so it is stable, for example by passing updateCell and the row index separately and having the row call updateCell(r, c, v) itself. An alternative design pushes subscriptions down: Grid selects only the list of row IDs, which changes only when rows are added or removed; each Row selects its own row object by ID; and each Cell selects its own value. Now an edit notifies only the one cell whose value changed, and nothing else renders even without memoization. The cost is 4,000 store subscriptions, each with a selector that runs on every store change. For a grid this size that is fine, since 4,000 trivial selector calls take well under a millisecond, but for a grid ten times larger, the selector cost itself becomes measurable, and virtualization is the better answer anyway. The example illustrates a general pattern. Most re-render problems involve correct immutable data and wasteful component structure. The fixes are colocation, narrower subscriptions, and memoization, not a change of state library. A team that responded to this lag by migrating from Zustand to Redux, or from Zustand to signals, would have done a great deal of work and very likely reproduced the same problem. A second example: the overloaded context Not every problem involves a store. A common case involves a design system's ThemeProvider extended over time to hold more than the theme. Suppose it began holding the color mode and later accumulated the current viewport breakpoint, computed from a resize listener, and a "reduced motion" flag. Every themed component in the application, which is to say nearly every component, reads this context. Profiling a window resize shows the problem immediately: as the user drags the browser edge, the breakpoint value updates on each resize event, the context value changes identity, and every component reading the context renders, many times per second. The compiler cannot help, because the components genuinely read the context and the context genuinely changed. The fix has two parts. First, split the context by update frequency: color mode and reduced motion change rarely and stay together; the breakpoint moves to its own context, read only by the components that adapt their layout. Second, reduce the breakpoint's update frequency at the source: instead of storing the raw window width, store only the named breakpoint, such as 'sm', 'md', or 'lg', and update the context only when that name changes. A resize from 1,100 to 1,050 pixels then produces no update at all. Better still, many layout adaptations can be handled by CSS media or container queries without any JavaScript state, removing the need for the context entirely. After the change, a window resize produces a handful of updates at breakpoint boundaries, each re-rendering only layout-aware components. Nothing about the state approach changed; the fix was entirely about subscription granularity and the precision of the state itself. What the compiler handles and what it does not The React Compiler automates the fourth item in the hierarchy, memoization, thoroughly and more consistently than most developers manage by hand. In practice, it often hides the absence of the first three: a component tree with state placed too high may perform acceptably once compiled, because the cascade is cut off by memoized children. It does not automate the rest. It will not move state down for you. It will not split a Context whose consumers re-render on every change. It will not fix a store hook called without a selector, because the component genuinely reads the whole state and the compiler cannot know which fields matter at runtime. It will not virtualize a list or make an expensive calculation cheaper on the occasions when it must run. And it skips components that break the Rules of React, such as those that mutate props or read refs during render, leaving them unoptimized; the compiler-powered lint rules in eslint-plugin-react-hooks flag these so they can be fixed. There is also a subtle interaction with existing manual memoization. The React team's guidance is that useMemo and useCallback can stay as escape hatches, particularly when a memoized value is used as an effect dependency and its identity must be guaranteed, but that new code should generally rely on the compiler. When adopting the compiler in an existing codebase, removing manual memoization should be done carefully and with testing, because some of it may be load-bearing for effect behavior rather than performance. Anti-patterns worth naming Several patterns recur in codebases with re-render problems, regardless of which state library they use. Storing derived data. Keeping filteredItems in a store alongside items and filter means every change must update both, and any code that forgets creates an inconsistency. It also causes extra renders, since updating items triggers one notification and updating filteredItems another. Derive on read. Syncing state with effects. An effect that watches one piece of state and sets another, such as useEffect(() => setTotal(sum(items)), [items]), causes two renders for every change: one with the stale total and one with the updated value. The intermediate render can even flash incorrect data. Compute total during render instead. Giant context values. A single AppContext holding user, settings, cart, notifications, and their setters turns every change anywhere into a re-render everywhere. Split it. Unselected store hooks. Calling useStore() with no selector, often done for convenience during prototyping and never revisited, subscribes the component to every change in the store. Putting everything in a global store. Local UI state such as an open dropdown or a hover state in a global store means that toggling a tooltip runs every subscriber's selector across the application. It is not usually a measurable cost, but it is a sign that the classification from Chapter 1 was skipped, and it tends to accompany worse problems. Every one of these is a structural mistake that no library choice fixes. The reassuring corollary is that a well-structured React application, with state classified correctly, subscriptions narrowed, and the compiler enabled, is fast with any of the mainstream state libraries. Performance differences between Redux, Zustand, and signals matter at the margins and on hot paths; structure matters everywhere. The largest structural improvement available to most applications, however, has nothing to do with client state at all. It is moving server data out of client stores and into a dedicated cache, which is the subject of the next chapter. Chapter 8: Server State and Asynchronous Data The largest single improvement most React codebases can make to their state management has nothing to do with Redux, Zustand, or signals. It is recognizing that data fetched from a server is not client state and should not be managed as if it were. Server data is a cache: a local copy of something owned elsewhere, which may be stale at any moment and must be refreshed according to rules. Caches are a well-understood problem with well-understood hazards, and two libraries dominate the solution space in React: TanStack Query and Redux Toolkit's RTK Query. This chapter explains what problems a server-state cache solves, how both libraries work, how they handle mutations and optimistic updates, and the asynchronous failure modes that any approach must address. Why hand-rolled fetching fails The simplest way to fetch data in React is an effect that calls fetch and stores the result in state: function Product({ id }) { const [product, setProduct] = useState(null); useEffect(() => { fetch(`/api/products/${id}`).then(r => r.json()).then(setProduct); }, [id]); return product ? <Details product={product} /> : <Spinner />; } This code has at least six problems, all of which show up in production. It has a race condition. If id changes from 1 to 2 and the response for 1 arrives after the response for 2, the component displays product 1 under the heading for product 2. Fixing this requires tracking whether the effect has been superseded, typically with an ignore flag set in the effect's cleanup or an AbortController. It has no error handling: a failed request leaves the spinner showing forever. It has no caching. Navigate away and back, and the component refetches and shows a spinner, even though the data was on screen a moment ago. It has no deduplication. If three components on the page need product 1, three identical requests go out. It has no revalidation. If the product changes on the server, the client never finds out unless the component remounts. It has no coordination with mutations. After the user edits the product elsewhere in the application, this component continues to show the old data. Moving the same logic into a global store with thunks fixes caching and deduplication in principle, since the data lives somewhere shared, but every team ends up implementing staleness rules, in-flight tracking, error states, retry, and invalidation by hand. This is the hand-built cache described in Chapter 1, and it is consistently one of the buggiest parts of applications that have one. A dedicated server-state library makes all of these behaviors the default, configured declaratively. TanStack Query TanStack Query, created by Tanner Linsley and originally released as React Query, is the most widely used server-state library for React. It also supports Vue, Solid, Svelte, and Angular through framework adapters around a shared core. Version 5, released in October 2023, remains the current major version for React as of late 2026, with frequent minor releases. The central abstraction is the query, identified by a query key, an array that uniquely describes the data, and backed by a query function that fetches it. import { useQuery } from '@tanstack/react-query'; function Product({ id }: { id: string }) { const { data, isPending, isError, error } = useQuery({ queryKey: ['product', id], queryFn: ({ signal }) => fetch(`/api/products/${id}`, { signal }).then(r => { if (!r.ok) throw new Error(`HTTP ${r.status}`); return r.json(); }), staleTime: 30_000, }); if (isPending) return <Spinner />; if (isError) return <ErrorView error={error} />; return <Details product={data} />; } Every component that calls useQuery with the key ['product', '1'] shares one cache entry. If several mount at once, one request goes out. The signal passed to the query function is an AbortSignal that the library triggers when the query is no longer needed, cancelling the request. Because each key has its own cache entry, the race condition disappears: the response for product 1 is stored under product 1's key and cannot overwrite product 2. Two timing settings govern the cache, and understanding them is most of understanding TanStack Query. staleTime is how long fetched data is considered fresh. While fresh, the library serves it from the cache without refetching. The default is zero: data is considered stale immediately, though it is still displayed. Stale data is refetched in the background when certain triggers occur, such as a new component mounting with the same key, the browser window regaining focus, or the network reconnecting. This behavior, sometimes called stale-while-revalidate, shows cached data instantly and quietly replaces it with fresh data when it arrives. gcTime (called cacheTime before version 5) is how long an unused cache entry is kept in memory after the last component using it unmounts. The default is five minutes. If the user navigates back within that window, the cached data appears instantly while a background refetch runs. Teams new to the library are often surprised by the amount of refetching the defaults produce, particularly refetch on window focus. The defaults are deliberately aggressive about freshness. Raising staleTime for data that changes rarely, such as reference lists or user profiles, is the usual first configuration step. Version 5 brought several changes worth knowing. All hooks take a single options object rather than positional arguments. The isLoading status was renamed isPending to mean "no data yet," while isLoading now means pending and actively fetching. The keepPreviousData option was replaced by passing the keepPreviousData helper function to placeholderData, used for keeping the previous page visible while the next one loads in paginated views. Dedicated useSuspenseQuery hooks integrate with React Suspense, and infinite queries require an explicit initialPageParam. RTK Query RTK Query is a server-state cache that ships inside Redux Toolkit. It solves the same problems as TanStack Query with a different organizational style: rather than defining queries where they are used, you define an API slice that describes all the endpoints of a backend in one place, and the library generates typed hooks for each. import { createApi, fetchBaseQuery } from '@reduxjs/toolkit/query/react'; export const api = createApi({ baseQuery: fetchBaseQuery({ baseUrl: '/api' }), tagTypes: ['Product'], endpoints: build => ({ getProduct: build.query<Product, string>({ query: id => `products/${id}`, providesTags: (result, error, id) => [{ type: 'Product', id }], }), updateProduct: build.mutation<Product, Partial<Product> & { id: string }>({ query: ({ id, ...patch }) => ({ url: `products/${id}`, method: 'PATCH', body: patch }), invalidatesTags: (result, error, { id }) => [{ type: 'Product', id }], }), }), }); export const { useGetProductQuery, useUpdateProductMutation } = api; The cache lives in the Redux store, so it is visible in the Redux DevTools, and cache updates are ordinary Redux actions that other reducers and listener middleware can respond to. Unused data is kept for 60 seconds by default, configurable with keepUnusedDataFor. Refetch on focus and on reconnect are available but off by default, a more conservative starting point than TanStack Query's. The most distinctive feature is tag-based invalidation. Queries declare which tags their data provides; mutations declare which tags they invalidate. When the updateProduct mutation succeeds, RTK Query invalidates the tag { type: 'Product', id }, and every active query that provided that tag refetches automatically. This keeps the relationship between reads and writes declarative and centralized. For teams with OpenAPI specifications, the @rtk-query/codegen-openapi tool can generate the entire API slice from the spec. Redux Toolkit 2.6 added infinite query support, closing one of the larger feature gaps with TanStack Query. Choosing between them Table 3 compares the two libraries on the dimensions where they most often differ in practice. Table 3. TanStack Query and RTK Query compared. Dimension TanStack Query v5 RTK Query (RTK 2.x) Definition style Per-use query options Central API slice Cache location Own QueryClient Redux store Invalidation By query key prefix By provided/invalidated tags Default refetch on focus On Off Default unused retention 5 minutes 60 seconds Best fit Any React app Apps already using Redux The decision is usually straightforward. If the application already uses Redux Toolkit, RTK Query integrates naturally and adds little to bundle size. If it does not, TanStack Query is the more common choice, has a larger ecosystem of plugins and examples, and does not require adopting Redux. Both are mature and well maintained, and teams rarely regret either choice. The decision that matters is using one of them rather than a hand-built cache. Mutations, invalidation, and optimistic updates Reading data is half the problem. When the user changes something, the cache must reflect it. The simplest strategy is invalidate and refetch. After a mutation succeeds, mark affected queries as stale and let them refetch. In TanStack Query this is queryClient.invalidateQueries({ queryKey: ['product'] }), which matches every key beginning with 'product'. Invalidation is robust because the server remains the source of truth, but the user waits for a round trip before seeing the change. Optimistic updates update the cache immediately, assuming the mutation will succeed, and roll back if it fails. This makes interfaces feel instantaneous, and it is how applications like task managers and chat clients keep up with fast user input. The pattern in TanStack Query uses the mutation's lifecycle callbacks: in onMutate, cancel any in-flight refetches for the affected key so they cannot overwrite the optimistic value, snapshot the current cached value, and write the optimistic value with setQueryData; in onError, restore the snapshot; and in onSettled, invalidate the key to reconcile with the server. RTK Query's equivalent uses an endpoint's onQueryStarted callback with api.util.updateQueryData, which returns a patch object with an undo method for rollback. Version 5 of TanStack Query also offers a simpler form for cases where the optimistic value appears in only one place: render the mutation's variables, the data being submitted, directly in the UI while the mutation is pending, without touching the cache at all. If the mutation fails, the pending item simply disappears or is shown with an error state. This avoids rollback logic entirely and is often the better choice. It parallels React 19's useOptimistic, discussed in the next chapter. Optimistic updates carry real risks. If two mutations to the same record are in flight, rollbacks can restore the wrong snapshot. If the server transforms data, for example by assigning an ID or normalizing text, the optimistic value differs from the final one, and the UI may visibly flicker on reconciliation. And for operations whose failure is consequential, such as payments, an interface that briefly shows success and then reverses is worse than one that waits. Use optimistic updates where the operation is low-stakes, failure is rare, and speed matters. Polling, push, and real-time data Some server state changes too often for focus-based revalidation to keep up: a job queue's status, a delivery's location, a chat thread. Server-state libraries offer two strategies, and the choice between them is an architectural one. Polling is the simplest. TanStack Query's refetchInterval option refetches a query on a timer, and it can be a function that computes the interval from the current data, so a build-status query can poll every two seconds while a build is running and stop once it finishes. RTK Query offers pollingInterval with equivalent behavior. By default, TanStack Query pauses interval refetching while the browser tab is in the background, which avoids wasting requests on tabs nobody is looking at. Polling is robust, works through any proxy or firewall, and needs no server changes, but it trades latency against load: a shorter interval means fresher data and more requests. Push delivers changes from the server as they happen, over a WebSocket or a Server-Sent Events stream. The question then is how pushed messages interact with the cache. There are two patterns. In the first, the message carries only a notification, such as "order 42 changed," and the handler calls invalidateQueries for the affected key, letting the normal fetch machinery retrieve the new data. This keeps the server as the source of truth and reuses existing query functions, at the cost of an extra request per change. In the second, the message carries the new data, and the handler writes it into the cache with setQueryData, or updateQueryData in RTK Query, which RTK Query's documentation demonstrates through an endpoint's onCacheEntryAdded lifecycle hook for streaming updates. This is faster and lighter on the server but duplicates data-shaping logic between the fetch path and the push path, and the two can drift. For most applications, notification-plus-invalidation is the better default. It is easy to reason about, it degrades gracefully if a message is missed, since the next focus or mount revalidates anyway, and it keeps one code path for shaping data. Direct cache writes are worth their complexity for very high-frequency streams, such as live prices or collaborative cursors, where an extra request per message is untenable. At that frequency, as Chapter 6 discussed, the data may not belong in a request cache at all, and a dedicated real-time store or a signal per value can be the better home. Request waterfalls and prefetching The final major hazard in asynchronous data is the waterfall: a sequence of requests where each starts only after the previous finishes, even though they could run in parallel. Component-level fetching makes waterfalls easy to create by accident. A page component fetches a user, renders a profile component once the user arrives, which fetches the user's projects, and renders a project list once those arrive, which fetches each project's latest activity. Three sequential round trips, each perhaps 150 milliseconds on a mobile network, add almost half a second before the page is complete. Server-state libraries do not prevent waterfalls by themselves; they cache what is fetched but cannot start a request for a component that has not rendered yet. The remedies are structural. Fetch data at the route level rather than deep in the component tree, so that all queries for a page start together; modern routers such as React Router and TanStack Router provide route loaders for this. Prefetch data you can predict the user will need, for example with queryClient.prefetchQuery when the user hovers over a link. Restructure APIs so that one request returns what a view needs, if the backend allows it. Or move data fetching to the server entirely, where requests to a database or internal service are fast and can be parallelized before any HTML or component payload is sent to the browser. That last option is the direction React itself has taken, and it is where the next chapter begins. What remains for the client store Once server data has moved into a dedicated cache, the client store is often startlingly small. In applications that have made this migration, slices that once held products, orders, users, and their loading flags disappear, and what remains is a handful of UI preferences, session details, and genuinely client-owned workflow state. Teams frequently discover they no longer need Redux at all, and either keep a small Zustand store or use React state and Context. That outcome is not universal. Applications with substantial client-owned state, such as editors, design tools, and offline-first applications, still need a real store alongside their server cache, and the two must cooperate: the store may hold a document being edited while the server cache holds the last saved version. But the separation itself is almost always an improvement, because it gives each kind of state a mechanism matched to its hazards, which is the central argument of this book applied to its largest category. Chapter 9: React 19, Actions, and Server Components React 19, released in December 2024, and the server-centric architecture that frameworks like Next.js built around it change where state lives in a React application more than any library release in the previous five years. The headline features are use(), Actions, useActionState, useOptimistic, and stable support for Server Components and Server Functions. Each takes over a job that used to require either a state library or hand-written effect code. This chapter explains what each primitive does, which kinds of state it absorbs, and what is left for client stores afterward. use(): reading promises and context during render use is a new API that reads the value of a resource during render. It accepts either a Promise or a Context. When passed a Promise, use suspends the component if the promise has not yet resolved, letting the nearest <Suspense> boundary show a fallback, and returns the resolved value once it is available. If the promise rejects, the nearest error boundary catches the error. function Comments({ commentsPromise }: { commentsPromise: Promise<Comment[]> }) { const comments = use(commentsPromise); return comments.map(c => <p key={c.id}>{c.text}</p>); } // Parent, often a Server Component, starts the request early: // <Suspense fallback={<Spinner />}> // <Comments commentsPromise={fetchComments(postId)} /> // </Suspense> Unlike hooks, use can be called conditionally and inside loops. It solves a specific problem: consuming asynchronous data inside a component without effects, loading flags, or race conditions. The loading state is expressed structurally by Suspense boundaries rather than imperatively with isLoading variables. There is an important constraint. The promise must be stable across renders. If a client component creates a new promise during render, as in use(fetch(url)), every render creates a new request, the component suspends again, and it never settles. Promises passed to use should come from a Server Component, from a cache, or from a library that memoizes them. React's documentation is explicit that use does not support promises created in client components during render. This is why use complements rather than replaces server-state libraries on the client: TanStack Query and RTK Query provide the caching that makes promises stable, and TanStack Query's useSuspenseQuery integrates with the same Suspense boundaries. When passed a Context, use behaves like useContext but can be called conditionally. It does not change the broadcast semantics described in Chapter 2. Actions and useActionState React 19 introduced the term Action for a function that performs an asynchronous transition, typically a data mutation, and lets React manage its pending state, errors, and sequencing. Any async function passed to startTransition becomes an Action. React tracks it as pending for its full duration, including after await, and keeps the current UI interactive. The <form> element gained direct support: its action prop can be a function, which React calls with the form's FormData on submission. Forms submitted this way are automatically reset after a successful Action. useActionState wraps an Action and manages the state it produces: async function addToCart(prevState: CartResult, formData: FormData) { const res = await fetch('/api/cart', { method: 'POST', body: formData }); if (!res.ok) return { error: 'Could not add item' }; return { error: null, count: (await res.json()).count }; } function AddButton({ productId }: { productId: string }) { const [state, formAction, isPending] = useActionState(addToCart, { error: null }); return ( <form action={formAction}> <input type="hidden" name="productId" value={productId} /> <button disabled={isPending}>Add to cart</button> {state.error && <p role="alert">{state.error}</p>} </form> ); } The hook returns the latest result of the Action, a wrapped action to pass to the form, and an isPending flag. Submissions are queued and run in order, each receiving the previous result as its first argument. A companion hook, useFormStatus from react-dom, lets a component inside a form read its parent form's pending status without prop drilling, which is useful for reusable submit buttons. What this absorbs is the mutation boilerplate that used to live in stores or in components: isSubmitting flags, error state, disabling buttons, preventing double submissions. A large amount of state that teams once put in Redux slices, such as checkout.status = 'submitting', is now handled by React directly and scoped to the form where it is relevant. useOptimistic useOptimistic provides a temporary, optimistic version of some state while an Action is in progress. const [optimisticTodos, addOptimistic] = useOptimistic( todos, (current, newTodo: Todo) => [...current, { ...newTodo, pending: true }], ); Inside an Action, you call addOptimistic(newTodo) before awaiting the server. React renders optimisticTodos with the pending item included. When the Action completes, React discards the optimistic state and renders from the real todos, which by then should include the server-confirmed item, either because the parent refetched it or because the Action updated the source state. If the Action fails, the optimistic item simply disappears, because the base state never changed. There is no rollback code to write. This is the same insight behind TanStack Query's pattern of rendering mutation variables: an optimistic value layered over the real state and discarded when the operation settles is simpler and safer than writing into the cache and undoing it. useOptimistic builds that pattern into React and ties it to the Action lifecycle. Server Components and Server Functions React Server Components are components that run only on the server, during a request or at build time. They can be async, can read directly from a database or file system or call internal services, and never ship their code to the browser. Their output is serialized and sent to the client, where it is merged into the tree alongside Client Components, ordinary interactive React components marked with the 'use client' directive. The implications for state management are large. In a Server Components architecture, a significant fraction of what used to be server state fetched by the client never reaches the client as data at all. A product page Server Component queries the database, renders the product details to markup, and sends the result. No client-side fetch, no cache entry, no loading state in a store. When the data changes, for example after a mutation, the framework re-renders the relevant Server Components on the server and streams the new output to the client. Server Functions, previously called Server Actions, are async functions marked with 'use server' that run on the server but can be called from client code, including as a form's action. Combined with useActionState and useOptimistic, they make a mutation flow possible with no client-side API layer at all: the form calls a server function, the function writes to the database and asks the framework to revalidate the affected data, and the updated Server Component output arrives with the response. Frameworks shape the details. Next.js's App Router has been the most widely used Server Components implementation, with its own caching and revalidation APIs, and other frameworks, including React Router in its framework mode, have adopted or are adopting Server Components support. The underlying React primitives are stable in React 19, but the bundler and framework integration that makes them usable is still maturing outside the largest frameworks, and React's documentation notes that the APIs bundlers use to implement Server Components do not yet follow semantic versioning. Effects after React 19 The primitives above share a theme: each removes a reason to write useEffect. Data loading moves into Server Components or use with a cached promise; mutation state moves into Actions; derived values are computed during render and memoized by the compiler. React's documentation had argued for years, in its "You Might Not Need an Effect" guide, that effects are an escape hatch for synchronizing with systems outside React, such as a browser API, a third-party widget, or a network connection, and not a general tool for reacting to state changes. React 19 makes that advice much easier to follow. React 19.2 added one more piece in the same direction: useEffectEvent. A common bug arises when an effect needs to read the latest value of some prop or state without re-running whenever that value changes. A chat connection effect that logs analytics with the current theme, for example, would reconnect every time the theme changed if theme appeared in its dependency array. useEffectEvent declares a function that always sees the latest props and state but is not itself a dependency, so the effect can call it without listing what it reads. This removes a category of workarounds, such as mirroring values into refs, that frequently confused state flow in components connected to stores. For state management, the net effect of React 19 and 19.2 is that fewer values need to be held anywhere for the purpose of coordinating effects, and the effects that remain are clearer about what they synchronize. Where server and client caches meet Most real applications do not fit cleanly into either "everything on the server" or "everything on the client." A dashboard might render its initial data in Server Components for a fast first paint, but then need client-side polling, filtering, and optimistic edits that only a client cache handles well. Both major server-state libraries support this hybrid by letting the server prefetch data into a cache that is then handed to the client. In TanStack Query, the pattern is to create a query client on the server, call prefetchQuery for the data a page needs, serialize the cache with dehydrate, and wrap the client part of the tree in a HydrationBoundary that receives the dehydrated state. On the client, useQuery calls with matching keys find the data already present and render immediately without a loading state, then follow the usual staleness rules from that point. Version 5 also supports streaming: a Server Component can start a prefetch without awaiting it, and the pending query is dehydrated and resolved on the client as the data streams in. RTK Query offers comparable server-side prefetching and rehydration through its extractRehydrationInfo option. Three pitfalls recur in this hybrid setup. The first is staleness on hydration. With TanStack Query's default staleTime of zero, data prefetched on the server is considered stale as soon as it reaches the client, and the client may immediately refetch it, duplicating the request the server just made. The library's server-rendering guide recommends setting a default staleTime above zero, for example a minute, to avoid this. The second is duplicate sources of truth. If a Server Component renders a product's name directly from the database and a Client Component elsewhere on the page renders the same product from a client query, a mutation that invalidates the client query will update one rendering but not the other until the server output is also refreshed. Choose one source per piece of data on a page, or ensure that mutations trigger both server revalidation and client invalidation. The third is leaking state across requests. A query client or store created at module scope on the server is shared by all requests the server process handles, the same trap described for Zustand in Chapter 4. Query clients and stores used during server rendering must be created per request. TanStack Query's documentation shows creating the client inside a component with useState for this reason, and React's cache function can scope a client to a single server request in Server Components. None of these pitfalls is exotic, and each has a documented solution. They are worth naming because they arise precisely at the boundary this chapter describes, where the server-centric model and the client-cache model overlap, and that boundary is where most production applications on React 19 actually live. What is left for client stores Put these primitives together and the territory for client-side state libraries shrinks in three ways. First, much server state moves to the server. Data that is displayed but not interactively manipulated, such as page content, product details, and lists that change only on navigation, can be fetched in Server Components and never enter a client cache. Data that is interactively manipulated on the client, such as an infinite feed with client-side filtering or a dashboard that polls, still benefits from TanStack Query or RTK Query, which both support hydrating their caches from data prefetched on the server. Second, much mutation state moves into Actions. Pending flags, errors, and optimistic values for form submissions are handled by useActionState, useFormStatus, and useOptimistic, scoped to the forms that need them. Third, derived memoization moves into the compiler, as described in Chapter 7, reducing the performance motivation for elaborate selector layers. What remains is the category that was always the real job of a client store: shared client state, which the client owns, which lives across interactions, and which several distant components read and write. Examples include: • An editor's in-progress document, its selection, and its undo history. • A canvas or diagram tool's shapes, viewport, and active tool. • A multi-step workflow that accumulates answers across several screens before a single submission. • Real-time collaborative state that merges local edits with remote ones. • Offline queues of actions waiting to sync. • Client preferences such as theme, density, and panel layout that should apply immediately without a round trip. None of these is served well by Server Components, which are not interactive, or by Actions, which model discrete submissions rather than continuous editing. These are exactly the cases where Redux Toolkit, Zustand, or signals still earn their place. Note too that Server Components cannot use client state at all: hooks, Context, and stores are available only in Client Components. A store's Provider, if it has one, must live in a Client Component that wraps the parts of the tree that need it, and per-request store creation, as described in Chapter 4, becomes mandatory in server-rendered setups. A small case: rebuilding a checkout To make the shift concrete, consider a checkout flow as a team might have built it around 2020 and as it might look on React 19. In the earlier design, a Redux store held a cart slice with items loaded from the API, a checkout slice with the shipping form's values, validation errors, a status field cycling through idle, submitting, succeeded, and failed, and an error message. Thunks loaded the cart, submitted the order, and handled errors. Every checkout component connected to the store. Adding a coupon code involved a new thunk, new actions, new reducer cases, and new selectors. In the redesigned flow, the cart page is a Server Component that reads the cart from the database and renders it. Quantity changes are forms whose action is a Server Function; useOptimistic shows the new quantity immediately while the server confirms. The shipping form uses a form library or plain uncontrolled inputs with HTML validation, and its submission is a Server Function wrapped in useActionState, which provides the pending state and any error. The coupon field is another small form with its own Action. The payment step, which must be interactive and integrates a third-party payment widget, is a Client Component. What is left of the Redux store? Possibly nothing. If the checkout spans several screens and the shipping answers must survive moving between them before final submission, a small Zustand store or a Context-scoped reducer might hold the in-progress answers, which is genuine shared client state. Everything else moved to the server, to Actions, or to forms. This is not a story about Redux being obsolete. It is a story about Redux having been used, in that earlier design, for state that was not client state. The new primitives make the classification harder to ignore by giving each kind of state an obvious home. The remaining question, for teams with real shared client state or with existing stores full of misclassified data, is how to choose and how to migrate. That is the subject of the final chapter. Chapter 10: Choosing, Combining, and Migrating The previous chapters described mechanisms. This one is about decisions: how to choose state tools for a new application, how to combine them coherently, how to tell when an existing setup needs to change, and how to migrate without halting feature work. The guidance is opinionated, but each recommendation follows from the cost models already developed rather than from fashion. Start with the inventory, not the library The decision process begins where Chapter 1 left off. Take the application's main screens and classify each changing value as server, URL, form, local UI, or shared client state. For a new application, do this from wireframes and the API design. For an existing one, do it from the current stores, slice by slice. The result usually looks like this: a large volume of server state, a modest amount of URL state that is currently scattered between the router and stores, several forms, a great deal of local UI state, and a small core of shared client state. The tooling decision then splits into independent sub-decisions, one per category. Table 4 sets out sensible defaults for each in a React 19 application. Table 4. Default mechanisms by kind of state in a React 19 application. Kind Default Reach for instead when Server Server Components or a query cache Heavy client interaction: TanStack Query or RTK Query URL Router search params Complex typed params: a typed router or URL-state library Form Actions plus uncontrolled inputs Complex validation: a form library Local UI useState, useReducer Never lift it without a reason Shared client React state plus small Contexts Frequent updates or many readers: Zustand, Redux Toolkit, or signals Notice that the default for shared client state is not a library. If the shared state is small and changes rarely, as with a theme, a signed-in user, or a feature-flag set, React state held near the root and exposed through a few narrowly scoped Contexts is entirely adequate. A library becomes worthwhile when the state changes often enough, or is read by enough components, that Context's broadcast semantics produce measurable re-render cost, or when the state's logic is complex enough to benefit from a dedicated home. Choosing a client store When a store is warranted, the choice among the three families covered in this book comes down to a handful of questions. Do you need an auditable history of changes? If the application benefits from undo and redo across many kinds of edits, from replaying a user's session to reproduce bugs, or from logging every state change for compliance or debugging, Redux Toolkit's action-based model provides this almost for free. Action logs are the natural substrate for these features, and the Redux DevTools are the most capable debugging tools in the ecosystem. Zustand can approximate them with middleware; signals libraries offer little by default. Do many independent parts of the application need to react to the same events? Redux's model of actions as events handled by many reducers, plus listener middleware for side effects, decouples features cleanly. A session/expired action can clear a dozen slices and trigger a redirect without the code that detects expiry knowing about any of them. In Zustand, that coordination is explicit function calls; in signals, it is effects reacting to a shared signal. Both work, but Redux makes the pattern structural. Is the shared state modest, and does the team value directness? Zustand is the pragmatic default here. A store is a hook, updates are functions, and there is almost nothing to learn beyond selector discipline. Its small API surface is an advantage when the state itself is small. Is there a genuine hot path with high-frequency updates rendered in many places? Signals, used surgically, can bypass component rendering for those values. Before adopting them, check whether colocation, narrow selectors, the compiler, or a ref-based direct DOM update solves the problem, because each of those adds less conceptual weight to the codebase. What does the team already know? This is not a minor consideration. A team fluent in Redux Toolkit will be more productive with it than with an unfamiliar library that is theoretically a better fit. Migration has costs, discussed below, and they must be weighed against the benefits. Several common arguments should carry less weight than they often do. Bundle size differences between these libraries are small: Redux Toolkit and React-Redux together are larger than Zustand, but all of them are modest next to a typical application's own code and its UI component library, and RTK Query can replace a separate fetching library. Raw update throughput differences rarely matter outside hot paths, for the reasons covered in Chapter 7. And popularity trends, whether measured by downloads or survey enthusiasm, describe what is fashionable, not what fits a given application. Three applications, three answers The questions above become clearer when applied to concrete cases. Consider three applications a team might plausibly build, each on React 19. The first is a B2B administration console: lists of customers, invoices, and subscriptions, with filters, detail pages, and edit forms. Its inventory is dominated by server state. URL state covers filters, sorting, and pagination. Forms are numerous but independent. Shared client state amounts to the signed-in user's session, a theme preference, and perhaps a collapsed or expanded sidebar. The right setup is a server-state library or Server Components for the data, router search parameters for the filters, Actions or a form library for the edit forms, and React Context for the tiny amount of shared state. No client store is needed. Adding Redux or Zustand here would provide a place to put things that already have better homes, and would invite exactly the misclassification this book warns against. The second is a browser-based diagram editor, where users drag shapes, connect them, group and align them, and undo and redo. Its inventory is the opposite: the document being edited is a large, intricate body of client-owned state that changes on nearly every pointer event, and several distant parts of the interface, such as the canvas, the layers panel, the property inspector, and the minimap, read and write it. Undo and redo across all edit types is a core requirement, and reproducing user-reported bugs from a sequence of operations would be enormously valuable. Redux Toolkit fits well. Edits become actions; undo can be implemented as a stack of past states that structural sharing makes affordable, or as inverse actions; the DevTools show exactly which operation produced a bad state. The hot path, a shape being dragged, can bypass the store entirely during the drag, with the shape's position held in a ref or a signal and committed to the store as a single action on pointer release. Server state, such as the saved document and collaborators' presence, goes through a query library or a realtime channel, with clear rules about when the local document syncs. The third is a consumer analytics dashboard with a dozen live-updating tiles, each showing a metric that refreshes every few seconds, plus filters that apply across tiles. The tiles' data is server state, best handled by a query library with polling intervals per tile. The filters belong in the URL. The interesting question is the rendering of rapidly changing numbers. If each tile is a separate component with its own query, React's normal model already limits each update to one tile, and with the compiler enabled, no further work is likely to be needed. If profiling showed otherwise, perhaps because each tile includes an expensive chart, the fix would be in chart rendering, not state management. A team tempted to adopt signals for this dashboard should first confirm that tile updates are actually expensive, which in a well-structured implementation they usually are not. The three cases share a pattern: the inventory determined the answer, and the client store question arose only in the one application whose inventory was dominated by shared client state. Combining tools coherently A mature React application will almost certainly use several state mechanisms. That is not a failure of architecture; it is the architecture. The risk is incoherence, where the same kind of data ends up in different places in different features. Three rules keep a multi-tool setup coherent. Each fact has one home. Never copy server data into a client store "for convenience." If a component needs server data and client state together, it reads both and combines them during render. A common violation is copying a fetched user profile into a Zustand store so that it is "globally available"; the query cache is already globally available, and the copy will go stale. Each category has one default mechanism, documented. A short architecture note stating that server data goes through TanStack Query, filters go in the URL, forms use the chosen form approach, and shared client state goes in a named store prevents each developer from making a fresh choice. Linting can enforce some of it, for example by restricting direct fetch calls in components. Cross-mechanism coordination is explicit. When a mutation through the query cache must also update client state, or a client-state change must trigger a refetch, put that coordination in one place, such as the mutation's onSuccess callback or a listener middleware, rather than scattering it across components. Recognizing when to change Existing applications rarely need a full state-management rewrite, but certain symptoms indicate that the current setup is misclassifying state or fighting the framework. Loading and error flags proliferating in a client store, alongside hand-written logic for when to refetch, mean server state is being managed as client state. The fix is a server-state cache, and it is almost always the highest-value change available. The back button behaving unexpectedly, or links to filtered or paginated views not working, mean URL state is in a store. Typing in a form feeling sluggish, with profiling showing large parts of the page rendering per keystroke, means form state is held too high or in a global store. A store that new team members find intimidating, with many slices that are mostly loading flags and copies of API responses, is the composite of all of the above. Notice that none of these symptoms is "we are using the wrong client store." The rarer symptom that does point at the store itself is a sustained mismatch between its model and the application's needs: for example, an editor built on a minimal store that has accumulated a home-made action log and undo system, which suggests the structure of Redux would serve better, or a small application with three pieces of shared state carrying the full weight of a Redux setup inherited from a template, which suggests a lighter tool. Migrating incrementally When a migration is warranted, do it incrementally. Big-bang rewrites of state management stall feature work for months, introduce regressions in behavior nobody documented, and frequently stop halfway, leaving two systems. The most valuable migration, from a hand-built server cache in a client store to a server-state library, can proceed one endpoint at a time. Pick a read-heavy, low-risk resource. Add a query for it. Change the components that read it to use the query instead of the store. Change the mutations that touch it to invalidate the query. Delete the slice. Repeat. Each step is independently shippable and immediately reduces code. Teams that undertake this migration commonly delete a large share of their store code, and in many cases the remaining client state is small enough that the original store library can be removed or replaced with something lighter. If the application uses Redux and the team adopts RTK Query, the migration is especially smooth, because the API slice lives in the same store and the existing DevTools continue to work. If the team adopts TanStack Query instead, the two can coexist indefinitely; nothing about Redux prevents a query cache from running alongside it. Moving URL state out of a store follows a similar per-feature pattern: replace the store field with a router search parameter, update the components to read and write it through the router, and delete the field. Migrating between client stores, say from Redux to Zustand or the reverse, is lower value and should be done only when the store's model is genuinely wrong for the application. When it is done, an adapter approach works well: wrap the old store's state and actions behind a small set of custom hooks, so that components depend on the hooks rather than on the library. Then reimplement the hooks on top of the new store, one domain at a time. This decouples the component code from the store choice, which is worth doing even without a migration, since it localizes any future change. Testing state logic A final practical consideration concerns testing, because the testability of state logic differs between approaches. Redux reducers are pure functions of state and action, which makes them trivially unit-testable: construct a state, apply an action, assert on the result. Selectors are equally easy. Thunks and listeners need a configured store, but configureStore makes that cheap. The Redux maintainers recommend favoring integration-style tests that render components with a real store over isolated tests of each piece. Zustand stores are plain JavaScript and can be tested by calling their actions and reading getState(). Because module-level stores are singletons, tests must reset them between cases; the Zustand documentation describes a mocking pattern that resets every store after each test, using the store's initial state. Signals can be tested in the same direct way, by writing signals and reading computeds, though effects require care to flush and dispose. Server-state libraries are best tested by mocking the network rather than the library. Tools that intercept requests at the network level, such as Mock Service Worker, let tests exercise the real caching, retry, and invalidation behavior. Tests should create a fresh query client per test, with retries disabled, to avoid shared cache state and slow failures. Across all approaches, the most robust tests are those that render real components against real state mechanisms and interact with them as a user would. Such tests survive refactoring of state internals, including migrations between libraries, which is a strong argument for writing them before a migration begins. Conclusion The question this book set out to reframe was "which state library should we use?" Having worked through the mechanisms, it is possible to say more precisely what follows from them, what remains genuinely unsettled, and what teams should do differently. The library question was always downstream The most important conclusion is that the long-running debate between state libraries was largely a proxy for a different problem: applications put several kinds of state with different hazards into one mechanism, and then blamed the mechanism for the resulting pain. Server data managed as client state produced hand-built caches with race conditions and stale views. URL state held in stores broke navigation. Form state lifted to global stores made typing slow. When teams migrated from one library to another without reclassifying their state, they tended to rebuild the same problems in a new syntax. Seen this way, the most consequential tools of the past several years were not the newest client stores. They were the server-state caches, TanStack Query and RTK Query, which gave the largest category of state a mechanism matched to its actual behavior, and React's own async primitives and Server Components, which moved much of that category off the client entirely. Every application that adopts them sees its client store shrink. That shrinkage is the clearest evidence that much of what used to be called "state management" was really cache management performed badly. Snapshots versus signals is a question of where you pay On the question the book's title raises directly, the comparison between immutable state trees and signals, the chapters support a conclusion more specific than "it depends." Snapshot stores and signals both solve the propagation problem correctly. They differ in where they pay. Snapshot stores pay at notification time: every subscriber runs its selector on every change, and correctness depends on selectors returning stable references. Signals pay at read time and in memory: every tracked read records a dependency edge, and correctness depends on all relevant reads being tracked. In a renderer built around signals, such as Solid or Svelte 5, the signal model's cost is low and its benefit, updates that skip components entirely, is large. In React, which was built around snapshots and reference equality, the balance is different. Integration requires either a build transform or a hook in every reading component; the fine-grained benefit is available only for values rendered directly as text; and the React Compiler now removes most of the re-render cascade that signals once sidestepped. The practical result is that inside React, the snapshot family remains the sensible default for shared client state, and signals are a precise tool for specific hot paths. That is a conclusion about React, not about signals in general. Outside React, signals have effectively won the framework design argument: nearly every major framework other than React now builds on them. What remains unsettled Three questions remain open, and teams should plan with them in mind rather than assume an answer. The first is whether React will ever adopt signal-like reactivity. The React team has consistently chosen compilation over runtime reactivity, and React Compiler 1.0 is the fullest expression of that choice so far. A compiler can, in principle, infer much of what a signals runtime tracks dynamically. But the compiler works at the granularity of components and hooks, not individual DOM bindings, and the performance ceiling of a virtual DOM with perfect memoization is still higher than that of direct fine-grained updates. Whether that gap matters enough for React to change course is not known, and nothing in React's public direction as of late 2026 suggests it will. The second is the fate of the TC39 Signals proposal. If it advances and lands in browsers, it would give frameworks a shared reactive substrate and could make signals interoperable across libraries, including, potentially, in React through libraries that build on the native primitive. But it remains at Stage 1, its champions have deliberately set a slow pace, and native availability is years away at best. Building an architecture on the assumption that native signals will arrive soon would be a mistake. The third is how far the server-centric model will spread. Server Components shift a great deal of state off the client, but they depend on framework and bundler integration that is mature mainly in the largest frameworks, and they suit content-heavy and form-heavy applications better than highly interactive ones. Applications that are mostly interaction, such as editors, design tools, and real-time collaboration, will continue to carry substantial client state. The split between server-shaped and client-shaped applications may become a sharper architectural fork than it is today. What to do differently The chapters point to a short list of changes that most teams would benefit from, roughly in order of impact. Classify state before choosing tools, and write the classification down. A one-page note naming the home for each kind of state prevents most of the structural mistakes described in this book from recurring. Move server data out of client stores. Adopt TanStack Query or RTK Query, or Server Components where the framework supports them, and migrate one resource at a time. This is almost always the highest-value change available. Put shareable, reload-surviving state in the URL. Filters, pagination, selected tabs, and search queries belong there. Treat selectors as subscriptions. In Redux or Zustand, every selector defines what a component reacts to. Narrow ones produce precise renders; broad ones produce cascades. Most performance problems in snapshot stores are selector problems. Enable the React Compiler, and fix what its lint rules report. It removes a large class of re-render problems with little code change, and the rules it enforces are good discipline regardless. Profile before optimizing, and fix structure before reaching for new tools. Colocation, composition, and narrow subscriptions solve most problems; library migrations solve few. Reserve client stores for genuinely shared client state, and choose among them by the guarantees you need: Redux Toolkit for event-driven history and decoupling, Zustand for directness with modest state, and signals for surgical use on hot paths. None of these recommendations depends on which library is fashionable, and none will be invalidated by the next major release of any of them. They follow from what state is, how React renders, and where each propagation model spends its effort. Those fundamentals have been stable for a decade, and they are the most durable basis for decisions in a field whose tools will keep changing. Further Reading Redux documentation, "Redux Essentials" and "Redux Fundamentals" tutorials (redux.js.org). The official tutorials teach modern Redux Toolkit from first principles, including RTK Query. The "Style Guide" page, with its advice on modeling actions as events and keeping state minimal, is worth reading even for teams that do not use Redux. Redux Toolkit documentation, "Usage Guide" and "RTK Query Overview" (redux-toolkit.js.org). Covers createSlice, createEntityAdapter, the listener middleware, and RTK Query's tag-based invalidation in detail, with migration notes for the 2.0 release. Zustand documentation (zustand.docs.pmnd.rs). Short and practical. The guides on the slices pattern, preventing re-renders with useShallow, Next.js setup with per-request stores, and testing are the most useful pages, along with the migration guide to version 5. TanStack Query documentation (tanstack.com/query). The "Important Defaults" page explains the staleness and refetch behavior that surprises new users. The guides on query keys, invalidation, optimistic updates, and the version 5 migration are essential. TkDodo's blog (tkdodo.eu). Dominik Dorfmeister, a TanStack Query maintainer, has written the most thorough practical commentary on server state, query keys, and the distinction between server and client state. His "Practical React Query" series is the standard reference. React documentation, "Managing State," "You Might Not Need an Effect," and the useSyncExternalStore reference (react.dev). The clearest official explanation of colocation, derived data, avoiding effect-based synchronization, and how external stores integrate with concurrent rendering. React blog, "React v19" (December 2024), "React 19.2" (October 2025), and "React Compiler v1.0" (October 2025) (react.dev/blog). Primary sources for use, Actions, useOptimistic, the Performance Tracks, and the compiler's design and reported results. TC39 Signals proposal repository (github.com/tc39/proposal-signals). The README explains the proposal's motivation, API, design decisions, and the champions' plan for advancement, along with links to the polyfill. Preact Signals documentation and repository (preactjs.com/guide/v10/signals and github.com/preactjs/signals). Explains the core primitives, the React integration through the Babel transform and useSignals, and the integration's documented limitations. Ryan Carniato's writing on fine-grained reactivity (dev.to and the Solid blog). Solid's creator has written extensively on how signals work internally, push-pull evaluation, and how fine-grained rendering differs from the virtual DOM. Accessible and technically precise. Mark Erikson, "Blogged Answers: Why React Context is Not a State Management Tool" (blog.isquaredsoftware.com). A Redux maintainer's careful explanation of the difference between dependency injection and state management, and of Context's propagation behavior. Hashtags: #StateManagement #ModernWebApps #ReactStateManagement #Redux #ReduxToolkit #Zustand #JavaScriptSignals #FineGrainedReactivity #FrontendDevelopment #ReactJS #React19 #ReactCompiler #ImmutableState #ReactiveProgramming #ClientSideState #ServerStateManagement #TanStackQuery #RTKQuery #ReactContext #ReactHooks #UseSyncExternalStore #StateSelectors #Memoization #RenderOptimization #ReactPerformance #ComputedSignals #OptimisticUpdates #ReactServerComponents #WebApplicationArchitecture #FrontendArchitecture

Download the Book (PDF): Introduction In October 2019 a team at Google reported that a 53-qubit superconducting processor called Sycamore had sampled from a random quantum circuit in about two hundred seconds, a task they estimated would take the world's largest supercomputer ten thousand years. Within days IBM argued that a cleverer classical simulation, using disk storage, could do it in two and a half days. Within three years, tensor-network methods running on ordinary GPU clusters had brought the classical time down further still. The episode set a pattern that anyone working in quantum machine learning needs to understand from the outset. A claim of quantum superiority is published; classical algorithm designers, newly motivated, look harder at the problem; the gap narrows or closes. Sometimes it does not close, and those cases are the ones worth studying. Quantum machine learning sits right on the fault line of that pattern. It promises to marry two of the most heavily funded technologies of the decade, and the marriage is attractive to every audience at once. Physicists get an application for noisy hardware that exists today. Data scientists get a story about exponentially large feature spaces. Executives get a slide that says "quantum AI." The result has been a literature of many thousands of papers, a crowd of start-ups, cloud services from IBM, Amazon, Google and Microsoft, and software frameworks such as Qiskit, PennyLane, Cirq and TensorFlow Quantum that let any graduate student train a "quantum neural network" on a laptop before lunch. What that literature has not yet produced is a single practically relevant learning or optimization task on which a quantum computer has been shown to beat the best classical method. That is not an opinion; it is the conclusion of the field's own most careful researchers, several of whom spent years building the very techniques that now look less promising than they once did. The purpose of this book is to explain, to a reader who is serious but not a specialist, why that is so, what has actually been learned, and where the genuine opportunities lie. The controlling idea The argument of the book can be stated in one sentence. Hybrid quantum-classical algorithms are a bet about where the hard part of a computation lives, and they pay off only when the quantum processor does something that classical computation demonstrably cannot do cheaply; over the past decade, the evidence has shown that many of the properties that make variational quantum models trainable and easy to load with data are the same properties that make them classically simulable, so the credible route to value runs through problems that are natively quantum and, eventually, through error-corrected hardware, rather than through accelerating ordinary data science on today's machines. That sentence contains several claims, and each chapter defends or qualifies one of them. The phrase "where the hard part lives" matters because a hybrid algorithm is a division of labour. A classical computer chooses parameters, a quantum processor prepares a state and measures it, and the classical computer updates its choice. If the quantum step is easy to imitate classically, the quantum computer is an expensive random number generator. If the classical step is impossibly hard, as optimizing over a landscape of exponentially flat terrain can be, then the quantum step's power is irrelevant because nobody can steer it. The bet pays only when the quantum step is hard to simulate and the classical step remains tractable, and much of the field's history is the discovery of how narrow that window is. What the book covers The description promised Variational Quantum Eigensolvers, quantum support vector machines and the use of near-term hardware to accelerate optimization. All three receive a full chapter, but they are placed inside an argument rather than presented as a catalogue. Chapter 1 establishes what a quantum computer actually gives a learning algorithm: a state space that grows exponentially, access to it only through destructive and noisy measurement, and a serious problem getting classical data in. Most confusion in the subject comes from forgetting the second and third of these while admiring the first. Chapter 2 describes the machinery shared by every hybrid algorithm: parameterized circuits, cost functions built from measured expectation values, the parameter-shift rule for computing gradients on hardware, and the shot budgets that determine what any of this costs. Chapter 3 is devoted to the Variational Quantum Eigensolver, the oldest and best-motivated hybrid algorithm, which estimates ground-state energies of molecules and materials. It is the place where quantum computers have the most natural claim to relevance, and it is also where the measurement overheads and noise problems are best quantified. Chapter 4 confronts trainability. The discovery of barren plateaus in 2018, the later proofs that variational training is in general NP-hard and riddled with poor local minima, and the 2024 Lie-algebraic theory that unified the various causes of vanishing gradients have changed how researchers think about the whole enterprise. Chapter 5 treats quantum kernels and quantum support vector machines, from the 2014 proposal built on a fault-tolerant linear-systems algorithm to the 2019 IBM experiment that estimated kernels on hardware, and on to the theorems about exponential concentration that explain why naive quantum kernels generalize badly. Chapter 6 is about the classical response: dequantization, classical surrogates, tensor-network simulation and the 2025 argument that provable absence of barren plateaus usually implies efficient classical simulation. It is the chapter most practitioners have not read and most need to. Chapter 7 turns to optimization: the Quantum Approximate Optimization Algorithm, quantum annealing, Rydberg-atom experiments on maximum independent set, and the financial-services pilots that have produced the most widely reported "quantum advantage" headlines. It asks what an honest benchmark of a quantum optimizer would have to show. Chapter 8 surveys the hardware as it stands in late 2026 and explains what error correction changes and what it does not. Fault tolerance revives some algorithms and leaves others exactly as unpromising as before. Chapter 9 asks where quantum learning wins on present theory and evidence. The answer, perhaps surprisingly, is clearest when the data are themselves quantum states produced by physical experiments, and it closes with a protocol for evaluating any quantum machine learning claim a reader is likely to meet. Who this book is for and what it assumes The reader I have in mind is a data scientist, engineer, analyst, technical manager or student who knows what a support vector machine and gradient descent are, has heard that quantum computers use superposition and entanglement, and wants to know whether quantum machine learning deserves their time, their budget or their research career. No prior quantum mechanics is assumed, although a reader who remembers linear algebra will move faster. There is occasional notation and two short code examples; the argument never depends on them. I have tried to be precise about the status of every claim. Some results are theorems, true under stated assumptions. Some are numerical studies on simulated devices of a few dozen qubits, which tell us how small instances behave and very little about large ones. Some are hardware experiments, which are valuable precisely because they include all the noise and overheads that theory leaves out, and which are routinely over-interpreted by press releases. Where a claim is disputed, the book says so and says by whom. A note on tone It would be easy to write a cheerful book about this subject, and easy to write a dismissive one. Neither would be honest. The negative results in quantum machine learning are real and in several cases rigorous; anyone who tells you that quantum computers will soon speed up your recommendation engine or your credit-risk model has not read them. But the same research that closed off easy routes has also clarified which routes remain open, and some of those, particularly in chemistry, materials and the analysis of quantum experiments, are substantial. The field is healthier now than it was in 2019, precisely because it has absorbed its disappointments. The aim here is to give the reader the same education the field has given itself, compressed into a short book, so that they can tell the difference between a result and a sales pitch. CHAPTER 1 What a Quantum Computer Offers a Learner Every popular account of quantum computing begins with the same fact: a register of n qubits is described by 2ⁿ complex numbers. Fifty qubits require about a quadrillion amplitudes; three hundred require more numbers than there are atoms in the observable universe. From there it is a short and seductive step to the idea that a quantum computer is a machine that holds and manipulates exponentially large vectors, and therefore that it should be an exponentially powerful engine for machine learning, a discipline that is, after all, mostly linear algebra on large vectors. The step is wrong, and understanding exactly why it is wrong is the foundation of everything else in this book. A quantum computer does manipulate an exponentially large vector. But it gives you almost no direct access to that vector, it is very hard to put your own vector into it, and every operation it performs is constrained to a particular geometry. Those three facts, read out, load in, and the shape of the allowed operations, determine where quantum machine learning can and cannot help. The state, the gates and the measurement A single classical bit is 0 or 1. A single qubit is described by two complex amplitudes, one attached to 0 and one attached to 1, whose squared magnitudes sum to one. When you measure the qubit you obtain 0 with probability equal to the squared magnitude of the first amplitude and 1 with probability equal to the squared magnitude of the second, and the qubit is left in whichever state you observed. Two qubits have four amplitudes, for 00, 01, 10 and 11, and n qubits have 2ⁿ. When the amplitudes cannot be written as a product of separate amplitudes for each qubit, the qubits are entangled, which means that the joint state contains correlations that no assignment of individual states can reproduce. Computation proceeds by applying gates. A gate is a unitary matrix, a rotation in the complex vector space that preserves the length of the state. Single-qubit gates rotate one qubit; two-qubit gates such as the controlled-NOT or the controlled-Z create entanglement. Any unitary on n qubits can in principle be built from these, but most require exponentially many gates, so the interesting unitaries are those that can be built from a number of gates growing only polynomially with n. A quantum circuit is a sequence of such gates, and its depth is the number of layers of gates that must be applied one after another. At the end comes measurement. In the standard model, you measure every qubit in the computational basis and obtain an n-bit string, sampled from the probability distribution given by the squared amplitudes. Then the state is gone. To learn anything more, you must prepare it again from scratch and measure again. Each repetition is called a shot. This is the heart of the matter. The machine holds 2ⁿ amplitudes, but each run yields n bits. To estimate the probability of one particular outcome to within an additive error ε, you need on the order of 1/ε² shots. To estimate the expectation value of an observable, a weighted average over outcomes such as the energy of a molecule or the output of a classifier, you face the same 1/ε² scaling. There is no way to read out the full vector of amplitudes without a number of measurements that grows exponentially with n, a procedure called quantum state tomography that is feasible for a handful of qubits and hopeless for fifty. Holevo's theorem, proved in 1973, makes the limitation precise: n qubits can carry at most n bits of classical information to a receiver who measures them. The exponential state space is real, but it is not an exponential memory you can query at will. Quantum algorithms that achieve large speedups, such as Shor's factoring algorithm, do so by arranging interference so that the amplitudes of wrong answers cancel and the amplitude of the right answer grows, and then reading out a short answer. The art lies in choreographing the interference so that the information you want ends up concentrated in a few measurable bits. Why an exponential state space is not exponential power There is a useful comparison with classical probability. A collection of n classical coins that can be in any joint random configuration is also described by 2ⁿ numbers, the probabilities of each configuration. Nobody thinks that a computer that flips coins is exponentially powerful because the probability distribution over its states is exponentially large. Sampling from such a distribution gives you one configuration at a time, just as measuring a quantum state does. What distinguishes quantum mechanics is that amplitudes are complex numbers that can interfere, adding constructively or cancelling, whereas probabilities are non-negative and only ever add. Interference is the resource. It is also fragile. Every unwanted interaction between a qubit and its environment leaks information out and degrades the interference pattern, which is why present-day machines are limited to circuits of modest depth before noise drowns the signal. The complexity-theoretic picture reflects this. The class of problems a quantum computer can solve efficiently, called BQP, is believed to contain problems outside the classical class BPP, factoring being the famous example. But BQP is not believed to contain the NP-complete problems, the class that includes most hard combinatorial optimization tasks. Grover's search algorithm gives a quadratic speedup for unstructured search, reducing N steps to roughly the square root of N, and it is known that no quantum algorithm can do better for that problem in general. So a quantum computer is not a machine that tries all answers in parallel and picks the best. It is a machine that can exploit certain kinds of mathematical structure, especially periodicity, symmetry and the structure of physical Hamiltonians, far better than any known classical method. For machine learning this distinction is decisive. A learning problem gives a quantum computer an advantage only if the quantum computer can use structure in the problem that classical methods cannot. Size alone is not structure. Most real-world data sets, images, transaction records, text, have structure that is spectacularly well exploited by classical deep learning, and there is no general reason to think that structure resembles the interference patterns a quantum circuit can produce. The data-loading problem Suppose you have a classical data set: a million customer records, each described by a hundred numerical features. To process it on a quantum computer, the data must somehow enter the quantum state. There are three broad ways to do this, and each has a cost that is often ignored in optimistic accounts. The first is basis encoding, in which each data point is written as a bit string and loaded into the computational basis. This uses one qubit per bit and gives no compression at all. The second is amplitude encoding, in which a vector of 2ⁿ numbers is stored as the amplitudes of an n-qubit state. This is the encoding that produces exponential compression: a million-dimensional vector fits in twenty qubits. Many of the celebrated quantum machine learning algorithms from the early 2010s, including the quantum support vector machine of Rebentrost, Mohseni and Lloyd and the quantum principal component analysis of Lloyd, Mohseni and Rebentrost, assume data arrive in this form. The catch is that preparing an arbitrary amplitude-encoded state requires, in general, a number of gates proportional to the length of the vector. Loading a million numbers costs about a million operations, which is already as much work as a classical algorithm needs to read the data once. The exponential compression is spent before any computation begins. The proposed escape was quantum random access memory, or QRAM, a hypothetical device that would load amplitude-encoded data in time logarithmic in the vector length by querying a memory structure in superposition. QRAM requires a number of physical components proportional to the size of the data, all of which must operate coherently, and analyses of noise in proposed architectures such as the bucket-brigade design suggest that while some designs are more forgiving than first feared, no one has built a QRAM of useful size, and it is unclear that one could be built at a cost competitive with simply storing the data classically. Scott Aaronson's 2015 essay in Nature Physics, "Read the fine print," catalogued this and four other caveats attached to the exponential speedups claimed for quantum linear-algebra algorithms: the input must be loadable quickly, the matrix must be well conditioned and sparse or low rank, and the output is a quantum state from which you can extract only a few numbers rather than the full solution vector. Those caveats have not been lifted in the decade since. A concrete case makes the arithmetic vivid. The MNIST data set of handwritten digits, the traditional first exercise in machine learning, consists of 70,000 greyscale images of 28 by 28 pixels, or 784 numbers each. Padded to 1,024, a single image fits in the amplitudes of ten qubits, which sounds like a spectacular compression. But a general-purpose state-preparation circuit for a 1,024-amplitude state needs on the order of a thousand two-qubit gates, more than a noisy processor can execute reliably, and it must be rebuilt and rerun for every image and every shot. The ten qubits hold the image only for the duration of one run; to classify the image, the circuit must be executed thousands of times to estimate a single output probability. A classical convolutional network, by contrast, reads each pixel once and classifies the image in a fraction of a millisecond on commodity hardware. The compression exists, but there is nowhere for its benefits to go. The third route, and the one used by almost all near-term quantum machine learning, is angle encoding or feature-map encoding. Each feature of a data point sets the angle of one or more rotation gates. A data point with d features might be encoded in d qubits, each rotated by an amount proportional to its feature, followed by entangling gates and possibly repeated layers in which features are encoded again. This is cheap: the circuit depth grows only modestly with d. But it gives no compression. Each data point is processed separately, the quantum computer never sees the whole data set at once, and a million data points require at least a million separate circuit executions, each repeated for many shots. The practical consequence is stark. On present hardware, the time to execute a shot is set by gate times and, more importantly, by the overheads of control electronics, readout and cloud job submission. Superconducting processors can in principle run thousands of shots per second, and trapped-ion machines considerably fewer. A classical GPU evaluates a small neural network on a million data points in well under a second. For a quantum model to compete on throughput alone, it would need to extract dramatically more value per evaluation than a classical model, and nothing in the theory of angle-encoded circuits guarantees that. Recent theoretical work has tried to reopen the question of classical data from a different angle. In April 2026 Haimeng Zhao, Hsin-Yuan Huang, John Preskill, Jarrod McClean and colleagues at Caltech, MIT and Google Quantum AI posted a paper titled "Exponential quantum advantage in processing massive classical data," which proposes a technique they call quantum oracle sketching. Instead of loading a whole data set coherently, a small quantum computer consumes a stream of random classical samples and builds up, in superposition, an approximate oracle for the data, which it then uses for classification or dimension reduction. The advantage they claim is in memory rather than time: a quantum machine with fewer than sixty logical qubits could, in their numerical studies on tasks such as single-cell RNA sequencing and sentiment analysis, match classical methods that would need four to six orders of magnitude more memory. The work is theoretical, its demonstrations are classical simulations, and it requires error-corrected logical qubits rather than today's noisy physical ones. It is nonetheless significant because it shows that the data-loading barrier is a statement about specific encodings and resources, not a law of nature, and it points to memory and streaming settings as the place to look. Measurement, noise and the shape of the allowed operations Two further constraints complete the picture. The first is statistical. Every quantity a hybrid algorithm uses is estimated from finite shots, and the error decreases only as the inverse square root of the number of shots. To estimate an energy or a classifier output to three decimal places requires on the order of a million shots. For a variational algorithm with hundreds of parameters, each requiring separate gradient estimates at each optimization step, over hundreds of steps, the total shot count quickly reaches the billions. This overhead, which classical simulators of quantum circuits often hide by computing exact expectation values, is one of the main reasons that experiments lag far behind simulations. The second is noise. Present-day quantum processors are what John Preskill in 2018 named Noisy Intermediate-Scale Quantum, or NISQ, devices: tens to a few hundred physical qubits without full error correction. The best two-qubit gates on commercial systems in late 2026 have error rates around one part in a thousand, roughly 99.9 percent fidelity, on Quantinuum's 98-qubit trapped-ion Helios system and on the best superconducting processors. That sounds good until one multiplies: a circuit with a thousand two-qubit gates at 99.9 percent fidelity succeeds without any error only about 37 percent of the time, and a circuit with ten thousand such gates almost never. Error-mitigation techniques recover some accuracy at the cost of more shots, but as Chapter 8 explains, those costs grow exponentially with circuit size. Finally there is the shape of the allowed operations. A quantum circuit implements a unitary transformation followed by measurement. A quantum machine learning model therefore computes functions of a particular form: expectation values of observables in states prepared by data-dependent and parameter-dependent circuits. Maria Schuld, Ryan Sweke and Johannes Jakob Meyer showed in 2021 that when data are encoded by rotation gates, such models compute truncated Fourier series in the input features, with the available frequencies determined by how many times and in what way the data are encoded. That result is illuminating in both directions. It explains what quantum models can express, and it immediately suggests that a classical model built from the same Fourier features, of which there may be many but which can be sampled, might do just as well. Chapter 6 shows that this suspicion is often justified. What this means for the rest of the book Putting these constraints together gives a short list of conditions that any genuine quantum advantage in learning must satisfy. The data must either be loadable cheaply or already be quantum, as when the input is the output state of a quantum sensor or simulation. The quantity to be learned must be extractable from a modest number of measurements. The computation in between must involve interference that classical methods cannot reproduce efficiently, which in practice means circuits that are deep enough, entangled enough and unstructured enough to resist tensor networks, Pauli propagation and other simulation techniques. And the training procedure must be able to find good parameters, which, as Chapter 4 explains, pulls in exactly the opposite direction, toward shallow, structured, weakly entangled circuits. Those conditions do not rule out quantum advantage in learning. They rule out the naive version, in which a large classical data set is loaded into a small quantum computer and processed exponentially faster. Once that version is set aside, the questions become sharper and more interesting. Which problems carry quantum structure? Which algorithms exploit it without becoming untrainable? Which tasks survive the classical community's best efforts at simulation? The rest of the book takes them in turn, starting with the common machinery that all hybrid algorithms share. CHAPTER 2 The Hybrid Loop Every algorithm in this book, whether it estimates the energy of a molecule, classifies data or searches for a good portfolio, follows the same basic loop. A classical computer holds a vector of parameters. It sends those parameters to a quantum processor, which prepares a state using a circuit whose gates depend on them, measures the state many times and returns averaged results. The classical computer combines those results into a single number, the cost, and decides how to change the parameters to make the cost smaller. The loop repeats until the cost stops improving or the budget runs out. This architecture was laid out in its general form by Jarrod McClean, Jonathan Romero, Ryan Babbush and Alán Aspuru-Guzik in their 2016 paper "The theory of variational hybrid quantum-classical algorithms," which generalized the Variational Quantum Eigensolver introduced two years earlier. By 2021, when Marco Cerezo and colleagues published a comprehensive review titled "Variational quantum algorithms" in Nature Reviews Physics, the loop had been applied to chemistry, optimization, classification, generative modelling, linear systems, factoring, error correction and quantum compiling. Its appeal is obvious. It uses the quantum computer only for short circuits that fit within the coherence time of noisy hardware, and it hands everything else, the bookkeeping, the optimization, the decisions, to a classical computer that is good at those things. Its weaknesses are less obvious and are the subject of much of this book. This chapter describes the loop's four components, the parameterized circuit, the cost function, the gradient estimator and the optimizer, and the resource that binds them together: the number of shots. Parameterized circuits A parameterized quantum circuit, often called an ansatz from the German word for an educated guess, is a fixed arrangement of gates in which some gates carry adjustable angles. The most common parameterized gates are single-qubit rotations about the x, y or z axis of the Bloch sphere, which rotate a qubit's state by an angle θ. Entangling gates such as controlled-NOT are usually fixed. A typical circuit consists of repeated layers: a column of parameterized rotations on every qubit, followed by a pattern of entangling gates between neighbouring qubits, repeated L times. The design of the ansatz is the most consequential choice in any variational algorithm, and there are two broad philosophies. Problem-inspired ansätze build in knowledge of the task. In chemistry, the unitary coupled-cluster ansatz translates a well-established classical method for electronic structure into a quantum circuit, so that its parameters correspond to amplitudes for electrons to move between orbitals. In optimization, the Quantum Approximate Optimization Algorithm alternates between a layer generated by the problem's own cost function and a mixing layer, with only two parameters per layer. Such ansätze respect the symmetries of the problem, such as conservation of particle number, and they restrict the search to a physically meaningful region of the vast state space. Hardware-efficient ansätze, introduced in an influential 2017 IBM experiment by Abhinav Kandala and colleagues, instead use whatever gates and connectivity the device performs best, arranged in generic layers. They are shallow and easy to run, but they know nothing about the problem. Their expressive power comes from depth and entanglement, and, as Chapter 4 explains, that same expressiveness is what leads to vanishing gradients at scale. For machine learning on classical data, the circuit contains two kinds of parameterized gates: those whose angles are set by data features, forming the feature map or encoding, and those whose angles are trainable weights. Adrián Pérez-Salinas and colleagues showed in 2020 that interleaving encoding and trainable layers, which they called data re-uploading, lets even a single qubit approximate complicated functions. The Fourier-series analysis by Schuld, Sweke and Meyer, mentioned in Chapter 1, explains why: each repetition of the encoding adds new frequencies to the series the model can represent. Cost functions and observables The output of a quantum circuit is a random bit string. To get a number that can be optimized, the algorithm defines an observable, a Hermitian operator such as the Hamiltonian of a molecule, the parity of a few qubits, or the Pauli Z operator on a single readout qubit, and estimates its expectation value in the prepared state. The cost function is then built from one or more such expectation values. In a Variational Quantum Eigensolver, the cost is the energy, the expectation value of the Hamiltonian. In a classifier, the cost might be the mean squared error between the expectation value of a readout observable and the target labels, summed over a training set. Observables that are not diagonal in the computational basis require rotating the state before measurement. A molecular Hamiltonian, when mapped to qubits, becomes a weighted sum of Pauli strings, products of the Pauli X, Y and Z operators acting on various qubits. Each string can be measured by applying the right single-qubit rotations and then reading out in the computational basis. Strings that commute qubit-wise can be measured together; others require separate circuit runs. For a molecule represented on N qubits, the number of Pauli strings grows roughly as N⁴, so a 50-qubit chemistry problem may involve hundreds of thousands of terms. Grouping strategies, classical shadows and related techniques reduce the number of distinct measurement settings substantially, but the overhead remains a central practical constraint. The choice between global and local cost functions turns out to matter enormously. A global cost compares the whole n-qubit state with a target, for example by asking for the probability that every qubit is measured as zero. A local cost averages over small groups of qubits, for example by asking for the average probability that each qubit individually is zero. The two can have the same minimum, but Cerezo and colleagues proved in 2021 that global costs suffer exponentially vanishing gradients even for shallow circuits, whereas local costs need not. That result, discussed further in Chapter 4, is an early sign that innocuous-looking design choices can make the difference between a trainable model and a useless one. Gradients on quantum hardware Classical deep learning depends on backpropagation, which computes the gradient of the cost with respect to every parameter at roughly the cost of one forward evaluation. Quantum circuits do not permit backpropagation in that form, because the intermediate quantum states cannot be stored and re-used: measuring them would destroy them. Hybrid algorithms therefore use other gradient estimators. The most important is the parameter-shift rule, developed by Kosuke Mitarai and colleagues in 2018 and generalized by Maria Schuld, Ville Bergholm, Christian Gogolin, Josh Izaac and Nathan Killoran in 2019. For a gate of the form exp(−iθG/2), where G is an operator with eigenvalues ±1, as is true of the standard rotation gates, the expectation value is a sinusoidal function of θ. Its exact derivative can therefore be obtained by evaluating the circuit at two shifted values of θ, typically θ + π/2 and θ − π/2, and taking half the difference. This is not a finite-difference approximation; it is exact, up to shot noise. It requires two circuit evaluations per parameter, so a model with P parameters needs 2P evaluations per gradient, each estimated with many shots. Later work by David Wierichs and colleagues extended shift rules to gates with more complicated spectra, at the cost of more evaluations. The linear scaling of gradient cost with parameter count is a sharp contrast with classical backpropagation, whose cost is essentially independent of the number of parameters. Amira Abbas and collaborators showed in 2023 that, under general conditions, quantum models cannot achieve backpropagation-like scaling with the resources available on near-term devices, because doing so would require storing multiple copies of the quantum state. A quantum model with a million parameters, a small number by the standards of classical deep learning, would need two million circuit evaluations per gradient step. This alone confines practical variational circuits to hundreds or at most thousands of parameters. Alternatives exist. Simultaneous perturbation stochastic approximation, or SPSA, an older classical technique due to James Spall, estimates a gradient from just two evaluations by perturbing all parameters at once in a random direction; it is noisy but cheap and widely used on hardware. Gradient-free methods such as COBYLA and Nelder-Mead treat the circuit as a black box. The quantum natural gradient, proposed by James Stokes and colleagues in 2020, adjusts the step direction using the geometry of the space of quantum states, analogous to natural gradient methods in classical machine learning; it can converge in fewer steps but each step requires estimating a metric tensor, which adds measurement overhead. The following short example, written for the PennyLane library, shows how compact the hybrid loop has become in software. A two-qubit circuit is trained to minimize the expectation of an observable; the library computes gradients with the parameter-shift rule when run on hardware. import pennylane as qml from pennylane import numpy as np dev = qml.device("default.qubit", wires=2, shots=2000) @qml.qnode(dev, diff_method="parameter-shift") def cost(params): qml.RY(params[0], wires=0) qml.RY(params[1], wires=1) qml.CNOT(wires=[0, 1]) return qml.expval(qml.PauliZ(0) @ qml.PauliZ(1)) opt = qml.GradientDescentOptimizer(stepsize=0.2) params = np.array([0.1, 0.4], requires_grad=True) for step in range(100): params = opt.step(cost, params) The brevity is a strength for teaching and a hazard for evaluation. The same few lines run identically whether the device is an exact simulator or a noisy processor in a data centre, and it is easy to forget that the simulator computes in microseconds what the hardware estimates over minutes, with statistical error the simulator does not show unless asked. Optimizers and shot budgets The classical optimizer in the loop faces an unusual problem. Its function evaluations are noisy, because of finite shots, and biased, because of hardware errors. They are expensive, because each requires a queue submission and thousands of circuit repetitions. And the landscape it must navigate may be highly non-convex. Standard choices include Adam and plain gradient descent using parameter-shift gradients, SPSA, and quasi-Newton methods with careful noise handling. Ryan Sweke and colleagues showed in 2020 that stochastic gradient descent converges even when gradients are estimated from very few shots, sometimes a single one, because the noise averages out over many steps; this can reduce total cost compared with estimating each gradient precisely. It helps to put rough numbers on the whole loop. Consider a modest variational classifier with 100 trainable parameters, a training set of 1,000 examples, 1,000 shots per expectation value and 200 optimization steps. Each gradient on one example needs 200 circuit evaluations by parameter shift, or 200,000 shots. Over the training set that is 200 million shots per step, and over 200 steps, 40 billion shots. Even at an optimistic thousand shots per second, that is more than a year of continuous machine time. Mini-batching, fewer shots per estimate and cheaper gradient estimators cut this by orders of magnitude, which is why real experiments use tiny data sets and few parameters, but the arithmetic explains why almost all published quantum machine learning results with realistic data sizes are simulations. Latency compounds the problem. A hybrid loop is iterative: the classical optimizer cannot choose the next parameters until it has the results of the current circuits. When the quantum processor sits in a remote data centre behind a shared queue, each round trip can take seconds to minutes even if the circuits themselves run in milliseconds. Cloud providers have responded with execution modes that reserve a processor for a session and run the classical optimizer close to it, and with batch interfaces that submit many parameter settings at once. These help considerably, but they do not change the fundamental fact that every step of a hybrid algorithm waits on a statistical estimate from a physical device, whereas every step of a classical training loop waits only on arithmetic. Practitioners have also learned that the optimizer and the circuit interact in subtle ways. Initialization matters: parameters drawn uniformly at random from the full range of angles often place the circuit in a region of vanishing gradients, while initializations close to the identity or close to a classically computed approximate solution, such as Hartree-Fock in chemistry, start in a better region. Layer-wise training, in which a few layers are optimized before more are added, sometimes helps. Warm-starting QAOA parameters from smaller instances or from classical relaxations is common. Each of these techniques has the same underlying logic: give the quantum circuit a starting point from which a short, local optimization suffices, because a long global search over its parameter space is generally hopeless. Simulators and what they hide Because hardware time is scarce and noisy, most hybrid algorithms are developed and evaluated on classical simulators, and it is worth understanding what those simulators can and cannot do. The simplest kind, a statevector simulator, stores all 2ⁿ complex amplitudes. At sixteen bytes per amplitude in double precision, thirty qubits need sixteen gigabytes, a large laptop; forty qubits need sixteen terabytes, a substantial cluster; fifty qubits need sixteen petabytes, beyond the memory of any single machine in existence. Exact simulation of arbitrary circuits therefore stops somewhere in the forties. That limit has two consequences for quantum machine learning research. The first is that the great majority of published studies use twenty qubits or fewer, often far fewer, because training requires thousands of circuit evaluations and each must be fast. At those sizes everything about the model, including any supposed quantum advantage, is classically computable by construction, and results say little about how the model behaves at sizes where it might matter. Scaling trends extrapolated from eight to sixteen qubits are fragile, because the phenomena that dominate at large sizes, barren plateaus and concentration among them, appear only gradually. The second consequence runs the other way. Simulators that do not store the full state, such as tensor-network simulators that exploit limited entanglement, Clifford simulators that exploit special gate sets, and Pauli-propagation methods that track only the parts of an observable that survive through a circuit, can handle hundreds or thousands of qubits when the circuit is of the right kind. These are exactly the methods that, as Chapter 6 shows, have repeatedly matched results from quantum hardware. A researcher who reports that a model "cannot be simulated" because it uses more than fifty qubits has only ruled out the crudest form of simulation. Finally, simulators hide costs unless they are asked to show them. An exact simulator returns expectation values with no shot noise, gradients with no statistical error, and no queueing delay. Training curves from exact simulators can look smooth and fast when the same training on hardware, with finite shots and real noise, would stall. Careful studies run simulations with finite shots and realistic noise models, and report both. What the loop does and does not buy The hybrid loop was designed for a world of short coherence times. It runs circuits that are short enough to finish before the qubits decohere, and it tolerates some systematic errors because the optimizer can partially compensate for them: if a gate over-rotates consistently, the optimizer will learn a slightly different angle. Peruzzo and colleagues emphasized this robustness in the first VQE paper, and Kandala's 2017 IBM experiment exploited it. What the loop does not buy is any guarantee that the quantum circuit is doing something classically hard. A hybrid algorithm is a heuristic: it comes with no proof that it will find a good answer, no bound on how long it will take, and no assurance that the state it prepares could not be prepared or approximated classically. The quantum processor is a subroutine whose value depends entirely on what it is asked to compute. If the optimized circuit prepares a state with low entanglement, a matrix product state simulation will reproduce it. If the observable is local and the circuit shallow, its expectation can often be computed classically by tracking only the part of the circuit inside the observable's light cone. If the model is a Fourier series with a manageable number of terms, a classical regression on random Fourier features may match it. This is the sense in which a hybrid algorithm is a bet about where the hardness lives. The loop concentrates all the quantum work in the preparation and measurement of a parameterized state, and it relies on the classical side to find good parameters. The next chapter examines the case where that bet looks most attractive, the computation of molecular ground-state energies, where the hardness of the quantum state is not an assumption but a well-documented feature of nature. CHAPTER 3 The Variational Quantum Eigensolver Richard Feynman's original argument for quantum computers, made in a 1981 lecture and published the following year, was not about machine learning or optimization. It was about simulating nature. Quantum systems of many interacting particles are hard to simulate classically because their states live in exponentially large spaces, Feynman observed, so perhaps a computer built from quantum parts could simulate them efficiently. Four decades later, the strongest case for any near-term quantum algorithm still rests on that observation. The Variational Quantum Eigensolver, or VQE, is its most direct descendant, and it is the place to see the hybrid loop at its most promising and to see, quantitatively, what stands in its way. The problem VQE addresses The central problem of quantum chemistry is to find the ground state of a molecule's electrons: the lowest-energy arrangement, given the positions of the nuclei. Its energy determines bond strengths, reaction barriers, binding affinities and spectra. The exact answer requires solving the Schrödinger equation for many interacting electrons, and the cost of doing so exactly, by full configuration interaction, grows exponentially with the number of orbitals. Exact calculations top out at around twenty-odd electrons in a similar number of orbitals. Chemists have spent seventy years building approximate methods that work extraordinarily well for most molecules. Density functional theory handles systems of thousands of atoms with useful accuracy. Coupled-cluster theory with single, double and perturbative triple excitations, known as CCSD(T), is called the gold standard because for molecules near their equilibrium geometry it reaches chemical accuracy, about one kilocalorie per mole or 1.6 milliHartree, at a cost growing as the seventh power of system size. The density matrix renormalization group, introduced by Steven White in 1992, handles systems with one-dimensional or weakly entangled structure. Quantum Monte Carlo methods and selected configuration interaction methods fill other niches. What these methods find hard are strongly correlated systems, in which several electronic configurations contribute comparably to the ground state and no single reference picture works. Transition-metal complexes, many catalytic active sites, bond-breaking processes and certain magnetic materials fall in this category. The iron-molybdenum cofactor of nitrogenase, called FeMoco, which catalyses the conversion of atmospheric nitrogen to ammonia at room temperature, became the standard example after Markus Reiher, Matthias Troyer and colleagues published a 2017 analysis of what a fault-tolerant quantum computer would need to compute its energy reliably. If a quantum computer could produce accurate energies for such systems, it would answer questions that matter for catalysis, batteries and drug design. How VQE works VQE applies the hybrid loop to a single principle of quantum mechanics: the variational principle, which states that the expectation value of the energy in any trial state is at least as large as the true ground-state energy. Minimizing the energy over a family of trial states therefore gives an upper bound that approaches the ground state as the family becomes rich enough. The procedure begins classically. The molecule's geometry is fixed, a basis set of orbitals is chosen, and the electronic Hamiltonian is computed in that basis. A fermion-to-qubit mapping, such as the Jordan-Wigner or Bravyi-Kitaev transformation, then turns the Hamiltonian into a weighted sum of Pauli strings acting on qubits, typically one qubit per spin orbital. A parameterized circuit prepares a trial state, usually starting from the Hartree-Fock state, the best single-configuration approximation, which is trivial to prepare. The quantum processor estimates the energy by measuring each group of Pauli strings, and a classical optimizer adjusts the parameters. Alberto Peruzzo, Jarrod McClean and colleagues demonstrated the idea in 2014 on a photonic chip, computing the ground-state energy of the helium hydride ion with two qubits. In 2016 Peter O'Malley and colleagues at Google computed the energy curve of molecular hydrogen on a superconducting processor and compared VQE with the phase-estimation algorithm it was designed to replace in the near term. In 2017 Kandala and colleagues at IBM used six superconducting qubits and a hardware-efficient ansatz to compute energy curves for hydrogen, lithium hydride and beryllium hydride. Cornelius Hempel and colleagues at Innsbruck ran comparable calculations on trapped ions in 2018. In 2020 Google's team ran a twelve-qubit Hartree-Fock calculation on its Sycamore processor, studying hydrogen chains and the isomerization of diazene. That last experiment was a careful demonstration of hardware calibration and error mitigation, but Hartree-Fock is a mean-field method that classical computers solve routinely, so it was a benchmark of the hardware rather than of chemistry. The hydrogen molecule shows the whole procedure in miniature. In the smallest standard basis set, called STO-3G, each hydrogen atom contributes one spatial orbital, giving two spatial orbitals and four spin orbitals for the molecule's two electrons. The Jordan-Wigner mapping turns the electronic Hamiltonian into a sum of fifteen Pauli strings on four qubits, including a constant term, with coefficients that depend on the bond length. Because of symmetries, only one excitation matters: both electrons moving together from the bonding orbital to the antibonding orbital. The unitary coupled-cluster ansatz therefore has a single parameter, an angle that controls how much of the doubly excited configuration is mixed into the Hartree-Fock state. The quantum processor prepares the state for a given angle, measures the Pauli terms in a few groups, and reports the energy; the classical optimizer adjusts the angle; and within a handful of iterations the energy converges. Repeating at different bond lengths traces out the potential energy curve, which near equilibrium is captured well by Hartree-Fock and at long bond lengths, where the bond is breaking and the two configurations contribute equally, requires the correlation that the single parameter provides. Symmetry reductions shrink the problem further, to two qubits or even one, which is how the earliest experiments fitted it onto their devices. Everything about this example can be computed exactly with pencil and paper or a few lines of classical code. It is a perfect teaching case, and it is also a reminder of how far a demonstration on hydrogen is from a calculation that matters. All these experiments addressed molecules small enough that the exact answer was known classically, which is necessary for validation but means none of them computed anything new. That is the first general feature of VQE experiments to keep in mind: a successful VQE demonstration on today's hardware is one whose result could be checked with a classical calculation. Choosing the trial states The choice of ansatz shapes everything else, and the main families trade physical faithfulness against circuit cost, as Table 1 summarizes. Table 1. Main families of VQE ansatz and their trade-offs. Ansatz family Built from Respects particle number Typical circuit cost Main weakness Unitary coupled cluster (UCCSD) Coupled-cluster excitations Yes Deep; grows steeply with orbitals Too deep for current hardware beyond small molecules Hardware-efficient Native gates in repeated layers Not in general Shallow per layer Barren plateaus; unphysical states ADAPT-VQE Operators added one by one by gradient Yes, with fermionic pools Grows only as needed Many measurement rounds to select operators Symmetry-preserving and Givens rotations Particle-conserving two-qubit gates Yes Moderate Less expressive per layer; still deep for large systems Qubit coupled cluster and qubit-ADAPT Pauli-string excitations Partially Shallower than UCCSD Weaker chemical intuition; symmetry breaking The unitary coupled-cluster ansatz with single and double excitations, UCCSD, translates the classical coupled-cluster idea into a unitary operator that can be implemented as a circuit. It respects the symmetries of the molecule and has a clear chemical interpretation, and it is exact for two-electron systems. Its weakness is cost. The number of double excitations grows roughly as the fourth power of the number of orbitals, and each is implemented with a ladder of entangling gates, so even small molecules require thousands of two-qubit gates. On present hardware, with error rates around one in a thousand per gate, that means circuits that fail far more often than they succeed. Hardware-efficient ansätze are shallow but generic. Kandala's 2017 experiment used them precisely because UCCSD was out of reach, and they worked for six qubits. At larger sizes they run into the barren plateaus described in the next chapter, and because they do not conserve particle number or spin, the optimizer can drift into states that correspond to the wrong number of electrons unless penalty terms are added. Adaptive methods try to get the best of both. ADAPT-VQE, introduced by Harper Grimsley, Sophia Economou, Edwin Barnes and Nicholas Mayhall in 2019, starts from the Hartree-Fock state and grows the circuit one operator at a time, at each step adding the operator from a pool whose gradient is largest. It typically yields much shorter circuits than UCCSD for the same accuracy. The cost moves to measurement: every growth step requires estimating gradients for every operator in the pool, which may contain thousands of candidates. The measurement bottleneck Even if a good ansatz can be run without errors, VQE faces a statistical bottleneck. The energy is a sum of many Pauli terms with different weights, and each must be estimated from shots. To reach chemical accuracy, the variance of the overall estimate must be pushed below roughly the square of 1.6 milliHartree, and the number of shots needed scales with the sum of the magnitudes of the Hamiltonian's coefficients divided by that target precision, squared. Dave Wecker, Matthew Hastings and Matthias Troyer pointed out the scale of this problem in 2015, estimating that for molecules of practical interest the number of measurements per energy evaluation could be astronomical with naive strategies. Subsequent work reduced the constants considerably by grouping commuting terms, by measuring in bases tailored to the Hamiltonian's structure, by factorizing the two-electron integrals, and by classical-shadow techniques. Yet a 2022 resource analysis by Jérôme Gonthier and colleagues at Zapata Computing, titled "Measurements as a roadblock to near-term practical quantum advantage in chemistry," concluded that for molecules relevant to combustion chemistry, even with the best available grouping methods, a single energy evaluation to chemical accuracy would require days of measurement time on hardware with realistic shot rates. A full VQE optimization needs hundreds or thousands of such evaluations. The bottleneck has a deeper interpretation. Classical methods compute energies to many digits essentially for free once they have the wavefunction. VQE pays a statistical cost for every digit, and the cost per digit grows quadratically. This is the opposite of the situation in which quantum computers shine, where a short output, such as the factors of a number, encodes the answer to a hard problem. In VQE the output is a real number that must be known precisely, and precision is expensive. Noise and error mitigation The third obstacle is noise. Every gate error biases the energy estimate, usually upward, because noise drives the state toward a mixture that has higher energy than the ground state. Researchers have developed a toolkit of error-mitigation techniques that reduce bias without full error correction. Zero-noise extrapolation, proposed by Kristan Temme, Sergey Bravyi and Jay Gambetta in 2017 and demonstrated by Kandala and colleagues in 2019, deliberately amplifies the noise, for example by stretching gate pulses or inserting pairs of gates that cancel ideally but add errors, measures the energy at several noise levels, and extrapolates back to zero noise. Probabilistic error cancellation learns a model of the noise and inverts it statistically by sampling from a set of corrective circuits with positive and negative weights. Symmetry verification discards measurement outcomes that violate known conservation laws, such as the wrong number of electrons. Each of these techniques can dramatically improve small experiments. Each also has a cost that grows exponentially with circuit size. Probabilistic error cancellation inflates the variance of the estimate by a factor that grows exponentially with the total error in the circuit; zero-noise extrapolation becomes unreliable when the noise is too large to extrapolate from. Yihui Quek, Daniel Stilck França, Sumeet Khatri, Johannes Jakob Meyer and Jens Eisert proved in a 2024 Nature Physics paper that, for a broad class of mitigation schemes, the number of samples needed grows exponentially with circuit depth and, for some families of circuits, exponentially with the number of qubits once the depth exceeds a surprisingly small threshold that grows only very slowly with system size. Mitigation can extend the reach of NISQ devices, but it cannot turn them into scalable machines. Where VQE stands Put the three obstacles together and the outlook for VQE as a route to practical quantum advantage in chemistry is sobering. Circuits deep enough to represent strongly correlated states accurately are too deep for current noise levels. Circuits shallow enough to run are often either classically simulable or untrainable. And whatever circuit one chooses, the measurement cost for chemical accuracy on industrially relevant molecules is prohibitive with current shot rates. There is a further, more fundamental question, raised in a 2023 Nature Communications paper by Seunghoon Lee, Garnet Chan, Ryan Babbush, John Preskill, Lin Lin and a large group of colleagues, titled "Evaluating the evidence for exponential quantum advantage in ground-state quantum chemistry." The authors examined whether there is evidence that quantum computers will offer exponential speedups for generic ground-state chemistry problems, even with full error correction. Their answer was that such evidence is lacking. Fault-tolerant algorithms like quantum phase estimation require an initial state with substantial overlap with the true ground state, and for systems where classical heuristics fail, preparing such a state may itself be hard; where such a state is easy to prepare, classical heuristics often work too. They found that classical methods appear to scale well enough on the systems examined that the quantum advantage is more likely to be polynomial than exponential. That does not make quantum chemistry pointless, because polynomial advantages with large exponents can be very valuable, but it reframes the question from whether quantum computers will make chemistry exponentially easier to whether they will deliver better constant factors and scaling on specific, carefully chosen strongly correlated problems. Recent experiments have moved in a direction that acknowledges these limits. IBM and the RIKEN institute in Japan have developed sample-based quantum diagonalization, in which the quantum processor is used not to estimate energies directly but to sample electronic configurations that are then used to build a subspace in which a classical supercomputer diagonalizes the Hamiltonian exactly. The method sidesteps the measurement bottleneck by treating the quantum device as a proposal generator, and published demonstrations in 2024 and 2025 used up to several dozen qubits on problems such as nitrogen bond dissociation and iron-sulfur clusters. The energies were competitive with some classical methods but not better than the best ones, and the question of whether the quantum-proposed configurations are harder to find than classically proposed ones remains open. The honest summary is this. VQE is the hybrid algorithm with the best physical motivation, because molecular ground states are genuinely quantum objects whose difficulty is not in doubt. It has been an invaluable testbed for hardware, error mitigation and measurement techniques. It has not yet computed a single chemically useful quantity that classical methods could not, and there are good reasons, rooted in measurement statistics, noise and trainability, to think that it will not do so in its original form on noisy hardware. Its legacy may be the toolkit it built, which feeds directly into the fault-tolerant chemistry algorithms discussed in Chapter 8, rather than the algorithm itself. That conclusion carries a lesson for quantum machine learning in general. If the most naturally quantum application, computing quantum ground states, struggles to show advantage in the variational framework, then applications in which quantum structure has to be imposed on classical data face a steeper climb. The next chapter explains one reason why: the landscape that the classical optimizer must navigate. CHAPTER 4 Trainability and the Barren Plateau In classical deep learning there was a period, roughly the 1990s and early 2000s, when deep networks were widely believed to be untrainable. Gradients vanished or exploded as they propagated through many layers, and researchers moved to shallower models and to support vector machines. The problem was solved, mostly, by a combination of better activation functions, careful initialization, normalization layers and residual connections, together with far more data and computation. It is natural to hope that quantum machine learning's training problems will yield to a similar campaign of engineering. The evidence of the last eight years suggests that the quantum version of the problem is different in kind, and that its solutions come at a price the classical version never exacted. The discovery In 2018 Jarrod McClean, Sergio Boixo, Vadim Smelyanskiy, Ryan Babbush and Hartmut Neven, all then at Google, published a paper in Nature Communications titled "Barren plateaus in quantum neural network training landscapes." Their argument was short and general. Suppose a parameterized circuit is expressive enough that, as its parameters vary randomly, the unitaries it produces are spread approximately uniformly over the space of all unitaries, or at least uniformly enough to match the first two moments of the uniform distribution, a property called being a unitary 2-design. Then for a broad class of cost functions, the expected gradient with respect to any parameter is zero, and, crucially, the variance of the gradient decreases exponentially with the number of qubits. To see what that means in practice, consider a circuit on fifty qubits. If the gradient's variance scales as 2 to the power minus fifty, its typical magnitude is around three parts in a hundred million. Estimating a quantity that small from shot noise, which decreases only as the inverse square root of the number of shots, would require on the order of 2⁵⁰, roughly 10¹⁵, shots for each gradient component. The landscape is not merely difficult; it is, for all practical purposes, perfectly flat everywhere except in an exponentially small neighbourhood of the minimum. The optimizer cannot tell which way is downhill. The name stuck because the image fits. A barren plateau is a vast featureless region of parameter space where the cost is essentially constant, concentrated sharply around its average value, and a randomly initialized model will almost certainly start on it. It was quickly established that the problem is not a quirk of gradient-based optimization. Andrew Arrasmith, Marco Cerezo and colleagues showed in 2021 that gradient-free optimizers suffer just as much, because the differences in cost between nearby points are exponentially small whatever method compares them. Higher-order derivatives vanish too. The problem lies in the concentration of the cost function itself, not in how one probes it. The many causes of flatness Over the following years, researchers identified a series of distinct mechanisms that produce barren plateaus, each initially treated as its own phenomenon. The first was expressiveness. Zoë Holmes, Kunal Sharma, Marco Cerezo and Patrick Coles showed in 2022 that the more expressive an ansatz, meaning the more closely its random parameter settings cover the full unitary group, the smaller its gradients. Expressiveness is exactly what one wants from a model in the abstract, because it guarantees that the optimal solution is representable, but beyond a point it guarantees that the optimal solution cannot be found. The second was the cost function. As mentioned in Chapter 2, Cerezo and colleagues proved in 2021 that global cost functions, which compare entire n-qubit states, exhibit barren plateaus even for shallow circuits of constant depth, while local cost functions, built from observables that act on a few qubits, retain gradients that vanish only polynomially for circuits of logarithmic depth. A very simple example, in the spirit of the one Cerezo and colleagues used, shows why the cost function matters even without any entanglement. Take n qubits, each rotated independently by its own angle, and suppose the goal is to return every qubit to zero. A global cost asks for the probability that all n qubits are measured as zero, which is the product of the n individual probabilities. If the angles start at random, each individual probability averages one half, so the product, and every derivative of it, is typically of order one over 2ⁿ: at fifty qubits, a number around one in a quadrillion. A local cost asks instead for the average, over qubits, of the probability that each one is zero. That average starts around one half regardless of n, and its derivative with respect to any angle depends only on that qubit's own rotation, divided by n. The two costs have the same minimum, but one presents the optimizer with a flat desert and the other with a gentle slope. The circuit here is trivial and classically simulable; the point is that the choice of what to measure, by itself, determines whether optimization is possible. The third was entanglement. Carlos Ortiz Marrero, Mária Kieferová and Nathan Wiebe showed in 2021 that when the hidden units of a quantum neural network become highly entangled with the visible ones, the reduced states that determine the cost become close to maximally mixed, and gradients vanish. Too much entanglement, the very resource that is meant to make quantum models powerful, leaves the observable part of the system looking like featureless noise. The fourth was noise. Samson Wang and colleagues showed in 2021 that local Pauli noise acting throughout a circuit causes gradients to vanish exponentially with circuit depth, irrespective of the ansatz or the initialization. This noise-induced barren plateau is especially damaging because none of the structural remedies discussed below address it. If the circuit must be deep, as many interesting circuits must, noise flattens the landscape regardless. The fifth was data. Supanut Thanasilp and colleagues showed in 2023 that the feature map used to encode classical data can itself induce concentration: if the encoding produces states that are too spread out over Hilbert space, the model's outputs concentrate around a constant for every input, and training stalls before any trainable layer is reached. The unifying theory For several years these results were proved one at a time, under different assumptions, with different techniques. In 2024 two groups unified them. Michael Ragone, Bojko Bakalov, Frédéric Sauvage, Alexander Kemper, Carlos Ortiz Marrero, Martín Larocca and Marco Cerezo, in a paper titled "A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits," and Enrico Fontana, Dylan Herman and colleagues, in "Characterizing barren plateaus in quantum ansätze with the adjoint representation," both published in Nature Communications that year, showed that the variance of the cost function for deep circuits is governed by a single algebraic object: the dynamical Lie algebra of the circuit. The idea can be explained without the algebra. The gates of a parameterized circuit are generated by a set of operators, one for each kind of parameterized gate. Repeatedly combining those generators, by taking their commutators, produces a space of operators that describes every direction in which the circuit can move the quantum state. That space is the dynamical Lie algebra. If it is small, growing only polynomially with the number of qubits, the circuit can only explore a polynomially sized portion of the full space of unitaries, and gradients need not vanish exponentially. If it is large, growing exponentially, the circuit explores an exponentially large space, its outputs concentrate, and a barren plateau follows. The theory also accounts for the roles of the initial state and the measured observable: if either is spread across a large part of the algebra, concentration follows even when the algebra is modest. This picture brought order to the zoo. Expressiveness, entanglement, global measurements and data-induced concentration are all ways of saying that some part of the computation, the circuit, the input state or the observable, occupies an exponentially large space. A comprehensive review by Martín Larocca, Supanut Thanasilp, Samson Wang and colleagues, published in Nature Reviews Physics in 2025, consolidated this view and set out the remaining open questions, chiefly concerning circuits of intermediate depth and the precise effects of noise. Remedies and their price The barren plateau literature produced a well-stocked list of remedies, and each one helps in its own domain. Local cost functions avoid the plateaus caused by global measurement. Initializing circuits near the identity, as proposed by Edward Grant and colleagues in 2019, or drawing initial angles from narrow Gaussian distributions, as studied by Kaining Zhang and colleagues in 2022, places the model in a region where gradients are large, at least at the start of training. Layer-wise training builds a deep circuit gradually. Quantum convolutional neural networks, proposed by Iris Cong, Soonwon Choi and Mikhail Lukin in 2019, have a hierarchical structure in which the number of qubits halves at each pooling layer; Arthur Pesah and colleagues proved in 2021 that they do not suffer from barren plateaus. Equivariant or symmetry-respecting architectures, developed by Larocca, Johannes Jakob Meyer and others, constrain the circuit to respect known symmetries of the problem, which shrinks the dynamical Lie algebra and keeps gradients alive. Warm starts from classical solutions, such as initializing a chemistry ansatz at the Hartree-Fock state, place the optimizer near a good region from the outset. All of these remedies work by the same mechanism, which the Lie-algebraic theory makes explicit. They restrict the computation to a small corner of the full quantum state space: a small Lie algebra, a local observable, a narrow region around a known starting point, a symmetry-constrained subspace. Within that corner gradients survive. The question that follows is obvious once stated, and it took the field until late 2023 to state it sharply. If the computation is confined to a small, polynomially sized region of the full space, can a classical computer simulate what happens in that region? Chapter 6 takes up the answer, which is in many cases yes. For now, it is enough to see the structure of the dilemma. A quantum model that explores the full space of quantum states is untrainable. A quantum model that is trainable explores only a small part of that space. Whether the small part is still classically hard is the question on which near-term quantum machine learning hinges. Identity-near initialization illustrates the subtlety. It ensures large gradients at the start of training, but only because the circuit begins very close to the identity, which is trivially simulable. As training progresses and parameters move away from small values, the model may wander into the plateau, or it may stay in a region where classical methods that expand around the identity, such as truncated Pauli propagation, still work. Either outcome is unwelcome for advantage. For a practitioner, the theory translates into a simple diagnostic that should precede any serious training run. Fix the ansatz, the observable and the encoding, draw many random initializations, and estimate the variance of a few gradient components at each of several qubit counts, say four, six, eight, ten and twelve. Plot the logarithm of the variance against the number of qubits. If the points fall on a straight line with negative slope, the gradient is vanishing exponentially and the model will not train at scale, however well it behaves at eight qubits. If the variance decays polynomially or not at all, the model may be trainable, and the next question is why. The diagnostic costs little in simulation and has saved many projects from scaling studies that could only have failed. It also, uncomfortably, tends to point to the reason for any good behaviour, a small algebra, a local observable, a restricted initialization, and therefore to the classical simulation strategy that the next stage of scrutiny will try. Beyond plateaus: traps and hardness Even a landscape without barren plateaus can be hard to optimize. Two results make this concrete. Lennart Bittel and Martin Kliesch proved in 2021 that training variational quantum algorithms is NP-hard in general. Their construction shows that finding optimal parameters is hard even for circuits on a logarithmic number of qubits, and even for classes of circuits whose quantum dynamics are classically simulable. The hardness comes from the classical optimization problem itself, not from any quantum effect. A variational algorithm, in other words, can turn an easy quantum problem into a hard classical one. Eric Anschuetz and Bobak Kiani showed in 2022, in a paper titled "Beyond barren plateaus: quantum variational algorithms are swamped with traps," that a broad class of variational models that avoid barren plateaus still have landscapes riddled with local minima far from the global minimum, so many that a local optimizer started at random almost surely gets stuck in a poor one. The exception is when the model is heavily overparameterized relative to the problem, which, for most models, is possible only when the relevant Lie algebra is small. Martín Larocca and colleagues' 2023 theory of overparameterization in quantum neural networks makes the same link: a model becomes overparameterized, with a benign landscape, when its number of parameters reaches the dimension of its dynamical Lie algebra. If that dimension is exponential, overparameterization is impossible. If it is polynomial, overparameterization is achievable, and the model is once again confined to a small space. In classical deep learning, overparameterization famously makes optimization easier: networks with more parameters than training examples find good minima readily. The quantum analogue exists, but it becomes available only on precisely the small-algebra models that are most vulnerable to classical simulation. Noise adds a final constraint. Daniel Stilck França and Raúl García-Patrón showed in 2021 that for noisy quantum devices running variational optimization, beyond a certain circuit depth the states produced become so mixed that an efficient classical algorithm can find solutions of comparable quality. Their analysis yields explicit depths beyond which noisy quantum optimization offers no advantage, for realistic noise rates. Together with noise-induced barren plateaus, this places an upper limit on the useful depth of a variational circuit on noisy hardware, while the need to avoid classical simulability places a lower limit. At the error rates of current devices, the window between the two is narrow and, for many problems, empty. What trainability tells us There is a temptation to read the barren plateau literature as a list of engineering problems to be solved one by one, the way the classical vanishing-gradient problem was. That reading misses its main lesson. The classical vanishing-gradient problem was a property of particular architectures and activation functions, and changing them did not weaken the models; modern deep networks are both trainable and enormously powerful. The quantum barren plateau, as the Lie-algebraic theory shows, is a property of how much of the quantum state space a model explores. The remedies work by exploring less of it. Trainability and quantum power are therefore in tension in a way that has no clear classical analogue. This does not mean every trainable quantum model is classically simulable. There may be models with small dynamical Lie algebras or other trainable structures whose outputs are nonetheless classically hard to compute, and there may be good warm starts, from classical solutions or from physical intuition, that place the optimizer in a region of a large space where gradients are informative and simulation is hard. Finding such models is now one of the central research problems of the field. But it means that the burden of proof has shifted. A new variational architecture that trains well on twenty simulated qubits should no longer be greeted as evidence of quantum power. It should prompt the question of why it trains well, and whether the same reason allows a classical computer to do the same job. With that question in hand, we can turn to the quantum support vector machine, a model that was designed to avoid variational training altogether and that meets its own version of the same difficulty. CHAPTER 5 Quantum Kernels and Support Vector Machines The support vector machine was the dominant tool of applied machine learning for most of the 2000s, and it remains one of the best understood. Its appeal for quantum computing is easy to see. A support vector machine separates data into classes by finding the hyperplane with the largest margin in some feature space, and through the kernel trick it can work in feature spaces of enormous, even infinite, dimension without ever writing down the features. All it needs is the inner product between pairs of feature vectors, the kernel. A quantum computer prepares vectors in an exponentially large space and can estimate their inner products by measurement. The match seems almost too neat. It is also attractive for a reason that goes beyond the geometry. Training a support vector machine, once the kernel is known, is a convex optimization problem with a unique solution. If the quantum computer's only job is to supply kernel values, the barren plateaus and local minima of Chapter 4 disappear from the training step entirely. The quantum support vector machine therefore looked, for a while, like the cleanest route to quantum machine learning: hard quantum feature maps, easy classical training. This chapter traces how that idea developed, from a fault-tolerant proposal in 2014 to the hardware experiments that followed, and explains why, despite one rigorous proof of advantage, the quantum kernel has turned out to suffer from its own form of flatness. Two different quantum support vector machines The phrase "quantum support vector machine" refers to two quite different algorithms, and confusion between them has caused much misunderstanding. Table 2 sets out the main variants. Table 2. Variants of the quantum support vector machine and the evidence for advantage. Variant Introduced Where the quantum computer acts Hardware required Status of advantage claim Linear-algebra QSVM Rebentrost, Mohseni, Lloyd, 2014 Solves the training linear system Fault-tolerant, with QRAM Largely dequantized for low-rank data Quantum kernel estimation Havlíček et al.; Schuld and Killoran, 2019 Estimates each kernel entry Near-term No practical advantage shown Discrete-logarithm kernel Liu, Arunachalam, Temme, 2021 Kernel based on a hard group problem Fault-tolerant Proven, for an artificial task Projected quantum kernel Huang et al., 2021 Kernel from reduced states Near-term Advantage only on engineered labels Covariant quantum kernel Glick et al., 2024 Kernel respecting a group symmetry Near-term Demonstrated on structured synthetic data The first, proposed by Patrick Rebentrost, Masoud Mohseni and Seth Lloyd in a 2014 Physical Review Letters paper, is a quantum algorithm for training a least-squares variant of the support vector machine. It reformulates training as the solution of a system of linear equations and solves that system with a quantum linear-systems routine descended from the 2009 algorithm of Aram Harrow, Avinatan Hassidim and Seth Lloyd. Under its assumptions, data loaded through quantum random access memory and a kernel matrix that is well conditioned and dominated by a few large eigenvalues, the training time scales logarithmically in both the number of data points and the dimension of the features. That was an exponential speedup on paper. It inherited every caveat that Scott Aaronson later collected in "Read the fine print." The data must be loaded in superposition, which requires a QRAM no one has built. The output is not a classifier one can inspect but a quantum state encoding the classifier's parameters, which must be re-prepared for every new prediction. And, as Chapter 6 describes, Ewin Tang's dequantization techniques, adapted to support vector machines by Chen Ding, Tian-Yi Bao and He-Liang Huang in 2019, showed that under comparable assumptions about data access and low rank a classical algorithm can achieve polylogarithmic dependence on dimension too, with a polynomial rather than exponential overhead. The exponential separation dissolved into a polynomial one, and quite possibly a large-degree polynomial in the classical algorithm's favour on small problems but against it on large ones. The second algorithm, and the one that most people now mean by the term, uses the quantum computer only to compute kernel values. Maria Schuld and Nathan Killoran laid out the idea in a 2019 paper titled "Quantum machine learning in feature Hilbert spaces," and Vojtěch Havlíček, Antonio Córcoles, Kristan Temme, Aram Harrow, Abhinav Kandala, Jerry Chow and Jay Gambetta demonstrated it the same year in Nature, in a paper titled "Supervised learning with quantum-enhanced feature spaces." Kernel estimation on hardware The quantum kernel estimator works as follows. A feature map is a circuit U(x) whose gates depend on the features of a data point x. Applied to a fixed starting state, it produces a quantum state that serves as the feature vector of x. The kernel between two data points x and x′ is defined as the squared overlap between their states: the probability that the state prepared for x would be found in the state prepared for x′. On hardware, this is estimated by running U(x) followed by the inverse of U(x′) and measuring the probability that all qubits return to zero. Repeating this for every pair of training points fills in the kernel matrix, which is handed to a classical support vector machine solver. Classifying a new point requires estimating its kernel with each of the support vectors. Havlíček and colleagues ran this on two qubits of a five-qubit IBM superconducting processor, using a feature map that encodes each feature in a rotation and products of pairs of features in entangling ZZ interactions, now commonly called the ZZ feature map. They classified an artificial data set constructed so that its labels were defined by the feature map itself, and achieved near-perfect accuracy. They argued that kernels of this type were likely hard to estimate classically for larger numbers of qubits, by connection to a class of circuits, instantaneous quantum polynomial-time circuits, whose output distributions are believed hard to sample. The paper was carefully worded and widely misread. Its data were synthetic and designed to suit the feature map; it did not show that the quantum kernel performed better than classical kernels on any natural problem; and its classical-hardness argument concerned exact computation of kernel entries for larger circuits, not the learning task. What it established was that the quantum kernel pipeline could be run on hardware, and it set off a large wave of follow-up studies applying quantum kernels to data sets from finance, medicine, high-energy physics and remote sensing. The cost structure of quantum kernel estimation deserves emphasis. A training set of N points requires about N²/2 kernel entries, each estimated with enough shots to reach the needed precision. For N = 1,000 that is half a million circuits; for N = 100,000, a modest data set by classical standards, it is five billion. Classical kernel methods face the same quadratic scaling, which is one reason kernel methods lost ground to deep learning on large data sets, but classical kernel entries cost nanoseconds rather than thousands of shots. The rigorous result and the power of data In 2021 Yunchao Liu, Srinivasan Arunachalam and Kristan Temme published in Nature Physics "A rigorous and robust quantum speed-up in supervised machine learning." They constructed a classification problem based on the discrete logarithm, the problem Shor's algorithm solves, in which data points are integers and labels depend on whether their discrete logarithms fall into a certain interval. They proved that no efficient classical learner can do better than random guessing by more than an inverse-polynomial margin on this task, assuming the discrete logarithm is classically hard, while a quantum kernel method using a feature map built from Shor-style circuits learns it accurately, and robustly to finite sampling error. This was a landmark because it was the first end-to-end proof of a quantum advantage for a learning task with classical data, and its techniques are genuinely instructive. But its limits are equally instructive. The data set is artificial, constructed so that its structure is precisely the kind of hidden periodicity that quantum computers exploit. The feature map requires the circuits of Shor's algorithm, which need a fault-tolerant machine. And the advantage is, at heart, the advantage of Shor's algorithm in disguise. The result shows that quantum kernels can embed known quantum speedups. It does not suggest that they will find new ones in natural data. The same year, Hsin-Yuan Huang, Michael Broughton, Masoud Mohseni, Ryan Babbush, Sergio Boixo, Hartmut Neven and Jarrod McClean published "Power of data in quantum machine learning" in Nature Communications, which made a point that recast the whole field. A classical machine learning model given data is more powerful than a classical algorithm without data. Even if computing a quantum kernel exactly is classically hard, a classical learner trained on examples of the function in question may predict it just as well. The authors introduced a quantity they called the geometric difference between a quantum kernel and a classical one, which bounds how much better the quantum kernel can generalize. For most quantum kernels they examined, including standard feature maps on standard data, the geometric difference was small, meaning a classical kernel would do as well. Only when the labels were deliberately constructed from the quantum kernel's own structure did they see a large gap, which they demonstrated in simulations of up to thirty qubits. Exponential concentration The deepest problem with quantum kernels mirrors the barren plateau. If the feature map is expressive, entangling and deep, the quantum states it produces for different inputs are spread almost uniformly across an exponentially large Hilbert space. Two random states in such a space are almost orthogonal: their overlap is exponentially small in the number of qubits. The kernel matrix therefore approaches the identity matrix, with ones on the diagonal and nearly zeros everywhere else. A kernel matrix close to the identity says that every training point is dissimilar to every other, and a support vector machine trained on it can memorize the training labels but has no basis for generalizing to new points. Jonas Kübler, Simon Buchholz and Bernhard Schölkopf made this point in their 2021 paper "The inductive bias of quantum kernels," arguing that quantum kernels without a carefully chosen inductive bias behave like learning in a space so large that exponentially many training examples would be needed. Their recommendation was to project quantum states onto a smaller space, as Huang's projected kernels do, so that the kernel captures structure at a scale the data can resolve. Supanut Thanasilp, Samson Wang, Marco Cerezo and Zoë Holmes made the result precise in a 2024 Nature Communications paper titled "Exponential concentration in quantum kernel methods." They identified four sources of concentration, expressive feature maps, global measurements, entanglement and hardware noise, matching the causes of barren plateaus, and showed that under any of them the kernel values concentrate exponentially around a fixed value. With only polynomially many shots per entry, the estimated kernel matrix is then statistically indistinguishable from one that carries no information about the inputs, and the trained model's predictions become independent of the data. The problem afflicts not only estimation by overlap tests but also the projected kernels that Huang and colleagues had proposed as a partial remedy, when their feature maps are sufficiently expressive. The practical fix is to reduce the spread. Ruslan Shaydulin and Stefan Wild showed in 2022 that introducing a bandwidth parameter, which simply scales down the rotation angles by which features are encoded, restores non-trivial kernel values and improves generalization dramatically; Abdulkadir Canatar and colleagues reached a similar conclusion in 2023. With small angles, the quantum states for different inputs stay close together and the kernel varies smoothly with the data. The simplest possible feature map shows where this leads. Encode a single feature x by rotating one qubit about the x axis by the angle x. The overlap between the states for two inputs x and x′ is then the squared cosine of half their difference, cos²((x − x′)/2), which is easy to verify by hand. Encode d features on d separate qubits in the same way, without entanglement, and the kernel becomes the product of d such factors. Now introduce a bandwidth c by rotating each qubit by c times its feature instead. When c is small, each factor cos²(c(xᵢ − x′ᵢ)/2) is well approximated by an exponential of minus c² times the squared difference divided by four, and the product becomes a Gaussian kernel in the distance between the two data points: the radial basis function kernel that every classical machine learning library has offered for decades. When c is large, the factors oscillate rapidly, the product for distinct points is a product of many numbers less than one, and the kernel matrix approaches the identity. Entangling layers complicate the algebra but not the qualitative picture. Bandwidth tuning moves a quantum kernel between two failure modes: concentration at one end, and at the other, a close relative of the classical kernel it was supposed to improve on. But small angles also mean that each feature's rotation is small, the circuit stays close to the identity, and the resulting kernel resembles a smooth classical kernel, with low-order Fourier components dominating. As in Chapter 4, the remedy works by confining the computation to a small region, and the question of whether a classical kernel can imitate the result follows immediately. Structured kernels and the role of symmetry The most promising direction to come out of this work is to design quantum kernels around known structure in the data, rather than choosing generic feature maps and hoping. Jennifer Glick, Tanvi Gujarati, Antonio Córcoles and colleagues at IBM published "Covariant quantum kernels for data with group structure" in Nature Physics in 2024. They considered data whose labels are determined by a group symmetry, built a kernel that respects that symmetry, and used a technique called kernel alignment to tune the kernel's parameters. They demonstrated the approach on a 27-qubit superconducting processor for data generated with a particular group structure. This is the right instinct. It matches the lesson of the Liu, Arunachalam and Temme proof, that advantages come from structure quantum computers handle well, and it matches the lesson of the power-of-data analysis, that a quantum kernel must differ from classical kernels in a way the data can detect. Its weakness is that the data sets on which such kernels have been shown to excel are themselves generated from the group structure. For natural data, one would first have to know that the relevant structure is present and that it is classically hard to exploit, which is a substantial research problem in each domain. Maria Schuld's 2021 observation that supervised quantum models are kernel methods ties this chapter back to the previous ones. Any quantum model that encodes data with a feature map and then applies a trainable circuit and measurement computes a linear function in the feature space defined by that map; the trainable part merely selects a measurement. The kernel formulation finds the best such linear function by convex optimization, so in principle it should match or outperform the variational approach on the training data. Sofiene Jerbi and colleagues complicated this picture in 2023 by showing that variational, or explicit, models can generalize better from fewer examples in some settings, because the kernel approach may need far more training data to identify a function that an explicit model with the right structure can find quickly. The two approaches therefore trade off training cost against data efficiency, but they share the same underlying feature space, and with it the same exposure to concentration. The quantum support vector machine has therefore traced an arc familiar from the rest of this book. It began with an exponential speedup that depended on unrealistic data access. It became a near-term hybrid method whose kernels, if chosen generically, concentrate and carry no information. And it has been rescued, partially, by narrowing it to structured problems and smaller effective spaces, which reopens the question of classical competition. That question is the subject of the next chapter. CHAPTER 6 The Classical Response In 2016 Iordanis Kerenidis and Anupam Prakash proposed a quantum algorithm for recommendation systems, the problem of suggesting products to a user based on a large, sparse, partially observed matrix of preferences. Their algorithm ran in time polylogarithmic in the dimensions of the matrix, exponentially faster than any classical recommendation algorithm then known. It was among the most cited examples of a practical exponential quantum speedup for a machine learning task, and unlike many proposals it did not require the output to be read out in full: it sampled a recommendation, which is what a recommendation system needs. In 2018 Ewin Tang, then an undergraduate at the University of Texas at Austin working under Scott Aaronson, was given the task of proving that no classical algorithm could match it. Instead she found one that did. Her paper, "A quantum-inspired classical algorithm for recommendation systems," published at the ACM Symposium on Theory of Computing in 2019, showed that a classical computer given a kind of sampling access to the data comparable to what the quantum algorithm assumed could also produce recommendations in time polylogarithmic in the matrix dimensions. The exponential speedup vanished, leaving at most a polynomial one. That episode began a line of research now called dequantization, and it changed how the field thinks about quantum advantage in machine learning. This chapter describes dequantization and three related classical responses, classical surrogates for variational models, tensor-network and Pauli-propagation simulation of hardware experiments, and the 2025 argument linking trainability to simulability, which together form the most important body of evidence that a reader evaluating quantum machine learning needs to know. Dequantization: fair comparisons of data access Tang's key insight concerned the input model. Quantum linear-algebra algorithms assume that data can be loaded into amplitudes efficiently, through a QRAM. A QRAM of that kind requires the data to be stored in a binary tree structure that records partial sums of squared entries, so that the quantum computer can prepare a state whose amplitudes are proportional to the entries. Tang observed that the same data structure gives a classical computer the ability to sample an index with probability proportional to the square of its entry, known as length-squared sampling, and to query individual entries. Classical randomized linear algebra, especially the work of Alan Frieze, Ravi Kannan and Santosh Vempala in the late 1990s on low-rank approximation by sampling, had already developed tools for exploiting exactly that kind of access. When the matrix in question is approximately low rank, as recommendation matrices are assumed to be, sampling a modest number of rows and columns suffices to approximate its dominant singular vectors and to sample from the resulting recommendation, in time that depends polylogarithmically on dimension and polynomially on the rank, the condition number and the desired precision. Tang and collaborators quickly extended the technique. András Gilyén, Seth Lloyd and Tang dequantized quantum algorithms for low-rank linear regression, Tang dequantized quantum principal component analysis and supervised clustering, and in 2020 Nai-Hui Chia, András Gilyén, Tongyang Li, Han-Hsuan Lin, Ewin Tang and Chunhao Wang published a general framework that dequantizes a broad class of quantum machine learning algorithms built on low-rank matrix arithmetic. In 2022 Sevag Gharibian and François Le Gall showed that the quantum singular value transformation, the unifying primitive behind most quantum linear-algebra algorithms, can itself be dequantized for low-rank matrices with sampling access. Tang summarized the lesson for quantum principal component analysis in a 2021 Physical Review Letters paper whose title says it all: "Quantum principal component analysis only achieves an exponential speedup because of its state preparation assumptions." Table 3 places dequantization alongside the other classical responses discussed in this chapter. Table 3. Selected quantum machine learning claims and the classical responses to them. Quantum claim Original work Classical response Outcome Exponentially faster recommendation systems Kerenidis and Prakash, 2016 Tang, 2018 Dequantized; polynomial gap at most Quantum principal component analysis Lloyd, Mohseni, Rebentrost, 2014 Tang, 2018 and 2021 Dequantized for low-rank inputs Linear-algebra QSVM and regression Rebentrost et al., 2014 Chia et al., 2020; Ding, Bao, Huang, 2019 Dequantized for low-rank inputs Variational classifiers with angle encoding Many, from 2018 Schreiber, Eisert, Meyer, 2023; Landman et al., 2022 Classical surrogates often match 127-qubit "utility" expectation values Kim et al., 2023 Tindall et al.; Begušić and Chan, 2023 to 2024 Reproduced classically Quantum convolutional neural networks Cong, Choi, Lukin, 2019 Bermejo et al., 2024 Simulable on standard benchmarks Dequantization is often described as a refutation of quantum machine learning, and that is too strong. Its results apply to low-rank problems, and the classical algorithms they produce have polynomial running times of high degree. Juan Miguel Arrazola and colleagues at Xanadu implemented several quantum-inspired algorithms in 2020 and found that they performed well only for matrices of extremely low rank and very large size, and were otherwise slower than standard classical methods; later theoretical work, including a 2024 paper by Ainesh Bakshi and Ewin Tang, reduced the polynomial degrees substantially. Quantum algorithms for sparse, high-rank, well-conditioned linear systems are not dequantized; inverting such matrices is, in a precise sense, as hard as any quantum computation. What dequantization established is that the exponential speedups claimed for quantum machine learning on classical data were artifacts of comparing a quantum algorithm with strong input assumptions against classical algorithms without them. Compared fairly, the gap for low-rank data shrinks to a polynomial, and whether the polynomial favours the quantum algorithm in practice depends on details no one has yet worked out at scale. A common objection to dequantization is that it moves the goalposts by granting the classical algorithm an unusual form of data access. The reply is that the access is not unusual once the quantum algorithm's assumptions are taken seriously. Any QRAM capable of loading amplitude-encoded states must store the data in a structure that records partial sums of squared entries, because that is what the state preparation requires. The same structure, held in ordinary classical memory, supports length-squared sampling at logarithmic cost. If one is willing to assume the quantum machine has that structure available, it is fair to give the classical machine the same, and far cheaper, structure. The comparison that dequantization makes is therefore not a concession to the classical side but a correction of an asymmetry that had been built into the original claims. Where data arrive in some other form, for example as the output of a physical process with no classical record, the argument does not apply, and that boundary marks one of the settings where quantum advantages remain plausible. Classical surrogates for variational models Dequantization addressed fault-tolerant algorithms. Near-term variational models met a parallel challenge from a different direction. Recall from Chapter 1 that a quantum model encoding data in rotation angles computes a truncated Fourier series in the input features, with frequencies set by the encoding. Franz Schreiber, Jens Eisert and Johannes Jakob Meyer showed in a 2023 Physical Review Letters paper titled "Classical surrogates for quantum learning models" that, for many such models, one can efficiently construct a classical model, built directly from the relevant Fourier basis, that reproduces the quantum model's input-output behaviour. Once a quantum model is trained, its surrogate can make predictions without a quantum computer, and in many cases the surrogate can be trained directly without one. Jonas Landman, Slimane Thabet, Constantin Dalyac, Hela Mhiri and Elham Kashefi showed in 2022 that random Fourier features, a standard classical technique for approximating kernel methods introduced by Ali Rahimi and Benjamin Recht in 2007, can approximate variational quantum models efficiently when their frequency spectra are concentrated, which is common in practice. Ryan Sweke and colleagues analysed in 2023 when this approach succeeds and when it fails: it can fail if the quantum model's spectrum is both very large and spread out, and that failure condition marks out the regime in which variational models might retain an edge. The practical message is that a variational classifier trained on classical data with angle encoding should be presumed to have a classical surrogate unless one can show otherwise. This is not a statement that the quantum circuit is easy to simulate gate by gate; it may not be. It is a statement that the function it computes belongs to a class that classical models can represent and learn efficiently. Simulating the hardware experiments The third response concerns specific hardware experiments. In June 2023 Youngseok Kim, Abhinav Kandala and colleagues at IBM published "Evidence for the utility of quantum computing before fault tolerance" in Nature. Using a 127-qubit Eagle processor and zero-noise extrapolation, they estimated expectation values in the dynamics of a two-dimensional kicked Ising model, with circuits of up to sixty layers and thousands of two-qubit gates, and showed that their mitigated results agreed with exact classical results where those were available and, in regimes they claimed were beyond brute-force classical simulation, gave plausible values. They were careful to call this "utility" rather than advantage, but it was widely reported as a sign that noisy processors had begun to compute things classical computers could not. Within weeks, several groups showed otherwise. Joseph Tindall, Matthew Fishman, Miles Stoudenmire and Dries Sels at the Flatiron Institute used tensor networks with belief-propagation contraction, matched to the heavy-hexagon geometry of IBM's chip, to reproduce the experimental results accurately and, in the hardest regimes, apparently more accurately than the experiment, using modest computational resources; their paper appeared in PRX Quantum in 2024. Tomislav Begušić and Garnet Chan at Caltech used a Pauli-path approach, propagating the measured observable backwards through the circuit and truncating small contributions, with similar success. Other groups at Google and elsewhere obtained comparable results with different methods. The experiment remained a striking demonstration of what error mitigation can extract from a noisy device, but its computational task was classically tractable. The experiment is not itself machine learning, but the lesson transfers directly. Variational circuits on noisy hardware live in the same regime of moderate depth, local observables and limited entanglement that made the utility experiment simulable. Tensor-network and Pauli-propagation methods have improved rapidly under the pressure of these challenges, and each improvement raises the bar that a quantum model must clear. It is worth recording that not every hardware experiment has fallen to classical challenge. The original 2019 Sycamore random-circuit-sampling experiment was eventually reproduced classically: in 2022 Feng Pan, Keyang Chen and Pan Zhang used tensor-network contraction on a cluster of several hundred GPUs to generate comparable samples in roughly fifteen hours. But later sampling experiments on larger and higher-fidelity processors, including Google's Willow in 2024, pushed the estimated classical cost far beyond any existing supercomputer, and those estimates have so far held. The pattern is not that classical methods always win; it is that they win when the quantum computation has exploitable structure, low entanglement, shallow light cones, small effective subspaces, and lose when it does not. Random circuits are deliberately unstructured, which is why they resist simulation and also why they are useless for learning. The question for machine learning is whether a model can be structured enough to learn and unstructured enough to resist simulation at the same time. Absence of barren plateaus and classical simulability The most consequential classical response is conceptual. In December 2023 Marco Cerezo, Martín Larocca, Diego García-Martín, N. L. Diaz, Paolo Braccia, Enrico Fontana, Manuel Rudolph, Pablo Bermejo, Aroosa Ijaz, Supanut Thanasilp, Eric Anschuetz and Zoë Holmes posted a paper titled "Does provable absence of barren plateaus imply classical simulability? Or, why we need to rethink variational quantum computing." It was published in Nature Communications in 2025. The argument connects directly to the Lie-algebraic theory of Chapter 4. Barren plateaus arise when the computation explores an exponentially large space. Every known way of proving that a model does not have a barren plateau works by showing that some relevant part of the computation, the dynamical Lie algebra, the light cone of a local observable, a symmetry-restricted subspace, is only polynomially large. But if the action of the model on its inputs can be described within a polynomially large space, then one can often write down that description classically. The authors examined the main families of provably barren-plateau-free models, including shallow circuits with local observables, circuits with small dynamical Lie algebras, symmetry-restricted models and quantum convolutional networks, and showed that in each case the loss landscape could be computed efficiently by a classical algorithm, sometimes after an initial data-acquisition phase on a quantum computer. That last qualification introduced a useful new category. In what the authors called quantum-enhanced classical simulation, a quantum device is used once to gather information about a fixed set of states, for example through classical shadows, and thereafter all training and inference is classical. Such a scheme still uses a quantum computer, but not in the iterative hybrid loop, and not in a way that requires the device to be available during training. It changes the cost-benefit analysis substantially: the quantum resource becomes a one-time measurement campaign rather than a continuously used accelerator. A companion paper by Pablo Bermejo, Paolo Braccia, Manuel Rudolph, Zoë Holmes, Lukasz Cincio and Marco Cerezo, "Quantum convolutional neural networks are (effectively) classically simulable," posted in 2024, applied the reasoning to the quantum convolutional neural network, which had been one of the most promising barren-plateau-free architectures. They showed that on the benchmark tasks used in the literature, typically classifying phases of matter from ground states, the networks' performance rests on low-weight measurements that a classical algorithm equipped with classical shadows of the input states can reproduce. They built such a classical surrogate and matched or exceeded the quantum network's performance. Their conclusion was that the field's benchmarks were too easy: the quantum networks succeeded because the tasks were locally solvable, and harder tasks would be needed to test whether quantum networks offer anything more. The authors of the 2025 paper were explicit about what their argument does not show. It is not a theorem that every trainable quantum model is classically simulable. It leaves open models whose trainability rests on something other than a provable small-space structure, such as a good warm start in a large space, heuristic models with no proven guarantees either way, and models acting on quantum data generated by experiments or deep circuits that a classical computer cannot reproduce. But it shifted the default assumption. A variational model accompanied by a proof that it trains well should now be presumed simulable until shown otherwise. The benchmarking problem The final strand of classical response is empirical. Joseph Bowles, Shahnawaz Ahmed and Maria Schuld at Xanadu published in 2024 a large-scale benchmark study titled "Better than classical? The subtle art of benchmarking quantum machine learning models." They implemented twelve popular quantum machine learning models from the literature, tested them on six binary classification tasks that generated 160 individual data sets, and compared them with standard classical models under careful hyperparameter optimization. Out-of-the-box classical models systematically outperformed the quantum classifiers. Moreover, removing the entangling gates from the quantum models, which makes them trivially simulable, often did not degrade their performance and sometimes improved it, suggesting that whatever the models were doing that worked was not quantum. Their study also documented why so many earlier papers had reported positive results: small data sets, weak or untuned classical baselines, choice of data sets after the fact, and the absence of ablations that remove quantum features. Those failings are common in early-stage machine learning research of any kind, but quantum machine learning has been particularly exposed to them because experiments are expensive, the community is small, and there is strong institutional pressure to report quantum advantages. Taken together, the four responses in this chapter define the standard any new claim must now meet. A quantum machine learning result should be compared with dequantized algorithms if it relies on linear algebra over classical data, with classical surrogates if it uses angle encoding, with tensor-network and Pauli-propagation simulations if it runs on hardware, and with a well-tuned classical baseline under an honest benchmark in all cases. Very few published results meet that standard. The next chapter applies it to the area where quantum computers have been most heavily marketed to industry: optimization. CHAPTER 7 Optimization on Near-Term Hardware When companies describe why they are investing in quantum computing, optimization usually comes first. Logistics firms want better vehicle routing, airlines better crew schedules, banks better portfolios, manufacturers better production plans. These problems are commercially enormous, they are mathematically hard, and they can be written in a form that maps directly onto quantum hardware. Quantum annealers have been sold commercially since D-Wave's first machine in 2011, and the Quantum Approximate Optimization Algorithm has been run on almost every gate-based processor. No area of quantum computing has generated more press releases. It is also the area where the case for near-term advantage is weakest on theoretical grounds, and where classical competition is fiercest. This chapter explains why, examines the most serious experimental evidence, and describes what a convincing demonstration would need to show. Why optimization is harder for quantum computers than it looks Most commercially important optimization problems are NP-hard in their general form: no known algorithm, classical or quantum, solves every instance efficiently. Quantum computers are not believed to change this. The best general-purpose quantum speedup for unstructured search is Grover's algorithm, which is quadratic, turning a search over N possibilities into roughly the square root of N steps, and it is known to be optimal for that problem. Quantum speedups for combinatorial optimization therefore come in two kinds: quadratic or other polynomial speedups that apply broadly, and heuristic improvements on particular classes of instances that may or may not exist. Neither kind is easy to cash in. Ryan Babbush, Jarrod McClean, Michael Newman, Craig Gidney, Sergio Boixo and Hartmut Neven analysed quadratic speedups in a 2021 PRX Quantum paper titled "Focus beyond quadratic speedups for error-corrected quantum advantage." On a fault-tolerant machine, each logical operation takes far longer than a classical operation, because of the overheads of error correction, while classical processors run at billions of operations per second and parallelize cheaply. The authors estimated that, for plausible early fault-tolerant architectures, quadratic speedups for problems such as combinatorial optimization would need runtimes of many hours to days before the quantum algorithm overtook the classical one, and that cheaper improvements in classical hardware or algorithms would often erase the gap. Higher-order polynomial speedups, cubic or quartic, are much more promising, but few are known for optimization. Heuristic improvements are harder to assess because they cannot be proved. They must be demonstrated, on instances that classical methods find hard, against the best classical methods, at sizes large enough that scaling trends are meaningful. Classical optimization is a mature, heavily engineered field. Mixed-integer programming solvers such as Gurobi and CPLEX embody decades of algorithmic advances; heuristics such as simulated annealing, parallel tempering, tabu search and their many variants are cheap and highly parallel; and specialized algorithms exist for specific problem structures. Any quantum optimizer competes against all of them. Annealing and the QUBO formulation The common language of quantum optimization is the quadratic unconstrained binary optimization problem, or QUBO, which asks for a string of binary variables minimizing a quadratic function. QUBO is equivalent to finding the ground state of an Ising model, a system of interacting spins, and Andrew Lucas's 2014 paper "Ising formulations of many NP problems" showed how to express a long list of combinatorial problems, including graph colouring, travelling salesman and scheduling, in this form. Constraints become penalty terms, which often require additional variables and large coefficients, a translation cost that is frequently understated. A small example shows how that cost arises. Suppose a fund wants to choose exactly k assets out of n to maximize expected return minus a risk penalty. The objective, with a binary variable for each asset and a covariance matrix for the risk, is naturally quadratic, so it maps onto QUBO directly. The constraint that exactly k assets be chosen does not. It must be added as a penalty, a large constant multiplied by the square of the difference between the number of chosen assets and k. Expanding that square couples every variable to every other variable. The resulting problem graph is fully connected even if the original risk model was sparse, and the penalty constant must be large enough to enforce the constraint yet not so large that it swamps the objective, a balance that requires tuning. On a quantum processor whose qubits connect only to near neighbours, a fully connected problem must be embedded by chaining several physical qubits into one logical variable on an annealer, or by inserting many swap gates on a gate-based machine, consuming qubits, depth and fidelity. Classical solvers face no such translation cost: they handle the constraint directly. Real problems, with dozens of constraint types, magnify the effect. Quantum annealing, proposed by Tadashi Kadowaki and Hidetoshi Nishimori in 1998, solves an Ising problem by preparing the ground state of a simple Hamiltonian, all spins aligned with a transverse field, and slowly changing the Hamiltonian into the problem Hamiltonian. If the change is slow enough, the adiabatic theorem ensures the system stays in the ground state. In practice, the required slowness depends on the minimum energy gap encountered along the way, which can be exponentially small for hard instances, and real annealers operate at finite temperature with noise, so they function as physical heuristics rather than exact solvers. D-Wave's annealers have grown from 128 qubits in 2011 to more than five thousand in the Advantage line and the Advantage2 system made generally available in 2025. Their record on optimization has been tested repeatedly. A 2014 Science paper by Troels Rønnow, Matthias Troyer and colleagues titled "Defining and detecting quantum speedup" established careful methodology, insisting that speedups be measured in the scaling of time-to-solution with problem size against the best classical algorithms, and found no evidence of quantum speedup on random spin-glass instances. A 2016 study by Vasil Denchev and colleagues at Google reported that a D-Wave machine was about a hundred million times faster than simulated annealing on carefully crafted instances designed to reward tunnelling; but other classical algorithms, including quantum Monte Carlo and specialized heuristics, matched the annealer on those instances, so the comparison showed the weakness of simulated annealing rather than the strength of the annealer. In March 2025 Andrew King and colleagues at D-Wave published in Science a paper titled "Beyond-classical computation in quantum simulation," claiming that their annealer simulated the quench dynamics of certain spin-glass models more accurately than state-of-the-art tensor-network methods could within practical resources. This is a claim about simulating physics, not about optimization, and it was promptly contested: groups including one at the Flatiron Institute and one at EPFL posted preprints arguing that classical methods could reproduce at least some of the reported dynamics. The dispute shows that the arms race described in Chapter 6 operates on annealers too. Whatever its resolution, it does not bear directly on whether annealers solve commercial optimization problems faster than classical solvers, for which no convincing evidence exists. QAOA: theory and experiment The Quantum Approximate Optimization Algorithm, introduced by Edward Farhi, Jeffrey Goldstone and Sam Gutmann in 2014, is a gate-based relative of annealing. It alternates p times between applying the problem Hamiltonian for a duration γ and applying a mixing Hamiltonian, a transverse field, for a duration β, then measures. The 2p angles are optimized in a hybrid loop. As p grows, QAOA can approximate adiabatic evolution arbitrarily well, so in the limit it finds optimal solutions; at small p it is a short, structured circuit that might find good approximate solutions. The early history of QAOA is a miniature of the whole field. Farhi, Goldstone and Gutmann showed in a second 2014 paper that QAOA at depth one achieved a better approximation ratio for a particular constraint satisfaction problem, called Max E3LIN2, than any known classical algorithm. Within months, a group of ten classical computer scientists, Boaz Barak, Ankur Moitra, Ryan O'Donnell, Prasad Raghavendra, Oded Regev, David Steurer, Luca Trevisan, Aravindan Vijayaraghavan, David Witmer and John Wright, found a classical algorithm that did better. The quantum algorithm had prompted the classical improvement, which is a real contribution to science but not a quantum advantage. Further theory has generally constrained QAOA at low depth. For Max-Cut on graphs where each vertex has three neighbours, depth-one QAOA guarantees an approximation ratio of about 0.69, while the classical Goemans-Williamson algorithm of 1995 guarantees about 0.878 for all graphs. Matthew Hastings showed in 2019 that simple local classical algorithms match or beat low-depth QAOA on bounded-degree problems. Edward Farhi, David Gamarnik and Sam Gutmann showed in 2020 that QAOA at depth too low for the algorithm to "see" the whole graph is limited by its locality, and Sergey Bravyi, Alexander Kliesch, Robert König and Eugene Tang showed in 2020 that symmetry protection prevents low-depth QAOA from beating Goemans-Williamson on certain graph families. None of this rules out advantage at larger depths, which are harder to analyse, but the depths at which QAOA could plausibly compete exceed what noisy hardware can run reliably. The most informative hardware experiment remains a 2021 Nature Physics paper by Matthew Harrigan and colleagues at Google, which ran QAOA on the Sycamore processor with up to 23 qubits. On problems whose graph matched the chip's native connectivity, performance held up roughly independently of size. On problems that required routing, such as Max-Cut on three-regular graphs and the Sherrington-Kirkpatrick spin glass, the additional gates needed to move information around the chip degraded performance toward that of random guessing as size increased. Since commercial problems rarely match a chip's connectivity, this is the regime that matters, and it is the regime where noise is most damaging. The strongest recent evidence for QAOA comes from a 2024 Science Advances paper by Ruslan Shaydulin and colleagues at JPMorgan Chase, Argonne National Laboratory and Quantinuum, titled "Evidence of scaling advantage for the quantum approximate optimization algorithm on a classically intractable problem." They studied the low-autocorrelation binary sequences problem, which has applications in radar and communications and is notoriously hard for classical exact solvers. Using noiseless simulations of up to forty qubits, they found that QAOA with fixed, transferable parameters showed better empirical scaling of time-to-solution than the leading branch-and-bound exact solvers, and that QAOA combined with quantum minimum finding, a Grover-style technique, gave the best empirical scaling of any algorithm they examined. They also ran small instances on Quantinuum trapped-ion hardware using a problem-specific error-detection scheme. The study is careful and the comparison serious. Its caveats are also important: scaling extrapolated from forty qubits may not persist; the quantum minimum-finding component requires fault tolerance; and the classical baselines for heuristic rather than exact solving are strong. It is evidence that QAOA may offer a polynomial scaling advantage on a specific hard problem with error-corrected hardware, not that it does so now. Rydberg atoms and maximum independent sets Neutral-atom platforms offer another path. In a 2022 Science paper, Sepehr Ebadi, Alexander Keesling, Mikhail Lukin and colleagues at Harvard, MIT and QuEra used arrays of up to 289 rubidium atoms, excited to Rydberg states, to search for maximum independent sets on unit-disk graphs, a problem that maps naturally onto the physics of the Rydberg blockade, which prevents neighbouring atoms from being simultaneously excited. Using a variational quantum annealing schedule, they found that on the hardest instances in their set, the quantum algorithm's success probability scaled better than that of simulated annealing tuned to the same problem, a superlinear speedup in a specific sense. As with the D-Wave work, the comparison with simulated annealing was the weak point. A 2023 study by Ruben Andrist and colleagues, "Hardness of the maximum-independent-set problem on unit-disk graphs and prospects for quantum speedups," found that state-of-the-art classical solvers handle unit-disk graphs of these sizes with little difficulty, and that such graphs are generally easier than the worst cases of the problem. The experiment was an impressive demonstration of controllable many-body physics, and the hardware has since advanced, but it did not establish that the quantum device was a better optimizer than the best classical tools. The financial pilots The most widely reported recent claim of near-term quantum value in industry came not from optimization proper but from prediction. In September 2025 HSBC and IBM announced what they described as the world's first known quantum-enabled algorithmic trading, with results posted to arXiv. The task was predicting whether a request for quote in the European corporate bond market would be filled at the quoted price. The team took features describing each request, passed them through a projected quantum feature map, a parameterized circuit whose single-qubit reduced states serve as new features, run on IBM Heron processors using 109 qubits, and fed the transformed features to standard classical models such as logistic regression, random forests and neural networks. On historical data covering roughly 1.1 million quote requests for more than 5,000 bonds, the best configurations improved prediction performance by up to 34 percent relative to models using the original classical features. The study's own text contains the most important caveats. The improvement was measured in backtesting on one data set. The authors stated that they could not explain the magnitude of the gain and did not claim a generalizable theory. Most strikingly, features produced by the noisy hardware outperformed features produced by an exact, noiseless simulation of the same circuits, and adding artificial noise to the simulation did not reproduce the effect. Whatever made the hardware features useful, it was not the ideal quantum computation the circuit was designed to perform. A transformation whose value depends on specific uncontrolled imperfections of a particular device is not a quantum algorithm in any sense that can be reproduced, scaled or improved. From the perspective of Chapters 5 and 6, there is a further question the study did not settle: whether a classical random-feature transformation of comparable dimension and nonlinearity, tuned with comparable effort, would have produced comparable gains. Feature engineering that injects random nonlinear projections can help classical models on noisy financial data, and a fair benchmark would compare the quantum features with rich classical random-feature baselines and report the variation across random seeds and time periods. The HSBC and IBM work is best read as an exploratory study that produced an anomalous result worth investigating, not as a demonstration of quantum advantage in finance. What a convincing demonstration would require The experience of the past decade suggests a set of requirements for any claim that a quantum device accelerates optimization. The problem class must be one that matters and whose hard instances are hard for the best classical methods, not just for simulated annealing or a default solver configuration. The comparison must be of time-to-solution, or time-to-target-quality, including all overheads: problem translation into QUBO form, compilation and routing, queueing, readout and repetition. It must be made on instances large enough that scaling is visible, with the classical side given comparable engineering effort, including parallel hardware. The quantum result must be reproducible on new instances and, ideally, by other groups. And the explanation of the advantage must be consistent with what is known about the algorithm, rather than depending on unexplained artifacts. No published demonstration meets all of these requirements for a practical optimization problem. The most rigorous studies, like the work on low-autocorrelation sequences, point to modest polynomial advantages that would materialize, if at all, on error-corrected machines. That is a meaningful finding, and it suggests that the realistic timeline for quantum optimization advantage is tied to the timeline for fault tolerance, which is the subject of the next chapter. For an organization deciding what to do now, this suggests a measured stance. It is reasonable to build expertise, to formulate key problems in QUBO form, and to track progress on problem classes where theory suggests large speedups. It is not reasonable to expect near-term quantum hardware to improve production optimization, and claims that it does should be checked against the requirements above. The most productive outcome of many corporate quantum optimization projects has been better classical optimization, prompted by the careful problem formulation that a quantum project requires, which is a genuine benefit worth naming honestly. CHAPTER 8 Hardware and the Road to Fault Tolerance The variational algorithms in the earlier chapters were designed for a specific technological moment: processors with tens to hundreds of qubits, error rates around one in a hundred to one in a thousand per two-qubit gate, and no error correction. That moment is ending. Between late 2024 and late 2025, several groups demonstrated error correction working as theory predicts, and the major hardware companies published roadmaps aimed at machines with hundreds of logical qubits before the end of the decade. This chapter describes where the hardware stands, what error correction is and costs, and, most importantly for the argument of this book, which quantum machine learning ideas fault tolerance revives and which it leaves exactly where they were. The state of the hardware Five hardware families now lead the field, each with characteristic strengths. Superconducting circuits, used by Google and IBM among others, have fast gates, measured in tens of nanoseconds, and are fabricated with techniques borrowed from the semiconductor industry, but they connect only to near neighbours on a chip and operate at millikelvin temperatures. Trapped ions, used by Quantinuum and IonQ, have the highest gate fidelities and can connect any qubit to any other, but their gates are slower by two or three orders of magnitude. Neutral atoms held in optical tweezers, used by QuEra, Atom Computing, Pasqal and academic groups, can be rearranged during a computation and scaled to large arrays. Photonic platforms, pursued by PsiQuantum and Xanadu, aim at large-scale fault tolerance through mass-manufactured optical chips. Quantum annealers, built by D-Wave, are special-purpose devices discussed in the previous chapter. Table 4 summarizes some of the most significant systems and results of the period; the superconducting systems listed all use transmon qubits. Table 4. Selected quantum hardware systems and results, 2024 to 2025. System Qubit type Physical qubits Notable result Google Willow Transmon 105 Error correction below threshold (2024) IBM Heron r2 Transmon 156 Utility-scale experiments (2024 to 2025) IBM Nighthawk Transmon 120 About 5,000 two-qubit gates (2025) Quantinuum Helios Trapped ions 98 99.92% two-qubit fidelity (2025) Harvard and QuEra Neutral atoms Up to 280 48 logical qubits (2023) D-Wave Advantage2 Annealing Over 4,400 General availability (2025) Source: Acharya et al., Nature (2025); Bluvstein et al., Nature (2024); IBM announcements, November 2025; Quantinuum announcement, November 2025; D-Wave announcement, May 2025. Three developments stand out. In December 2024 Google announced its 105-qubit Willow processor, and the accompanying Nature paper by Rajeev Acharya and many colleagues reported a surface-code quantum memory operating below the error-correction threshold: as the code distance increased from three to five to seven, using more physical qubits per logical qubit, the logical error rate fell by a factor of roughly two at each step. That was the first clear experimental demonstration of the exponential suppression of errors on which all of fault-tolerant quantum computing depends. In October 2025 Google followed with an experiment it called Quantum Echoes, measuring out-of-time-order correlators on Willow and claiming a computation roughly 13,000 times faster than the best classical simulation on a leading supercomputer, together with a proof-of-principle application to molecular structure. Like every advantage claim, it invites classical challenge, but it targets a physically meaningful, verifiable quantity rather than random sampling. In November 2025 Quantinuum launched Helios, a 98-qubit trapped-ion system using barium ions, with two-qubit gate fidelity of about 99.92 percent across all pairs and single-qubit fidelity above 99.997 percent. The company reported 48 error-corrected logical qubits operating at a two-to-one encoding rate, and larger numbers of error-detected logical qubits. The same month IBM introduced Nighthawk, a 120-qubit superconducting processor with a square lattice and higher connectivity than its earlier heavy-hexagon designs, together with an experimental processor called Loon for testing the components of its planned fault-tolerant architecture, and reiterated a target of a fault-tolerant machine named Starling, with around 200 logical qubits capable of running 100 million gates, by 2029. IBM also stated that it expects quantum advantage, verified by the wider community, by the end of 2026, and with partners it established a public tracker of candidate advantage experiments so that classical simulation groups can test them openly. That institutional acknowledgement that advantage claims must survive classical challenge is itself a product of the episodes described in Chapter 6. Neutral atoms deserve particular mention because their progress in qubit count has been the fastest. In late 2023 Dolev Bluvstein, Mikhail Lukin and colleagues at Harvard, MIT and QuEra ran circuits on up to 48 logical qubits encoded in 280 physical atoms, moving atoms between zones of the array during the computation to entangle logical qubits in parallel. In 2025 a group at Caltech reported holding more than six thousand atoms in a single optical tweezer array with long coherence times, a demonstration of scale rather than of computation, but a sign that the qubit counts needed for error-corrected machines are not out of reach for this platform. The trade-off is speed: atom-based gates and, especially, measurements and rearrangements are slow, so the logical clock rate of a neutral-atom machine will be lower than that of a superconducting one with the same number of logical qubits. For learning algorithms whose cost is dominated by repetitions, that difference matters as much as qubit count. What error correction costs Quantum error correction encodes one logical qubit in many physical qubits, so that errors on individual physical qubits can be detected and corrected without disturbing the encoded information. The surface code, the most widely studied scheme, arranges physical qubits on a two-dimensional grid and repeatedly measures parity checks. Below a threshold error rate of roughly one percent, increasing the size of the grid suppresses logical errors exponentially. The cost is overhead. For the logical error rates needed for long algorithms, of order one in a billion or lower, a surface-code logical qubit at realistic physical error rates requires hundreds to over a thousand physical qubits. Logical operations are also slower than physical ones. A surface-code cycle on a superconducting processor takes about a microsecond, and a logical operation takes several cycles. Some gates, notably the non-Clifford T gate needed for universality, cannot be implemented directly in the surface code and require magic-state distillation, consuming additional qubits and time. Trapped-ion and neutral-atom systems have different trade-offs but similar or larger time overheads. A fault-tolerant quantum computer in the early 2030s will therefore perform perhaps millions to billions of logical operations per second at best, compared with the trillions of operations per second available from classical accelerators. Newer codes reduce overhead. IBM's architecture is built around quantum low-density parity-check codes; a 2024 Nature paper by Sergey Bravyi and colleagues described a code that encodes twelve logical qubits in 144 data qubits with error suppression comparable to surface codes that would need many more. Such codes demand longer-range connections between qubits, which is why IBM's recent hardware focuses on couplers and connectivity. Resource estimates have fallen accordingly. In 2017 Reiher and colleagues estimated that computing FeMoco's ground-state energy reliably would require many millions of physical qubits and long runtimes; in 2021 Joonho Lee, Dominic Berry, Craig Gidney, William Huggins, Jarrod McClean, Nathan Wiebe and Ryan Babbush used tensor hypercontraction of the Hamiltonian to bring the estimate down to roughly four million physical qubits running for a few days; subsequent work has continued to cut such numbers. In a different domain, Craig Gidney's 2025 estimate that a 2048-bit RSA key could be factored with fewer than a million noisy physical qubits in under a week illustrates how quickly algorithmic and architectural improvements compound. Reading roadmaps and runtimes Hardware roadmaps are useful but must be read with care. They state targets, not commitments, and the history of the field contains both delivered milestones and quietly missed ones. Three habits help. The first is to look for logical rather than physical qubit counts, and for logical error rates and gate budgets alongside them. A processor with a thousand physical qubits and no error correction can run circuits of only a few thousand gates before noise dominates; a machine with a hundred logical qubits and a budget of a hundred million logical gates is a different kind of instrument. IBM's stated target for Starling, around two hundred logical qubits and a hundred million gates, is useful precisely because it is expressed in those terms. The second is to translate gate budgets into runtimes. Suppose a fault-tolerant machine completes one logical operation every ten microseconds, a plausible figure for early superconducting architectures once the overheads of error correction and magic-state supply are included. A hundred million operations then take about seventeen minutes. A billion take nearly three hours. A machine-learning algorithm that needs a billion logical operations per training step, and thousands of steps, would run for months. Trapped-ion and neutral-atom machines, with slower physical gates, face longer times for the same logical operation count unless their connectivity allows offsetting savings. Any proposed fault-tolerant learning algorithm should come with an estimate of its logical gate count, and the reader should do this conversion. The third is to ask what the machine will be compared against when it arrives. Classical hardware and algorithms will have improved too. The relevant comparison for a 2030 fault-tolerant learning algorithm is with 2030 classical methods on 2030 accelerators, not with today's. With those habits, the roadmaps support a modest but definite expectation: machines with on the order of a hundred logical qubits, capable of tens to hundreds of millions of logical operations, are a plausible target for the end of this decade from more than one company. That is enough for scientifically meaningful quantum simulations and for testing the first fault-tolerant learning proposals at small scale. It is not enough for the large chemistry calculations whose resource estimates still run to millions of physical qubits. What fault tolerance revives Error correction changes the calculus of quantum machine learning in specific ways. First, it makes deep circuits possible. The noise-induced barren plateaus of Chapter 4, and the depth limits derived by Stilck França and García-Patrón, are consequences of physical noise. On a fault-tolerant machine, logical noise can be made small enough that circuits of millions of gates run reliably. Algorithms that were never realistic on noisy hardware, quantum phase estimation for chemistry, quantum linear-systems solvers, Grover-type search and amplitude estimation, become available. In chemistry, this means VQE is likely to be replaced by phase-estimation methods that compute energies to high precision at a cost that grows as the inverse of the target precision rather than as its inverse square, as shot noise forces on VQE, although, as the Lee and colleagues analysis discussed in Chapter 3 emphasized, phase estimation still requires a good initial state. Second, it makes the measurement bottleneck tractable. Amplitude estimation, a fault-tolerant technique, estimates expectation values with error decreasing as the inverse of the number of circuit repetitions rather than the inverse square root, a quadratic improvement known as Heisenberg scaling. For VQE-like tasks, this transforms the shot budget of Chapter 3. Third, it enables the kind of theoretically grounded algorithms that offer provable advantages. The discrete-logarithm kernel of Liu, Arunachalam and Temme requires fault tolerance. So does quantum oracle sketching, the 2026 proposal by Zhao, Huang, Preskill and colleagues described in Chapter 1, whose claimed memory advantage for classifying massive classical data sets is achieved with fewer than sixty logical qubits. A follow-up paper later in 2026 examined practical considerations for that approach. Sixty logical qubits is well within the stated targets of the current roadmaps for the end of the decade, which makes oracle sketching one of the few quantum machine learning proposals with a theoretical advantage on classical data and a plausible hardware target. Whether its advantage survives the full costs of data streaming, error correction and readout in practice is unknown. Fourth, fault tolerance allows quantum computers to generate quantum data that classical computers cannot reproduce: the output states of long quantum simulations, which can then be analysed with learning algorithms. This setting, discussed in the next chapter, is where theory suggests the clearest learning advantages. Between the noisy and the fully fault-tolerant regimes lies an intermediate period, sometimes called early fault tolerance, that the hardware is now entering. Machines in this period can detect or correct some errors on a modest number of logical qubits, but cannot yet run arbitrarily long computations. Algorithms designed for it use shallower circuits than full phase estimation, accept a constant probability of failure, and combine partial error correction with error mitigation. For machine learning, the significance of this regime is that it softens the noise-induced limits on depth without lifting the structural ones. A variational classifier run on error-detected logical qubits will suffer less from noise and exactly as much from expressiveness-induced plateaus, kernel concentration and classical surrogates. The algorithms that benefit most are those whose only obstacle was noise, which brings us back to the division this chapter has been drawing. What fault tolerance does not change Error correction removes physical noise. It does not change the mathematics of the models themselves. Barren plateaus arising from expressiveness, global cost functions, entanglement or data encoding are properties of the ideal, noiseless circuit. A fault-tolerant variational model built on a hardware-efficient ansatz suffers them just as much as a noisy one, and the Lie-algebraic theory applies unchanged. The trade-off between trainability and simulability described in Chapters 4 and 6 is a property of the model class, not of the hardware. Dequantization is unaffected. The classical algorithms of Tang and her collaborators compete with fault-tolerant quantum linear algebra on low-rank data, and error correction cannot help a quantum algorithm whose advantage was an artifact of the input model. Data loading remains expensive. Fault tolerance makes a QRAM conceivable in principle but no cheaper: it still requires a number of physical components proportional to the data size, all operating fault-tolerantly. The overheads of error correction make large QRAMs even less attractive. For classical data at scale, fault-tolerant quantum machine learning must find ways around loading the full data set coherently, which is exactly what oracle sketching and related streaming approaches attempt. Quadratic speedups remain weak. Babbush and colleagues' analysis of the overhead of error correction applies directly, and any quantum machine learning algorithm whose advantage over classical methods is only quadratic will struggle to deliver a practical speedup on early fault-tolerant machines. And the speed gap remains. A fault-tolerant processor executing a million logical operations per second faces classical hardware executing well over a quadrillion floating-point operations per second at scale. Quantum algorithms must therefore offer large asymptotic advantages, exponential or high-degree polynomial, and the problems must be large enough that those asymptotic advantages dominate the constant factors. The upshot is that fault tolerance divides the quantum machine learning literature in two. On one side are proposals whose difficulties were mainly about noise, depth and precision: phase estimation for chemistry, amplitude estimation, algorithms with provable advantages that need deep circuits. Error correction makes these feasible, though not necessarily practical. On the other side are proposals whose difficulties are structural: generic variational classifiers, kernels with generic feature maps, linear algebra on low-rank classical data. Error correction does nothing for them. For a practitioner deciding what to learn and build, the implication is that the skills developed for noisy variational algorithms transfer only partly to the fault-tolerant era. Familiarity with circuits, Hamiltonians, measurement strategies and error mitigation remains valuable. A portfolio of near-term variational classifiers does not. The final chapter turns to the settings in which, on present theory and evidence, quantum learning genuinely wins, and to the practical question of how to evaluate claims as the hardware continues to improve. CHAPTER 9 Where Quantum Learning Wins After eight chapters of constraints, a reader could be forgiven for concluding that quantum machine learning has no future. That conclusion would be wrong, and the reason it is wrong is instructive. The negative results of the past decade apply most forcefully to one particular programme: taking classical data, encoding it into a near-term quantum circuit and hoping the circuit learns something a classical model cannot. When the problem changes, when the data are quantum, when the learning task has hidden structure of the kind quantum algorithms exploit, or when fault-tolerant resources are available, the picture changes too. This chapter describes the settings where the theory and evidence for quantum learning advantages are strongest, and it ends with a practical protocol for evaluating any claim a reader is likely to meet. Learning from quantum data Most of the world's data are classical: images, text, records, prices. But a growing share of scientific data originates in quantum systems: the states of molecules and materials in experiments, the outputs of quantum sensors, the states produced by quantum simulators and, increasingly, the outputs of quantum computers themselves. For such data, the usual pipeline measures each quantum system, records classical outcomes and analyses them with classical methods. Measurement collapses the state, so information not captured by the chosen measurements is lost. A quantum computer that receives the quantum states directly, without first measuring them, can process them coherently, entangling multiple copies, applying circuits that depend on earlier observations and measuring in clever joint bases. Theory shows that this can make an exponential difference. The foundation was laid by Hsin-Yuan Huang, Richard Kueng and John Preskill. Their 2020 Nature Physics paper introduced classical shadows, a technique that measures a quantum state in randomly chosen bases and constructs a compact classical description from which many properties can be predicted. For local observables, a number of measurements growing only logarithmically with the number of properties suffices. Classical shadows have become a workhorse of experimental quantum science, and, as Chapter 6 showed, they are also the tool that makes many quantum learning models classically simulable. In 2021 the same authors established information-theoretic bounds on quantum advantage in learning. For predicting average properties of a quantum process, a classical learner that receives measurement data needs at most polynomially more experiments than a quantum learner with full quantum access. But for predicting properties accurately on all inputs, or for certain tasks such as estimating many non-commuting observables, the quantum learner can need exponentially fewer experiments. Sitan Chen, Jordan Cotler, Huang and Jerry Li developed this into a theory of exponential separations between learning with and without quantum memory, and Dorit Aharonov, Jordan Cotler and Xiao-Liang Qi reached related conclusions in their 2022 work on quantum algorithmic measurement. The experimental test came in a 2022 Science paper, "Quantum advantage in learning from experiments," by Huang, Michael Broughton, Jordan Cotler, Sitan Chen, Jerry Li, Masoud Mohseni, Hartmut Neven, Ryan Babbush, Richard Kueng, John Preskill and Jarrod McClean. Working on Google's Sycamore processor, they compared two strategies for learning properties of unknown quantum states and processes: a conventional strategy that measures each copy separately and processes the results classically, and a quantum-enhanced strategy that stores two copies in quantum memory, entangles them and measures jointly. For tasks such as predicting certain properties of physical states and distinguishing between classes of quantum processes, the quantum-enhanced strategy needed exponentially fewer experiments, and the hardware results showed a large advantage even with noise, using dozens of qubits. This is a different kind of advantage from the computational speedups of earlier chapters. It concerns sample complexity: how many experiments, or copies of a state, a learner needs. It cannot be dequantized in the usual sense, because the classical learner's handicap is not a lack of computation but a lack of quantum access to the data. It is provable, rather than heuristic. And it has been demonstrated on hardware. Its limitation is scope. It applies when the object of study is itself a quantum system that can be coupled to a quantum processor, which requires either transducing physical quantum signals into a processor's qubits or generating the states on the processor in the first place. For applications in quantum sensing and in characterizing quantum devices, that coupling is a natural goal; for analysing a bank's transaction records, it is meaningless. A smaller experiment illustrates both the promise and the trap. In 2022 Johannes Herrmann, Andreas Wallraff and colleagues at ETH Zurich ran a quantum convolutional neural network on a seven-qubit superconducting processor to recognize quantum phases of matter: the network received ground states of a spin chain, prepared directly on the device, and classified which phase each belonged to, without the states ever being converted into classical data. That is exactly the pipeline the theory favours, quantum states in, a short classical answer out. Yet, as Chapter 6 described, Bermejo and colleagues later showed that on benchmark phase-recognition tasks of this kind, the network's success rests on low-weight measurements that a classical learner equipped with classical shadows can reproduce. Quantum input is necessary for an advantage in learning from quantum data, but not sufficient: the task must also require information that local, independent measurements cannot capture. It would be a mistake to overstate even this setting. Huang, Kueng, Giacomo Torlai, Victor Albert and Preskill showed in a 2022 Science paper that classical machine learning trained on classical-shadow data can efficiently predict ground-state properties of gapped many-body Hamiltonians across a phase, and classify quantum phases of matter, with provable guarantees. Classical learners equipped with good measurement data are powerful even on quantum problems. The quantum advantage in learning from experiments is real but specific: it applies to tasks where coherent processing of multiple copies, or adaptive quantum measurements, extract information that independent measurements cannot. Embedded structure and massive data The second credible setting is learning problems whose structure matches known quantum algorithms. The discrete-logarithm kernel of Liu, Arunachalam and Temme, discussed in Chapter 5, is the cleanest example: a learning task whose labels hide a group structure that Shor's algorithm reveals. Other constructions embed hidden subgroup problems, period finding or simulations of quantum dynamics into learning tasks. In each case the advantage is provable under standard complexity assumptions, and in each case it requires fault-tolerant hardware and data with the right structure. Whether natural data contain such structure is an open empirical question. The general view among researchers is that most commercially important data sets do not, which is why classical deep learning works so well on them: their structure is hierarchical, local and compositional in ways neural networks exploit efficiently. But there may be scientific data, for instance from physical systems governed by quantum dynamics, whose patterns are generated by processes that quantum algorithms handle naturally. Finding them is a research task for domain scientists and quantum algorithm designers working together. The third setting is the most recent: memory-constrained processing of massive classical data. The 2026 quantum oracle sketching proposal by Zhao, Huang, Preskill and colleagues argues that a small fault-tolerant quantum computer, consuming random samples from a data stream, can perform classification and dimension reduction with exponentially less memory than any classical machine achieving the same performance, a separation they prove rests on quantum mechanics alone rather than on unproven complexity conjectures. Their numerical experiments on real data, including single-cell RNA sequencing and sentiment analysis, suggested memory reductions of four to six orders of magnitude with fewer than sixty logical qubits. This proposal deserves careful attention precisely because it targets classical data, the regime where other approaches have failed, and does so with a modest logical-qubit count. It deserves careful scrutiny for the same reason: memory is rarely the binding constraint in commercial data analysis, where storage is cheap, and the practical value will depend on whether there are streaming or edge settings where memory is genuinely scarce and the time and error-correction overheads are acceptable. Table 5 draws these settings together with the near-term approaches discussed earlier. Table 5. Settings for quantum learning advantage and the strength of the evidence. Setting Representative work Kind of advantage Evidence Hardware needed Learning from quantum experiments Huang et al., 2022 Exponentially fewer experiments Theorem and hardware demonstration Near-term to early fault-tolerant Embedded algebraic structure Liu, Arunachalam, Temme, 2021 Exponential, under hardness assumption Theorem; artificial data Fault-tolerant Streaming massive classical data Zhao et al., 2026 Exponential memory saving Theorem and simulation Tens of logical qubits Ground-state chemistry Lee et al., 2023 analysis Probably polynomial Debated Fault-tolerant Variational models on classical data Bowles, Ahmed, Schuld, 2024 benchmark None demonstrated Negative benchmarks Near-term The contrast between the first and last rows summarizes the book. The settings with the strongest evidence are those in which the quantum computer has privileged access to something, quantum states or hidden quantum structure, that a classical learner lacks. The settings with the weakest evidence are those in which the quantum computer and the classical learner see the same classical data and the quantum computer is expected to process it better by virtue of a larger state space. A note on generalization One area of quantum learning theory has produced results that look like good news, and they deserve a careful reading. In 2022 Matthias Caro, Hsin-Yuan Huang, Marco Cerezo, Kunal Sharma, Andrew Sornborger, Lukasz Cincio and Patrick Coles proved in Nature Communications that the generalization error of a parameterized quantum model, the gap between its performance on training data and on new data, grows roughly as the square root of the number of trainable gates divided by the number of training examples. A model with a modest number of gates therefore generalizes from a modest number of examples, which was taken as evidence that quantum models are data-efficient. The bound is real but limited in what it implies. It says that a small quantum model will not overfit badly; it says nothing about whether the model will fit the data well in the first place, nor whether a classical model of comparable size would do better. Classical models with few parameters also generalize well by the same kind of argument. In 2024 Elies Gil-Fuster, Jens Eisert and Carlos Bravo-Prieto showed in Nature Communications, in a paper titled "Understanding quantum machine learning also requires rethinking generalization," that quantum models of the kind studied can fit randomly labelled data, which means that uniform generalization bounds of this type cannot by themselves explain why such models generalize when they do. The paper deliberately echoed a famous 2017 result about classical deep networks, and its lesson is the same: generalization in practice depends on the interaction between model, data and training procedure, not on parameter counts alone. Data efficiency, like speed, has to be demonstrated against classical baselines rather than inferred from a bound. The reverse direction A brief detour is warranted, because the most productive marriage of machine learning and quantum computing so far has run the other way: classical machine learning improving quantum computers. Neural networks now help calibrate and control quantum hardware, discover pulse sequences and design experiments. In 2024 Johannes Bausch and colleagues at Google DeepMind and Google Quantum AI published in Nature AlphaQubit, a transformer-based neural decoder for the surface code that, trained on simulated and real data from the Sycamore processor, outperformed the leading conventional decoders in accuracy on real experimental data. Decoding speed remains a challenge for neural approaches at the cycle rates of superconducting hardware, but the result shows machine learning contributing directly to the fault-tolerance programme. Since Giuseppe Carleo and Matthias Troyer's 2017 Science paper introducing neural-network quantum states, classical neural networks have also become a powerful method for representing and simulating quantum many-body systems, and therefore one of the classical competitors that quantum algorithms must beat. The relationship between the two fields is thus asymmetric at present. Classical machine learning is helping build quantum computers and is setting the classical baselines they must exceed. Quantum computing's contributions to machine learning remain mostly theoretical, with the important exception of learning from quantum experiments. A protocol for evaluating claims Readers will continue to encounter claims of quantum machine learning success in papers, press releases and sales presentations. The following questions, drawn from the arguments of this book, are the ones to ask. 1. What is the data, and how does it get in? If the data are classical, how are they encoded, and what does loading cost? If the claim relies on amplitude encoding or QRAM, has the cost of loading been counted? If the claim uses angle encoding, is there a reason to think a classical surrogate cannot match the model? 1. What is the classical baseline? Was the comparison with a well-tuned, state-of-the-art classical method, or with a default configuration, a simple heuristic, or simulated annealing? Were the classical models given the same hyperparameter search and feature engineering effort? Did the authors test classical surrogates, random Fourier features or tensor-network simulation of the quantum model? 1. Was there an ablation? Does removing entanglement, or replacing the quantum circuit with a classical random feature map of similar dimension, eliminate the improvement? If not, the improvement is not evidence of quantum computation. 1. How does it scale? Is the evidence a single-size comparison, or a scaling trend? Over what range of sizes? Are the problem instances hard for classical methods at those sizes? 1. What did it cost? Has the comparison counted wall-clock time, including shots, queueing, compilation and error mitigation, or only idealized circuit evaluations? How many shots per estimate, and how many estimates per training step? 1. Why should it work? Is there a theoretical reason, such as provable separation, embedded hardness or quantum data access, to expect an advantage? Does the explanation hold up, or does the advantage depend on unexplained hardware artifacts? 1. Is it trainable and not simulable? If the model avoids barren plateaus, is the reason one that also makes it classically simulable, such as a small Lie algebra, local observables or near-identity circuits? 1. Has it been reproduced? Has the result been confirmed on new data, new instances, other hardware or by other groups? Has it survived a classical simulation challenge? A claim that answers these questions satisfactorily is a significant result. Very few current claims do. That is not a reason for cynicism, but it is a reason for patience and precision. What a practitioner should do now For data scientists and engineers, the practical guidance follows directly. Learning the fundamentals of quantum computation is worthwhile for those whose work touches chemistry, materials, sensing, cryptography or the development of quantum technology itself, and the variational toolkit is a reasonable way to learn them. Investing in near-term quantum machine learning for production use on classical business data is not justified by current evidence. Organizations that want strategic exposure should focus on understanding which of their problems have the structure that fault-tolerant algorithms exploit, building partnerships with groups doing rigorous resource estimation, and maintaining a realistic view of timelines. For researchers, the open problems are sharp and important. Identifying trainable quantum models that are provably not classically simulable, or at least that resist every known simulation method at relevant scales, is a central challenge. Finding natural data sets with structure that quantum algorithms exploit would change the field. Developing protocols for coupling quantum sensors and simulators directly to quantum processors would turn the theoretical advantages of learning from quantum data into practical tools. And producing honest end-to-end resource estimates for fault-tolerant learning algorithms, including data streaming, error correction and readout, would tell us which proposals can matter and when. These are not the questions the field was asking in 2018. That they are being asked now, with clarity and in public, is the main achievement of the past several years of quantum machine learning research. Conclusion The question this book set out to answer was whether hybrid quantum-classical algorithms can accelerate machine learning and optimization on the quantum hardware that exists or will soon exist. The chapters have assembled the evidence piece by piece. What follows from putting the pieces together is something none of them says alone. The tension at the centre Read separately, the results of the last decade look like a list of unrelated obstacles: barren plateaus in one chapter, kernel concentration in another, dequantization in a third, classical simulation of hardware experiments in a fourth, weak benchmarks in a fifth. Read together, they describe a single structural tension. A quantum model can be powerful only if its computation ranges over a space too large for classical methods to track. A quantum model can be trained, loaded with data and read out efficiently only if its computation is confined to a space small enough that gradients survive, kernels do not concentrate and a modest number of measurements suffice. For the generic variational models and quantum kernels applied to classical data, these two requirements pick out regions that do not overlap. The remedies for each obstacle, local cost functions, small Lie algebras, narrow bandwidths, symmetry constraints, near-identity initialization, all work by moving the model into the small region, and the classical responses then find it there. This is why the bet described in the introduction has so rarely paid off. The hybrid loop places the hardness in the quantum state and relies on the classical optimizer to steer it. When the state is hard enough to matter, the optimizer cannot steer; when the optimizer can steer, the state is usually not hard. The tension is not a temporary engineering problem that better hardware will solve, because it is a property of the models rather than of the noise. Error correction removes noise-induced flatness and the depth limits that noise imposes, and it does nothing to the rest. That conclusion has a corollary that is easy to miss. The settings where quantum learning advantages are best supported, learning from quantum experiments, structured problems with embedded quantum algorithms, and streaming data with oracle sketching, all escape the tension in the same way. They do not rely on an optimizer to discover a hard quantum state by search. Either the hard state is supplied by nature, as in quantum experiments, or the algorithm that produces it is designed from the structure of the problem, as in Shor-based kernels and sketching, rather than trained. The advantage lives in access or in design, not in optimization. That is the deepest lesson of the period: quantum advantage in learning has so far come from knowing what quantum state to prepare, not from learning it. What remains unsettled Several questions are open, and their answers will determine how the field looks at the end of this decade. The first is whether there are trainable quantum models that are not classically simulable. The 2025 argument linking provable absence of barren plateaus to simulability left room for models trained from good warm starts in large spaces, and for heuristic models with no proven guarantees. If such models exist and can be identified, the near-term variational programme gains a reason to continue. If careful searches keep finding classical surrogates, the programme will narrow to a tool for physics experiments. The second is how large the quantum advantage in chemistry and materials will be once fault-tolerant machines arrive. The 2023 analysis by Lee, Chan, Preskill and colleagues found no evidence for generic exponential advantage in ground-state chemistry; the remaining question is whether the polynomial advantages on strongly correlated systems are large enough, and the target systems valuable enough, to justify the cost of a fault-tolerant machine. Resource estimates are falling fast enough that this question may have a concrete answer within a few years. The third is whether any of the industrial anomalies hold up. The HSBC and IBM bond-trading result, in which noisy hardware features outperformed noiseless simulations, is either a statistical artifact, a disguised classical effect such as useful random nonlinearity, or a phenomenon nobody understands. Only replication with strong classical baselines and ablations will tell, and the field would benefit from treating such results as puzzles rather than as milestones. The fourth is whether IBM's stated target of a community-verified quantum advantage by the end of 2026, and Google's claims for verifiable physical simulations, will survive the classical challenge that has followed every previous claim. Those are not machine learning results, but if they hold, they will mark the point at which quantum processors begin to generate data that classical computers cannot reproduce, which is the precondition for the quantum-data learning advantages described in Chapter 9. The fifth is whether quantum oracle sketching, or something like it, turns into a practical advantage on classical data. It is the first proposal to claim a provable, unconditional advantage for processing large classical data sets with a modest number of logical qubits, and it will be among the first serious tests of what early fault-tolerant machines can do for data science. What the reader should do differently For anyone evaluating quantum machine learning, the main change is to ask a different first question. The usual first question is whether a quantum model can do as well as a classical one on a given task. The better first question is where the hardness lives. Is the quantum state classically hard to describe, and if so, how will the optimizer find good parameters? Is the data quantum, or does it carry structure a quantum algorithm is known to exploit? If the answer to these questions is no, a positive benchmark result is more likely to reflect weak baselines, small sizes or a classical effect in quantum clothing than a quantum advantage, and the protocol in Chapter 9 will usually show which. For organizations, the implication is to separate two investments that are often bundled. Building quantum literacy, formulating key problems precisely and tracking fault-tolerant resource estimates for problems that matter is inexpensive and prudent. Deploying near-term quantum machine learning on business data is not supported by the evidence, and resources spent on it are better spent on classical methods, which have repeatedly been the main beneficiaries of quantum projects through the scrutiny those projects bring to problem formulation. For researchers and students, the implication is to design from the hardness backward. Start with a source of classical intractability, a quantum state that nature or a deep circuit produces, an algebraic structure a known quantum algorithm exploits, a resource such as memory or sample count for which quantum separations are provable, and build the learning method around it. Then ask whether it can be trained, loaded and read out. The field spent its first decade doing the opposite, building trainable models and hoping they would be hard, and the results of that strategy are now well documented. None of this diminishes the scientific achievement of the period. The barren plateau theory, the dequantization results, the classical shadow formalism, the provable separations for learning from experiments and the careful benchmarks are lasting contributions to physics, computer science and machine learning. They have turned a field built on hope into one built on theorems, experiments and honest comparisons. That is a better foundation for whatever quantum learning eventually becomes, and anyone who wants to take part in building it now knows where to begin. Notes Introduction 1. F. Arute et al., "Quantum supremacy using a programmable superconducting processor," Nature 574, 505–510 (2019). 2. E. Pednault, J. Gunnels, G. Nannicini, L. Horesh and R. Wisnieff, "Leveraging secondary storage to simulate deep 54-qubit Sycamore circuits," arXiv:1910.09534 (2019). 3. F. Pan, K. Chen and P. Zhang, "Solving the sampling problem of the Sycamore quantum circuits," Physical Review Letters 129, 090502 (2022). Chapter 1 1. A. S. Holevo, "Bounds for the quantity of information transmitted by a quantum communication channel," Problems of Information Transmission 9, 177–183 (1973). 2. P. Rebentrost, M. Mohseni and S. Lloyd, "Quantum support vector machine for big data classification," Physical Review Letters 113, 130503 (2014). 3. S. Lloyd, M. Mohseni and P. Rebentrost, "Quantum principal component analysis," Nature Physics 10, 631–633 (2014). 4. S. Aaronson, "Read the fine print," Nature Physics 11, 291–293 (2015). 5. H. Zhao, A. Zlokapa, H. Neven, R. Babbush, J. Preskill, J. R. McClean and H.-Y. Huang, "Exponential quantum advantage in processing massive classical data," arXiv:2604.07639 (2026). 6. J. Preskill, "Quantum computing in the NISQ era and beyond," Quantum 2, 79 (2018). 7. Quantinuum, "Introducing Helios: the most accurate quantum computer in the world," company announcement, November 2025. 8. M. Schuld, R. Sweke and J. J. Meyer, "Effect of data encoding on the expressive power of variational quantum-machine-learning models," Physical Review A 103, 032430 (2021). Chapter 2 1. J. R. McClean, J. Romero, R. Babbush and A. Aspuru-Guzik, "The theory of variational hybrid quantum-classical algorithms," New Journal of Physics 18, 023023 (2016). 2. M. Cerezo et al., "Variational quantum algorithms," Nature Reviews Physics 3, 625–644 (2021). 3. A. Kandala et al., "Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets," Nature 549, 242–246 (2017). 4. A. Pérez-Salinas, A. Cervera-Lierta, E. Gil-Fuster and J. I. Latorre, "Data re-uploading for a universal quantum classifier," Quantum 4, 226 (2020). 5. M. Cerezo, A. Sone, T. Volkoff, L. Cincio and P. J. Coles, "Cost function dependent barren plateaus in shallow parametrized quantum circuits," Nature Communications 12, 1791 (2021). 6. K. Mitarai, M. Negoro, M. Kitagawa and K. Fujii, "Quantum circuit learning," Physical Review A 98, 032309 (2018). 7. M. Schuld, V. Bergholm, C. Gogolin, J. Izaac and N. Killoran, "Evaluating analytic gradients on quantum hardware," Physical Review A 99, 032331 (2019). 8. D. Wierichs, J. Izaac, C. Wang and C. Y.-Y. Lin, "General parameter-shift rules for quantum gradients," Quantum 6, 677 (2022). 9. A. Abbas et al., "On quantum backpropagation, information reuse, and cheating measurement collapse," Advances in Neural Information Processing Systems 36 (2023). 10. J. Stokes, J. Izaac, N. Killoran and G. Carleo, "Quantum natural gradient," Quantum 4, 269 (2020). 11. R. Sweke et al., "Stochastic gradient descent for hybrid quantum-classical optimization," Quantum 4, 314 (2020). Chapter 3 1. R. P. Feynman, "Simulating physics with computers," International Journal of Theoretical Physics 21, 467–488 (1982). 2. S. R. White, "Density matrix formulation for quantum renormalization groups," Physical Review Letters 69, 2863 (1992). 3. M. Reiher, N. Wiebe, K. M. Svore, D. Wecker and M. Troyer, "Elucidating reaction mechanisms on quantum computers," Proceedings of the National Academy of Sciences 114, 7555–7560 (2017). 4. A. Peruzzo et al., "A variational eigenvalue solver on a photonic quantum processor," Nature Communications 5, 4213 (2014). 5. P. J. J. O'Malley et al., "Scalable quantum simulation of molecular energies," Physical Review X 6, 031007 (2016). 6. C. Hempel et al., "Quantum chemistry calculations on a trapped-ion quantum simulator," Physical Review X 8, 031022 (2018). 7. Google AI Quantum and collaborators, "Hartree-Fock on a superconducting qubit quantum computer," Science 369, 1084–1089 (2020). 8. H. R. Grimsley, S. E. Economou, E. Barnes and N. J. Mayhall, "An adaptive variational algorithm for exact molecular simulations on a quantum computer," Nature Communications 10, 3007 (2019). 9. D. Wecker, M. B. Hastings and M. Troyer, "Progress towards practical quantum variational algorithms," Physical Review A 92, 042303 (2015). 10. J. F. Gonthier et al., "Measurements as a roadblock to near-term practical quantum advantage in chemistry: resource analysis," Physical Review Research 4, 033154 (2022). 11. K. Temme, S. Bravyi and J. M. Gambetta, "Error mitigation for short-depth quantum circuits," Physical Review Letters 119, 180509 (2017). 12. A. Kandala et al., "Error mitigation extends the computational reach of a noisy quantum processor," Nature 567, 491–495 (2019). 13. Y. Quek, D. Stilck França, S. Khatri, J. J. Meyer and J. Eisert, "Exponentially tighter bounds on limitations of quantum error mitigation," Nature Physics 20 (2024). 14. S. Lee et al., "Evaluating the evidence for exponential quantum advantage in ground-state quantum chemistry," Nature Communications 14, 1952 (2023). 15. J. Robledo-Moreno et al., "Chemistry beyond the scale of exact diagonalization on a quantum-centric supercomputer," arXiv:2405.05068 (2024); published in Science Advances (2025). Chapter 4 1. J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush and H. Neven, "Barren plateaus in quantum neural network training landscapes," Nature Communications 9, 4812 (2018). 2. A. Arrasmith, M. Cerezo, P. Czarnik, L. Cincio and P. J. Coles, "Effect of barren plateaus on gradient-free optimization," Quantum 5, 558 (2021). 3. Z. Holmes, K. Sharma, M. Cerezo and P. J. Coles, "Connecting ansatz expressibility to gradient magnitudes and barren plateaus," PRX Quantum 3, 010313 (2022). 4. C. Ortiz Marrero, M. Kieferová and N. Wiebe, "Entanglement-induced barren plateaus," PRX Quantum 2 (2021). 5. S. Wang et al., "Noise-induced barren plateaus in variational quantum algorithms," Nature Communications 12, 6961 (2021). 6. S. Thanasilp, S. Wang, N. A. Nghiem, P. J. Coles and M. Cerezo, "Subtleties in the trainability of quantum machine learning models," Quantum Machine Intelligence 5 (2023). 7. M. Ragone et al., "A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits," Nature Communications 15, 7172 (2024). 8. E. Fontana et al., "Characterizing barren plateaus in quantum ansätze with the adjoint representation," Nature Communications 15, 7171 (2024). 9. M. Larocca et al., "Barren plateaus in variational quantum computing," Nature Reviews Physics 7 (2025). 10. E. Grant, L. Wossnig, M. Ostaszewski and M. Benedetti, "An initialization strategy for addressing barren plateaus in parametrized quantum circuits," Quantum 3, 214 (2019). 11. K. Zhang, L. Liu, M.-H. Hsieh and D. Tao, "Escaping from the barren plateau via Gaussian initializations in deep variational quantum circuits," Advances in Neural Information Processing Systems 35 (2022). 12. I. Cong, S. Choi and M. D. Lukin, "Quantum convolutional neural networks," Nature Physics 15, 1273–1278 (2019). 13. A. Pesah et al., "Absence of barren plateaus in quantum convolutional neural networks," Physical Review X 11, 041011 (2021). 14. L. Bittel and M. Kliesch, "Training variational quantum algorithms is NP-hard," Physical Review Letters 127, 120502 (2021). 15. E. R. Anschuetz and B. T. Kiani, "Quantum variational algorithms are swamped with traps," Nature Communications 13, 7760 (2022). 16. M. Larocca et al., "Theory of overparametrization in quantum neural networks," Nature Computational Science 3 (2023). 17. D. Stilck França and R. García-Patrón, "Limitations of optimization algorithms on noisy quantum devices," Nature Physics 17 (2021). Chapter 5 1. A. W. Harrow, A. Hassidim and S. Lloyd, "Quantum algorithm for linear systems of equations," Physical Review Letters 103, 150502 (2009). 2. C. Ding, T.-Y. Bao and H.-L. Huang, "Quantum-inspired support vector machine," arXiv:1906.08902 (2019); later in IEEE Transactions on Neural Networks and Learning Systems. 3. M. Schuld and N. Killoran, "Quantum machine learning in feature Hilbert spaces," Physical Review Letters 122, 040504 (2019). 4. V. Havlíček et al., "Supervised learning with quantum-enhanced feature spaces," Nature 567, 209–212 (2019). 5. Y. Liu, S. Arunachalam and K. Temme, "A rigorous and robust quantum speed-up in supervised machine learning," Nature Physics 17, 1013–1017 (2021). 6. H.-Y. Huang et al., "Power of data in quantum machine learning," Nature Communications 12, 2631 (2021). 7. J. M. Kübler, S. Buchholz and B. Schölkopf, "The inductive bias of quantum kernels," Advances in Neural Information Processing Systems 34 (2021). 8. S. Thanasilp, S. Wang, M. Cerezo and Z. Holmes, "Exponential concentration in quantum kernel methods," Nature Communications 15, 5200 (2024). 9. R. Shaydulin and S. M. Wild, "Importance of kernel bandwidth in quantum machine learning," Physical Review A 106, 042407 (2022). 10. A. Canatar et al., "Bandwidth enables generalization in quantum kernel models," Transactions on Machine Learning Research (2023). 11. J. R. Glick et al., "Covariant quantum kernels for data with group structure," Nature Physics 20 (2024). 12. M. Schuld, "Supervised quantum machine learning models are kernel methods," arXiv:2101.11020 (2021). 13. S. Jerbi et al., "Quantum machine learning beyond kernel methods," Nature Communications 14, 517 (2023). Chapter 6 1. I. Kerenidis and A. Prakash, "Quantum recommendation systems," Proceedings of the 8th Innovations in Theoretical Computer Science Conference (2017); arXiv:1603.08675 (2016). 2. E. Tang, "A quantum-inspired classical algorithm for recommendation systems," Proceedings of the 51st ACM Symposium on Theory of Computing (2019). 3. A. Frieze, R. Kannan and S. Vempala, "Fast Monte-Carlo algorithms for finding low-rank approximations," Journal of the ACM 51, 1025–1041 (2004); conference version 1998. 4. A. Gilyén, S. Lloyd and E. Tang, "Quantum-inspired low-rank stochastic regression with logarithmic dependence on the dimension," arXiv:1811.04909 (2018). 5. N.-H. Chia et al., "Sampling-based sublinear low-rank matrix arithmetic framework for dequantizing quantum machine learning," Proceedings of the 52nd ACM Symposium on Theory of Computing (2020); Journal of the ACM 69 (2022). 6. S. Gharibian and F. Le Gall, "Dequantizing the quantum singular value transformation: hardness and applications to quantum chemistry and the quantum PCP conjecture," Proceedings of the 54th ACM Symposium on Theory of Computing (2022). 7. E. Tang, "Quantum principal component analysis only achieves an exponential speedup because of its state preparation assumptions," Physical Review Letters 127, 060503 (2021). 8. J. M. Arrazola, A. Delgado, B. R. Bardhan and S. Lloyd, "Quantum-inspired algorithms in practice," Quantum 4, 307 (2020). 9. A. Bakshi and E. Tang, "An improved classical singular value transformation for quantum machine learning," Proceedings of the 2024 ACM-SIAM Symposium on Discrete Algorithms (2024). 10. F. J. Schreiber, J. Eisert and J. J. Meyer, "Classical surrogates for quantum learning models," Physical Review Letters 131, 100803 (2023). 11. J. Landman, S. Thabet, C. Dalyac, H. Mhiri and E. Kashefi, "Classically approximating variational quantum machine learning with random Fourier features," arXiv:2210.13200 (2022). 12. A. Rahimi and B. Recht, "Random features for large-scale kernel machines," Advances in Neural Information Processing Systems 20 (2007). 13. R. Sweke et al., "Potential and limitations of random Fourier features for dequantizing quantum machine learning," arXiv:2309.11647 (2023). 14. Y. Kim et al., "Evidence for the utility of quantum computing before fault tolerance," Nature 618, 500–505 (2023). 15. J. Tindall, M. Fishman, E. M. Stoudenmire and D. Sels, "Efficient tensor network simulation of IBM's Eagle kicked Ising experiment," PRX Quantum 5, 010308 (2024). 16. T. Begušić and G. K.-L. Chan, "Fast and converged classical simulations of evidence for the utility of quantum computing before fault tolerance," Science Advances 10, eadk4321 (2024). 17. F. Pan, K. Chen and P. Zhang, "Solving the sampling problem of the Sycamore quantum circuits," Physical Review Letters 129, 090502 (2022). 18. M. Cerezo et al., "Does provable absence of barren plateaus imply classical simulability? Or, why we need to rethink variational quantum computing," arXiv:2312.09121 (2023); Nature Communications (2025). 19. P. Bermejo et al., "Quantum convolutional neural networks are (effectively) classically simulable," arXiv:2408.12739 (2024). 20. J. Bowles, S. Ahmed and M. Schuld, "Better than classical? The subtle art of benchmarking quantum machine learning models," arXiv:2403.07059 (2024). Chapter 7 1. R. Babbush et al., "Focus beyond quadratic speedups for error-corrected quantum advantage," PRX Quantum 2, 010103 (2021). 2. A. Lucas, "Ising formulations of many NP problems," Frontiers in Physics 2, 5 (2014). 3. T. Kadowaki and H. Nishimori, "Quantum annealing in the transverse Ising model," Physical Review E 58, 5355 (1998). 4. T. F. Rønnow et al., "Defining and detecting quantum speedup," Science 345, 420–424 (2014). 5. V. S. Denchev et al., "What is the computational value of finite-range tunneling?," Physical Review X 6, 031015 (2016). 6. A. D. King et al., "Beyond-classical computation in quantum simulation," Science 388 (2025). 7. E. Farhi, J. Goldstone and S. Gutmann, "A quantum approximate optimization algorithm," arXiv:1411.4028 (2014). 8. E. Farhi, J. Goldstone and S. Gutmann, "A quantum approximate optimization algorithm applied to a bounded occurrence constraint problem," arXiv:1412.6062 (2014). 9. B. Barak et al., "Beating the random assignment on constraint satisfaction problems of bounded degree," arXiv:1505.03424 (2015). 10. M. X. Goemans and D. P. Williamson, "Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming," Journal of the ACM 42, 1115–1145 (1995). 11. M. B. Hastings, "Classical and quantum bounded depth approximation algorithms," arXiv:1905.07047 (2019). 12. E. Farhi, D. Gamarnik and S. Gutmann, "The quantum approximate optimization algorithm needs to see the whole graph," arXiv:2004.09002 (2020). 13. S. Bravyi, A. Kliesch, R. Koenig and E. Tang, "Obstacles to variational quantum optimization from symmetry protection," Physical Review Letters 125, 260505 (2020). 14. M. P. Harrigan et al., "Quantum approximate optimization of non-planar graph problems on a planar superconducting processor," Nature Physics 17, 332–336 (2021). 15. R. Shaydulin et al., "Evidence of scaling advantage for the quantum approximate optimization algorithm on a classically intractable problem," Science Advances 10, eadm6761 (2024). 16. S. Ebadi et al., "Quantum optimization of maximum independent set using Rydberg atom arrays," Science 376, 1209–1215 (2022). 17. R. S. Andrist et al., "Hardness of the maximum-independent-set problem on unit-disk graphs and prospects for quantum speedups," Physical Review Research 5 (2023). 18. HSBC, "HSBC demonstrates world's first-known quantum-enabled algorithmic trading with IBM," media release, 25 September 2025; accompanying preprint arXiv:2509.17715 (2025). Chapter 8 1. Google Quantum AI and collaborators (R. Acharya et al.), "Quantum error correction below the surface code threshold," Nature 638 (2025). 2. Google Quantum AI, "Our Quantum Echoes algorithm is a big step toward real-world applications for quantum computing," announcement and accompanying Nature paper, October 2025. 3. Quantinuum, Helios launch announcement, 5 November 2025. 4. IBM, announcements of the Nighthawk and Loon processors and the Starling roadmap, November 2025. 5. D. Bluvstein et al., "Logical quantum processor based on reconfigurable atom arrays," Nature 626 (2024). 6. S. Bravyi et al., "High-threshold and low-overhead fault-tolerant quantum memory," Nature 627 (2024). 7. J. Lee et al., "Even more efficient quantum computations of chemistry through tensor hypercontraction," PRX Quantum 2, 030305 (2021). 8. C. Gidney, "How to factor 2048 bit RSA integers with less than a million noisy qubits," arXiv:2505.15917 (2025). 9. "Practical considerations for processing massive classical data via quantum oracle sketching," arXiv:2609.13549 (2026). Chapter 9 1. H.-Y. Huang, R. Kueng and J. Preskill, "Predicting many properties of a quantum system from very few measurements," Nature Physics 16, 1050–1057 (2020). 2. H.-Y. Huang, R. Kueng and J. Preskill, "Information-theoretic bounds on quantum advantage in machine learning," Physical Review Letters 126, 190505 (2021). 3. S. Chen, J. Cotler, H.-Y. Huang and J. Li, "Exponential separations between learning with and without quantum memory," Proceedings of the 62nd IEEE Symposium on Foundations of Computer Science (2022). 4. D. Aharonov, J. Cotler and X.-L. Qi, "Quantum algorithmic measurement," Nature Communications 13, 887 (2022). 5. H.-Y. Huang et al., "Quantum advantage in learning from experiments," Science 376, 1182–1186 (2022). 6. J. Herrmann et al., "Realizing quantum convolutional neural networks on a superconducting quantum processor to recognize quantum phases," Nature Communications 13 (2022). 7. H.-Y. Huang, R. Kueng, G. Torlai, V. V. Albert and J. Preskill, "Provably efficient machine learning for quantum many-body problems," Science 377, eabk3333 (2022). 8. M. C. Caro et al., "Generalization in quantum machine learning from few training data," Nature Communications 13, 4919 (2022). 9. E. Gil-Fuster, J. Eisert and C. Bravo-Prieto, "Understanding quantum machine learning also requires rethinking generalization," Nature Communications 15, 2277 (2024). 10. J. Bausch et al., "Learning high-accuracy error decoding for quantum processors," Nature 635 (2024). 11. G. Carleo and M. Troyer, "Solving the quantum many-body problem with artificial neural networks," Science 355, 602–606 (2017). Glossary Ansatz. The fixed arrangement of gates, with adjustable angles, that defines the family of quantum states a variational algorithm can prepare. Barren plateau. A region of a variational model's parameter space in which the cost function is almost constant and its gradients shrink exponentially with the number of qubits, so that no practical number of measurements can tell an optimizer which way to move. Classical shadow. A compact classical description of a quantum state built from measurements in randomly chosen bases, from which many properties of the state can be predicted. Dequantization. The construction of a classical algorithm that matches a claimed quantum speedup once both are given comparable access to the input data, typically by sampling. Dynamical Lie algebra. The space of operators generated by repeatedly combining a circuit's gate generators; its size governs whether the circuit's gradients vanish. Error mitigation. Techniques that reduce the bias caused by hardware noise in estimated quantities, such as zero-noise extrapolation, without correcting errors during the computation; their sampling costs grow exponentially with circuit size. Fault tolerance. The ability to run arbitrarily long quantum computations reliably by encoding logical qubits in many physical qubits and correcting errors continuously. Feature map. A circuit whose gate angles depend on the features of a data point, used to encode classical data into a quantum state. Parameter-shift rule. A method for computing the exact gradient of a circuit's expectation value by evaluating it at two shifted values of each parameter. QRAM. Quantum random access memory: a hypothetical device that would load classical data into a quantum state in superposition at a cost logarithmic in the data size. QUBO. Quadratic unconstrained binary optimization, the problem of minimizing a quadratic function of binary variables, equivalent to finding the ground state of an Ising model. Shot. A single preparation and measurement of a quantum circuit; estimates of expectation values require many shots, with error falling as the inverse square root of their number. Hashtags: #QuantumMachineLearning #QML #QuantumComputing #HybridQuantumClassicalAlgorithms #QuantumArtificialIntelligence #QuantumAlgorithms #VariationalQuantumAlgorithms #VQA #VariationalQuantumEigensolver #VQE #QuantumNeuralNetworks #ParameterizedQuantumCircuits #QuantumSupportVectorMachines #QuantumKernels #QuantumFeatureMaps #QuantumApproximateOptimizationAlgorithm #QAOA #QuantumOptimization #QuantumAnnealing #QuantumChemistry #QuantumSimulation #QuantumEntanglement #BarrenPlateaus #QuantumErrorMitigation #NISQComputing #QuantumAdvantage #QuantumClassicalOptimization #QuantumCircuitLearning #PennyLane #Qiskit

Download the Book (PDF): Introduction Somewhere in most factories there is a machine everyone is a little afraid of. It might be a forty-year-old hydraulic press, a gearbox on the main conveyor, or a compressor that feeds the whole plant's air supply. It was built before anyone used the word "connected," it has no network port, and its manual, if one still exists, lives in a binder in the maintenance office. Nobody knows exactly when it will fail. Everybody knows what happens when it does: the line stops, the overtime starts, and the plant manager spends the next week explaining to customers why their orders are late. This booklet is about that machine, and about the practical work of learning to hear it before it breaks. The idea is simple to state. Almost every mechanical failure announces itself. A bearing that is starting to spall produces a faint, rhythmic impact long before it seizes. A motor winding whose insulation is breaking down runs hotter than its neighbours for weeks. A pump drawing air through a worn seal changes the pattern of its own vibration. These signals are physical and measurable, and in most cases they appear well before the failure that ends production. If you can capture them, interpret them correctly, and act on them in time, you can replace a part on a planned Tuesday afternoon instead of during an unplanned Saturday night shutdown. That is predictive maintenance. It is not new. Vibration analysts with handheld data collectors have been walking routes through paper mills and power stations since the 1970s, and infrared thermography has been a standard inspection tool for decades. What has changed in the past ten years is cost and reach. Sensors that once cost hundreds or thousands of dollars per measurement point now cost a fraction of that. Battery-powered wireless nodes can be glued or bolted to a motor housing in minutes with no cabling. Cloud platforms and cheap edge computers can store and process continuous data that a technician with a clipboard could only sample once a month. The industrial internet of things, usually shortened to IIoT, is the name that has attached itself to this shift: ordinary industrial equipment, most of it never designed to communicate, fitted with sensors and connected to software that watches it continuously. Alongside IIoT has come a second term, the digital twin, which promises a living software replica of a physical machine or an entire production line, updated in real time and capable of answering questions about the future. The phrase has been stretched to cover almost anything, from a dashboard showing live temperatures to a high-fidelity physics simulation of a turbine. Some of what is sold under that label is valuable. Much of it is a relabelled dashboard. The argument of this booklet The central claim of what follows is this: on the factory floor, predictive maintenance succeeds or fails on unglamorous decisions made before any algorithm runs. Which machines are worth monitoring at all. Which specific failure modes those machines suffer. Which physical signal reveals each failure mode early enough to act. Where the sensor goes and how it is mounted. How the data leaves the machine without opening a hole in the plant's security. And, above all, whether the maintenance organisation has the discipline and authority to act on a warning when it arrives. Get those decisions right, and fairly simple analytics, such as trending a vibration level against an alarm limit, will catch most developing faults with weeks of notice. Get them wrong, and no amount of machine learning will rescue the project. The most common failure pattern in industrial IoT is not a bad model. It is a well-instrumented machine whose alerts nobody reads, or a pilot that monitors whatever was easiest to instrument rather than what was most likely to stop the line. The same discipline applies to digital twins. A twin is worth building when it answers a specific question that cannot be answered more cheaply, such as how much useful life remains in a component given the loads it has actually seen. It is not worth building as a statement of ambition. The most useful twins on most factory floors are modest: a well-structured model of an asset, its failure modes, its sensor data and its maintenance history, joined so that people can reason about the machine with evidence rather than memory. Who this is for The booklet is written for people who need to make or influence decisions about modernising existing plants: maintenance and reliability engineers who have heard the pitch and want to know what is real; plant and operations managers weighing the cost of a pilot against the cost of the next breakdown; engineers in IT or data roles who have been asked to "do something with IoT" on equipment they do not know; and students and newcomers who want a grounded picture of how this work is actually done. It assumes no specialist background. Terms are defined where they first appear. It does assume a willingness to think about machines as physical objects, because the physics is where the argument starts. Vibration, heat and electrical current are not abstractions; they are what a failing bearing, a misaligned shaft or a degrading motor actually produce, and understanding why they change is what lets you decide what to measure. The focus on legacy equipment The emphasis throughout is on retrofit: adding sensing and connectivity to equipment that already exists and was never designed for it. That is where most of the world's manufacturing capacity sits. A new machine bought today may arrive with condition monitoring built in, but factories are rarely replaced wholesale. They are accumulations of equipment from different decades, different manufacturers and different engineering philosophies, held together by maintenance crews who know each machine's habits. The economic case for predictive maintenance is usually strongest exactly where the equipment is oldest, most critical and least documented, because those are the machines whose failures are both likely and expensive. Retrofit also imposes useful constraints. You cannot redesign the machine. You often cannot run new cable through a live production area without a shutdown. You may not be allowed to touch the machine's controller at all, either because it is under a vendor's warranty or because nobody is confident they could restore it if something went wrong. These constraints shape what is possible, and they explain why so much modern IIoT practice relies on external, non-invasive sensors: an accelerometer on a bearing housing, a clamp-on current transformer around a motor lead, a thermal sensor aimed at a panel, none of which require touching the control system. How the booklet is organised The chapters follow the path a real project should take. The first chapter sets out how machines actually fail, why that matters for any maintenance strategy, and what "predictive" really means compared with reactive and preventive approaches. The second turns to the decision most projects skip: choosing which assets and which failure modes are worth monitoring, using the long-established tools of criticality ranking and failure modes analysis. The third and fourth chapters are about the signals themselves. Vibration gets a chapter of its own because it is the most informative and most widely used signal for rotating machinery, and because understanding it explains the logic of the whole discipline. Temperature, electrical current, ultrasound and oil condition follow, each with the faults it reveals best and the traps it sets. The fifth chapter deals with getting data off the factory floor: wireless options, gateways, edge computing, the standard protocols that let different systems talk to each other, and the cybersecurity obligations that come with connecting anything to an industrial network. The sixth covers analytics, from simple thresholds to machine learning, and spends time on the problem that undermines more programmes than any other, which is false alarms. The seventh chapter takes digital twins seriously enough to be sceptical about them, separating the useful core of the idea from the marketing. The eighth is about the organisation: how alerts become work orders, how pilots become programmes, how the economics are measured honestly, and why the human side determines most of the outcome. The conclusion draws out what follows from all of this for anyone about to begin. A final note on tone. There is a great deal of enthusiasm in this field, some of it justified. The aim here is not to dampen it but to direct it. The technology works. Sensors are cheap, wireless is reliable enough, software is capable. What remains hard is what has always been hard in maintenance: knowing your machines, knowing how they fail, and building an organisation that acts on what it knows. IIoT makes that work faster and more precise. It does not do it for you. Chapter 1: How Machines Actually Fail Every maintenance strategy rests on an assumption about how failure happens, whether or not anyone has written it down. The calendar-based schedule that replaces a gearbox's oil every six months assumes that the oil degrades at a predictable rate. The policy of running a pump until it breaks assumes that the cost of a breakdown is lower than the cost of preventing it. The decision to install vibration sensors assumes that the failures you care about give warning, and that the warning comes early enough to be useful. Before deciding what to monitor, it is worth being clear about which of these assumptions are true for the machines in front of you. The surprising shape of failure For most of the twentieth century, engineers worked with a simple picture of equipment life. A machine was thought to be reliable while new, to wear gradually as it aged, and eventually to reach a point where failure became likely. The obvious response was to overhaul or replace components before they reached that point. Preventive maintenance at fixed intervals was the natural expression of this belief, and it remains the backbone of many maintenance programmes today. The picture was challenged decisively by the civil aviation industry. In the 1960s and 1970s, airlines facing the arrival of the Boeing 747 realised that applying traditional time-based overhaul policies to a much larger and more complex aircraft would be prohibitively expensive, and there was little evidence the overhauls were making aircraft safer. Work at United Airlines led to a landmark 1978 report for the United States Department of Defense by F. Stanley Nowlan and Howard F. Heap, titled Reliability-Centered Maintenance. Analysing the failure histories of aircraft components, Nowlan and Heap described six distinct patterns of conditional probability of failure against age. Only a small minority of components followed the classic "bathtub curve," with high early failure, a long flat middle and a rising wear-out phase. A similar small share showed a clear wear-out zone. Some showed a slow, steady increase in failure probability with no distinct wear-out point. The great majority, however, showed no relationship between age and failure at all once they had survived an initial period: the probability of failure was roughly constant, or high when new and then constant. In the United Airlines data, the age-related patterns together accounted for only a small fraction of the components studied, while random and infant-mortality patterns dominated. Later studies in other industries, including naval and industrial settings, have produced different proportions, and the exact figures should not be carried uncritically from aircraft to a bottling plant. But the qualitative lesson has held up across sectors: for complex equipment, much failure is not driven by age in any simple way. Many failures are effectively random with respect to calendar time, and a significant share are caused by maintenance itself, through incorrect reassembly, contamination introduced during servicing, or the disturbance of a system that was running well. This has an uncomfortable implication. If a component's probability of failure does not increase with age, replacing it at a fixed interval does nothing to reduce failures. It throws away useful life, costs labour and parts, and, because every intervention carries some risk of introducing a new fault, may actually make things worse. The bearing replaced on schedule might have run for another five years. The new one may have been installed with a slight misalignment that kills it in six months. The P–F interval If time does not predict failure, what does? The answer the reliability community converged on is condition. Many failure modes, even those that occur at random times, do not happen instantly. They develop. There is a point at which a fault begins to form and becomes detectable, and a later point at which it becomes a functional failure, meaning the equipment can no longer do what it is required to do. The interval between those two points is called the P–F interval, where P is the point of potential failure, the earliest moment at which the developing fault can be detected by some means, and F is the point of functional failure. The concept was developed within reliability-centred maintenance and popularised by John Moubray in his book Reliability-Centred Maintenance, which adapted the aviation work for industry. The P–F interval is the single most important idea in predictive maintenance, so it is worth being precise about it. Imagine a rolling-element bearing in an electric motor. At some point, a microscopic crack forms beneath the surface of the outer race, perhaps because of a lubrication problem or a manufacturing flaw. The crack grows with each rotation until a small piece of metal breaks away, leaving a pit. Every time a rolling element passes over that pit, it produces a tiny impact. As the pit grows, the impacts become stronger, debris contaminates the lubricant, the surfaces roughen further, heat rises, and eventually the bearing becomes noisy, hot and loose. If nothing is done, the cage may break, the rolling elements may skid, and the bearing seizes or the shaft is damaged. Along that path, different measurement techniques detect the fault at different stages. High-frequency vibration and ultrasound techniques can pick up the earliest impacts when the defect is still small. Overall vibration levels rise somewhat later. Debris appears in oil analysis at some stage. Audible noise comes later still. A rise in temperature that a hand on the housing can feel usually arrives near the end, shortly before the bearing fails. Each technique therefore has its own P point and its own P–F interval for the same failure mode. This is why choosing what to measure matters so much. A temperature sensor on a bearing housing will detect a bearing failure, but often with only days or hours of warning, which may not be enough to order a part and schedule a shutdown. A vibration sensor sampling at high frequency may provide weeks or months. The difference is not in the sophistication of the software; it is in the physics of which signal changes first. The P–F interval also sets the monitoring frequency. If a failure mode typically develops over eight weeks from detectable to failed, then checking once a quarter will often miss it entirely. A common rule in reliability practice is to inspect at an interval no longer than half the P–F interval, so that at least one inspection falls inside the window with enough time remaining to act. This rule was one of the original economic arguments for continuous wireless monitoring: when a route-based technician can only visit a machine monthly, any failure mode with a P–F interval shorter than about two months slips through. Continuous sensors collapse the inspection interval to minutes, which makes even short P–F intervals usable, provided someone acts quickly on the warning. Finally, the P–F interval is not a fixed property of a component. It varies with load, speed, lubrication and operating conditions. A bearing on a slow, lightly loaded fan may degrade over many months. The same bearing type on a heavily loaded, high-speed spindle may go from first detectable defect to failure in days. Good practice is to treat published or vendor-quoted intervals as rough guides and to learn the real ones from your own equipment over time. Four ways to maintain a machine With failure patterns and the P–F interval in hand, the familiar maintenance strategies can be placed on a sounder footing. There are four broad approaches, and a mature plant uses all of them, each where it fits. Reactive maintenance, also called run-to-failure or breakdown maintenance, simply waits for something to break and then repairs it. It sounds irresponsible, but it is the right choice for equipment whose failure is cheap, safe and has no knock-on consequences: a light fitting, a redundant pump with an automatic standby, a small fan whose failure is noticed immediately and fixed in minutes. Applying sensors to such equipment wastes money. Preventive maintenance, also called time-based or planned maintenance, performs tasks at fixed intervals of time or usage: change the oil every 2,000 operating hours, replace the filter every quarter, overhaul the gearbox every five years. It works well for failure modes that genuinely are age-related, such as wear of a cutting tool, exhaustion of a consumable, or degradation of a lubricant. It fails, as Nowlan and Heap showed, for the many failure modes that are not. Condition-based maintenance acts on the measured condition of the equipment rather than on time. Inspections or measurements are carried out, and work is triggered when the condition crosses some limit. A vibration analyst's monthly route, oil sampling and thermographic surveys of electrical panels are all forms of condition-based maintenance. Predictive maintenance is best understood as a refinement of this approach: it uses condition data not only to detect that a fault is present but also to estimate how it will develop, and therefore when intervention should happen. Finally, there is redesign or proactive maintenance, which removes the root cause of failure: improving lubrication practice, correcting chronic misalignment, upgrading a seal design, or changing how a machine is operated. Condition monitoring frequently feeds this approach, because once you can see faults developing you can often work out why, and fixing the cause is better than predicting the symptom repeatedly. These approaches differ on what triggers work, what failure patterns they suit and what they demand of the organisation, as Table 1 summarises. Table 1. The four broad maintenance strategies compared. Strategy What triggers work Suits failure modes that are Main cost or risk Reactive (run-to-failure) Breakdown Cheap, safe, non-critical Unplanned downtime, secondary damage Preventive (time-based) Calendar or usage interval Clearly age-related, e.g. wear, consumables Wasted life, maintenance-induced faults Condition-based / predictive Measured condition crossing a limit or trend Detectable with a usable P–F interval Sensors, data handling, skilled interpretation Proactive (root cause) Analysis of recurring failures Chronic and repeatable Engineering time, change management Hidden failures and failures we cause Two further categories of failure deserve attention before going further, because both are common on older plants and neither is addressed by putting a sensor on a bearing. The first is the hidden failure. Some equipment exists only to act when something else goes wrong: a pressure relief valve, a standby pump, a high-temperature trip, an emergency stop circuit, a fire suppression system, a backup generator. In normal operation these devices do nothing, so when they fail, nobody notices. The failure becomes visible only on the day they are needed, when the relief valve does not lift or the standby pump does not start. Reliability-centred maintenance treats these separately, through what it calls failure-finding tasks: periodic tests designed to discover whether a protective device still works. On legacy plants, where trips have sometimes been bypassed during a past emergency and never reinstated, or where a standby pump has sat unused for years, a review of hidden functions often reveals risks far larger than anything a vibration programme would catch. Condition monitoring can help, for instance by running standby equipment periodically and recording its vibration and current during the test, but the essential task is the test itself. The second category is maintenance-induced failure. Every time a machine is opened, there is a chance of introducing a fault: a bearing installed with a hammer instead of a press or heater, a coupling reassembled with poor alignment, a seal pinched during fitting, dirt introduced into a gearbox, the wrong grease, too much grease, a bolt left loose. Many experienced reliability engineers consider these among the largest single sources of failure in industry, and the infant-mortality pattern in the aviation data is consistent with that view. It has a direct implication for predictive maintenance. Condition monitoring is very good at revealing them, because a machine that ran smoothly before an overhaul and vibrates badly afterwards is obviously telling you something. A baseline measurement taken immediately after every significant repair, compared with the measurement before, is one of the cheapest and most effective uses of monitoring equipment. Over time, it also reveals which crews, procedures and contractors leave machines in better or worse condition than they found them, which is uncomfortable information but valuable. Both categories reinforce the same point. The value of monitoring lies not in monitoring as such but in the match between the technique and the specific way a specific function can fail. A plant that understands its failure modes will put sensors where they help, schedule tests where sensors cannot see, and fix the practices that cause failures in the first place. What failure costs The economic case for any of this rests on the cost of unplanned stoppages, which is routinely underestimated because much of it never appears on a maintenance budget. The direct repair cost of a failed gearbox may be modest. The cost of the line standing for fourteen hours, the scrap from a batch interrupted mid-process, the overtime to catch up, the expedited freight for a replacement part, the penalties or lost goodwill from late deliveries, and the safety risk of crews working under pressure to restart can all be far larger. Siemens's True Cost of Downtime 2024 report, produced by its Senseye predictive maintenance business, estimated that unplanned downtime costs the world's 500 largest companies around 1.4 trillion dollars a year, equivalent to roughly 11 percent of their revenues. Such headline figures are estimates built on surveys and assumptions, and they should be treated as indications of scale rather than precise accounting. The more useful exercise is local: work out what an hour of stoppage costs on your own critical lines, including the indirect costs, and keep that number in front of every decision about where to invest in monitoring. There is a second economic point that is easy to miss. The benefit of predictive maintenance does not come from avoiding repair. The bearing still has to be replaced. The benefit comes from changing when and how the repair happens: during a planned window rather than in the middle of a production run, with the part already on the shelf, with the right crew and tools, before the failure has damaged the shaft, the housing or the product. A bearing replaced at the first sign of an outer-race defect costs a bearing and a few hours of labour. The same bearing allowed to run to seizure can cost a shaft, a motor, a coupling, a damaged gearbox and a lost shift. What "predictive" can and cannot promise The word "predictive" invites overconfidence. It suggests that software can say when a machine will fail, to the day. For a few well-understood failure modes on well-instrumented assets with good histories, estimates of remaining useful life are achievable within useful bounds. For most equipment on most factory floors, what monitoring really provides is earlier detection and a trend: the fault is present, it is getting worse at roughly this rate, and action is warranted within roughly this window. That is enormously valuable, but it is different from prophecy. A programme sold on the promise of precise failure dates will disappoint. A programme built on the more honest promise of turning surprises into planned work, and of widening the P–F interval so that the planned work can be scheduled sensibly, can deliver reliably. There are also failure modes that give no useful warning. A shaft that fractures from a sudden overload, a fuse that blows, an electronic component that fails abruptly, or a fault whose P–F interval is shorter than the time needed to respond cannot be predicted in any practical sense. For these, the right responses are design changes, redundancy, protective devices or simply accepting run-to-failure. Recognising them early prevents a great deal of wasted effort trying to monitor the unmonitorable. The rest of this booklet builds on these foundations. If failures are often random in time but gradual in development, then the job is to find the failure modes that give warning, select the signal that gives the earliest warning, measure it often enough to catch the P–F interval, and make sure warnings turn into well-timed work. Each of those steps has its own difficulties, and the first of them, deciding what to monitor in the first place, is where most projects go wrong. Chapter 2: Choosing What to Watch Most industrial IoT pilots begin in the wrong place. A vendor offers a starter kit of a few dozen wireless sensors, and the plant team, keen to see results, attaches them to whatever is convenient: motors near the maintenance office, a pump that is easy to reach, a fan with a flat surface for mounting. Data starts flowing within a day. Dashboards light up. And six months later, the pilot has detected nothing of consequence, because none of the monitored machines was likely to fail in a way that mattered, and the machine that did fail, a gearbox deep inside an enclosure on the packaging line, had no sensor on it at all. The cure is old and unglamorous. Before buying a single sensor, decide which assets matter, how they fail, and which of those failure modes are both damaging and detectable. Reliability engineers have tools for exactly this, and they long predate the internet of things. Ranking criticality A typical mid-sized plant has thousands of maintainable assets: motors, pumps, fans, gearboxes, compressors, conveyors, presses, valves, heat exchangers and countless smaller components. Monitoring all of them is neither affordable nor useful. The first job is to sort them by how much their failure matters. Asset criticality analysis does this by scoring each asset on the consequences of its failure and, usually, on the likelihood of failure. Consequences typically cover several dimensions. Safety: could failure injure someone? Environment: could it cause a release, spill or regulatory breach? Production: does failure stop a line, slow it, or have no effect because there is redundancy or buffer stock? Quality: does failure produce scrap or defective product that might reach customers? Cost: what does repair cost, including secondary damage and spare parts with long lead times? Likelihood can be estimated from failure history in the maintenance management system, from the experience of technicians, or from the asset's age, duty and environment. Scores are combined, often as consequence multiplied by likelihood, to produce a ranking. The method is deliberately rough. Its value lies less in the precision of the numbers than in forcing a structured conversation between production, maintenance, safety and engineering staff, who often have very different views of which equipment matters. A production manager may never have thought about the cooling-water pump that, if it fails, forces an emergency shutdown of three furnaces. A maintenance planner may be spending large effort on a machine that production could happily do without for a day. Several features of an asset typically push it to the top of the list. It is a single point of failure, with no installed spare or bypass. It sits on the constraint of the line, the step whose capacity limits the whole plant's output, so that any hour lost there is an hour lost for the factory. Its spare parts are expensive or have long lead times; a custom gearbox from a manufacturer that no longer exists may take months to rebuild. It has a history of failure, or it is old enough that failures are expected. Its failure causes secondary damage, as when a failed bearing destroys an expensive shaft. Or its failure carries safety or environmental consequences. In practice, criticality analysis usually reveals that a small fraction of assets accounts for most of the risk. In many plants, perhaps ten to twenty percent of assets end up classified as highly critical. That is the population where condition monitoring belongs first. Criticality is not static. A line that was the plant's bottleneck last year may not be after a capacity upgrade elsewhere. An asset whose spare was used and not replaced has just become more critical. Good practice is to revisit the ranking periodically, and certainly after significant changes in production, equipment or spares holdings. Failure modes, not machines Knowing that a machine is critical is necessary but not sufficient. A sensor does not monitor a machine; it monitors a physical quantity that changes when particular failure modes develop. To choose the right sensor, you need to know how the machine fails. Failure modes and effects analysis, usually abbreviated to FMEA, is the classic method. For each critical asset, a small team of people who know the equipment works through its functions, the ways each function can fail, the specific failure modes that cause those functional failures, and the effects of each. For a centrifugal pump, the function might be to deliver a certain flow at a certain pressure. Functional failures include delivering no flow, insufficient flow or leaking. Failure modes behind those functional failures include bearing wear, mechanical seal failure, impeller erosion, cavitation damage, shaft misalignment with the motor, coupling wear, motor winding failure, blocked suction strainer and so on. Each failure mode has effects: the line stops, product is contaminated, the pump overheats, the motor trips. An FMEA done properly for the purpose of maintenance planning is the core of reliability-centred maintenance, the approach that grew out of the Nowlan and Heap work. In its formal versions, such as the process defined in the SAE JA1011 standard for RCM evaluation, each failure mode is then run through a decision logic: is there a condition-based task that is technically feasible and worth doing? If not, is there a time-based task? If not, should the equipment be redesigned or run to failure? Many plants find full RCM too heavy to apply across all their assets, and lighter-weight variants exist. What matters for predictive maintenance is the essential sequence: list the failure modes of critical assets, estimate how likely and how damaging each is, and for each one ask what physical signal would reveal it early. This is where the analysis starts to pay off. Consider a critical gearbox on a conveyor drive. Its plausible failure modes include gear tooth wear, tooth cracking, bearing defects, lubricant degradation, lubricant contamination, oil leaks, and misalignment with the motor or the driven shaft. Several of these are visible in vibration: gear mesh problems and bearing defects produce characteristic frequencies, and misalignment produces distinctive patterns at multiples of shaft speed. Lubricant degradation and contamination are better caught by oil analysis or oil condition sensors. Leaks may be caught by a level sensor or by an operator's inspection round. Tooth cracking can give limited warning and may be best addressed by reducing shock loads. A single vibration sensor on the gearbox will cover some of these failure modes well, others poorly, and some not at all. Knowing which is which prevents false confidence. Detectability and the P–F test For each important failure mode, three questions decide whether condition monitoring is worth applying. First, is there a measurable physical change before functional failure? Some failures give no warning: a sudden fracture under overload, or many electronic failures. If there is no precursor, no sensor can help. Second, is the P–F interval long enough to be useful? If the earliest detectable signal arrives three hours before failure, and it takes two days to obtain a part and schedule the work, monitoring offers little except a slightly more orderly emergency. Short intervals may still be worthwhile if they allow a controlled shutdown that prevents secondary damage or a safety incident, but the benefit is smaller. Third, is the P–F interval consistent enough? If a failure mode sometimes develops over months and sometimes over hours, monitoring will catch some instances and miss others. That might still be worthwhile, but expectations should be set accordingly. These questions frequently produce surprising answers. Some failure modes that look obvious targets turn out to be poorly suited to monitoring, while others that seem mundane, such as belt slippage on a fan drive, prove to be cheap to detect and very common. The question is never "can we put a sensor on it?" but "will this sensor tell us something we can act on, in time, for a failure that matters?" Starting with the data you already have Before installing anything, it pays to mine what already exists. The computerised maintenance management system, often called the CMMS, holds years of work orders: what failed, when, how long repair took, which parts were used. The quality of this data is often poor. Failure codes may be missing or generic, descriptions may say only "fixed" or "replaced bearing," and technicians may have logged emergency work under preventive orders to save time. Even so, a careful review of work orders for critical assets usually reveals the recurring problems: the pump whose seals keep failing, the motor that has been rewound three times, the conveyor whose gearbox is replaced every eighteen months. Production systems hold other clues. Downtime logs, where operators record stoppages and their causes, show which assets actually stop the line. Process historians, which record temperatures, pressures, flows and speeds from the control system, may already contain signals that would have revealed past failures if anyone had looked. A motor's current draw, recorded by its variable-frequency drive, can show developing mechanical problems. A cooling loop's temperature might have drifted for weeks before a heat exchanger fouled badly enough to trip a machine. People are the third source. Experienced technicians and operators know which machines are temperamental, which noises mean trouble and which faults recur. Their knowledge is rarely written down, and many of them are approaching retirement. A structured interview programme, asking each experienced person which failures they most fear and what signs they look for, often produces a better first list of monitoring targets than any software tool. It also builds the trust that will be needed later, when those same people are asked to act on alerts from sensors they did not choose. The particular problems of old machines Legacy equipment complicates every step of this analysis in ways worth naming, because they shape what a realistic plan looks like. Documentation is often incomplete or wrong. The original manual may be lost, or may describe a machine that has since been modified many times. Motors have been replaced with different models, bearings with equivalents from other manufacturers, gearboxes rebuilt by local workshops. The nameplate may be painted over or missing. Before a vibration analyst can calculate bearing defect frequencies, someone has to find out which bearing is actually fitted, and sometimes the only way to do that is to wait until the machine is next opened and read the part number off the bearing itself. A useful habit is to make every repair an opportunity to collect this information: photograph nameplates and components, record bearing numbers and gear tooth counts, and add them to the asset record. Original manufacturers may no longer exist, or may no longer support the model. Spare parts become harder to obtain, so lead times stretch and the consequences of failure grow. An asset that was moderately critical when its spares were on a supplier's shelf can become highly critical when the only source is a specialist who can machine a replacement in twelve weeks. Criticality analysis on legacy plants must take obsolescence into account explicitly. Knowledge is concentrated in people. On many older lines, one or two technicians understand a particular machine well enough to diagnose it by sound and feel. That knowledge is valuable and precarious. Monitoring programmes can help capture it, by recording which signals these technicians trust and what thresholds they would set, but only if they are involved from the beginning rather than presented with a finished system. Physical access can be awkward. Bearings may be hidden behind guards, inside enclosures, beneath conveyors, or in hazardous areas. Some measurement points are reachable only during shutdowns. Wireless sensors make previously inaccessible points monitorable, which is one of their greatest advantages, but installation still requires a planned outage, and the plan should be ready before that outage arrives. Finally, old machines often operate in conditions they were not designed for: faster speeds, heavier loads, different products, longer shifts. Their failure modes may therefore differ from what the original designers expected, and generic failure data for their equipment class may be misleading. Local failure history and technicians' experience matter more than ever. None of these problems is a reason to avoid monitoring legacy equipment. On the contrary, they are often reasons to prioritise it, since old, undocumented, hard-to-replace machines are exactly the ones whose sudden failure is most damaging. But they mean that the analysis takes longer, that the information model of each asset has to be built rather than imported, and that a good plan is iterative, improving as each repair reveals more about the machine. A worked example Consider a plant producing packaged food on four lines. A criticality exercise identifies, among others, the following high-ranking assets: the spiral freezer's refrigeration compressor, which has no backup and serves all four lines; the main ammonia pump in the refrigeration loop; the drive gearboxes on two high-speed packaging machines; the plant's main air compressor; and a set of conveyor motors that, though individually cheap, stop a line when they fail. The FMEA for the refrigeration compressor, a large screw compressor, lists bearing wear, rotor contact, oil contamination, motor faults and coupling misalignment as key failure modes. Vibration monitoring at the bearing housings covers bearing wear, rotor problems and misalignment well. Oil analysis covers contamination and wear debris. Motor current analysis and winding temperature cover motor faults. The compressor gets a multi-signal approach because its failure would stop the whole plant. The packaging gearboxes are fast, critical and have a history of bearing failures. A vibration sensor on each, sampling at a frequency high enough to see bearing defects, is the obvious choice. The ammonia pump has a history of seal leaks; seal failure is better detected by leak detection and pressure monitoring than by vibration, though vibration helps with bearing and cavitation issues. The air compressor is a packaged unit whose own controller already monitors temperatures and pressures; the cheapest first step is to connect that controller's existing data rather than add sensors. The conveyor motors, individually inexpensive and quickly replaced, are better handled by holding spares and perhaps monitoring current at the motor control centre, rather than placing a vibration sensor on each. The result is a monitoring plan that looks quite different from the one a sensor starter kit would have produced. It concentrates on a handful of assets, uses several kinds of signal, reuses existing data where possible, and deliberately leaves many machines unmonitored. It is also defensible: every sensor has a named failure mode it is there to catch, and every unmonitored critical asset has a reason. The discipline of leaving things out The hardest part of this stage is restraint. Sensors are cheap enough that it is tempting to instrument broadly and let analytics find the problems. That temptation should be resisted, for three reasons. First, every sensor creates ongoing cost and attention: batteries to replace, mounts to check, data to store, alerts to triage. A thousand poorly chosen sensors produce a thousand streams of noise that someone must eventually look at or ignore. Second, broad, unfocused instrumentation makes it harder to demonstrate value. When a pilot is judged on whether it caught failures, monitoring low-risk equipment dilutes the evidence. Third, and most importantly, unfocused monitoring encourages the belief that data will reveal its own meaning. It rarely does. A vibration trend means something because an analyst knows what failure mode it is watching and what changes in that failure mode look like. Without that framing, data becomes a lake into which effort disappears. Starting narrow does not mean staying narrow. A focused first phase that catches real faults on genuinely critical assets earns the credibility and the knowledge to expand. It also teaches the organisation what its own P–F intervals look like, which sensors work in its environment, and how its people respond to alerts, all of which inform the next phase better than any vendor's brochure. The next two chapters turn to the signals themselves, beginning with the most informative of them. Chapter 3: Listening to Vibration If you could choose only one signal to monitor rotating machinery, it would be vibration. Every rotating machine vibrates, and the pattern of that vibration is a detailed record of what is happening inside it. Imbalance, misalignment, looseness, bearing defects, gear damage, electrical problems in motors, flow problems in pumps and fans: nearly all of them leave a recognisable signature. Vibration analysis has been the core technique of industrial condition monitoring for half a century, and the arrival of cheap wireless sensors has made it accessible on equipment that could never justify a dedicated analyst's attention before. Understanding vibration well enough to make good decisions about it does not require a degree in signal processing. It does require a grasp of a few physical ideas, because those ideas determine which sensor to buy, where to put it, how often to sample, and what the resulting numbers mean. What vibration is and how it is measured Vibration is oscillating motion. A point on a bearing housing moves back and forth, very slightly, many times a second. That motion can be described in three related ways. Displacement is how far the surface moves, usually measured in micrometres. Velocity is how fast it moves, usually in millimetres per second. Acceleration is how quickly that velocity changes, usually in metres per second squared or in multiples of g, the acceleration due to gravity. These three quantities are mathematically linked, but they emphasise different frequencies. For the same severity of vibration, displacement is largest at low frequencies, acceleration is largest at high frequencies, and velocity sits in between. In practice, displacement is useful for slow-moving machinery and for measuring shaft motion inside journal bearings, typically using non-contact proximity probes. Velocity has long been the standard measure of overall machine vibration severity across the range of frequencies where imbalance, misalignment and looseness show up. Acceleration is essential for detecting early-stage bearing and gear faults, which produce short, sharp impacts that live at high frequencies. The most common sensor in industrial vibration monitoring is the accelerometer, which produces a signal proportional to acceleration. Velocity can then be calculated by integrating that signal mathematically. Two main families of accelerometer matter for retrofit projects. Piezoelectric accelerometers contain a crystal that produces an electrical charge when stressed. A small mass sits on the crystal; when the sensor vibrates, the mass presses on the crystal and generates a signal. These sensors are robust, have a wide frequency range, often extending to ten kilohertz or beyond, and low noise. Many industrial versions include built-in electronics following a standard commonly called IEPE (integrated electronics piezoelectric), which allows them to be powered and read over a simple two-wire cable. They are the workhorse of wired online monitoring systems and of portable data collectors. MEMS accelerometers (micro-electromechanical systems) are tiny silicon devices of the kind found in phones. They are cheap, consume little power and can measure down to zero frequency, which piezoelectric sensors cannot. Historically they had higher noise and narrower bandwidth than piezoelectric sensors, which limited their usefulness for detecting early bearing defects. Their performance has improved substantially, and they now sit at the heart of many battery-powered wireless vibration nodes. The trade-off still matters, however: a node built around a low-bandwidth MEMS sensor may be excellent for detecting imbalance and misalignment yet blind to the high-frequency signatures of an incipient bearing fault. Datasheets should be read for the usable frequency range and noise floor, not only for the headline features. From waveform to spectrum The raw output of an accelerometer is a time waveform: a wiggly line showing acceleration changing over time. For a healthy machine it looks like a fairly regular oscillation with some noise. For a faulty one it may show repetitive spikes, beating patterns or sudden changes. Experienced analysts read waveforms directly, particularly for impacts. But the most powerful view of vibration is the frequency spectrum. A spectrum breaks the complex vibration signal down into the individual frequencies it contains and shows how strong each one is. The mathematical tool that does this is the Fourier transform, computed efficiently by an algorithm called the fast Fourier transform, or FFT. Running an FFT on a vibration waveform produces a list of frequencies and their strengths: a strong component at the shaft's rotational speed, perhaps smaller components at twice and three times that speed, a component at the gear mesh frequency, and so on. The reason the spectrum is so useful is that machine faults occur at predictable frequencies, tied to the geometry and speed of the machine. Shaft speed is the anchor. If a motor runs at 1,480 revolutions per minute, its rotational frequency is about 24.7 hertz, and analysts refer to this as "1×". Many faults appear at multiples or fractions of 1×, or at frequencies that can be calculated from component geometry. Imbalance, where the mass of a rotor is not evenly distributed around its axis, produces a strong component at 1×, mainly in the radial direction. Misalignment between a motor and the machine it drives typically produces strong components at 1× and 2×, often with significant axial vibration. Mechanical looseness, such as a cracked foundation or loose mounting bolts, produces a series of harmonics, many multiples of running speed, and sometimes half-multiples. Gear problems appear at the gear mesh frequency, which is the number of teeth multiplied by shaft speed, together with sidebands spaced at the speed of the damaged gear. Electrical problems in induction motors show up at frequencies related to the electrical supply frequency, often at twice line frequency. Rolling-element bearings have their own characteristic frequencies, determined by the number of rolling elements, their diameter, the pitch diameter of the bearing and the contact angle. There are four main ones: the ball pass frequency of the outer race (BPFO), the rate at which rolling elements pass a given point on the outer race; the ball pass frequency of the inner race (BPFI); the ball spin frequency (BSF); and the fundamental train frequency (FTF), which is the rotation rate of the cage. A defect on the outer race produces an impact every time a rolling element passes over it, so it appears at BPFO. Bearing manufacturers publish these frequencies for their bearings, or provide calculators, and condition monitoring software often contains libraries of bearing geometries. The main signatures that analysts use to distinguish common faults are set out in Table 2. Table 2. Common rotating-machine faults and their typical vibration signatures. Fault Dominant frequency Typical direction Notes Imbalance 1× running speed Radial Phase steady; amplitude rises with speed Misalignment 1× and 2× running speed Axial and radial Phase differs across coupling Mechanical looseness Many harmonics of 1×, sometimes ½× multiples Often vertical Waveform may look truncated or erratic Rolling bearing defect BPFO, BPFI, BSF, FTF and harmonics Radial Best seen early in high-frequency envelope Gear damage Gear mesh frequency with sidebands Radial and axial Sideband spacing identifies the damaged gear Induction motor electrical fault Twice line frequency and related bands Radial Disappears instantly when power is removed Seeing bearing faults early: envelope analysis Bearing defects present a puzzle. The impacts they produce are small and repeat at relatively low frequencies, perhaps a hundred or a few hundred hertz. In a standard spectrum, those frequencies can be buried under the much larger vibration from imbalance, misalignment and the process itself. Yet each impact also excites the natural resonances of the bearing and its housing, ringing them like a struck bell at frequencies of several kilohertz. Envelope analysis, sometimes called demodulation, exploits this. The vibration signal is filtered to keep only a high-frequency band where these resonances live, then processed to extract the envelope, the outline of how the high-frequency energy rises and falls. If a bearing has an outer-race defect, the envelope rises and falls at BPFO, and an FFT of the envelope shows a clear peak there, even when the ordinary spectrum shows nothing unusual. Related techniques sold under trade names, such as spike energy or shock pulse methods, work on similar principles. Envelope analysis is the reason sensor bandwidth matters so much. If a wireless node can only sample up to, say, one kilohertz, it cannot capture the resonances on which envelope analysis depends, and it will typically detect bearing faults only at a later stage when they produce broader changes in vibration. Some modern nodes perform envelope processing on the device itself and transmit only the result, which saves battery power while preserving the early warning. A bearing failure in stages Practitioners often describe the progression of a rolling-element bearing failure in four broad stages, and the scheme is a useful way of connecting the physics to monitoring decisions. In the first stage, the defect is microscopic, often below the surface. The only detectable signals are at very high frequencies, in the ultrasonic range and above, picked up by ultrasound instruments or specialised high-frequency vibration measurements. Ordinary spectra show nothing, and the bearing may run for a long time in this condition. In the second stage, small defects begin to produce impacts that excite the natural frequencies of the bearing components and housing. Envelope analysis now reveals the defect frequencies clearly, while overall velocity readings may still look normal. This is the ideal point for detection: the fault is confirmed, the bearing still has considerable life, and there is time to plan a replacement calmly. In the third stage, the defect frequencies and their harmonics appear directly in the ordinary spectrum, often with sidebands at running speed as the defect moves in and out of the load zone. Wear debris increases, noise may become audible, and temperature may begin to rise slightly. Replacement should now be scheduled with some urgency. In the fourth stage, the bearing is close to the end. The discrete defect frequencies may paradoxically become less distinct as damage spreads around the races and the spectrum fills with broadband noise. Overall vibration rises sharply, temperature climbs, and running speed components may grow as clearances open. Failure can follow quickly. The stage scheme explains a common misunderstanding about severity. A falling bearing defect peak is not always good news; it can mean that a localised defect has spread into general damage. It also explains why measurement capability matters: a system that only sees stage three and four signals has thrown away most of the P–F interval that stage two would have provided. Severity standards How much vibration is too much? The International Organization for Standardization publishes guidance in the ISO 20816 series, which superseded the older ISO 10816 and ISO 7919 series. ISO 20816-1 sets out general principles for measuring and evaluating vibration on non-rotating parts such as bearing housings, and other parts of the series cover specific machine classes. For many common industrial machines, the guidance defines zones based on broadband velocity, from newly commissioned machinery through acceptable long-term operation to levels considered sufficient to cause damage. These zones are useful starting points, particularly for setting initial alarm limits on machines with no history. But experienced analysts treat them as a backstop rather than the main tool. A machine that has run at a steady low level for years and suddenly doubles is telling you something important, even if it remains within the "acceptable" zone. Conversely, some machines run reliably for decades at levels that a generic severity guide would flag. Trends relative to the machine's own baseline are usually more informative than absolute limits. Where and how to mount a sensor A vibration sensor can only report what reaches it. The path from the source of vibration, typically a bearing, to the sensor should be as short and stiff as possible. That means mounting on the bearing housing, close to the load zone of the bearing, rather than on a thin cover, a fan shroud or a cooling fin. On a horizontal motor, the radial direction, either horizontal or vertical, is the usual primary measurement, with axial measurements added where misalignment or thrust problems are a concern. Triaxial sensors measure all three directions at once and are common in wireless nodes. Mounting method has a large effect on usable frequency range. A sensor screwed onto a flat, spot-faced surface with a threaded stud gives the best high-frequency response. An adhesive-mounted pad, onto which the sensor is screwed, comes close and avoids drilling the machine, which is often important on legacy equipment where drilling a housing may void a warranty or require engineering approval. A magnetic mount is convenient for portable data collection but reduces the usable high-frequency range and can shift if knocked. For permanent retrofit installations, epoxy-bonded mounting pads are a common compromise. Consistency matters as much as quality. If a sensor is moved or remounted, its readings may change for reasons that have nothing to do with the machine. Mark positions clearly, record them in the asset model, and treat any remount as a new baseline. Environment also matters. Sensors must survive temperature, moisture, washdown, chemicals and physical knocks. Food plants that clean equipment with high-pressure water need appropriately sealed housings; hazardous areas with flammable gases or dust require sensors certified for those zones under the relevant schemes. These requirements narrow the field of suitable products more than many teams expect. Sampling, bandwidth and battery life Digital vibration measurement involves a basic constraint: to capture a frequency, you must sample at more than twice that frequency, a requirement known as the Nyquist criterion. In practice, a margin is added, so capturing content up to ten kilohertz requires sampling at something like twenty-five thousand samples per second. The length of the recording determines the frequency resolution of the spectrum: separating two closely spaced frequencies, such as a bearing frequency and a nearby harmonic of running speed, requires a longer recording. Wired online systems can sample continuously at high rates. Battery-powered wireless nodes cannot, because sampling, processing and transmitting all consume energy. Most wireless nodes therefore work on a schedule: they wake periodically, take a short high-rate recording, calculate summary values such as overall velocity, peak acceleration and envelope levels, transmit those, and go back to sleep. Full waveforms or spectra may be transmitted less often or on demand. Battery life of several years is commonly quoted, but it depends heavily on settings, and a node configured to send full spectra every few minutes will exhaust its battery far sooner than one that sends overall values twice an hour. This creates design choices that need to be made consciously. For a slow-developing failure mode with a P–F interval of months, a few measurements a day may be ample. For a fast-developing problem on a critical high-speed machine, wired continuous monitoring may be justified. Machines that run intermittently create another complication: a measurement scheduled while the machine is stopped is useless, and one taken during start-up may be misleading. Good systems trigger measurements on machine state, using speed, current or a signal from the controller, so that readings are comparable over time. Variable-speed machines add a final complication. Many modern drives vary motor speed continuously, which moves every speed-related frequency in the spectrum. Analysis then requires knowing the speed at the moment of measurement, either from a tachometer, from the drive, or by estimating it from the vibration signal itself. Techniques such as order tracking, which resamples the signal relative to shaft rotation rather than time, make spectra comparable across speeds. What vibration cannot tell you Vibration is powerful but not universal. It reveals mechanical faults in rotating equipment far better than it reveals electrical insulation degradation, fluid contamination, or slow thermal problems in stationary equipment. Very slow machines, such as large kilns or slow-turning mixers, produce low-energy signals that are hard to analyse with standard methods. Reciprocating machines, such as piston compressors and presses, produce complex, impulsive vibration in normal operation that makes simple spectral analysis less effective. Gearboxes with many stages generate dense spectra that require skilled interpretation. It also requires interpretation. A wireless vibration system can report that overall vibration on a motor has risen by forty percent over three weeks. It takes analysis, either by a person or by well-designed software, to say whether that rise is caused by a developing bearing defect, a change in process load, a loosened mount or a new pump installed nearby. The next chapter turns to the other signals that fill vibration's blind spots and that, combined with it, make diagnosis more certain. Chapter 4: Heat, Current and the Other Signals Vibration is the backbone of mechanical condition monitoring, but a factory is not only a collection of rotating shafts. It is also wiring, switchgear, motors, hydraulics, lubricants, compressed air, steam, heat exchangers and processes whose own behaviour reveals the health of the equipment carrying them out. Each of these has failure modes that vibration sees poorly or not at all. A sound monitoring plan picks the signal that fits the failure mode, and on the most critical assets combines several so that one confirms or questions another. This chapter covers the main alternatives: temperature, electrical current, ultrasound, lubricant condition and process data. None of them is new. What IIoT has changed is the ability to measure them continuously, cheaply and without a technician on site. Temperature and thermal imaging Heat is the universal by-product of things going wrong. Friction from a poorly lubricated bearing, resistance at a loose electrical connection, excess current in an overloaded motor, a blocked cooling passage, a failing steam trap, a degrading insulator: all show up as temperature that is higher, or sometimes lower, than it should be. Temperature can be measured in two broad ways. Contact sensors, chiefly resistance temperature detectors (RTDs) and thermocouples, measure the temperature of whatever they touch. Many large motors and machines already have RTDs embedded in windings or bearings, connected to protection relays that trip the machine on high temperature. Connecting the readings from these existing sensors to a monitoring system, so that trends are visible long before the trip point, is one of the cheapest upgrades available. For equipment without them, small wireless temperature sensors can be attached to bearing housings, motor frames, gearbox casings or pipework. Non-contact sensors measure the infrared radiation emitted by a surface. Spot infrared sensors measure one point. Thermal imaging cameras produce a picture in which each pixel represents a temperature, revealing patterns that a single sensor would miss. Infrared thermography has been used for decades in periodic surveys, particularly of electrical distribution equipment, where loose or corroded connections and overloaded circuits show up clearly as hot spots. In the United States, the 2023 edition of NFPA 70B, the National Fire Protection Association's document on electrical equipment maintenance, was converted from a recommended practice into a standard, which has raised the profile of regular thermographic inspection of electrical equipment. Thermography has its own pitfalls. The radiation a camera receives depends not only on temperature but on the surface's emissivity, its ability to emit infrared energy. Shiny, unpainted metal has low emissivity and reflects heat from other sources, which can make a hot component look cool or a cool one look hot. Analysts compensate by adjusting emissivity settings, applying high-emissivity tape or paint at measurement points, and comparing similar components under similar load. Comparison is key: three phases of the same circuit should be at similar temperatures, and a phase markedly hotter than its neighbours indicates a problem even when its absolute temperature looks unremarkable. Load matters too: a connection that is barely warm at light load may be dangerously hot at full load, so inspections should be done when equipment is working hard. Fixed thermal cameras and arrays of low-resolution thermal sensors, mounted permanently to watch switchgear, conveyor belts, kilns or electrical cabinets, have become more affordable. They can trigger alerts on temperature rise without anyone opening a panel, which also reduces exposure to electrical hazards. Infrared windows installed in switchgear doors allow periodic inspection without removing covers. The limitation of temperature as a signal for mechanical faults has already appeared in the previous chapter: in rolling-element bearings, significant temperature rise usually comes late in the failure progression. Temperature is excellent for electrical faults, lubrication starvation, cooling problems and overload, but for early bearing and gear faults it is a backstop rather than an early warning. Electrical current and motor signature analysis Electric motors drive most rotating equipment in industry, and the current they draw carries information not only about the motor but also about the load it is driving. Measuring current is attractive for retrofit because it can be done at the motor control centre, often far from the machine itself, using clamp-on current transformers that need no connection to the motor and no shutdown of the machine. The simplest use is trending. A pump whose impeller is wearing or whose filter is clogging may draw a different current for the same output. A conveyor whose bearings or gearbox are binding draws more. A fan whose belt is slipping draws less. Combined with knowledge of the process, a change in current can be an early indicator of trouble. Many variable-frequency drives already measure current, torque and power, and simply collecting this data from the drive can provide a valuable stream at almost no cost. The more sophisticated technique is motor current signature analysis, often abbreviated MCSA. Here the current waveform is captured at a high sampling rate and analysed in the frequency domain, much as vibration is. A healthy induction motor drawing current at a 50 or 60 hertz supply frequency shows a strong peak at that frequency. Certain faults modulate the current and add sidebands around it. The classic example is broken rotor bars in a squirrel-cage induction motor: cracked or broken bars produce sidebands spaced at twice the slip frequency around the supply frequency, and the size of those sidebands relative to the main peak indicates severity. Air-gap eccentricity, where the rotor is not centred in the stator, produces its own characteristic frequencies. Some mechanical faults in the driven equipment, such as certain load fluctuations, also appear in the current. MCSA can detect electrical rotor problems that vibration finds hard to identify, and it can work from a location that is safe and accessible. It is less sensitive than vibration to early bearing defects, and it requires knowledge of motor details such as the number of poles and rotor bars for accurate diagnosis. Related techniques analyse voltage as well as current, which allows power quality issues, supply imbalance and stator winding problems to be assessed. Stator insulation deserves particular mention, because winding insulation failure is a common cause of motor failure and gives warning through means other than vibration. Offline tests, such as insulation resistance and polarisation index tests carried out with the motor stopped, have been standard for decades. Online partial discharge monitoring, which detects small electrical discharges within degrading insulation, is used on larger medium- and high-voltage machines and switchgear, where the cost of a failure justifies the specialist instrumentation. Ultrasound Ultrasound refers to sound at frequencies above the range of human hearing, roughly above twenty kilohertz. Many fault processes generate it: friction and impacting in bearings, turbulent flow through a leak, electrical arcing, tracking and corona discharge, and the internal leakage of valves and steam traps. Because ultrasound is short-wavelength and highly directional, and attenuates quickly with distance, it can be used to locate the source of a problem precisely. Ultrasonic instruments convert the high-frequency signal into an audible sound and a measured level. Technicians use them in several ways. Compressed air leak surveys are perhaps the most common, and often pay for an ultrasound programme on their own: compressed air is one of the most expensive utilities in a plant, and systems commonly leak a substantial share of what is produced. Leaks hiss strongly in the ultrasonic range even in noisy environments. Steam trap surveys detect traps that are stuck open, wasting steam, or stuck closed, flooding equipment with condensate. Electrical inspections can detect arcing and corona in enclosed switchgear, sometimes where thermography cannot see because the fault is not yet producing heat. For bearings, ultrasound is particularly useful for lubrication. Over-greasing is a widespread cause of bearing failure, as excess grease raises temperature and can damage seals, while under-greasing leads to friction and wear. Technicians can use an ultrasonic instrument to listen to a bearing while adding grease a little at a time, stopping when friction noise falls to a minimum. Continuous ultrasonic sensors and smart lubrication systems that combine sensing with automatic grease dispensing are available and are spreading in industries with large numbers of bearings. Ultrasound and high-frequency vibration overlap in what they detect. Ultrasonic monitoring is often favoured for very slow machinery, where conventional vibration analysis struggles, and for its ability to detect friction and lubrication problems early. Lubricant condition Oil is a machine's bloodstream, and like blood it carries evidence of what is happening throughout the system. Laboratory oil analysis has long been a standard tool for gearboxes, hydraulics, compressors, turbines and engines. A sample is drawn from the system and analysed for several properties. The condition of the oil itself is assessed through measures such as viscosity, oxidation, acidity and additive depletion. Contamination is measured as water content and particle counts, often reported against the ISO 4406 cleanliness code, which describes the number of particles above specific sizes per millilitre of fluid. Wear is measured by the types and quantities of metal particles: iron from gears and bearings, copper from bushings, chromium, aluminium and others. Analysis of the shape and size of wear particles, called ferrography, can indicate the type and severity of wear. Oil analysis has a long P–F interval for many failure modes. Wear metals and contamination often rise well before a gearbox produces significant changes in vibration, and contamination can be corrected before it causes damage at all, which makes oil analysis a proactive tool as well as a predictive one. Its weakness is frequency: samples are typically taken monthly or quarterly, and results take days to return from the laboratory. Poor sampling practice, such as drawing oil from a drain point where sediment collects, or from a system that has been stationary, can render results meaningless. Inline oil sensors have emerged to fill the gap between laboratory samples. Sensors that measure water content, dielectric properties, particle counts or ferrous debris can be fitted permanently to gearboxes, hydraulic power units and turbines, and connected to monitoring systems. They are generally less precise than laboratory analysis and measure fewer properties, but they provide continuous trends and can trigger a laboratory sample when something changes. For critical gearboxes and hydraulic systems, the combination of inline sensing and periodic laboratory analysis is a strong approach. Hydraulics and compressed air Two kinds of fluid power system appear in almost every factory and deserve brief separate treatment, because their failure modes are poorly served by vibration alone. Hydraulic systems power presses, injection moulding machines, clamps, lifts and many machine tools. Their typical failure modes include pump wear, internal leakage past worn seals and valves, contamination of the fluid, overheating, and cavitation. Many of these show first as changes in pressure, flow, temperature and cycle time rather than in vibration. A press whose ram takes longer to reach pressure, or whose hydraulic oil runs hotter for the same duty, is often suffering internal leakage that wastes energy and will eventually degrade performance. Pressure transducers at key points, an oil temperature sensor, an inline particle counter and the cycle time already recorded by the machine controller together give a good picture of hydraulic health. Fluid cleanliness is especially important: hydraulic components have small clearances, and contamination is widely regarded as the dominant cause of hydraulic failures, so monitoring and controlling it is both predictive and proactive. Compressed air systems supply tools, actuators and process equipment throughout a plant. Compressors themselves are rotating machines suited to vibration and temperature monitoring, and many packaged compressors have controllers that already record pressures, temperatures, running hours and load cycles. The distribution system downstream is where much of the waste lies. Leaks, which ultrasound finds well, are one problem; others include pressure drops across clogged filters, failed condensate drains and dryers that no longer remove moisture. Flow meters and pressure sensors at key points in the distribution network, combined with compressor power monitoring, can reveal rising leakage as a growing baseline flow when production is idle, such as at weekends. This is a case where condition monitoring delivers energy savings as well as reliability, and where the business case can often be made on electricity costs alone. Process data as condition data The final signal source is often the richest and most overlooked: the process itself. Plant control systems already measure pressures, flows, temperatures, speeds, positions, cycle times and power consumption, and often store them in process historians. Many of these measurements reveal equipment condition indirectly. A centrifugal pump has a performance curve relating flow, head and power. As an impeller wears or a suction line clogs, the pump's operating point drifts from its curve; tracking flow, discharge pressure and power over time can reveal degradation long before the pump fails. A heat exchanger's effectiveness, calculated from inlet and outlet temperatures and flows, falls as it fouls, allowing cleaning to be scheduled before efficiency loss becomes costly. A pneumatic actuator that takes longer to stroke than it used to may have a leaking seal or a sticking valve. A hydraulic press whose cycle time slowly increases, or whose pressure takes longer to build, may have internal leakage. A compressor whose discharge temperature creeps upward may have failing valves or cooling problems. The advantage of process data is that it often already exists and requires no new sensors, only connection and analysis. The difficulty is that process signals vary for many reasons unrelated to equipment health: changes in product, recipe, throughput, ambient temperature and operator settings. Separating equipment degradation from normal process variation requires context, which is why process-based condition monitoring often uses models of expected behaviour, comparing what the equipment actually does with what it should do under the current conditions. Combining signals No single signal catches everything, and the most reliable diagnoses come from combining them. A rise in vibration at bearing frequencies, followed weeks later by a small temperature increase and an uptick in iron particles in the oil, tells a consistent story of bearing degradation and provides confidence to schedule a replacement. A rise in vibration with no change in any other signal, appearing suddenly after maintenance work, might instead indicate a loose sensor mount or a changed machine configuration. On critical assets, a combination of vibration, temperature, current and, where relevant, oil condition is common. On less critical assets, a single well-chosen signal may be enough. The selection should flow from the failure modes identified in the analysis described in Chapter 2: each important failure mode should have at least one signal with a useful P–F interval, and the most critical failure modes may justify more than one. Combining signals also makes analytics more robust. A model that looks at vibration alone is vulnerable to any change in vibration from causes other than faults. One that looks at vibration, temperature, load and speed together can distinguish a fault from a change in operating conditions far better. This is the practical basis for many claims about machine learning in predictive maintenance, a subject Chapter 6 takes up. First, however, the data has to get from the machine to wherever it will be analysed, which on a legacy factory floor is a problem in its own right. Chapter 5: Getting Data Off the Floor A sensor reading that stays on the machine is no more useful than a gauge nobody reads. To support predictive maintenance, measurements must travel from the sensor to somewhere they can be stored, analysed and seen by people who will act on them. On a modern greenfield plant, that path can be designed from the start. On a legacy factory floor, it must be threaded through a landscape of old controllers, proprietary networks, thick concrete, electrically noisy equipment, and organisational boundaries between engineering and IT that are often harder to cross than any wall. This chapter covers the practical choices: how to connect retrofitted sensors and existing systems, where to process data, which standards help different systems talk, and how to do all of it without making the plant more vulnerable to cyber attack. Two worlds: OT and IT Factories run on two kinds of technology that grew up separately. Information technology, or IT, covers business systems: email, enterprise resource planning, finance, the office network. Operational technology, or OT, covers the systems that control physical processes: programmable logic controllers (PLCs), distributed control systems, human-machine interface screens, drives, safety systems and the industrial networks that link them. The two worlds have different priorities. In IT, confidentiality usually comes first: protect the data. In OT, availability and safety come first: keep the process running and keep people safe. An IT team's instinct to patch and reboot systems promptly can be dangerous on a control system running a live process, where a reboot might stop a line or leave equipment in an unsafe state. Conversely, OT systems often run old operating systems and software that cannot be patched at all, because they are certified with a particular configuration or because the vendor no longer supports them. For decades, the standard reference for organising industrial networks has been the Purdue Enterprise Reference Architecture, usually called the Purdue model, developed at Purdue University in the early 1990s. It divides systems into levels: field devices such as sensors and actuators at the bottom; controllers above them; supervisory systems such as operator screens and historians; site operations systems; and business systems at the top. Traffic between levels is controlled, and a demilitarised zone, a buffer network, separates the plant from the enterprise network. IIoT challenges this layered picture. A wireless vibration sensor may send its data directly to a cloud platform, skipping every level in between. That is often efficient, and for sensors that are completely separate from the control system it can be acceptable. But it needs to be a deliberate decision, made with the people responsible for OT security, not a side-effect of a vendor's default configuration. Retrofit sensor connectivity Retrofit sensors can be connected in two basic ways: by cable or wirelessly. Wired connection gives the best performance and reliability. A wired vibration system can sample continuously at high rates and power its sensors over the cable. It is the norm for the most critical machinery, such as large turbines and compressors, which have long been protected by permanently installed monitoring systems. But running new cables through a working plant is expensive, especially where cable trays are full, distances are long, areas are hazardous, or work requires a shutdown. For a typical retrofit of dozens or hundreds of measurement points on general machinery, the cost of cabling can exceed the cost of the sensors many times over. Wireless connection removes most of that cost. Battery-powered sensors attach to the machine and transmit to a gateway, which connects to the plant network or directly to the internet. Installation can be quick, and adding or moving sensors is easy. The price is lower data rates, battery maintenance, and the need to ensure reliable radio coverage in an environment full of metal, which reflects and blocks radio waves. There is no single wireless standard for industrial sensing. The options trade off range, data rate, power consumption, network topology and ecosystem, as Table 3 summarises. Table 3. Common wireless options for retrofit sensor connectivity. Option Typical role Strengths Limitations Bluetooth Low Energy Short-range sensors to nearby gateway or phone Very low power, cheap, widespread Short range; gateway density needed Wi-Fi Sensors where plant Wi-Fi exists High data rate, familiar to IT Higher power; congestion; coverage gaps WirelessHART / ISA100.11a Process instrumentation in industrial mesh networks Designed for industry, mesh reliability, security Lower data rates; specialist ecosystem LoRaWAN and similar low-power wide-area Low-rate data over long distances Long range, years of battery life Very low data rate; summary values only Private LTE / 5G Plant-wide high-performance wireless Coverage, capacity, mobility Cost and complexity; spectrum licensing varies Vendor proprietary mesh Turnkey vibration and condition systems Optimised for the vendor's sensors Lock-in; limited interoperability In practice, most retrofit programmes use whatever wireless technology their chosen sensor vendor supports, which is often a proprietary protocol or a variant of a standard one. That is acceptable, provided the data can leave the vendor's system through an open interface. The questions to ask a vendor are less about the radio and more about what happens after the gateway: can raw measurements, not just alerts, be exported? In what format? Through what interface? At what cost? Radio surveys are worth doing before committing. Factory floors are hostile environments for radio: dense steel structures, moving machinery, electrical noise from variable-speed drives and welding equipment. A gateway that works perfectly in a demonstration may struggle to reach a sensor inside a steel enclosure. Mesh networks, where sensors relay each other's messages, help, as does placing gateways thoughtfully. A small pilot in the actual environment reveals more than any data sheet. Connecting existing systems Not all useful data needs new sensors. As Chapter 4 noted, PLCs, drives, motor protection relays, packaged machine controllers and process historians often already hold valuable condition information. Extracting it is a matter of connecting to those systems, which on a legacy floor may be the hardest part of the job. Older equipment speaks older languages. Modbus, a simple protocol from the late 1970s, remains widespread. Profibus, DeviceNet, older Ethernet-based protocols and many proprietary vendor protocols are common. Some machines expose no data at all except through their front-panel display. Gateways and protocol converters translate these into modern formats. Connecting to a controller must be done carefully: reading data from a PLC is generally low-risk if done correctly, but poorly configured polling can load a controller's processor, and any write access to a control system is a different order of risk that should not be part of a monitoring project. Where touching the controller is not allowed, either for warranty reasons or because of the risk of disturbing a validated system in regulated industries such as pharmaceuticals, external sensing remains the fallback. A clamp-on current transformer, a vibration sensor or a temperature sensor can provide a useful picture without involving the controller at all. This independence is one of the main attractions of the external retrofit approach. Edge, fog and cloud Once data leaves the sensor, where should it be processed? There are three broad options, and most practical architectures use a mix. Processing on the sensor itself, increasingly common in modern wireless nodes, reduces the volume of data to be transmitted. A node might compute overall vibration levels, envelope values and spectral bands, and send only these, along with full waveforms when a threshold is crossed. This saves battery and bandwidth but means raw data is not available for later analysis unless explicitly requested. Processing at the edge, on an industrial computer or gateway located in or near the plant, allows more complex analysis close to the equipment. Edge computing reduces latency, keeps working if the internet connection fails, can filter or aggregate data before it leaves the site, and helps meet requirements to keep certain data on premises. It is a natural home for high-rate data, such as continuous vibration waveforms, that would be expensive to send to the cloud. Processing in the cloud, on a remote platform, provides scale, ease of management, access from anywhere, and the ability to compare similar equipment across many sites. Most commercial predictive maintenance platforms are cloud-based. The cost is dependence on connectivity, ongoing subscription fees, and questions about data ownership and security that need clear answers. The right architecture depends on data rates, latency requirements, security policy and the organisation's IT capabilities. A common pattern is to compute features on the sensor or at the edge, send those features and occasional raw data to the cloud for trending and analytics, and keep any action that affects the control system strictly local. Speaking a common language As the number of data sources grows, interoperability becomes the central problem. A plant with three sensor vendors, two PLC brands, a historian, a maintenance management system and a cloud analytics platform can quickly end up with a tangle of point-to-point integrations, each fragile and each a barrier to change. Two standards have become central to industrial interoperability. OPC UA (Open Platform Communications Unified Architecture), published as the IEC 62541 series and maintained by the OPC Foundation, provides a vendor-neutral way for industrial systems to expose data along with its meaning. Beyond moving values, OPC UA defines information models that describe what the data represents: this value is a bearing temperature on this motor, in these units. Industry groups have produced companion specifications that define standard models for particular types of equipment, so that different vendors' machines can describe themselves consistently. MQTT (Message Queuing Telemetry Transport) is a lightweight publish-and-subscribe messaging protocol, originally developed in the late 1990s for monitoring oil pipelines over unreliable satellite links, and now an OASIS standard widely used in IoT. Devices publish messages to a central broker under named topics, and any authorised system can subscribe to the topics it needs. MQTT's simplicity and efficiency make it popular for sending sensor data to edge and cloud systems. The Sparkplug specification, now managed by the Eclipse Foundation, adds structure on top of MQTT for industrial use, defining payload formats and device state management. These technologies underpin an architectural idea that has gained popularity in manufacturing, often called the unified namespace: rather than wiring every system to every other, all systems publish their data to a central broker using a consistent, hierarchical naming structure organised by site, area, line and asset, and any application can subscribe to what it needs. Whether or not a plant adopts that pattern formally, the principle is sound: decide on a consistent model of your assets and their data early, and require every vendor to fit into it, rather than letting each vendor's model become a silo. An architecture for a typical retrofit To make these choices concrete, return to the food plant described in Chapter 2, which has decided to monitor a refrigeration compressor, an ammonia pump, two packaging gearboxes and a main air compressor, with a plan to expand later. The packaging gearboxes and the pump get battery-powered wireless vibration and temperature sensors, mounted on epoxy-bonded pads at the bearing housings. These sensors compute overall velocity, acceleration and envelope values on board, transmit them several times an hour, and send a full waveform once a day or whenever a threshold is crossed. Two gateways, positioned after a radio survey to reach sensors inside the packaging enclosures, collect the data. The refrigeration compressor, as the plant's single most critical asset, gets wired accelerometers at each bearing connected to a small online monitoring unit that samples continuously, plus an inline oil condition sensor. Its existing winding RTDs, previously wired only to a protection relay, are connected to the same unit through a spare output on the relay. The air compressor's packaged controller exposes its data over Modbus. A small protocol gateway reads it, along with current from clamp-on transformers at the motor control centre for the larger conveyor motors. All of these feed an edge server in the plant's OT network, located in a zone separated from the control system by a firewall that permits only the specific read-only connections needed. The edge server stores high-rate data locally, computes trends, and publishes summary values and alerts to an MQTT broker using a consistent naming structure of site, area, line, asset and measurement point. From the broker, data flows outward through the plant's demilitarised zone to a cloud analytics platform and to the maintenance management system, where alerts create draft work orders. No path exists from the cloud back into the control network. This is not the only reasonable design, and it is more elaborate than a first pilot strictly needs. But it shows the principles in action: the most critical asset gets the highest-performance monitoring, existing data is reused, external sensors avoid touching controllers, data is modelled consistently from the start, and the security boundary is designed rather than accidental. A simpler first phase might use the vendor's cloud directly for the wireless sensors, provided the plant has agreed the security arrangements and can later move the data into its own structure. Security is not optional Connecting industrial equipment to networks creates paths that attackers can use. The consequences of an OT security breach are not limited to data theft: they can include production stoppages, equipment damage and safety incidents. Real incidents illustrate the stakes. In 2019, the aluminium producer Norsk Hydro was hit by the LockerGoga ransomware, which forced parts of its operations to switch to manual working. In 2017, the malware known as Triton or Trisis targeted safety instrumented systems at a petrochemical facility in Saudi Arabia, the first publicly reported attack designed to interfere with industrial safety systems. In 2021, a ransomware attack on the IT systems of Colonial Pipeline in the United States led the company to shut down pipeline operations as a precaution, causing fuel shortages on the east coast. In none of these cases was a predictive maintenance sensor the point of entry, but each shows how closely business and operational systems are now coupled, and how costly disruption can be. The international reference for industrial cybersecurity is the IEC 62443 series of standards, developed with the ISA99 committee of the International Society of Automation. Among its core ideas is the division of systems into zones, groups of assets with similar security requirements, connected through conduits, controlled communication paths with defined security measures. A predictive maintenance architecture should be designed within that framework. Several practical principles follow. Keep monitoring systems separate from control systems wherever possible; a vibration network that has no path to a PLC cannot be used to attack it. Where data must flow from OT to IT or the cloud, make the flow one-way where feasible, using firewalls configured to permit only outbound traffic or, in high-security environments, hardware data diodes that physically enforce one-way communication. Require encryption and authentication for sensor and gateway communications. Change default credentials, which remain a common weakness. Keep an inventory of every connected device, including its firmware version and who is responsible for it. Plan for patching and updating gateways and edge devices, which are computers and need maintenance like any other. And treat vendor remote access with particular care, because remote support connections have been the route for many intrusions. Regulation is tightening. In the European Union, the revised Network and Information Security Directive, known as NIS2, extends cybersecurity obligations to many more sectors, including parts of manufacturing, and the Cyber Resilience Act imposes security requirements on products with digital elements sold in the EU, with obligations phasing in over the coming years. Manufacturers buying connected sensors and platforms should expect vendors to demonstrate secure design and support for security updates. The organisational connection The technical problems of connectivity are real but solvable. The harder obstacle in many companies is organisational: IT and OT teams with different priorities, budgets and vocabularies, each reluctant to own something that crosses their boundary. Predictive maintenance projects that start in the maintenance department, using a vendor's cloud platform and cellular gateway to avoid involving IT, can deliver quick results. They can also create unmanaged security risks and data silos that become expensive to untangle later. The better approach is to bring both groups in early. Agree on an architecture, a security model and an asset data model before scaling. Decide who owns the sensors, gateways and platforms, who patches them, and who responds when they fail. These conversations are slower than plugging in a cellular modem, but they are what turns a pilot into infrastructure. With data flowing, the next problem is extracting warnings from it that are early, accurate and believable. Chapter 6: From Signal to Warning The promise most often attached to industrial IoT is that artificial intelligence will watch the machines and tell you which one is about to fail. Some of that promise is real. But the path from a stream of sensor readings to a warning someone can act on has several steps, and the hardest of them are not the ones that involve the most sophisticated mathematics. This chapter follows that path, from simple thresholds through machine learning to estimates of remaining life, and spends deliberate time on the problem that sinks more programmes than any algorithmic weakness: alarms that people stop believing. A layered view of analysis ISO 13374, the international standard on condition monitoring and diagnostics data processing, describes a sequence of functional layers that is a useful map. Data acquisition converts the sensor's physical signal into digital values. Data manipulation processes those values into useful forms: filtering, spectra, envelope values, statistical features. State detection compares the processed data with expectations and flags abnormal conditions. Health assessment diagnoses what is wrong and how severe it is. Prognostic assessment estimates how the condition will develop and how much time remains. Advisory generation turns all of this into recommendations for action. The layering matters because different tools fit different layers, and because weakness in an early layer cannot be fixed by sophistication in a later one. A remaining-life model fed by a poorly mounted, under-sampled sensor will produce confident nonsense. Most of the practical value in industrial programmes comes from doing the first three layers well: good data, sensible features, and reliable detection of change. Thresholds and trends The simplest form of state detection is the threshold: alert when a value exceeds a limit. For vibration, initial limits often come from the ISO 20816 zones discussed in Chapter 3. For temperature, they may come from the manufacturer's maximum operating temperature or from the trip settings of protection relays. Fixed thresholds have two weaknesses. Set conservatively, they produce alerts only when a fault is well advanced, wasting most of the P–F interval. Set aggressively, they produce frequent false alerts on machines that simply run a little rougher than average. The usual improvement is to make thresholds relative to each machine's own history. After a baseline period of normal operation, statistical limits are set at some number of standard deviations above the machine's typical level, or at a multiple of its baseline value. A machine that has run at 1.5 millimetres per second for a year and climbs to 3 deserves attention, even if 3 is well within a generic acceptable zone. Trend-based alerts go a step further. Instead of asking whether the current value is high, they ask whether the value is rising, and how fast. A steadily increasing envelope level at a bearing's outer race frequency is a clearer warning than any single reading, and its slope provides a first, crude estimate of how quickly the fault is developing. Thresholds and trends are often dismissed as unsophisticated. They are also transparent, easy to explain to technicians, and, when applied to well-chosen features on well-mounted sensors, remarkably effective. Many experienced reliability engineers argue that the majority of the value of continuous monitoring comes from these simple methods applied consistently across many machines. The problem of operating context Machines do not run in a single condition. A pump's vibration depends on flow and pressure; a motor's temperature depends on load and ambient temperature; a gearbox's vibration depends on speed and torque; an extruder's behaviour depends on the material it is processing. A threshold that ignores these variations will either miss faults when the machine is lightly loaded or raise false alarms when it is working hard. There are several ways to handle this. The simplest is to take measurements only in a defined operating state, for example only when a motor is running above a certain load, so that readings are comparable. Another is to set different baselines for different operating modes, such as different products or speeds. More advanced is to model the expected value of a signal as a function of operating conditions, and alert on the difference between actual and expected. If a model predicts that a bearing should run at 62 degrees given the current load and ambient temperature, and it is running at 70, the residual of eight degrees is a cleaner indicator of trouble than the raw temperature. This modelling of expected behaviour is where machine learning often adds genuine value, because the relationships between operating conditions and normal sensor readings can be complex and machine-specific. Machine learning: what it does well Machine learning is a broad label for methods that learn patterns from data rather than being explicitly programmed. In predictive maintenance, three broad uses can be distinguished. The first is anomaly detection, also called novelty detection. A model learns what normal operation looks like from historical data, often across many sensors and operating conditions at once, and flags data that does not fit. Techniques range from straightforward statistical methods, such as principal component analysis and multivariate distance measures, to clustering, isolation forests and neural network approaches such as autoencoders, which learn to reconstruct normal data and flag inputs they reconstruct poorly. The great advantage of anomaly detection is that it does not need examples of failures; it needs only examples of normal operation, which are abundant. Its weakness is that it says only that something is unusual, not what. An anomaly might be a developing bearing fault, a new product, a sensor problem, or a change made during maintenance. The second use is fault classification. Given labelled examples of different faults, a supervised model learns to distinguish them: this pattern is imbalance, this one misalignment, this one an outer-race bearing defect. Research datasets, such as the widely used bearing fault data from Case Western Reserve University, have spawned a very large literature of classification models with impressive reported accuracy. Translating that accuracy to the factory floor is harder than the literature suggests. Laboratory datasets often contain artificially seeded faults on a single test rig under controlled conditions; real machines have different geometries, mounting, loads and noise, and models trained on one rig often perform poorly on another. The third use is prognostics, estimating remaining useful life, which is discussed below. The scarcity of failure data Underlying all of these difficulties is a fundamental problem: on well-maintained equipment, failures are rare. A plant may have hundreds of motors, but in any given year only a handful will develop a particular fault, and each fault will look a little different because of differences in machine, load and environment. Supervised machine learning thrives on many labelled examples; predictive maintenance generally offers very few. When failures do occur, they are often poorly documented in the maintenance management system, so the exact fault, the date it began, and the action taken are uncertain. Several approaches help. Anomaly detection sidesteps the need for failure labels. Physics-informed features, such as bearing defect frequencies and gear mesh sidebands, encode expert knowledge so that a model needs fewer examples to learn what matters. Pooling data across many similar machines, sometimes across many companies, as large vendors with installed bases of sensors do, increases the number of examples. Transfer learning, where a model trained on one machine or dataset is adapted to another, is an active research area. Simulation and digital twin models can generate synthetic failure data, a subject for the next chapter. The most important practical response, however, is organisational: record failures properly. Every time a monitored machine fails or a fault is found during maintenance, the work order should state what was found, where, and how severe it was, ideally with a photograph of the damaged component. Over time, this turns a monitoring programme into a learning system. Without it, no algorithm can improve. Remaining useful life Prognostics aims to estimate how long a component will continue to function before failure, its remaining useful life (RUL). This is the most ambitious form of predictive maintenance and the one most often promised in marketing. Research in this area has been shaped by benchmark datasets, most famously the turbofan engine degradation simulation data generated with NASA's C-MAPSS software and released by the NASA Ames Prognostics Center of Excellence, introduced in a 2008 paper by Abhinav Saxena and colleagues. These datasets simulate engines degrading to failure under various conditions and have been used to develop and compare thousands of prognostic models. Prognostic methods fall roughly into three groups. Physics-based models describe the degradation mechanism directly, such as crack growth under cyclic stress or wear rate under load, and project it forward. Data-driven models learn degradation trajectories from historical run-to-failure data and match current trends against them. Hybrid models combine the two, using physics to constrain the form of the degradation and data to calibrate it for a particular asset. For industrial practice, two caveats are essential. First, RUL estimates are inherently uncertain, and any useful estimate must come with a range rather than a single number. An estimate of "between three and eight weeks, with high confidence that it is more than two" is actionable; "failure in 34 days" is false precision. Second, for most general industrial machinery, the information needed for reliable RUL estimation, including good run-to-failure histories and detailed load data, does not exist. In those cases, a trend-based estimate of how quickly a fault is worsening, interpreted by an experienced analyst, is often as good as a more formal prognostic model and easier to trust. False alarms and the arithmetic of trust Imagine a plant with 200 monitored machines. In a typical month, two of them develop a significant fault. The monitoring system is good: it detects 90 percent of real faults, and on a healthy machine it raises a false alert only 2 percent of the time in a month. These sound like excellent figures. Now do the arithmetic. Of the two machines with real faults, the system will detect, on average, 1.8. Of the 198 healthy machines, 2 percent, or about four, will raise false alerts. So in a typical month the maintenance team receives around six alerts, of which fewer than a third are real. If each alert requires a technician to investigate, two out of three investigations will find nothing. This is the base-rate problem, familiar from medical screening, and it explains why so many monitoring programmes lose credibility. When faults are rare, even a small false positive rate produces more false alarms than true ones. After a few months of investigating alerts that turn out to be nothing, technicians start to ignore them, and the next real fault is missed. Several remedies help, and the strongest programmes use them all. Reduce the false positive rate through better data quality: secure mounts, measurements taken in consistent operating states, automatic detection of sensor faults such as a disconnected cable or a flat battery. Require confirmation, such as a sustained trend over several readings or agreement between two signals, before escalating. Grade alerts by severity and confidence, so that a minor anomaly is logged for review while a strong, rising, multi-signal warning triggers immediate action. Route alerts first to a trained analyst, whether in-house or a remote monitoring service, who screens them before they reach the shop floor. And feed back the outcome of every investigation, so that thresholds and models can be tuned to the machines' real behaviour. The goal is not to eliminate false alarms, which is impossible without also missing real faults, but to keep them at a level where every alert reaching a technician is worth his or her time. Credibility is the currency of a predictive maintenance programme, and false alarms spend it fast. Explainability and the analyst's role A technician asked to shut down a production line on the basis of an alert will reasonably ask why. "The model says so" is not a satisfying answer, particularly when the shutdown costs money and the technician's own experience says the machine sounds fine. This is why explainability matters in industrial analytics. A good alert says not only that something is wrong but what: elevated envelope energy at the outer-race defect frequency of the drive-end bearing, rising for three weeks, with a corresponding increase in temperature. That gives the technician something to check with a portable instrument, something to look for when the bearing is removed, and a reason to trust the next alert. This also defines the continuing role of the human analyst. Even as software has taken over routine data collection and screening, the diagnosis of complex faults, the setting of priorities, and the communication of findings to maintenance and production still benefit greatly from expert judgement. Certification schemes, such as those defined under the ISO 18436 series for vibration analysts and other condition monitoring specialists, set out levels of competence for this work. Many companies that cannot justify an in-house analyst use remote monitoring services, in which a vendor's analysts review data from many plants and send diagnosed reports. Generative AI tools have recently been added to many monitoring platforms, offering natural-language summaries of alerts and suggested actions drawn from manuals and maintenance history. Used carefully, they can make expert knowledge more accessible to less experienced staff. They do not change the underlying requirements: good data, sound features, honest uncertainty and a clear line from the evidence to the recommendation. A fluent explanation of an unreliable alert is still an unreliable alert. What good analytics looks like Stripped of jargon, a sound analytics approach for a retrofit programme looks something like this. Measure the right signals on the right machines, in consistent operating states. Compute physically meaningful features. Establish baselines for each machine. Detect change with a combination of baseline-relative thresholds, trends and, where justified, anomaly models that account for operating conditions. Screen alerts through people who understand the machines. Diagnose with the help of spectra and multi-signal evidence. Estimate urgency from the rate of change. Record what was found when the machine is opened. And use those records to improve the system. None of this precludes advanced methods. On fleets of similar machines with rich data, machine learning can genuinely improve detection and reduce false alarms. But advanced methods are built on the foundation of these basics, not a substitute for them. The same is true of the most ambitious concept in the field, the digital twin. Chapter 7: Digital Twins Without the Hype Few terms in industrial technology have been stretched as far as "digital twin." It has been applied to three-dimensional models of factories, live dashboards of sensor readings, physics simulations of jet engines, scheduling optimisers, virtual commissioning environments for automation software and the entire data platform of a manufacturing company. When a word means everything, it means nothing, and a team asked to "build a digital twin" can spend a year and a large budget without ever agreeing on what it was supposed to do. Yet underneath the noise lies a genuinely useful idea. Applied with discipline, it can make predictive maintenance more accurate, more explainable and more valuable. This chapter tries to separate that idea from the marketing, and to show where twins earn their keep on a legacy factory floor and where they do not. Where the idea came from The concept is generally traced to Michael Grieves, who in 2002 presented a model for product lifecycle management at the University of Michigan that included a real space, a virtual space and a flow of data and information between them. He later described this as the conceptual basis of the digital twin. The term itself gained prominence through NASA, which used it in technology roadmap documents around 2010, envisioning a highly detailed, continuously updated simulation of a specific spacecraft that would mirror the life of its physical counterpart and predict its future performance. NASA's thinking built on a long history in the space programme of maintaining ground-based replicas of spacecraft systems, used to reproduce and solve problems in flight. From aerospace, the idea spread into manufacturing, energy and infrastructure, and with the rise of IIoT platforms it became a central marketing term. In the process it acquired a range of meanings, from the very ambitious to the very modest. A useful set of distinctions One of the clearest attempts to impose order came from a 2018 review by Werner Kritzinger and colleagues, published in the proceedings of an IFAC symposium on manufacturing. They distinguished three levels of integration between a physical object and its digital counterpart, based on how data flows between them. A digital model is a digital representation of a physical object with no automatic data exchange. A CAD model or a simulation into which engineers enter data by hand is a digital model; changes in the physical object are not reflected unless someone updates the model. A digital shadow has an automatic one-way flow of data from the physical object to the digital one. Sensor readings update the digital representation, but nothing flows back automatically. Most condition monitoring systems, including the dashboards often sold as twins, are digital shadows. A digital twin, in this stricter sense, has automatic data flow in both directions: the physical object updates the digital one, and the digital one can influence the physical, for example by adjusting control settings. By this definition, very few industrial systems described as twins are twins at all, and for predictive maintenance on legacy equipment, a two-way automatic link would rarely be desirable. Most plants do not want a maintenance model altering the settings of a production machine on its own. The useful distinction, however, is not about the label. It is about what the digital representation contains and what questions it can answer. Another dimension cuts across the data-flow distinction: the kind of model inside the twin, and whether it describes the asset's physics, its data, or simply its identity and history. The main variants are set out in Table 4. Table 4. Kinds of digital representation relevant to maintenance. Kind What it contains Typical use in maintenance Effort to build and keep current Information model Asset identity, structure, components, documents, history Single source of truth for an asset; context for alerts Low to moderate; mostly data discipline Data-driven model Statistical or machine learning model of normal behaviour Anomaly detection, expected-value residuals Moderate; needs good historical data Physics-based model Equations or simulation of mechanics, heat, fluids, wear Expected behaviour under new conditions; fault signatures; life consumption High; needs engineering expertise Hybrid model Physics structure calibrated with live data Remaining-life estimates, what-if analysis High; needs both data and expertise The information twin: the unglamorous core The most valuable digital representation for most plants is also the least exciting: a structured, accurate model of each asset and everything known about it. What it is, where it is, what it is made of, which bearings are fitted at which positions, which sensors are mounted where and in which direction, what its operating limits are, what its failure modes are, what work has been done on it, what was found, and what its condition data looks like over time. Without this, condition data floats free of context. An alert from sensor 0x4F7A on "Motor 12" is meaningless to a technician who does not know which machine that is, which bearing the sensor sits nearest to, or that the bearing was changed last month. Bearing defect frequencies cannot be calculated if nobody knows which bearing is fitted. Diagnosis cannot draw on history if work orders are not linked to the asset. Fleet comparisons cannot be made if similar machines are not identified as similar. Building an information model on a legacy floor is laborious. Nameplates must be read, sometimes cleaned first. Drawings must be found or recreated. Bearing types must be identified, sometimes only when a machine is next opened. Asset hierarchies in the maintenance management system must be checked and corrected. Sensor positions must be photographed and recorded. None of this involves advanced technology, and all of it is essential. Standards are emerging to make such models consistent and portable. In Germany's Industrie 4.0 programme, the Asset Administration Shell has been developed as a standardised digital representation of an asset, holding submodels for its identification, technical data, documentation, condition and so on, and has been progressing through international standardisation as IEC 63278. OPC UA companion specifications, mentioned in Chapter 5, define standard information models for many classes of equipment. ISO 23247 provides a framework for digital twins in manufacturing, setting out reference architecture rather than a specific technology. Even where a plant does not adopt these standards fully, borrowing their structure helps avoid building yet another proprietary silo. Physics-based twins and what they add At the other end of the spectrum, physics-based models describe how an asset behaves according to engineering principles. A model of a rotor-bearing system can predict vibration responses to imbalance or misalignment. A thermal model of a motor can predict winding temperature from load, ambient temperature and cooling conditions. A fatigue model can estimate how much of a component's life has been consumed by the loads it has actually experienced. A fluid model of a pump can predict its performance curve and how wear changes it. These models can add value to predictive maintenance in several specific ways. They can provide expected behaviour under conditions not seen in historical data. A data-driven model trained on last year's operating conditions may not know what normal looks like after a process change. A physics model can extrapolate, within limits, to new loads, speeds and temperatures. They can generate synthetic fault data. Because real failures are rare, physics models can simulate how particular faults, such as a bearing defect of a given size, would appear in the vibration signal on a specific machine, providing training examples for diagnostic models. Results depend heavily on the model's fidelity, and synthetic data must be validated against real faults where possible. They can estimate life consumption. For components whose degradation depends on accumulated load, such as bearings under variable loading, gears, shafts subject to fatigue, or insulation subject to thermal ageing, a model fed with measured loads can estimate how much life has been used and how much remains, independently of whether a fault is yet detectable. The bearing life calculations in ISO 281, based on load and speed, are a simple example of this kind of reasoning, and more detailed models extend it. Thermal ageing of motor insulation, which roughly follows the principle that a sustained increase in operating temperature substantially shortens insulation life, can be tracked from measured winding temperatures. They can answer what-if questions. What happens to bearing life if line speed increases by ten percent? If maintenance is deferred to the next planned shutdown in six weeks, what is the risk of failure? Such questions are the realm of decision support, where the value of a twin is not in the model itself but in the decisions it informs. Physics-based models are expensive to build and require engineering expertise and accurate information about the asset: dimensions, materials, component specifications. They must also be maintained as the asset is modified. For legacy equipment whose drawings are lost and whose components have been replaced with substitutes over the years, building an accurate physics model can be very difficult. The investment is justified for high-value assets with well-understood degradation mechanisms, such as large turbines, compressors, critical gearboxes and expensive spindles, and rarely for a general population of pumps and fans. Hybrid twins and calibration The most promising practical approach for critical assets combines physics and data. A physics model provides the structure, the relationships that should hold between loads, speeds, temperatures and vibration, while live data calibrates it to the specific machine and updates it as the machine ages. Techniques for this updating range from simple parameter fitting to formal methods such as Kalman filtering and Bayesian updating, which combine model predictions with measurements while tracking uncertainty. A hybrid model of a motor might, for instance, use a thermal network model to predict winding temperature, calibrated against measured temperatures so that it reflects the motor's actual cooling performance. If measured temperatures begin to exceed predictions at the same load, the residual suggests degraded cooling or increased losses, and the model can estimate the effect on insulation life. A hybrid gearbox model might use load measurements and a fatigue model to estimate life consumption, while vibration analysis detects when actual damage begins, at which point the forecast is updated. Hybrid models are also more explainable than purely data-driven ones. When the model's structure reflects engineering principles, deviations can be traced to physical causes, which makes their outputs easier for maintenance staff to trust. A worked example: the main drive of a rolling line Consider a steel processing plant with a cold rolling line whose main drive, a large motor and gearbox installed several decades ago, is the constraint on the whole site. The gearbox was last overhauled long enough ago that nobody now working at the plant was involved. A replacement would take many months to obtain. The plant's question is simple and expensive: can the gearbox safely run until the line's next major shutdown, eighteen months away, or must it be opened sooner? Vibration monitoring is already in place, and it shows no clear signs of gear or bearing damage. But that only says that detectable damage has not yet begun; it says nothing about how much fatigue life the gear teeth and bearings have consumed. The loads on the drive vary widely with the product being rolled, and over recent years the plant has rolled harder grades at higher speeds than the line was designed for. A hybrid model offers a way to answer the question more rigorously. Engineers begin with an information model: gear tooth counts, bearing types and positions, materials where known, the overhaul history and the record of past inspections. They then build a load history from data the line already records, such as motor torque and speed from the drive and the product schedule, to estimate the loads on each gear stage and bearing over the years. Using standard methods for gear and bearing life calculation, they estimate the fraction of design life each component has consumed under those loads. Finally, they calibrate the model against what is actually observed, the vibration levels and the oil analysis results, and against the condition found at the last inspection. The result is not a precise prediction, but it is far better evidence than intuition. It might show, for example, that the intermediate-stage bearings have consumed much more of their calculated life than the others because of the heavier products, while the gears have ample margin. That would point to a targeted decision: increase monitoring frequency and oil sampling on those bearings, prepare replacement bearings and a job plan for a short, focused intervention at a minor shutdown, and defer the full overhaul to the planned major shutdown. It would also allow a what-if analysis of how changing the product mix would affect the timeline. This is a twin earning its keep: a specific, high-stakes question, an asset whose failure would be catastrophic, a degradation mechanism that depends on accumulated load, and a decision that the model genuinely improves. The same effort spent on a population of small pumps would be wasted. When a twin is worth building The useful test for any proposed twin is the question it answers. If the answer is vague, such as "provide visibility" or "enable insights," the twin will probably become an expensive dashboard. If the answer is specific, such as "estimate the remaining life of the main compressor's thrust bearings given actual load history, to decide whether the overhaul can be deferred to next year's shutdown," it can be designed, built and evaluated against that purpose. A second test is the cost of keeping it current. A twin is only valuable while it reflects the physical asset. Every modification, component substitution, sensor change and process change must be reflected in the model. Organisations that underestimate this maintenance burden end up with twins that drift away from reality and are quietly abandoned. A third test is whether a cheaper method would answer the same question well enough. If trending a vibration feature against a baseline would give adequate warning of the failure modes that matter, a physics model adds cost without adding value. Twins earn their keep where simpler methods fall short: where failure modes depend on accumulated loads rather than detectable damage, where decisions about deferral or life extension carry high stakes, or where operating conditions change in ways that make historical baselines unreliable. For most legacy plants, the sensible sequence is clear. Build the information model first, because everything else depends on it. Add data-driven models of normal behaviour for monitored assets as data accumulates. Reserve physics-based and hybrid models for the small number of assets where the stakes and the questions justify them. Call any of these a digital twin if it helps win budget, but judge them by the decisions they improve. That judgement is made not by the technology but by the organisation that has to act on it, which is the subject of the final chapter. Chapter 8: Making It Stick A predictive maintenance programme produces exactly one thing of value: a repair carried out at the right time instead of the wrong one. Everything else, sensors, gateways, platforms, models and twins, exists to make that happen. If the warning arrives and the repair does not follow, the investment has bought nothing but data. That is why the outcome of most IIoT maintenance projects is decided not by the technology but by the organisation that receives its output. The pattern is familiar enough to have a name in industry, "pilot purgatory": a promising trial that detects a few faults, impresses the steering committee, and then never scales, because nobody redesigned the work around it. This chapter is about avoiding that fate. From alert to work order Follow a single alert through a plant. A wireless sensor on the drive-end bearing of a packaging machine gearbox reports rising high-frequency envelope energy. The monitoring platform flags it. What happens next? In a plant where predictive maintenance works, the path is defined in advance. An analyst, perhaps an in-house reliability engineer or a remote monitoring service, reviews the alert within a set time, examines the spectrum, confirms a probable outer-race defect, estimates severity and urgency, and writes a recommendation: replace the bearing within four weeks, check lubrication in the meantime, increase monitoring frequency. That recommendation becomes a work order in the maintenance management system, linked to the asset, with the condition data attached. A planner checks that the bearing is in stores, or orders it, prepares a job plan with the tools and procedures needed, and estimates labour. A scheduler negotiates a window with production, perhaps during a planned changeover next week. The job is done. The technician records what was found when the bearing was removed, ideally with a photograph of the damage. The analyst compares that with the diagnosis, and the alert is closed. In a plant where it does not work, the alert arrives in an email inbox that nobody owns, or on a dashboard nobody looks at during the night shift. Or it reaches a supervisor who cannot judge its importance and has no authority to stop the line. Or it becomes a work order with no priority, sitting in a backlog behind dozens of others until the bearing fails anyway. Or the bearing is replaced, but nobody records what was found, so nobody learns whether the alert was right. Every step in the first path needs an owner, a process and a time limit. Who reviews alerts, and within how long? What are the criteria for each level of urgency? Who can approve stopping production to make a repair, and on what evidence? How are predictive work orders prioritised against other work? What must the technician record afterwards? These are not technical questions, and no vendor can answer them for you. Writing them down, agreeing on them with production and maintenance leaders, and checking that they are followed is a large part of the real work of implementation. The planning foundation Predictive maintenance only pays if the time it buys is used. The warning creates a window, and planning and scheduling turn that window into an efficient, low-risk repair. A plant whose maintenance organisation operates in permanent firefighting mode, with most work unplanned and the backlog uncontrolled, will find that predictive warnings simply become another stream of urgent jobs competing for attention. This is why experienced reliability practitioners often say that predictive maintenance should not come first. Basic maintenance discipline does: a functioning work order system, a planning function that prepares jobs before they reach technicians, a schedule agreed with production, stores that hold the right spares, and preventive tasks that are actually completed. Without this foundation, sensors tell you about problems you cannot respond to effectively. Spares deserve particular attention. One of the less obvious benefits of predictive maintenance is the possibility of holding fewer spares, because warnings give enough time to order parts as needed. But that benefit only materialises if lead times are known and the P–F interval comfortably exceeds them. For long-lead or obsolete parts on legacy equipment, the opposite may be true: monitoring may reveal that a critical spare should be held, or that a component should be reverse-engineered before it is needed. Designing a pilot that can scale A good pilot is designed to answer specific questions and to be extended if the answers are positive. That means choosing scope, success criteria and a path to scale before the first sensor is installed. Scope should follow the criticality and failure mode work described in Chapter 2. A pilot of perhaps twenty to fifty assets, selected because they are critical and prone to detectable failure modes, is large enough to have a fair chance of catching real faults within a few months and small enough to manage closely. Monitoring only low-risk equipment because it is easy guarantees an inconclusive result. Success criteria should be defined in advance and include both technical and organisational measures. How many real faults were detected, and with how much warning? How many alerts were false? How many detected faults resulted in planned repairs, and how quickly? What was found when equipment was opened, and did it match the diagnosis? How much time did analysts and technicians spend handling alerts? Did the technology hold up physically, with sensors staying mounted, batteries lasting and communication remaining reliable? The path to scale should be considered from the beginning. Will the architecture handle ten times as many sensors? Will the vendor's pricing remain affordable at scale? Does the platform integrate with the plant's maintenance management system, or will every alert have to be copied by hand? Is the data model consistent with what other sites use? Is IT security satisfied? A pilot that succeeds technically but cannot be extended because of cost, integration or security problems has wasted much of its value. The stages a programme typically passes through, and the questions that need answering at each, are summarised in Table 5. Table 5. Stages from pilot to sustained programme. Stage Main aim Key questions Typical failure Assessment Choose where to start Which assets and failure modes matter most? Starting with convenient rather than critical assets Pilot Prove detection and response Did we catch real faults and act on them? Technology tested, workflow ignored Integration Connect to maintenance systems Do alerts become planned work automatically? Manual hand-offs that break under load Scale Extend to more assets and sites Do cost, security and data models hold? Vendor lock-in, unsustainable cost per point Sustain Make it routine and self-improving Are failures recorded and models tuned? Sensors decay, alerts ignored, champions leave Measuring value honestly The economic case for predictive maintenance is usually presented as avoided downtime: a failure detected and fixed in a planned window instead of causing an unplanned stoppage. That is real, but measuring it honestly is surprisingly hard. When a fault is detected and repaired before failure, the failure never happens, so its cost is a counterfactual. Would the bearing really have failed? When? With what consequences? Enthusiastic programmes are tempted to count every detection as an avoided catastrophe, which undermines credibility with finance departments that have seen such claims before. A more defensible approach combines several kinds of evidence. For each detected fault, record the condition found when the equipment was opened, which provides direct evidence that intervention was needed. Estimate the likely consequence of failure conservatively, based on the plant's own history of similar failures. Track plant-level indicators over time, such as unplanned downtime hours on monitored assets, the share of maintenance work that is planned rather than reactive, mean time between failures, and maintenance cost per unit of output, while acknowledging that many other factors also affect them. Compare monitored and unmonitored assets of similar types where possible. The Society for Maintenance and Reliability Professionals publishes a set of standardised maintenance and reliability metrics, and using recognised definitions makes comparisons over time and between sites more credible. Overall equipment effectiveness, or OEE, which combines availability, performance and quality, is widely used in manufacturing and captures some of the benefit through the availability component. Costs should be counted fully: sensors, gateways, installation labour, platform subscriptions, integration work, analyst time, battery replacements, and the internal time spent on management and training. Cost per monitoring point per year is a useful metric for comparing approaches and planning scale. A simple worked example shows the shape of the calculation. Suppose a plant's history shows that a critical gearbox on a packaging line fails roughly once every two years, and each failure causes about twelve hours of unplanned downtime plus damage to the shaft and housing. Suppose the plant values an hour of downtime on that line, including lost margin, overtime and scrap, at a figure the finance team has agreed. The expected annual cost of failure is then half a failure per year times the cost of twelve hours of downtime, plus half the cost of secondary damage. If monitoring detects most such faults early enough to replace the bearing during a planned changeover, the avoided cost is most of that figure, minus the cost of the planned repair. Compare that with the annual cost of monitoring the gearbox, and the case is either strong or weak, with no need for industry-wide headline numbers. Multiply the exercise across the pilot assets, and the business case rests on the plant's own data. People and skills Predictive maintenance changes jobs. Technicians who were valued for their ability to fix breakdowns quickly are now asked to work to planned schedules on equipment that has not yet failed. Analysts who once collected data with handheld instruments now review alerts from hundreds of sensors. Planners must handle a new kind of work order. Production managers must agree to stop equipment that appears to be running perfectly well. Each of these changes can meet resistance, and the resistance is often rational. A technician who has been told by sensors that a machine was about to fail, opened it, and found nothing will be sceptical next time. A production manager who stops a line for a predicted failure that was not confirmed will be reluctant to do it again. Trust has to be built through accurate alerts, transparent reasoning and visible results, such as showing the damaged bearing that was removed before it failed, and through involving experienced staff in choosing what to monitor and setting alert criteria. Skills need investment. Vibration analysis, thermography, ultrasound and oil analysis each have established training and certification paths, and even organisations that outsource analysis need staff who understand the results well enough to act on them. Data and IT skills are needed to manage platforms, integrations and security. And the people with deep knowledge of the plant's legacy equipment, often the longest-serving technicians, are a resource to be captured before they retire, both by recording what they know in the asset information model and by pairing them with newer staff. Vendors and lock-in The IIoT market includes large automation and industrial companies, specialist condition monitoring firms, start-ups offering wireless sensors with cloud analytics, and cloud and software platforms. Many offer complete packages of sensors, gateways, platform, analytics and remote analyst services, sometimes on subscription. These can be the fastest route to value, particularly for plants without in-house expertise. The risk is dependence. If the sensors only work with one vendor's gateway, the gateway only sends data to that vendor's cloud, and the data can only be viewed in that vendor's application, then the plant's condition history is effectively owned by the vendor. Changing supplier means starting again. Pricing may rise once the vendor knows switching is costly. The vendor may be acquired or may discontinue the product. Sensible contracts and architecture reduce this risk. Insist on access to raw data, not just alerts, through documented open interfaces such as OPC UA, MQTT or standard APIs, and on the right to export complete history in usable formats. Keep the asset information model under the plant's own control. Prefer sensors and gateways that can work with more than one platform. And consider which capabilities are core to the business and should be developed in-house, and which can sensibly be bought as a service. Buying monitoring as a service A growing share of plants do not build monitoring capability at all, but buy it as a service. A provider installs and owns the sensors, operates the platform, has its own analysts review the data, and delivers diagnosed reports and recommendations, all for a subscription fee per asset or per monitoring point. Some contracts go further and link fees to outcomes such as availability. The model has real attractions for plants that lack analysts and cannot easily hire them. It converts a capital project into an operating cost, brings expertise that a single site could not justify, and gives access to the provider's experience across many similar machines. A remote analyst who has seen hundreds of similar gearboxes is often better placed to interpret an unusual spectrum than a generalist at the plant. The risks mirror those of lock-in. If the provider owns the sensors and the data, the plant's condition history may leave with the provider at the end of the contract. Outcome-based fees require careful definitions of what counts as an avoided failure, which returns to the measurement problem discussed above. And no external service can carry out the plant's own part of the workflow: the planning, scheduling, decision to stop equipment and recording of findings remain the plant's responsibility. A service that delivers excellent diagnoses to an organisation that does not act on them will fail just as surely as an in-house system. The sensible test is the same as for any purchase. Does the contract give the plant its data, in usable form, both during and after the arrangement? Are response times and alert quality measured? Is there a clear interface between the provider's recommendations and the plant's work order system? And is someone inside the plant accountable for making sure that recommendations are acted on and their outcomes reported back? Sustaining the programme The final failure mode is decay. Sensors fall off or are removed during repairs and not reinstalled. Batteries die. Gateways are unplugged during refurbishments. Assets are modified and the information model is not updated. Thresholds set during the pilot are never tuned. The champion who drove the project moves to another job, and nobody replaces him or her. Within two or three years, the system is still running but nobody trusts it, and alerts are quietly ignored. Preventing decay requires treating the monitoring system as an asset in its own right, with its own maintenance. Sensor health should be monitored automatically, with alerts for loss of communication, low battery and suspicious data. Sensor reinstallation should be part of every repair job plan on monitored equipment. Periodic reviews should check alert performance, update thresholds, and confirm that the list of monitored assets still reflects current criticality. And ownership should be institutional, with defined roles and budgets, rather than depending on one enthusiastic individual. When this is done, the programme becomes self-improving. Each detected fault, confirmed by what was found on opening the machine, sharpens the thresholds and models. Each recorded failure fills a gap in the plant's knowledge of its own P–F intervals. Each success builds trust and makes the next intervention easier to approve. That compounding of knowledge, more than any single technology, is what distinguishes a mature predictive maintenance programme from a collection of sensors. Conclusion The technology for predictive maintenance on legacy equipment is no longer the hard part. Accelerometers small enough to glue to a bearing housing, wireless links that run for years on a battery, clamp-on current sensors that need no shutdown, edge computers that cost less than a day of downtime, and cloud platforms that can trend thousands of measurement points: all of it is available, proven and affordable. A plant manager who wants to start monitoring an old machine next month can do so. What the preceding chapters argue is that this abundance changes where the difficulty lies rather than removing it. When sensors were expensive and analysts scarce, the constraint was capacity: there was only enough money and expertise to watch a few machines, so choices were forced. Now that capacity is cheap, the constraint is judgement. It is easy to instrument broadly, collect a great deal of data and still miss the failure that stops the line. The programmes that succeed are those that impose on themselves the discipline that cost once imposed. What follows Several practical conclusions follow from the argument, and they apply whether a plant is starting from nothing or trying to rescue a stalled pilot. Start from failure, not from technology. The question is never which sensor to buy but which failure, on which critical asset, you are trying to catch, and what physical signal reveals it earliest. Criticality ranking and failure modes analysis are old tools, and they remain the right first step. A plan built on them can explain why every sensor is where it is and why every unmonitored critical asset is unmonitored. Respect the P–F interval. The value of a measurement depends on how early in a failure's development it detects change and how often it is taken. A temperature sensor that sees a bearing fault two days before seizure and a high-frequency vibration measurement that sees it two months before are not equivalent, even though both will appear on a vendor's list of supported failure modes. Measure in a way that leaves time to act. Treat the information model as infrastructure. Knowing exactly what each asset is, what is fitted to it, where each sensor sits and what has been done to it is what turns data into diagnosis. On legacy equipment this knowledge has to be assembled patiently, one nameplate and one repair at a time. It is the unglamorous core of any digital twin worth the name, and it outlasts any particular vendor or platform. Design connectivity and security together. Data must leave the machine, but it must not open a path into the control system. Decisions about wireless technologies, edge processing, protocols and cloud platforms are best made once, with IT and OT in the same room, against a consistent data model and a security framework such as IEC 62443, rather than separately by every vendor that installs a gateway. Prefer simple analytics done well to sophisticated analytics done badly. Baseline-relative thresholds and trends on physically meaningful features, measured in consistent operating states, catch most developing faults on most general machinery. Machine learning and physics-based models add genuine value where data is rich and stakes are high, but they rest on the same foundations and cannot repair a weak one. Guard credibility as the programme's most valuable asset. Because faults are rare, false alarms will outnumber true ones unless the system is deliberately designed to prevent it. Every alert that reaches a technician should be worth his or her time, should explain itself, and should be checked against what is found when the machine is opened. Redesign the work, not only the monitoring. A warning has value only when it becomes a planned repair. That requires defined ownership of alerts, planning and scheduling capability, spares strategy, and the authority to stop equipment on evidence. Plants in permanent firefighting mode should expect predictive maintenance to deliver little until basic maintenance discipline is in place. What remains unsettled Some questions in the field are genuinely open. Remaining useful life estimation for general industrial machinery is still more promising in research than reliable in practice, largely because the run-to-failure data it needs is scarce and the operating conditions of real machines vary so widely. Fleet-level learning, in which vendors pool data across thousands of machines at many companies, may change that, but it raises questions of data ownership and competitive sensitivity that the industry has not resolved. Interoperability standards such as OPC UA information models and the Asset Administration Shell are maturing, but adoption on legacy equipment is uneven, and many plants will live with a patchwork for years. Generative AI tools that summarise alerts and draw on maintenance history are spreading quickly; whether they meaningfully improve decisions, or mainly make uncertain alerts sound more confident, will be clearer after more plants have used them for longer. And cybersecurity regulation, particularly in Europe, is tightening in ways that will raise the bar for connected industrial products and the plants that buy them. There is also a deeper uncertainty about skills. Much of the knowledge that makes condition monitoring work, how a particular gearbox sounds, which machine runs rough by nature, what a certain spectral pattern means on a certain asset, lives in experienced people who are retiring. Monitoring systems can capture some of this, through well-built asset models, recorded diagnoses and documented thresholds. They cannot capture all of it, and plants that fail to transfer knowledge to newer staff may find that their sensors are producing data that nobody remaining can fully interpret. The machine everyone is afraid of The introduction began with the old machine that everyone in the plant fears. The argument of this booklet is that the fear can be replaced by knowledge, and that the route to that knowledge is less exotic than the marketing suggests. Identify the machine as critical, because everyone already knows it is. Work out how it fails, with the people who have repaired it. Choose the signals that reveal those failures early, and mount the sensors properly. Get the data out safely. Watch for change against the machine's own history. Make sure that when a warning comes, someone with the authority to act receives it, understands it and acts in time. And when the machine is opened, record what was found, so that the next warning is believed. None of this is beyond any competent maintenance organisation. All of it is made faster, cheaper and more precise by the industrial internet of things. The machine will still eventually need repair. The difference is that the repair will happen on a planned Tuesday, with the part on the shelf, rather than on a Saturday night with the line stopped and the customers waiting. That difference, repeated across the critical assets of a plant, is what modernising the factory floor actually means. Notes Chapter 1. The six failure patterns are described in F. Stanley Nowlan and Howard F. Heap, Reliability-Centered Maintenance (United Airlines, report for the U.S. Department of Defense, 1978). The P–F interval and its use in setting inspection frequency are developed in John Moubray, Reliability-Centred Maintenance, 2nd edition (Butterworth-Heinemann, 1997). The downtime estimate is from Siemens, The True Cost of Downtime 2024 (Siemens AG / Senseye, 2024). Chapter 2. The criteria for a process to be called reliability-centred maintenance are set out in SAE JA1011, Evaluation Criteria for Reliability-Centered Maintenance (RCM) Processes (SAE International). General guidance on condition monitoring programmes is given in ISO 17359, Condition monitoring and diagnostics of machines — General guidelines. Chapter 3. Vibration severity evaluation on non-rotating parts is covered by ISO 20816-1, Mechanical vibration — Measurement and evaluation of machine vibration — Part 1: General guidelines (2016), which replaced ISO 10816-1. The theory of envelope analysis and bearing diagnostics is treated in depth in Robert B. Randall, Vibration-based Condition Monitoring: Industrial, Automotive and Aerospace Applications (Wiley, 2011), and in Robert B. Randall and Jérôme Antoni, "Rolling element bearing diagnostics — A tutorial," Mechanical Systems and Signal Processing 25, no. 2 (2011): 485–520. Chapter 4. Fluid cleanliness codes are defined in ISO 4406, Hydraulic fluid power — Fluids — Method for coding the level of contamination by solid particles. Electrical maintenance practice in the United States is addressed by NFPA 70B, Standard for Electrical Equipment Maintenance (National Fire Protection Association, 2023 edition). Chapter 5. The OPC UA specifications are published as IEC 62541; MQTT version 5.0 is an OASIS standard (2019). Industrial cybersecurity requirements are set out in the ISA/IEC 62443 series. The Triton malware incident is analysed in public reports by several security firms and government agencies, and the Norsk Hydro and Colonial Pipeline incidents were widely reported by the companies themselves and in the press. Chapter 6. The functional layers of condition monitoring data processing are defined in ISO 13374, Condition monitoring and diagnostics of machines — Data processing, communication and presentation. The turbofan degradation datasets are introduced in Abhinav Saxena, Kai Goebel, Don Simon and Neil Eklund, "Damage propagation modeling for aircraft engine run-to-failure simulation," International Conference on Prognostics and Health Management (IEEE, 2008). A widely cited survey is Andrew K. S. Jardine, Daming Lin and Dragan Banjevic, "A review on machinery diagnostics and prognostics implementing condition-based maintenance," Mechanical Systems and Signal Processing 20, no. 7 (2006): 1483–1510. Personnel certification for vibration analysts is covered by the ISO 18436 series. Chapter 7. The distinction between digital model, digital shadow and digital twin is from Werner Kritzinger, Matthias Karner, Georg Traar, Jan Henjes and Wilfried Sihn, "Digital Twin in manufacturing: A categorical literature review and classification," IFAC-PapersOnLine 51, no. 11 (2018): 1016–1022. Michael Grieves discusses the origins of the concept in Michael Grieves and John Vickers, "Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems," in Transdisciplinary Perspectives on Complex Systems (Springer, 2017). The manufacturing framework is ISO 23247, Automation systems and integration — Digital twin framework for manufacturing (2021). Rolling bearing life calculation is defined in ISO 281. Chapter 8. Standardised maintenance and reliability metrics are published by the Society for Maintenance and Reliability Professionals in its SMRP Best Practices metrics compendium. Further Reading John Moubray, Reliability-Centred Maintenance, 2nd edition (Butterworth-Heinemann, 1997). The standard industrial account of RCM, with the clearest treatment of failure patterns, the P–F interval, hidden failures and the logic of choosing maintenance tasks. Still the best single starting point for anyone deciding what to monitor. R. Keith Mobley, An Introduction to Predictive Maintenance, 2nd edition (Butterworth-Heinemann, 2002). A practical, plant-oriented survey of vibration, thermography, oil analysis and other techniques, with sound advice on setting up and running a programme. Older than the IIoT era, but the fundamentals have not changed. Robert B. Randall, Vibration-based Condition Monitoring: Industrial, Automotive and Aerospace Applications (Wiley, 2011).** The authoritative technical text on vibration diagnostics, including spectral analysis, envelope methods and gear and bearing diagnostics. Demanding but rewarding for engineers who want to understand what their software is doing. Andrew K. S. Jardine, Daming Lin and Dragan Banjevic, "A review on machinery diagnostics and prognostics implementing condition-based maintenance," Mechanical Systems and Signal Processing 20 (2006).** A widely cited academic survey that organises the field of data acquisition, processing, diagnostics and prognostics, and remains a good map of the methods. Werner Kritzinger and colleagues, "Digital Twin in manufacturing: A categorical literature review and classification," IFAC-PapersOnLine 51 (2018).** A short, clear paper whose distinction between digital models, shadows and twins is a useful antidote to vague usage of the term. F. Stanley Nowlan and Howard F. Heap, Reliability-Centered Maintenance (1978).** The original report from United Airlines. Long, but its reasoning about why time-based overhaul often fails is still worth reading in the original, and it is freely available from several public archives. ISA/IEC 62443 series. The reference framework for industrial automation and control system security. The overview parts and the ISA's own introductory material are the most accessible entry points for engineers who are not security specialists. Siemens, The True Cost of Downtime 2024. An industry report estimating the scale of unplanned downtime costs across major sectors. Useful for context, best read alongside a plant's own downtime figures. Hashtags: #IndustrialIoT #IIoT #PredictiveMaintenance #DigitalTwins #Industry40 #SmartManufacturing #IndustrialAutomation #ConditionMonitoring #AssetPerformanceManagement #ReliabilityEngineering #ReliabilityCenteredMaintenance #FailureModeAnalysis #FMEA #AssetCriticalityAnalysis #VibrationAnalysis #MachineConditionMonitoring #IndustrialSensors #WirelessSensorNetworks #EdgeComputing #IndustrialDataAnalytics #MachineLearning #AnomalyDetection #RemainingUsefulLife #EquipmentFailurePrediction #DigitalTwinTechnology #IndustrialConnectivity #OperationalTechnology #IndustrialCybersecurity #SmartFactories #IndustrialAssetManagement

Download the Book (PDF): Introduction In August 2016 a team at Google reported in the Journal of the American Medical Association that a deep neural network, trained on roughly 128,000 retinal photographs graded by ophthalmologists, could detect referable diabetic retinopathy with sensitivity and specificity comparable to the specialists who had labelled the images. Within two years the United States Food and Drug Administration had authorised a different system, IDx-DR, to make that call without a physician reviewing the image at all. It was the first time a regulator had permitted software to issue a diagnostic decision on its own. A decade of announcements followed: algorithms that read chest radiographs, that flag strokes on CT scans before a radiologist opens the study, that predict kidney injury two days ahead, that design molecules, that fold proteins. By the spring of 2026 the FDA's public list of AI-enabled medical devices ran to more than fifteen hundred entries, roughly three quarters of them in radiology. Yet anyone who has worked inside a hospital in that period knows a stranger fact. Most clinicians, most days, do not feel that medicine has been transformed. The sepsis alert fires and is dismissed. The chest radiograph is still read by a person, who may or may not glance at the software's heat map. The drugs reaching patients in 2026 were, with a handful of interesting exceptions, discovered by the methods of 2010. The distance between what machine learning can do on a curated dataset and what it does for a patient on a ward is the most important fact about the field, and it is the subject of this book. The argument The controlling idea is simple to state and surprisingly hard to act on. In medicine, a model's accuracy on held-out data is where the work begins, not where it ends. A prediction becomes useful only when it is coupled to an action that someone is able and willing to take, at a moment when taking it changes what happens to the patient, and only when the model has been shown to hold up in the place where that action occurs. Every successful application described in the following chapters satisfies those conditions. Nearly every disappointing one fails at least one of them, often while posting excellent numbers on a benchmark. That claim cuts across the three areas the book covers, which otherwise look as though they share nothing but a technique. In diagnostic imaging, convolutional networks and their successors reached specialist-level performance on narrowly defined tasks years ago. The open questions are no longer about whether a network can see a nodule but about who acts on what it sees, how its errors interact with human readers, what happens when the scanner or the population changes, and whether finding more abnormalities actually leads to better outcomes rather than more biopsies. The large Swedish randomised trial of AI-supported mammography screening, which reported its primary results in 2026, matters precisely because it measured something downstream of accuracy: whether women screened with AI support went on to develop fewer cancers between screening rounds. In predicting patient deterioration from electronic health records, the gap between benchmark and bedside is wider and more instructive. Health records are not measurements of patients; they are traces of what clinicians did and when they did it. A model trained on them can learn, without anyone intending it, that patients who are about to be treated for sepsis are patients for whom a blood culture has just been ordered. It will then predict sepsis brilliantly in retrospect and arrive too late to help in practice. The widely deployed Epic Sepsis Model, examined independently in 2021, performed far worse than its developer's figures suggested. A different system, TREWS, evaluated prospectively across five hospitals, was associated with lower mortality when clinicians responded to its alerts promptly. The difference between the two lies less in their algorithms than in how each was built, validated and woven into clinical work. In drug discovery, the most technically sophisticated of the three, graph neural networks treat molecules as networks of atoms and bonds and learn to predict properties such as how tightly a compound binds to a protein target. Structure prediction systems such as AlphaFold, recognised by the 2024 Nobel Prize in Chemistry, and newer co-folding models that predict binding affinity alongside structure have changed what computational chemists can attempt. But here, too, the benchmark is not the patient. Widely used affinity datasets turned out to contain enough overlap between training and test sets that models could score well by memorisation. And the hard part of drug development, the attrition of candidates in phase II trials because the biology was wrong or the drug was unsafe, sits downstream of anything a binding model predicts. The first AI-discovered drug candidates to report phase II efficacy data, including Insilico Medicine's rentosertib for idiopathic pulmonary fibrosis in 2025, are genuinely interesting results. They are also small, short trials that illustrate how far the pipeline extends beyond the model. Precision medicine, the promise of matching treatment to the individual rather than the average patient, connects all three. It requires reading images and genomes and records together, and it requires a kind of inference that most machine learning does not perform: estimating not who is at risk but who will benefit from intervening. That distinction, between prediction and causal effect, recurs throughout the book because it is where well-meant systems most often go wrong. What the book covers and what it leaves out The chapters run in a deliberate order. The first establishes what a medical prediction actually is: what counts as ground truth, how base rates change the meaning of an accurate test, and which performance measures matter for clinical decisions. Without that grounding the rest of the book cannot be read critically. Two chapters then take up computer vision in radiology, first as a technical achievement and its characteristic failure modes, then as a matter of deployment, regulation and human-machine interaction. Two more do the same for deterioration prediction from health records, moving from the peculiar nature of the data to the sociotechnical system into which alerts are delivered. Two chapters on drug discovery follow: one on how graph neural networks and structure models represent molecules and predict binding, and one on what has actually happened when those methods have been used to find drugs. A chapter on precision medicine draws the three strands together around the problem of individual treatment effects. The last chapter turns to evidence, regulation and governance: what it should take for a clinical AI system to be trusted, and who decides. Two omissions are worth stating plainly. The book says comparatively little about large language models, which since 2023 have attracted more attention than everything else in this field combined. They appear where they bear on the argument, chiefly in the final chapters, but the most consequential clinical deployments of machine learning to date have been narrower predictive systems, and the lessons those systems teach apply with greater force to general-purpose models whose failure modes are harder to characterise. The book also does not attempt a survey of products or companies. Products change quarterly; the reasons they succeed or fail do not. How to read it The reader assumed here is intelligent and curious but not necessarily trained in either medicine or machine learning: a clinician wondering what to make of the algorithm the hospital has just bought, a data scientist moving into health, a policymaker or investor trying to separate signal from noise, a patient who wants to know who is reading the scan. Technical ideas are explained from first principles where they are needed, and only where they are needed. Where the book reports a number, it is from a named, published source, listed in the notes at the end. The tone is neither boosterish nor dismissive. The achievements are real. A screening programme that finds more clinically relevant cancers while reducing radiologists' reading workload by close to half is not hype; it is a substantial public health result. A protein structure predictor that does in minutes what used to take a doctoral student years has changed biology. But the history of clinical prediction is also littered with systems that were accurate and useless, or accurate and harmful, and the field has learned, slowly and sometimes at patients' expense, why that happens. Understanding those reasons is the most practical knowledge a reader can take into a world where these tools will only become more common. CHAPTER 1 What a Medical Prediction Actually Is Every machine learning system in medicine, whatever its architecture, does the same basic thing. It takes some description of a patient, a molecule or an image, and it outputs a number: a probability that a lesion is malignant, an estimate of how strongly a compound binds, a risk that a patient will be in intensive care by tomorrow morning. Everything interesting about the field follows from three questions about that number. What exactly is it a prediction of? How was its accuracy measured? And what is anyone supposed to do when they see it? Most of the confusion in public discussion of medical AI, and a good deal of the confusion inside hospitals, comes from answering the second question carefully while leaving the first and third vague. The label is a decision, not a fact A supervised learning model learns a mapping from inputs to labels. The model's performance can never be better than the labels allow, and in medicine the labels are rarely clean facts about the world. They are themselves the products of human judgement, institutional process, and the passage of time. Consider the apparently simple task of training a model to detect pneumonia on chest radiographs. What is the label? It might be the radiologist's report, mined from free text by a language-processing tool. In that case the model learns to reproduce radiologists' reports, including their uncertainty, their variation between readers, and the errors of the text-mining tool. One of the most widely used public chest radiograph datasets, released by the US National Institutes of Health in 2017, was labelled this way; when independent radiologists later reviewed samples of its images, they found that a meaningful fraction of labels for some findings did not match what the images showed. The label might instead be a confirmed clinical diagnosis, which brings in information not visible on the image at all. Or it might be a consensus of several expert readers, which is closer to an image-based truth but costly to produce and still a matter of opinion for subtle findings. None of these choices is wrong. But each defines a different task, and a model that excels at one may not excel at the others. When a paper reports that an algorithm matched radiologists, the first question to ask is what the radiologists were being compared against. If the reference standard was the opinion of another group of radiologists, the study measured agreement, not correctness. The problem becomes sharper with outcomes defined in health records. "Sepsis" has several competing clinical definitions, and the one encoded in a dataset may be operationalised from billing codes, from combinations of antibiotic orders and blood cultures, or from organ dysfunction scores computed after the fact. Billing codes are assigned for reimbursement and are entered after discharge, sometimes by coders reading the clinicians' notes. A model trained to predict a billing code for sepsis is learning, in part, to predict documentation behaviour. A model trained on "antibiotics ordered plus culture drawn" has an outcome definition that includes the clinician's own decision to treat, which, as later chapters show, creates a dangerous circularity. In drug discovery the labels are experimental measurements, which sounds more objective. Binding affinities are reported as dissociation constants, inhibition constants, or half-maximal inhibitory concentrations, measured in different assays, by different laboratories, under different conditions. Two measurements of the same compound against the same protein in different laboratories can differ by an order of magnitude. A model whose predictions agree with experiment to within that spread may be as accurate as the data permit; claims of accuracy finer than the noise in the measurements are a warning sign rather than an achievement. So the first discipline in reading any claim about medical AI is to ask what was used as truth, who produced it, and when. A label assigned after the outcome was known, using information the model will not have when it is used in practice, is the single most common way that impressive results evaporate. Accuracy depends on who is being tested The second discipline concerns base rates. A test's sensitivity is the proportion of people with the condition whom it correctly flags. Its specificity is the proportion of people without the condition whom it correctly clears. These two numbers describe the test and are, at least in principle, stable across populations. What a clinician or patient actually wants to know is something different: given that the test is positive, how likely is it that the condition is present? That is the positive predictive value, and it depends heavily on how common the condition is in the population being tested. A worked example makes the point. Suppose a model detects a cancer with 90 percent sensitivity and 95 percent specificity, numbers that would look excellent in a paper. In a specialist clinic where a third of referred patients have the cancer, out of 300 patients 100 have the disease. The model flags 90 of them, and among the 200 without disease it wrongly flags 10. Of 100 positive results, 90 are true. In a screening population where only 1 in 200 people has the cancer, out of 20,000 people 100 have the disease. The model again flags 90 of them, but among the 19,900 without disease it wrongly flags 995. Now only 90 of 1,085 positive results are true, about 8 percent. The model has not changed. The meaning of its output has changed completely. This arithmetic explains why so many models that perform well in hospital datasets disappoint in screening or general practice, and why alarm systems in intensive care units, where events are common, can tolerate false positives that would be unbearable on general wards, where they are rare. It also explains a quieter problem. Datasets used to develop models are frequently enriched: they deliberately include more positive cases than occur naturally, so that the model has enough examples to learn from. Performance measured on an enriched test set says little about the positive predictive value that will be experienced in deployment unless it is recalculated for the real prevalence. The measures that matter, and the one everyone reports The most commonly reported performance measure for clinical prediction models is the area under the receiver operating characteristic curve, usually shortened to AUROC or AUC. It has a clean interpretation: if you pick one patient who had the outcome and one who did not, the AUROC is the probability that the model gives the first a higher score than the second. A model that guesses randomly scores 0.5; a perfect model scores 1.0. The AUROC is useful precisely because it is independent of prevalence and of any particular decision threshold. Those same properties make it a poor guide to clinical value. It summarises performance across every possible threshold, most of which no one would ever use. It says nothing about whether the predicted probabilities mean what they say. And it is notoriously insensitive in the range where clinical decisions are often made. Two models with AUROCs of 0.80 and 0.82 may differ substantially, or not at all, in how many patients they help at the threshold a hospital actually sets. Several other measures carry more of the information that clinical decisions need, and Table 1 sets out what each one answers. Table 1. Common performance measures and the questions they answer. Measure Question it answers Depends on prevalence? Typical pitfall AUROC Does the model rank cases above non-cases? No High values with useless thresholds Sensitivity What share of true cases are flagged? No Reported without specificity PPV If flagged, how likely is a true case? Yes Measured on enriched test sets AUPRC How well are rare positives found? Yes Hard to compare across datasets Calibration Do predicted risks match observed rates? Yes Rarely reported at all Net benefit Is acting on the model better than defaults? Yes Requires a stated threshold Calibration deserves particular attention because it is the measure most often missing from published work. A model is well calibrated if, among all patients to whom it assigns a 20 percent risk, about 20 percent go on to have the outcome. A model can rank patients perfectly and still be badly calibrated, assigning 60 percent risks to patients whose true risk is 15 percent. For a ranking task, such as deciding which radiographs a radiologist should read first, that may not matter. For any decision that depends on the absolute level of risk, such as whether to start a preventive drug, transfer a patient to intensive care, or counsel a family about prognosis, it matters enormously. Calibration also drifts more readily than discrimination when a model moves to a new hospital, because baseline risk differs between populations even when the relationships between predictors and outcomes are similar. Net benefit, the quantity plotted in decision curve analysis, was developed by Andrew Vickers and Elena Elkin in 2006 to answer the question clinicians actually face: given that acting on a false positive has a cost and missing a true positive has a cost, is it better to use the model, to treat everyone, or to treat no one? The threshold probability at which a clinician would act encodes how they weigh those costs. A model has value only over the range of thresholds where its net benefit exceeds both default strategies. Decision curves are not perfect, but they force the developer to say what decision the model is for, which is more than most performance tables do. Validation is a ladder, not a checkbox Performance measured on the data used to build a model is always optimistic. The question is how optimistic, and the answer depends on how the test data relate to the training data. It helps to think of validation as a ladder with several rungs. On the lowest rung is internal validation: the model is tested on a random held-out portion of the same dataset. This protects against the crudest overfitting but shares every quirk of the original data. Patients from the same hospital, imaged on the same machines, coded by the same staff, treated under the same protocols, appear on both sides of the split. If a single patient contributes several images or admissions, and the split is made by image or admission rather than by patient, the model may effectively be tested on patients it has already seen. This error has inflated results in a surprising number of published studies. Temporal validation, the next rung, tests the model on patients from a later period than those used in training. It captures some of the drift that real deployment will bring: changes in coding practice, new treatments, new equipment, shifting case mix. Geographic or external validation tests the model at different institutions altogether. Performance almost always falls at this rung, sometimes slightly and sometimes catastrophically, and the size of the drop is among the most informative numbers a study can report. Above these lie prospective validation, in which the model is run in real time on new patients without affecting their care, and then interventional evaluation, in which the model's outputs are used and outcomes are compared with care delivered without it, ideally in a randomised trial. Only that final rung answers the question that matters: does using the model make patients better off? Each rung is more expensive than the one below, which is why the literature is crowded at the bottom and sparse at the top. A systematic review of COVID-19 prediction models published in the BMJ during the pandemic, led by Laure Wynants, examined hundreds of models and found almost all at high risk of bias, with very few externally validated. That pattern, magnified by the urgency of the pandemic, is typical of the field rather than exceptional. What the model learns that no one asked it to The final discipline is the most subtle. A flexible model trained to minimise prediction error will use any regularity in the data that helps it do so, whether or not that regularity reflects the biology of disease. Machine learning researchers call this shortcut learning. In medicine the shortcuts are everywhere, because clinical data are produced by clinical processes. A model asked to predict mortality from health records will learn that patients visited by the palliative care team tend to die. A model asked to find fractures on radiographs may learn that images taken with portable machines in the emergency department are more likely to show fractures than routine outpatient films. A model asked to predict which hospital patients have a condition may learn which hospital the image came from, because prevalence differs between hospitals. Each of these associations is real in the data, improves the reported performance, and is useless or actively misleading as a basis for clinical action, because the shortcut either arrives too late or encodes an artefact that will change when the model is deployed. Shortcut learning is not a bug that can be patched once. It is a standing property of flexible models trained on observational data, and the defence against it is a habit of mind: asking, for every high-performing model, what information it actually used and whether that information will be available, unchanged, at the moment the prediction is needed. Interpretability tools such as saliency maps and feature attributions help, but they are imperfect; a heat map that highlights the lung does not prove that the network is reasoning about lung tissue rather than about the texture of a particular scanner's output in that region. The more reliable defences are careful construction of the prediction time, external validation on data produced by different processes, and deliberate stress tests in which suspected shortcuts are removed or reversed. Prediction is not the same as prescription One last distinction will run through every chapter that follows. Most clinical models predict risk: the probability that something will happen to a patient under current practice. But clinicians do not act on risk for its own sake; they act to change outcomes. The patient who matters most is not necessarily the one at highest risk but the one whose outcome would change most if something were done. Those are different quantities. A patient with metastatic cancer and failing organs may have the highest predicted mortality on the ward and the least to gain from transfer to intensive care. A patient with moderate risk and a treatable cause may have the most. A model trained on observational data predicts what happens under the treatment patterns in its training data. If clinicians in that data already treated high-risk patients aggressively, the model's estimate of risk already reflects that treatment. A famous illustration, published in the 1990s and revisited many times since, concerned a pneumonia risk model that learned that patients with asthma had lower mortality than other pneumonia patients. The reason was that asthmatic patients with pneumonia were routinely admitted to intensive care and treated intensively. Taken at face value, the model would have recommended sending them home. The lesson is not that risk models are useless but that their outputs must be read in light of the decision they are supporting. The rest of this book returns again and again to that point: in radiology, where finding more lesions is not the same as improving outcomes; in deterioration prediction, where the action an alert triggers determines whether it helps; in drug discovery, where predicting binding is not predicting therapeutic benefit; and in precision medicine, where the entire enterprise depends on estimating effects rather than risks. A model is a means of informing an action. Evaluating it without reference to the action is evaluating half of the system. CHAPTER 2 Teaching Machines to See: Computer Vision in Radiology Radiology was the first medical specialty to be told it would be automated, and the reasons were easy to see. Its raw material is digital and already stored in standard formats. Its core task, looking at an image and deciding what is there, resembles the image classification problems on which deep learning made its breakthrough around 2012. And its practitioners generate, as a by-product of every working day, millions of labelled examples in the form of reports. In 2016 the computer scientist Geoffrey Hinton remarked that people should stop training radiologists, because within five years deep learning would do better. A decade later there are more radiologists in training than there were then, the demand for imaging has grown faster than the workforce, and the majority of authorised AI medical devices are radiology tools that assist rather than replace. The prediction was wrong in an instructive way: the technology did roughly what was expected, and the work turned out to be larger than the technology. How a network learns to read an image A digital radiograph is a grid of numbers, each giving the brightness of one pixel. A CT or MRI study is a stack of such grids, forming a three-dimensional volume. The task of a diagnostic model is to map this array, often millions of numbers, to a much smaller output: a yes or no, a probability, an outline of a structure, or a measurement. The architecture that made this feasible was the convolutional neural network. Its central operation is simple. A small filter, perhaps three pixels by three, slides across the image, and at each position it computes a weighted sum of the pixels beneath it. The weights determine what the filter responds to: one set of weights produces a strong output where brightness changes sharply from left to right, marking a vertical edge; another responds to a horizontal edge, another to a spot. A layer of the network applies dozens or hundreds of such filters, producing a set of feature maps that record where in the image each pattern occurs. The next layer applies filters to those maps, combining edges into corners and curves, and the layer after that combines curves into textures and shapes. After many layers, the network's internal representation encodes complex visual concepts: the rounded density of a nodule, the blunted angle where fluid collects at the base of a lung, the branching pattern of vessels in a retina. The crucial point is that no one specifies what the filters should detect. Their weights start random and are adjusted, through millions of small steps, to reduce the difference between the network's outputs and the labels in the training data. The network discovers for itself which visual patterns are predictive. This is its power, because it can find patterns that human experts have not articulated, and its danger, because it will find any pattern that is predictive, including ones that have nothing to do with disease. Two refinements matter for medical imaging. The first is transfer learning. Training a large network from scratch requires enormous numbers of labelled images, far more than most medical datasets contain. In practice, many medical models begin with a network already trained on millions of everyday photographs, whose early layers have learned generic detectors for edges and textures, and then fine-tune it on medical images. This works surprisingly well, though it is less necessary now that large medical image collections and self-supervised methods exist. The second refinement is architectures designed for segmentation, the task of outlining structures pixel by pixel rather than classifying the whole image. The U-Net, introduced by Olaf Ronneberger and colleagues in 2015 for microscopy, became the workhorse for this task, used to outline tumours for radiotherapy planning, measure heart chambers, and quantify brain volume. Since around 2021, a different architecture, the vision transformer, has competed with convolutional networks. Instead of sliding filters, it divides the image into patches and uses an attention mechanism to let every patch exchange information with every other, which captures long-range relationships more directly. Alongside the architectural change came a change in training. Rather than relying solely on labelled examples, foundation models are pre-trained on very large collections of unlabelled images, learning by predicting masked portions of an image or by matching images with their accompanying reports, and are then adapted to specific tasks with relatively few labels. These models have improved performance on some tasks and reduced the labelling burden on many, but they have not changed the fundamental questions this chapter raises, which concern what the model has learned and whether it will hold up elsewhere. The landmark results The first wave of high-profile results, between 2016 and 2020, established that deep networks could match specialists on well-defined diagnostic tasks. Table 2 lists several of the studies that shaped the field's expectations. Table 2. Selected landmark studies in diagnostic imaging AI. Year Task Study Key finding 2016 Diabetic retinopathy Gulshan et al., JAMA Sensitivity about 87–90%, specificity about 98% on two validation sets 2017 Chest radiographs Rajpurkar et al., CheXNet Pneumonia detection compared with four radiologists 2018 Pneumonia shortcuts Zech et al., PLOS Medicine Models identified the source hospital; performance fell externally 2018 Autonomous retinal screening Abràmoff et al., npj Digital Medicine Prospective primary-care trial; first autonomous FDA authorisation 2020 Breast screening McKinney et al., Nature Fewer false positives and negatives than original readers in UK and US data 2026 Breast screening RCT Gommers et al., The Lancet (MASAI) Fewer interval cancers; higher sensitivity; lower workload The diabetic retinopathy study led by Varun Gulshan at Google is the archetype. The task was narrow and clinically meaningful, because diabetic retinopathy is a leading cause of preventable blindness and screening is limited by the availability of graders. The reference standard was a majority decision of several ophthalmologists. The network was validated on two separate datasets, one from the United States and one from France. And the results were strong enough to suggest real clinical use. Equally important, the task had a clear action attached: a positive result means referral to an eye specialist. The CheXNet paper, posted in late 2017 by Pranav Rajpurkar and colleagues at Stanford, trained a deep convolutional network on the NIH chest radiograph dataset of more than 100,000 frontal images and reported pneumonia detection exceeding the average of four radiologists on one metric. It attracted wide attention and also substantial criticism, much of it constructive: the labels were text-mined, the comparison radiologists were reading images without the clinical context they would ordinarily have, and pneumonia on a radiograph is a finding about which experienced readers frequently disagree. The episode taught the community to scrutinise reference standards and reading conditions in any comparison between model and human. Shortcuts in the pixels The study that did most to temper early enthusiasm was published in 2018 by John Zech, Eric Oermann and colleagues. They trained pneumonia-detection networks on chest radiographs from three hospital systems and tested them both within and across systems. The networks performed noticeably worse on hospitals they had not seen. More strikingly, the authors showed that a network could identify which hospital system an image came from with near-perfect accuracy, and which department within a hospital, from features such as the markers, text overlays and processing characteristics of the image. Because the prevalence of pneumonia differed between hospitals and departments, knowing where an image came from was itself predictive. The network had found a shortcut. The pandemic produced a dramatic demonstration of the same phenomenon. In 2020, hundreds of papers reported deep learning models that detected COVID-19 on chest radiographs with startling accuracy. Many had been trained on datasets in which COVID-positive images came from one set of sources and negative images from another, often including paediatric images from an entirely different public collection. A 2021 analysis by Alex DeGrave, Joseph Janizek and Su-In Lee in Nature Machine Intelligence showed that such models relied heavily on spurious features, including text markers and the positioning of the patient, and that their performance collapsed when tested on data from new sources. A systematic review led by Michael Roberts in the same journal, examining machine learning models for COVID-19 diagnosis and prognosis from chest imaging, concluded that none of the models it assessed was of potential clinical use, owing to methodological flaws and biases. Shortcuts can be subtler than hospital labels. In 2022 a team led by Judy Gichoya reported in The Lancet Digital Health that deep networks could predict patients' self-reported race from chest radiographs, CT scans and mammograms with high accuracy, even when the images were degraded to the point that clinicians could not identify the anatomy. The researchers could not determine what features the networks were using. The finding alarmed the field not because race prediction is itself useful but because it showed that models can encode demographic information that their developers do not know about and cannot easily remove, which means they can also learn to treat patients differently on that basis without anyone noticing. Shortcuts arise in pathology too, where stain colour, scanner type and tissue preparation differ between laboratories and correlate with diagnosis, and in dermatology, where images of malignant lesions are more likely to contain rulers because clinicians measure suspicious lesions. In every case the mechanism is the same: the training data contain correlations produced by the clinical process, and the network, which has no idea what disease is, uses them. Looking inside the network The natural response to shortcut learning is to ask the network what it is looking at. A family of techniques called saliency methods tries to answer that question by producing a map, overlaid on the image, of which pixels most influenced the output. The most widely used, Grad-CAM, traces the network's decision back through its final convolutional layer and highlights the regions whose features pushed the prediction up. Many commercial products display such maps alongside their outputs, and they have an intuitive appeal: if the highlighted region sits over the lesion, the network seems to be reasoning about the right thing; if it sits over a text marker in the corner, something is wrong. The appeal outruns the reliability. Saliency maps are coarse, because the final convolutional layer has much lower resolution than the image. They are unstable, sometimes changing substantially in response to small perturbations of the input that leave the prediction unchanged. Different saliency methods applied to the same network and image can highlight different regions. And a map that highlights the right region does not show that the network is using the right features within it; it may be responding to a texture or a processing artefact that happens to coincide with the lesion. A 2022 study led by Adriel Saporta and Pranav Rajpurkar in Nature Machine Intelligence systematically compared several saliency methods on chest radiographs against the localisations of radiologists and found that the methods localised pathologies considerably less accurately than a human benchmark, especially for small or irregularly shaped findings. Saliency maps remain useful as a debugging tool, a way for developers to spot gross shortcuts during development. They are much weaker as a guarantee to clinicians that a model is trustworthy, and a radiologist who treats a plausible heat map as evidence of correct reasoning may be reassured by exactly the wrong thing. The more dependable checks are behavioural: testing whether performance holds when suspected shortcut features are masked, when images come from a new site, or when the case mix changes. A model is understood by how it fails, not by what it highlights. Accuracy on what, compared with whom Comparisons between AI systems and human readers dominated the early literature and still appear in marketing materials. They deserve careful reading. A reader study typically assembles a set of cases, often enriched with positives, and asks several radiologists to read them under experimental conditions, then compares their performance with the model's. The design is useful but artificial. Radiologists in the study may lack prior images, clinical history, and the option to recommend further views, all of which they would use in practice. They know that the case set is enriched, which may change their thresholds. And the comparison of a model to individual readers is not the comparison that matters in practice, where the model will be used with, not instead of, those readers. The large mammography study published by Scott McKinney and colleagues in Nature in 2020 addressed some of these concerns by comparing an AI system with the original clinical decisions made in screening programmes in the United Kingdom and United States. It reported reductions in both false positives and false negatives relative to those decisions, and in a simulation of the UK double-reading system suggested that the AI could take the place of the second reader while maintaining performance. It was a retrospective analysis, however, and the authors were careful to frame it as a step towards prospective evaluation rather than a substitute for it. That prospective evaluation arrived from Sweden. The Mammography Screening with Artificial Intelligence trial, known as MASAI and led by Kristina Lång at Lund University, randomised more than 105,000 women attending population screening to either standard double reading by two radiologists or an AI-supported pathway. In the AI arm, a commercial system assigned each examination a risk score; low-risk examinations were read by a single radiologist, high-risk ones by two, and the AI highlighted suspicious regions for the readers. An early safety analysis published in The Lancet Oncology in 2023 found that the AI-supported pathway detected more cancers without increasing false positives while substantially reducing reading workload. The primary results, published in The Lancet in early 2026, addressed the question that matters most for screening: the interval cancer rate, meaning cancers that surface between screening rounds, which a good screen should have found. Interval cancers occurred at 1.55 per 1,000 women in the AI-supported group against 1.76 per 1,000 in the control group, a result that met the trial's non-inferiority criterion; sensitivity was 80.5 percent against 73.8 percent, with specificity the same in both arms; and the radiologists' screen-reading workload fell by 44 percent. MASAI is important not because its effect sizes are enormous but because it measured the right things, in the right setting, with the right design. It did not ask whether an algorithm could outperform a radiologist on a test set. It asked whether a screening programme using an algorithm in a specific role produced better outcomes for the population it served, and it found evidence that it did. That is the rung of the validation ladder most imaging AI has never reached. What seeing does not settle Even a well-validated detector leaves questions that accuracy cannot answer. The most important is overdiagnosis. A screening test that finds more cancers is valuable only if the extra cancers are ones that would have caused harm. Mammography already detects some ductal carcinoma in situ and slow-growing invasive tumours that would never have threatened the patient's life, and finding them leads to surgery, radiation and anxiety without benefit. An AI system that increases detection could disproportionately find such lesions. The MASAI investigators examined this concern by looking at the characteristics of the additional cancers detected and the interval cancers prevented, and reported fewer invasive and fewer larger tumours among interval cancers in the AI arm, which is reassuring. But for most imaging AI tools, which detect pulmonary nodules, incidental findings on CT, or small fractures, the question of whether increased detection improves outcomes is simply unstudied. A second question concerns what the model is not asked about. A network trained to detect pneumothorax looks for pneumothorax. It does not notice the incidental mass at the lung apex unless it was trained to. A radiologist reading the same film reports everything. Narrow tools can thus create an illusion of completeness, and a system that triages normal studies to a shorter queue may delay the reading of findings it was never designed to see. The third question is how the model's output changes human behaviour, and that question is large enough to need its own chapter. CHAPTER 3 From the Reading Room to the Workflow The question that decides whether an imaging algorithm helps patients is rarely how accurate it is. It is where in the chain of events between a scan being acquired and a patient being treated the algorithm's output enters, who sees it, and what they do differently as a result. The same detector can be beneficial in one position and harmful in another. This chapter is about those positions, about the history that should have taught the field to take them seriously, and about the regulatory and practical machinery that governs how imaging AI reaches patients. A cautionary precedent: computer-aided detection in mammography Imaging AI did not begin with deep learning. In 1998 the FDA approved the first computer-aided detection system for screening mammography, which marked suspicious calcifications and masses for the radiologist's attention. In 2001 Medicare in the United States began paying extra for mammograms read with computer-aided detection, and within a decade the large majority of screening mammograms in the country were read with it. The technology had shown promise in reader studies. Its effect in practice was another matter. A study published in the New England Journal of Medicine in 2007, led by Joshua Fenton, compared screening performance at facilities before and after they adopted computer-aided detection and found that its use was associated with lower specificity, meaning more women recalled for further testing, without a significant improvement in cancer detection. In 2015 a larger study in JAMA Internal Medicine, led by Constance Lehman and drawing on hundreds of thousands of screening mammograms in the Breast Cancer Surveillance Consortium, found that computer-aided detection did not improve diagnostic accuracy and that radiologists' sensitivity was in fact lower when they used it. By then the United States had been paying for the technology for well over a decade. Several things went wrong. The systems generated many marks per image, most of them false, so radiologists learned to ignore them, and in ignoring the false marks they sometimes ignored true ones. Some radiologists may have read less carefully, assuming the software would catch what they missed. And the decision to pay for the technology had been made on the basis of reader studies and technical performance rather than evidence of improved outcomes in screening populations. Every one of these failure modes is available to modern deep learning systems, which are more accurate but are deployed into the same human and institutional settings. Four roles for an algorithm It helps to distinguish the roles an imaging algorithm can play, because each brings different risks and requires different evidence. The first is triage: the algorithm reviews studies as they arrive and moves those likely to contain an urgent finding to the front of the radiologist's queue, or alerts a specialist directly. The best-known example is the detection of large vessel occlusion strokes on CT angiography. In 2018 the FDA granted a De Novo authorisation to Viz.ai's software for this purpose, creating a new category of computer-aided triage and notification devices. The logic is straightforward. A large vessel occlusion is treatable by mechanical thrombectomy, the benefit of which declines rapidly with time, and in many hospitals the scan may sit in a queue, or the neurointerventional team may not be notified, for precious minutes. An algorithm that notifies the stroke team directly when it detects a probable occlusion can shorten that delay. Because the radiologist still reads every study, a false negative does not remove the case from normal care; it simply fails to accelerate it. A false positive costs a specialist a few minutes reviewing images. The risk profile is favourable, and the action is clear. The second role is detection support: the algorithm marks suspicious regions on images the radiologist is reading. This was the role of old computer-aided detection and is the role of many current products, from lung nodule detection to fracture detection. Its value depends on how the radiologist integrates the marks, which is a question of human behaviour as much as of algorithm performance. The third is workload redistribution, as in the MASAI design, where the algorithm's risk score determines how much human attention each case receives. Low-risk examinations are read by one radiologist instead of two; high-risk ones get extra scrutiny. This role can free substantial specialist time but depends heavily on the algorithm's performance at the low-risk end, because a cancer scored as low risk receives less human attention than it would have under standard practice. The fourth is autonomy: the algorithm makes the call without any human reading the image. The first autonomous AI diagnostic system authorised by the FDA, in 2018, was IDx-DR for diabetic retinopathy. Its pivotal trial, published by Michael Abràmoff and colleagues in npj Digital Medicine the same year, was conducted prospectively in ten primary care sites across the United States, enrolling 900 people with diabetes who had not been diagnosed with retinopathy. Images were captured by staff without specialist training, and the system's output was compared with a rigorous reference standard based on wide-field stereoscopic photography and optical coherence tomography graded by an independent reading centre. The system reached a sensitivity of 87.2 percent and specificity of 90.7 percent against prespecified endpoints. The reason autonomy made sense here is that the alternative was not an ophthalmologist reading the image; for many people with diabetes, it was no screening at all. Most people with diabetes in the United States were not receiving their recommended annual eye examinations. A system that could be operated in a primary care office during a routine visit extended screening to people who would otherwise have gone without. That example shows the principle that should govern autonomy. The relevant comparison is not between the algorithm and the best specialist but between care with the algorithm and care as it would actually be delivered without it. Where specialists are scarce, the algorithm's accuracy need only be good enough, with errors manageable by downstream care, to produce a large net benefit. Where specialists are available and reading every case, autonomy offers little and risks much. Automation bias and the reader in the loop When a radiologist reads an image with an algorithm's output visible, two people are making a decision, and one of them is a machine whose errors have a different pattern from human errors. In principle, this complementarity is the source of benefit: the algorithm catches some things the human misses, and the human catches some things the algorithm misses. In practice, the combination performs better than either alone only if the human weighs the algorithm's output appropriately, trusting it when it is right and overriding it when it is wrong. Humans are not good at this. Automation bias, the tendency to over-rely on automated suggestions, has been documented in aviation, process control and clinical decision support for decades. In radiology it was demonstrated sharply in a study published in Radiology in 2023 by Thomas Dratsch and colleagues. Radiologists of varying experience read mammograms with suggestions from what they were told was an AI system; on some cases the suggestions were deliberately incorrect. When the suggested category was wrong, readers' accuracy fell substantially, and inexperienced readers were the most affected, though even experienced readers were influenced. The finding inverts a common assumption, that AI will be most helpful to less experienced clinicians. It may help them most when it is right and harm them most when it is wrong. The opposite failure also occurs. Clinicians who see many false alarms learn to disregard the tool, and their performance reverts to what it would be without it, minus the time spent dismissing marks. Which failure dominates depends on the algorithm's false positive rate, the way its output is displayed, whether the clinician sees it before or after forming an initial impression, and the time pressure under which they work. These are design choices, and they matter as much as the model's accuracy. One emerging practice is to present the algorithm's output only after the radiologist has recorded an initial reading, preserving an independent human judgement; another is to display calibrated probabilities rather than binary flags, giving the reader information about how confident the system is. Neither has been shown to be universally superior, which is itself a reason to evaluate the combination of human and machine in the setting where it will be used rather than assuming that a good algorithm makes good decisions. The skill that fades when the tool is present A further effect of AI assistance appears only over time, and it was documented first not in radiology but in a neighbouring field that faces the same issues. Computer-aided detection systems for colonoscopy, which highlight possible polyps on the video screen in real time, have been tested in many randomised trials and generally increase the adenoma detection rate, the proportion of procedures in which at least one precancerous adenoma is found and a standard measure of colonoscopy quality. They are among the best-evidenced AI tools in clinical use. In 2025 Krzysztof Budzyń and colleagues reported in The Lancet Gastroenterology and Hepatology an observational study at four endoscopy centres in Poland that had introduced such a system. They compared the performance of nineteen experienced endoscopists, each with more than two thousand procedures behind them, on colonoscopies performed without AI before and after the centres began using it routinely. In procedures performed without AI assistance, the adenoma detection rate fell from 28.4 percent before the tool was introduced to 22.4 percent afterwards. The study was observational, and other changes over the same period could have contributed. But the result is consistent with a plausible mechanism: clinicians who grow used to a system that draws their attention to abnormalities may become less vigilant on their own, so that when the system is absent, because it fails, because they move to a hospital without it, or because it was never designed for the case in front of them, their performance is worse than it was before they ever used it. The implications for radiology are direct. A radiologist trained in a department where an algorithm pre-marks every nodule may never develop the search pattern that an earlier generation learned by necessity. If the algorithm is reliable and always available, that may not matter. If it is not, the combined system has a hidden fragility. Deskilling is not an argument against assistive AI, whose benefits in trials are real, but it is an argument for measuring the human component of the system over time, for training that preserves independent skill, and for treating the clinician's unaided performance as something worth monitoring, alongside the algorithm's. How imaging AI is regulated In the United States, software intended to diagnose or treat disease is regulated by the FDA as a medical device. Devices reach the market by one of three main routes. The 510(k) pathway requires showing that the new device is substantially equivalent to a device already on the market, called a predicate. The De Novo pathway is used for novel devices of low to moderate risk that have no predicate; once granted, the device becomes a predicate for later ones. Premarket approval, the most demanding route, is required for the highest-risk devices and usually involves clinical trial data. The great majority of AI-enabled devices have been cleared through the 510(k) pathway. This has allowed rapid growth: the FDA's public list of AI-enabled devices, which it maintains and updates periodically, contained more than 1,500 entries by early 2026, of which about three quarters were in radiology. But substantial equivalence is a modest standard. It establishes that a new device performs comparably to a predicate on the evidence submitted, which is typically retrospective testing on a few hundred or a few thousand cases. Analyses of the list by independent researchers have repeatedly found that only a small minority of devices report prospective evaluation, that many summaries do not state how many sites contributed test data, and that information about performance across demographic groups is often absent. Authorisation tells a hospital that a device is legal to use and has met a performance bar on a test set. It does not tell the hospital that the device will perform at that level on its own patients and scanners, or that using it will improve outcomes. Europe regulates AI software as a medical device under the Medical Device Regulation, which since 2021 has classified most diagnostic software in risk classes that require assessment by a notified body. The European Union's Artificial Intelligence Act, which entered into force in 2024, adds a further layer: AI systems that are safety components of regulated medical devices are classed as high-risk and will be subject to additional requirements on data governance, documentation, human oversight and post-market monitoring, with those obligations phasing in over the following years. How the two regimes will interact in practice is still being worked out by regulators and manufacturers. Local validation and the problem of drift Because authorisation does not guarantee local performance, careful health systems conduct their own validation before going live. A typical approach runs the algorithm silently on several months of local studies, compares its outputs with radiologists' final reports and with any available follow-up, and checks performance across the hospital's scanners, protocols and patient groups. This frequently reveals problems: an algorithm that performs poorly on images from an older scanner model, or on portable radiographs from the intensive care unit, or on paediatric patients who fall outside its intended use but are routed to it by the hospital's systems. The work does not end at go-live. Imaging data change over time in ways that can degrade a model without anyone noticing. Scanner software is updated, changing image reconstruction. Acquisition protocols are revised. New patient populations are referred. The prevalence of the target condition shifts, as COVID-19 dramatically demonstrated. A model whose performance was validated last year may be performing differently this year, and because the radiologist is still reading every image, the degradation may be invisible until someone looks for it. Monitoring in practice involves tracking the rate at which the algorithm flags studies, the distribution of its scores, the rate at which radiologists agree or disagree with it, and wherever possible its performance against confirmed outcomes. Sudden shifts in any of these can signal a change in the input data. This is engineering work that sits uncomfortably within most hospital departments, which were not built to run software quality assurance, and it is one of the reasons the practical cost of deploying imaging AI is considerably higher than the licence fee. For models that are updated after deployment, regulators have had to rethink the traditional assumption that a device is fixed once authorised. In December 2024 the FDA finalised guidance on predetermined change control plans for AI-enabled device software, allowing manufacturers to specify in advance, as part of their marketing submission, the types of modification they intend to make, such as retraining on new data, and the methods they will use to validate those modifications. Changes made within an authorised plan do not require a new submission. The approach acknowledges that learning systems need to be updated and attempts to keep the updates within bounds that were reviewed in advance. What radiology teaches Radiology is the most mature domain of medical AI, and its lessons are by now reasonably clear. Narrow detection tasks can be learned to specialist level. Shortcut learning and distribution shift are pervasive and must be designed against. The benefit of a tool depends on its role in the workflow and on how human readers respond to it, and that response can make an accurate tool harmful. Authorisation is a floor, not a ceiling, and local validation and monitoring are the purchaser's responsibility. And the evidence that matters, that patients are better off, requires prospective evaluation in real screening or diagnostic pathways, which is expensive, slow and has been done for only a handful of applications. None of this means imaging AI has failed. It has become routine in stroke triage, it is entering breast screening programmes on the strength of trial evidence, and it is reducing reporting delays in departments overwhelmed by demand. But the pattern of success is specific. It comes where the task is well defined, the action is clear, the comparison is with care as it is actually delivered, and someone has taken the trouble to measure outcomes. The next two chapters turn to a domain where those conditions are much harder to satisfy. CHAPTER 4 The Record Speaks: Predicting Deterioration from Health Records Every hour on a hospital ward, somewhere in the building, a patient is quietly getting worse. Their breathing quickens a little, their blood pressure drifts down, their urine output falls, a laboratory value creeps out of range. Each change on its own is unremarkable. Together, over six or twelve hours, they announce a patient sliding towards sepsis, respiratory failure or cardiac arrest. When the slide is noticed early, a few relatively simple actions, such as fluids, antibiotics, a senior review or a transfer to a higher level of care, can often reverse it. When it is noticed late, the same patient may need intensive care, or may die. The appeal of predicting deterioration from electronic health records is that the signals of the slide are already in the computer, scattered across vital signs, laboratory results, medication charts and nursing notes, waiting to be assembled. The appeal is real, and the problem is harder than it looks, because health records are not what they appear to be. What a health record actually records An electronic health record looks like a database of measurements about patients. It is better understood as a log of what clinicians did. A blood pressure appears in the record because a nurse measured it, at a time the nurse chose, on a schedule determined by the patient's perceived acuity and the ward's staffing. A lactate level appears because a doctor was worried enough to order one. A diagnosis code appears because someone needed to bill for it, often after the patient had left. The record is a trace of human attention and institutional process, and the patient's physiology shows through only partly. This has consequences that surprise people trained on other kinds of data. In 2018 Denis Agniel, Isaac Kohane and Griffin Weber published an analysis in the BMJ of laboratory tests in a large hospital system. They found that the timing of a test, whether it was ordered and at what hour of the day, was in many cases more strongly associated with patient survival than the result of the test itself. A test ordered at three in the morning tells you that someone was worried at three in the morning. The value carries information about the patient; the fact of measurement carries information about the clinician's judgement of the patient. The same principle governs missing data. In most statistical settings, missing values are treated as a nuisance to be imputed or ignored. In health records, missingness is information. A patient with no lactate measurement is a patient whom no one thought needed one, which is a weak signal of good health. A patient whose vital signs are recorded every fifteen minutes is a patient whom the nurses are worried about. Models that use the pattern of measurement as well as the values often perform better, and they perform better because they are partly learning to read the clinicians' concern. That is not necessarily a problem. Clinicians' concern is a legitimate and often valuable signal, and a model that aggregates the subtle worries of many staff members may notice a pattern that no one of them would act on alone. But it creates two dangers. The first is that a model that learns from clinicians' actions may only confirm what clinicians already suspect, adding little to care. The second, more serious, is that if the outcome the model is trained to predict is itself partly defined by clinicians' actions, the model can learn to predict those actions rather than the underlying physiology. The leakage trap Suppose a team wants to build a model that predicts sepsis several hours before onset. They need a definition of onset. A common computational definition, derived from international consensus criteria, identifies suspected infection by the combination of a blood culture being drawn and antibiotics being given within a certain time window, then requires evidence of organ dysfunction. The time of onset is set by when these events occurred. Now consider what the model sees in the hours before onset. Shortly before the blood culture is drawn, a doctor typically examines the patient, perhaps orders a lactate, a complete blood count and a chest radiograph, and writes a note mentioning possible infection. All of these appear in the record. A model given access to them will learn that a lactate order followed by a radiograph order is strongly predictive of sepsis onset within the next hour or two. Its performance on retrospective data will be excellent. In deployment it will fire, reliably, at about the moment the clinician has already recognised the problem and started acting. It is predicting the clinician, not the disease. This is a form of label leakage, information about the outcome entering the inputs through the back door of the clinical process. It is extremely common in published deterioration models and is often not obvious from the paper. The defence is to define, with great care, what information would be available at the time a prediction is supposed to be useful, and to exclude anything that reflects a clinical decision already made about the outcome being predicted. Doing this rigorously usually reduces retrospective performance, sometimes dramatically, which creates an incentive not to do it. A model evaluated honestly against the question "would this have told the team something they did not already know, early enough to matter?" will almost always look worse on paper than one evaluated against the question "does this score correlate with the outcome?" Early warning scores and their descendants Hospitals were trying to detect deterioration long before machine learning. The early warning scores developed in the 1990s and 2000s assign points to abnormal values of a handful of vital signs, such as respiratory rate, oxygen saturation, temperature, blood pressure, heart rate and level of consciousness, and sum them. A high total triggers a response, typically a review by a senior clinician or a rapid response team. In the United Kingdom, the Royal College of Physicians introduced a standardised National Early Warning Score in 2012 and updated it as NEWS2 in 2017; it is now used across the National Health Service. These scores have important virtues. They are transparent, so a nurse can see why the score is high. They use data collected at the bedside, so they do not depend on the electronic record's integration. They are standardised, so a score means the same thing across wards and hospitals, which supports common escalation protocols. And they have been externally validated many times. Their weakness is that they are crude: a fixed set of variables with fixed thresholds cannot capture trends, interactions or the information in laboratory results. Statistical and machine learning models that use the full record can do better at discrimination, and several have been widely deployed. In the United States, the electronic health record vendor Epic distributed proprietary models, including a sepsis prediction model, to the hospitals that use its software, which together care for a large fraction of the country's inpatients. Academic groups developed their own, among them the eCART score from the University of Chicago and the Targeted Real-time Early Warning System, or TREWS, from Johns Hopkins. Kaiser Permanente in Northern California built a model called the Advance Alert Monitor. The DeepMind team, working with the US Department of Veterans Affairs, developed a deep learning model to predict acute kidney injury. Their stories, set side by side in Table 3, illustrate almost every lesson in this part of the book. Table 3. Selected deterioration prediction systems and how they were evaluated. System Target Evaluation Headline result NEWS2 General deterioration Many external validations Standard UK escalation score Epic Sepsis Model Sepsis External retrospective (Wong et al., 2021) AUROC 0.63; most sepsis cases missed DeepMind AKI Acute kidney injury Retrospective, US veterans (Tomašev et al., 2019) 55.8% of episodes predicted up to 48h ahead Kaiser AAM ICU transfer or death Staggered deployment, 19 hospitals (Escobar et al., 2020) Lower 30-day mortality after alerts TREWS Sepsis Prospective, 5 hospitals (Adams et al., 2022) Lower mortality when alerts confirmed in 3h The Epic Sepsis Model The Epic Sepsis Model was deployed at hundreds of hospitals. Its developer reported discrimination in the range of an AUROC of 0.76 to 0.83, based on its own validation. Because the model was proprietary, its inputs and training were not publicly documented in detail, and few hospitals had evaluated it independently. In 2021 Andrew Wong, Karandeep Singh and colleagues at the University of Michigan published an external validation in JAMA Internal Medicine. They examined nearly 28,000 patients across more than 38,000 hospitalisations at Michigan Medicine, where the model was in use. At the alerting threshold the hospital used, the model had an AUROC of 0.63. It failed to identify about two thirds of patients who developed sepsis. Of those it did identify, most had already been recognised by clinicians, in the sense that they had already received antibiotics, before the alert fired. And it generated alerts on about 18 percent of all hospitalised patients, most of whom did not have sepsis, creating a large burden of alerts that clinicians had to evaluate. Several factors probably contributed. The developer's validation and the Michigan validation used different definitions of sepsis onset, which can change performance substantially. The model may have been trained on data from hospitals with different practices and populations. And the evaluation question differed: a model judged by whether it correlates with eventual sepsis coding will look different from one judged by whether it alerts before clinicians act. The publication prompted other health systems to examine their own deployments, and the vendor later revised its approach, including allowing local retraining. But the episode exposed how a model could be deployed at enormous scale, influencing the care of millions of patients, on the strength of internal validation that no one outside the developer had been able to scrutinise. Predicting kidney injury two days ahead The DeepMind acute kidney injury model, published by Nenad Tomašev and colleagues in Nature in 2019, was a technical tour de force. Using records of more than 700,000 adult patients from the US Department of Veterans Affairs, a recurrent neural network processed each patient's record as a sequence of events and produced continuously updated predictions of acute kidney injury in the following 48 hours. It predicted 55.8 percent of all inpatient episodes of acute kidney injury, and 90.2 percent of those that subsequently required dialysis, up to 48 hours in advance, at an operating point where there were two false alerts for every true one. The study was widely admired and also carefully criticised. The Veterans Affairs population is predominantly male, and performance in women, who made up a small fraction of the dataset, was lower. The evaluation was retrospective, so it could not show whether the predictions would lead to actions that prevented injury; acute kidney injury is often caused by factors, such as surgery, contrast agents or sepsis, that may not be avoidable even with advance warning. And a ratio of two false alerts for each true one, applied across a hospital, generates a considerable number of alerts. None of this diminishes the technical achievement. It locates it accurately, as an excellent answer to the question of whether kidney injury can be predicted from records, which leaves open the different question of whether predicting it helps. Two systems that measured outcomes The two systems in Table 3 that reported effects on mortality share a feature that distinguishes them from the others: each was designed as part of a clinical process, not as a free-standing model. Kaiser Permanente's Advance Alert Monitor predicts the risk of unplanned transfer to intensive care or death within the next twelve hours, updating hourly for adult inpatients outside intensive care. Its alerts do not go directly to bedside staff. They go to a team of specially trained nurses working remotely, who review the patient's record and, if they judge the concern warranted, contact the rapid response team or the attending physician, following standardised workflows that include addressing goals of care. The system was rolled out in stages across nineteen hospitals, and an evaluation led by Gabriel Escobar, published in the New England Journal of Medicine in 2020, exploited the staggered deployment to compare outcomes before and after implementation. Thirty-day mortality was lower after deployment among patients whose risk reached the alert threshold, an effect the authors estimated as equivalent to roughly 520 deaths avoided per year across the system. The design was not a randomised trial, and some of the benefit may have come from the standardised response processes as much as from the model's predictions. That is not a criticism; it is the point. The model and the response were designed together. TREWS, developed by Suchi Saria and colleagues at Johns Hopkins, was evaluated prospectively at five hospitals. Its alerts appeared in the electronic record, and clinicians were asked to evaluate and either confirm or dismiss them. A study led by Roy Adams, published in Nature Medicine in 2022, examined more than 590,000 patients, among whom 6,877 had sepsis identified by the system before antibiotics were started. Patients whose alert was confirmed by a clinician within three hours had lower in-hospital mortality than those whose alert was not confirmed promptly, an adjusted absolute reduction of 3.3 percentage points and a relative reduction of 18.7 percent. A companion paper by Katharine Henry and colleagues examined clinicians' engagement with the system and reported high rates of evaluation and adoption across the hospitals. The TREWS findings are observational, comparing patients whose alerts were confirmed promptly with those whose alerts were not, and the two groups could differ in ways that adjustment does not fully capture. Critics raised exactly this concern. But the design of the system carried lessons that do not depend on the effect size. Its developers paid close attention to avoiding leakage from clinicians' actions, designed the alert to be reviewed by a clinician rather than to dictate action, and invested heavily in training and engagement. Their system achieved a precision, the proportion of alerts that were true sepsis cases, high enough that clinicians took the alerts seriously. What separates the successes Placed side by side, these systems suggest that what distinguishes a useful deterioration model from a useless one is less the sophistication of its algorithm than four other things. The first is honest construction of the prediction problem, so that the model predicts what clinicians do not already know. The second is enough precision that alerts are worth responding to; a model that fires on nearly a fifth of all patients will be ignored, however good its AUROC. The third is a defined response: someone specific receives the alert, with time and authority to act, following a process designed for that purpose. And the fourth is evaluation against outcomes in the setting of use, rather than reliance on a developer's retrospective validation. The second and third points lead into the subject of the next chapter. A deterioration model does not act on patients. It acts on clinicians, who act on patients. Its effects are therefore mediated by attention, trust, workload and the design of the organisation around it, and it can change those things in ways that its developers never intended. CHAPTER 5 Alerts, Actions and the Clinical System Around a Model A deterioration model's output is a number. To affect a patient, that number must become an alert, the alert must reach a person, the person must notice it, believe it, decide what to do, and have the time, authority and resources to do it. At every step the signal can be lost. And because the model is embedded in a system of people who learn and adapt, its presence changes the system: what clinicians pay attention to, what they document, which patients receive care, and eventually the data on which the next version of the model will be trained. This chapter is about those dynamics. They explain why models that look identical on paper can have opposite effects in practice, and why the most consequential decisions about clinical AI are often not technical decisions at all. The economics of attention Hospitals already generate more alerts than anyone can process. Drug interaction warnings, abnormal laboratory flags, bed alarms, infusion pump alarms, cardiac monitor alarms: the modern ward is a cacophony of automated warnings, the great majority of which require no action. Studies of medication alerts in electronic prescribing systems, summarised in a frequently cited 2006 review by Heleen van der Sijs and colleagues in the Journal of the American Medical Informatics Association, found that clinicians overrode most of them, with override rates commonly ranging from about half to over ninety percent. Monitor alarms in intensive care units are dominated by false and clinically irrelevant alerts. The phenomenon of alert fatigue, in which clinicians exposed to frequent irrelevant alerts become desensitised and miss the important ones, has been recognised as a patient safety hazard by regulators and accreditation bodies. Into this environment comes a new predictive alert. Its designers think of it as a valuable signal. To the clinician receiving it, it is one more interruption competing with dozens of others, and its credibility will be judged by experience. If the first ten alerts a nurse sees are for patients who are obviously fine, or obviously already being treated, the eleventh will be dismissed without much thought. This is why the positive predictive value of an alert matters more in practice than its AUROC. Suppose a hospital with 500 inpatients deploys a sepsis model and sets a threshold that flags 5 percent of patients each day, 25 alerts. If one in ten alerts identifies a patient who genuinely has sepsis and has not already been recognised, clinicians see two or three actionable alerts a day among more than twenty that are not. Whether that ratio is acceptable depends on how long each evaluation takes and who does it. The Epic Sepsis Model at Michigan, which alerted on nearly a fifth of hospitalisations, sat well past the point where most clinicians would sustain careful evaluation. TREWS, by contrast, was explicitly tuned for higher precision at the cost of catching fewer cases, on the reasoning that an alert that is trusted and acted on is worth more than two that are ignored. The threshold, then, is not a statistical parameter to be set by maximising some metric. It is a decision about how to allocate a scarce resource, clinical attention, and it should be set by asking how many alerts the responding team can actually evaluate well, then choosing the threshold that delivers that many. A model with excellent discrimination but a threshold set without reference to capacity will either flood its responders or miss most of the patients it could have helped. Who receives the alert, and what they can do The second variable is the design of the response. An alert that pops up in the electronic record of whichever physician next opens the chart, with no specified action, relies on that physician's interpretation and willingness to act. An alert that goes to a dedicated team with a defined protocol is a different intervention. Duke University's Sepsis Watch programme, which went live in 2018, is among the best-documented examples of the second approach. A deep learning model developed at Duke predicted sepsis risk for patients in the emergency department. Its outputs went not to the treating physicians directly but to rapid response team nurses, who monitored a dedicated application, reviewed flagged patients and telephoned the emergency department physicians to discuss them. Mark Sendak and colleagues described the implementation in detail, and a qualitative study by the anthropologists Madeleine Clare Elish and Elizabeth Anne Watkins, published by the research institute Data & Society in 2020, examined the work the nurses actually did. They found that the nurses' role went well beyond relaying a number. They had to translate the model's output into terms that busy physicians would find credible, to negotiate across professional hierarchies in which nurses do not usually direct physicians' decisions, and to build relationships of trust over months. The authors called this work "repair": the social labour required to fit a new technology into existing practice. None of it appeared in the model's performance metrics, and without it the model would have had no effect. The lesson generalises. Every successful deployment of a deterioration model documented in the literature has invested heavily in the human side: dedicated responders, training, feedback, and workflows that specify what should happen when an alert fires. Kaiser Permanente's remote nurses, TREWS's clinician engagement programme, and Duke's rapid response team all represent substantial ongoing costs, often larger than the cost of the model. Health systems that buy a model and switch it on, expecting the algorithm alone to improve outcomes, are buying the least important part of the intervention. Counting the work an alert creates A practical way to make these trade-offs concrete is to express a model's performance in terms of the work it creates. Santiago Romero-Brufau and colleagues at the Mayo Clinic proposed in 2015 a measure they called the number needed to evaluate: the number of patients a clinician must assess in response to alerts in order to find one true case. It is simply the reciprocal of the positive predictive value. A model with a positive predictive value of 20 percent has a number needed to evaluate of five; one with 5 percent has twenty. Unlike the AUROC, it speaks directly to the person who has to walk to the bedside. The measure can be extended to what matters even more, the number of evaluations needed to find one patient whose care actually changes as a result. Return to the leakage problem from the previous chapter. Suppose a sepsis model alerts on 100 patients, of whom 25 truly have sepsis, giving a number needed to evaluate of four, which sounds manageable. But suppose that 20 of those 25 had already been recognised by their clinicians and started on antibiotics when the alert fired. Only five alerts identified a patient whose care could be accelerated. The number needed to evaluate for a change in care is twenty, five times worse than the headline figure, and the clinicians receiving the alerts will notice the difference long before anyone calculates it. A model can therefore have respectable precision against its label and still be experienced as noise, because most of its true positives tell clinicians what they already know. The reverse calculation is equally informative. If the responding team can carefully assess, say, thirty patients a day, and the model at a given threshold produces fifty alerts, then twenty alerts will receive cursory attention or none, and the patients behind them will be no better off than without the model. The design question becomes which thirty patients the team should see, and whether the model's ranking chooses them better than the team's own judgement would. That comparison, the model's top thirty against the clinicians' top thirty, is a more honest test of value than any standard performance metric, and it is rarely made. Measured this way, outcome-oriented evaluation of a deterioration system tracks a short list of quantities over time: how many alerts fire, how many are evaluated and how quickly, how many lead to a change in management, what the change was, and what happened to the patients. These are process measures, not proof of benefit, but they reveal very quickly whether an alerting system is functioning as an intervention at all. A system in which most alerts are dismissed within seconds is not an intervention, whatever its AUROC. When the label encodes inequity In 2019 Ziad Obermeyer, Brian Powers, Christine Vogeli and Sendhil Mullainathan published a study in Science that has become the most cited example of algorithmic bias in medicine. They examined a commercial algorithm used by health systems to identify patients with complex health needs for enrolment in care management programmes, which provide extra nursing support, closer follow-up and coordination. The algorithm was applied to many millions of patients in the United States. The algorithm did not use race as an input. It predicted future healthcare costs, on the reasonable-seeming premise that patients who will need more care will cost more. The researchers found that at any given risk score, Black patients were considerably sicker than white patients, as measured by the number of active chronic conditions and by biomarkers of disease control. The reason was that, for a given level of illness, less money was spent on Black patients, because of differences in access to care, in how they were treated by the healthcare system, and in other structural factors. By predicting cost accurately, the algorithm faithfully reproduced the inequity in spending and concluded that Black patients were healthier than they were. The authors estimated that correcting this would raise the proportion of Black patients automatically identified for extra help from 17.7 percent to 46.5 percent. They worked with the developer, who reformulated the model to predict a combination of cost and health measures and substantially reduced the bias. The model was not inaccurate. It predicted exactly what it was trained to predict. The error lay in the choice of label, a proxy that was correlated with need in general but differed systematically between groups. This failure is invisible to standard validation, which checks whether a model predicts its label well, not whether the label measures what the user cares about. It recurs in many forms. Models that predict who will be diagnosed with a condition learn patterns of diagnosis, which reflect access to testing. Models that predict who will attend appointments learn the circumstances that make attendance hard. Models trained on measurements from devices that perform differently across groups inherit those differences: a study by Michael Sjoding and colleagues in the New England Journal of Medicine in 2020 found that pulse oximeters missed hypoxaemia, low blood oxygen confirmed by arterial blood gas, almost three times as often in Black patients as in white patients. Any deterioration model that uses oxygen saturation as an input absorbs that bias silently. Models that change the world they predict A subtler dynamic arises when a model is successful. Suppose a deterioration model is deployed and works: clinicians respond to its alerts, intervene early, and many patients who would have deteriorated do not. A year later, the hospital collects new data and retrains the model. What does the new data show? Patients whose features looked high-risk now frequently had good outcomes, because they were treated. The retrained model learns that these features are less dangerous than before and assigns them lower risk. It now alerts on fewer of them, they receive less early intervention, and outcomes worsen. The model's success has eroded the evidence on which it was built. This phenomenon, sometimes called performative prediction or the feedback loop problem, was described in a clinical context by Matthew Lenert, Michael Matheny and Colin Walsh in a 2019 article in the Journal of the American Medical Informatics Association with the apt title "Prognostic models will be victims of their own success, unless…". There is no simple fix. Options include retaining records of which patients were alerted and how they were treated, so that retraining can account for the intervention; reserving some form of controlled comparison; and treating the model's training data as a record of care under a particular policy, which changes when the policy changes. All of these require thinking of the model not as a static predictor but as one component of a changing system. The feedback can run in darker directions. A model that predicts poor prognosis can influence decisions to limit treatment, and if treatment is limited, the poor prognosis is fulfilled. Neurologists have worried for decades about this kind of self-fulfilling prophecy in predicting outcomes after severe brain injury, where early decisions to withdraw life support are informed by prognostic estimates that were themselves derived from patients whose life support was withdrawn. Machine learning models trained on such data inherit the prophecy and, by appearing objective, may strengthen it. Drift, shocks and maintenance Even without feedback loops, the world drifts. Clinical practice changes, populations change, documentation changes, and the electronic record itself is periodically upgraded or replaced. Studies of model calibration over time, such as work by Sharon Davis and colleagues at Vanderbilt, have shown that clinical prediction models lose calibration over periods of a few years, often in ways that standard monitoring would not detect until the error is large. The COVID-19 pandemic was a sudden and extreme shock. Hospital populations, workflows, testing practices and the meaning of many clinical signals changed within weeks. A 2021 article in the New England Journal of Medicine by Samuel Finlayson and colleagues, titled "The Clinician and Dataset Shift in Artificial Intelligence", used examples from this period to argue that clinicians needed to be able to recognise when a model's inputs had moved outside the conditions under which it was validated. Sepsis models, in particular, began alerting far more often as wards filled with patients with respiratory failure from a viral infection that the models had never seen, prompting several hospitals to suspend them. The practical implication is that a deployed clinical model needs an owner: someone responsible for monitoring its alert rate, calibration and performance against outcomes, for detecting drift, for deciding when to recalibrate, retrain or retire it, and for communicating changes to the clinicians who rely on it. In most hospitals this role did not exist when the first predictive models went live, and in many it still does not. Several academic health systems have created governance committees for clinical AI, with responsibility for approving deployments, setting monitoring requirements and reviewing incidents. This kind of institutional infrastructure is unglamorous and indispensable. A system, not a model The argument of the last two chapters can be summarised by an observation about where the leverage lies. Improving a deterioration model's AUROC from 0.80 to 0.85 is a respectable research achievement. Setting its threshold to match the responding team's capacity, routing alerts to people with time and authority to act, specifying the response, checking that the label measures what matters, watching for drift, and planning for the feedback loop will usually matter far more to patients. Those decisions belong to clinicians, nurses, managers and governance committees, not to data scientists alone. The model is the visible part of the intervention, and the smallest part. That perspective carries over directly into the second half of the book, where the setting changes from the hospital ward to the medicinal chemistry laboratory. The technology is different and so are the stakes, but the gap between a model's benchmark performance and its contribution to the real goal, a medicine that helps patients, turns out to have a very similar shape. CHAPTER 6 Molecules as Graphs: Learning to Predict Binding Most medicines work by binding to a protein. A small molecule fits into a pocket on an enzyme or receptor and blocks it, or stabilises it in a particular shape, or prevents it from binding to something else. How tightly the molecule binds, its binding affinity, is not the only property that matters for a drug, but it is usually the first one that medicinal chemists try to optimise, and it is the property on which much of the excitement about machine learning in drug discovery has centred. This chapter explains how machine learning models represent molecules and predict binding, why graph neural networks became the natural tool for the job, how those models have been evaluated, and why their benchmark results turned out to be less informative than they first appeared. What binding affinity is When a drug molecule and its target protein are mixed in solution, some molecules bind and some remain free, and at equilibrium the ratio settles at a value determined by the strength of the interaction. The dissociation constant, written Kd, is the concentration of free drug at which half of the protein molecules are occupied. A smaller dissociation constant means tighter binding: a compound with a Kd of one nanomolar binds a thousand times more tightly than one with a Kd of one micromolar. Because these values span many orders of magnitude, chemists usually work with their negative logarithm, the pKd, so that one nanomolar becomes 9 and one micromolar becomes 6. Many experimental datasets report related but not identical quantities, such as the inhibition constant Ki or the half-maximal inhibitory concentration IC50, which depends on the conditions of the assay. Binding affinity is a thermodynamic quantity. It reflects the free energy difference between the bound and unbound states, and at body temperature a tenfold change in affinity corresponds to a free energy difference of only about 1.4 kilocalories per mole, roughly the strength of a single good hydrogen bond. That free energy is the net result of many competing contributions: favourable contacts between drug and protein, the cost of stripping water molecules from both surfaces, the loss of flexibility when the drug is locked in place, changes in the protein's own motion. Small errors in estimating any of these can produce large errors in the predicted affinity. This is why predicting binding from first principles has been so difficult, and why the field has long hoped that learning from data might succeed where physics struggled. Three ways to estimate binding before machine learning Computational chemists had three main approaches before the deep learning era, and all three remain in use. Docking programs place a candidate molecule into a protein's binding site in many possible orientations and conformations, and score each pose with a fast function that sums approximate terms for shape complementarity, hydrogen bonding and other interactions. Docking is cheap enough to screen millions or even billions of compounds, and it is reasonably good at distinguishing plausible binders from implausible ones. Its scores, however, correlate only weakly with measured affinities. Free energy perturbation and related alchemical methods sit at the opposite extreme. They use molecular dynamics simulations, in which every atom of the protein, drug and surrounding water moves according to a physical force field, to compute the free energy difference between two closely related compounds by gradually transforming one into the other in simulation. When the force field is accurate and the simulation runs long enough, these relative binding free energy calculations can predict affinity differences within a series of related compounds to roughly one kilocalorie per mole, enough to be genuinely useful for deciding which analogue to make next. A widely cited 2015 study by Lingle Wang and colleagues at the software company Schrödinger, published in the Journal of the American Chemical Society, showed this level of accuracy across a range of targets, and such methods are now routine in many pharmaceutical companies. Their drawbacks are cost, since each comparison needs many hours of graphics processor time, and scope, since they work best for small changes to a known compound bound to a protein with a well-characterised structure. Between these extremes lie quantitative structure–activity relationship models, which have been used since the 1960s. They describe each molecule with a set of numerical descriptors, such as molecular weight, polarity, counts of particular chemical groups or binary fingerprints recording the presence of substructures, and fit a statistical model relating those descriptors to measured activity. These models can be accurate within a well-sampled chemical series, but their quality depends heavily on the choice of descriptors, which must be designed by hand. Why a molecule is a graph A molecule, to a chemist, is a set of atoms connected by bonds. Mathematically, that is a graph: a set of nodes connected by edges. Each node can carry features, such as element, charge, hybridisation and whether the atom is in a ring, and each edge can carry features, such as bond order and whether the bond is aromatic. This representation captures the molecule's structure directly, without the need to choose descriptors by hand. The difficulty is that standard neural networks are built for inputs of fixed size and fixed order, like the pixel grid of an image. Molecules vary in size, and there is no natural order in which to list their atoms; the same molecule can be written down with its atoms numbered in many different ways, and a sensible model must give the same answer for all of them. Graph neural networks solve this problem through an operation called message passing. The idea is easiest to grasp step by step. Begin by giving each atom a vector of numbers describing its own features. In the first round of message passing, each atom collects the vectors of its immediate neighbours, the atoms it is bonded to, combines them with information about the connecting bonds, and uses a small learned neural network to update its own vector. After one round, each atom's vector encodes information about itself and its immediate neighbourhood: a carbon now knows that it is bonded to an oxygen and two hydrogens. After a second round, each atom has received information that its neighbours gathered in the first round, so it now knows about atoms two bonds away. After several rounds, each atom's vector summarises a local chemical environment several bonds in radius, much as a chemist would recognise a carboxylic acid group or an aromatic ring. Finally, a readout step combines all the atom vectors, typically by summing or averaging them, into a single vector for the whole molecule, which a final network maps to the predicted property. Because every atom is treated by the same update rule, and the readout combines atoms in a way that does not depend on their order, the result is the same however the atoms are numbered. And because the update functions are learned from data, the network discovers for itself which local environments matter for the property being predicted. Justin Gilmer and colleagues at Google formalised this family of methods in 2017 under the name message passing neural networks, unifying several earlier architectures, among them the neural fingerprints proposed by David Duvenaud and colleagues in 2015. A variant using directed messages along bonds, developed by Kevin Yang, Regina Barzilay, Tommi Jaakkola and colleagues at MIT and released as the open-source package Chemprop, became one of the most widely used tools for molecular property prediction. Adding the protein and the third dimension A model that sees only the drug molecule can learn which chemical features tend to make compounds active against a particular target, given enough measured examples for that target. This ligand-based approach is powerful within a project where hundreds of analogues have been tested, but it cannot generalise to a new target, because it knows nothing about the protein. To predict binding to arbitrary proteins, the model must also represent the target. One family of methods, often called drug–target affinity models, encodes the protein as a sequence of amino acids, processes it with a separate network, and combines the protein and ligand representations to predict affinity. Early examples included DeepDTA, published by Hakime Öztürk and colleagues in 2018, which used convolutional networks on both protein sequences and the text strings chemists use to write molecules, and GraphDTA, which replaced the ligand string with a graph. These models are trained on large tables of measured affinities, such as the kinase inhibitor datasets assembled by Mindy Davis and colleagues and the KIBA dataset, and report strong correlations with held-out measurements. A second family uses three-dimensional structures. If the structure of the protein bound to the ligand is known, from X-ray crystallography or prediction, the complex can be represented as a graph in which nodes are atoms of the ligand and of the surrounding protein pocket, and edges connect atoms that lie within some distance of each other, including atoms that are not chemically bonded but interact through space. Message passing over this graph lets the model learn which spatial arrangements of ligand and protein atoms are associated with tight binding. Because physical interactions depend on distances and angles, not on the arbitrary orientation of the coordinate system, more recent architectures are built to be equivariant: their internal representations rotate consistently when the input is rotated, so they respect the geometry of the problem by construction rather than having to learn it from examples. The standard dataset for structure-based affinity prediction is PDBbind, curated since 2004 by Renxiao Wang and colleagues. It collects protein–ligand complexes from the Protein Data Bank for which binding affinities have been measured, and its later releases contain around twenty thousand complexes. A widely used benchmark, the Comparative Assessment of Scoring Functions known as CASF-2016, defines a core set of 285 high-quality complexes for testing. For years, papers reported steadily rising correlations between predicted and measured affinities on this core set, with deep learning models reaching Pearson correlations around 0.8 and above, well beyond classical scoring functions. The benchmark that measured the wrong thing Those results turned out to be much less impressive than they looked. The core set and the training set were not independent. Many proteins in the core set also appeared, sometimes with closely related ligands, in the training data, and many ligands in the core set had close chemical relatives among training examples. A model could therefore achieve high correlation by recognising familiar proteins and familiar ligands and recalling roughly how tightly their relatives bound, without learning anything about the physics of the interaction. Several groups demonstrated this from different angles. A 2022 study in the Journal of Medicinal Chemistry by Mikhail Volkov and colleagues examined state-of-the-art deep learning affinity models and showed that their predictions were driven largely by memorisation of ligand and protein features rather than by the protein–ligand interactions the models were supposedly learning; models given only the ligand, or only the protein, performed nearly as well as those given the full complex. Jie Li, Teresa Head-Gordon and colleagues at Berkeley reorganised PDBbind into a split they called Leak Proof PDBbind, controlling the similarity of proteins and ligands between training and test sets. Models evaluated on this stricter split scored lower, and those retrained on it generalised better to newly assembled complexes that none of them had seen. The drug–target affinity benchmarks had similar problems: random splits of the Davis and KIBA tables put measurements for the same proteins and closely related compounds on both sides, so a model could perform well by learning each protein's typical affinity range. Docking-style deep learning models faced a parallel reckoning. DiffDock, introduced by Gabriele Corso and colleagues at MIT in 2022, used a diffusion generative model to predict where and how a ligand binds and reported large improvements over classical docking in finding the correct pose. In 2024 Martin Buttenschoen, Garrett Morris and Charlotte Deane at Oxford published PoseBusters in Chemical Science, a suite of checks for whether predicted poses were physically plausible, with sensible bond lengths, no clashes between atoms and correct stereochemistry. Many deep learning methods produced a large share of poses that looked close to the right answer by a simple distance criterion but were chemically impossible, and classical docking programs remained competitive once physical validity was required. The point is not that these models are worthless. It is the same lesson radiology learned from shortcut learning and deterioration prediction learned from label leakage. A flexible model trained on data with exploitable regularities will exploit them, and a benchmark that shares those regularities between training and test will reward the exploitation. The question that matters to a medicinal chemist, how well the model will predict binding for a new compound against a target of current interest, is answered only by an evaluation designed to mimic that situation. The structure revolution Running alongside these developments was the change brought by protein structure prediction. AlphaFold 2, presented by John Jumper and colleagues at DeepMind and published in Nature in 2021, predicted the three-dimensional structures of single protein chains from their amino acid sequences with accuracy that in many cases rivalled experimental methods. The AlphaFold Protein Structure Database, built with the European Bioinformatics Institute, made predicted structures freely available for more than two hundred million proteins. In 2024 Demis Hassabis and John Jumper shared the Nobel Prize in Chemistry for this work with David Baker, recognised for computational protein design. AlphaFold 2 predicted proteins alone. Its successor, AlphaFold 3, described by Josh Abramson and colleagues in Nature in 2024, predicts the joint structure of complexes containing proteins, nucleic acids, small molecules and ions, using a diffusion-based architecture. On benchmarks of protein–ligand complexes it outperformed classical docking at predicting how a ligand sits in its pocket, even without being given the protein's structure in advance. Open models followed, among them RoseTTAFold All-Atom from Baker's group and the Boltz series from MIT. In mid-2025 Boltz-2, released by researchers at MIT with the company Recursion under a permissive open-source licence, added a binding affinity prediction alongside the predicted structure; its developers reported accuracy approaching that of free energy perturbation on a widely used benchmark of congeneric series, at a computational cost roughly a thousand times lower. These co-folding models are a genuine advance, and the most important shift is in what becomes possible: structure-based design for targets that have no experimental structure, and rapid triage of very large numbers of compounds with more physical realism than docking. But the history of the field counsels caution about benchmark claims. Co-folding models are trained on the Protein Data Bank, and their accuracy is markedly higher for ligands and pockets that resemble those in their training data. Independent analyses have found cases where predicted complexes look confident and plausible but place ligands in ways that ignore physical constraints, for instance when a binding site is mutated so that the ligand could no longer fit. Affinity prediction remains hardest exactly where it would be most valuable: for novel chemical scaffolds, for targets unlike those in the training data, and for the activity cliffs where a tiny structural change produces a large change in binding. Table 4 sets out how the main approaches to affinity prediction compare in their requirements and where each is typically used. Table 4. Approaches to predicting protein–ligand binding. Approach Needs Speed Typical use Docking Protein structure Very fast Screening large libraries Free energy perturbation Structure, known binder Slow Ranking close analogues Ligand-based GNN Measured data for the target Fast Optimising within a series Structure-based GNN Complex structure Fast Rescoring poses Co-folding models Sequence and ligand Moderate Pose and affinity for new targets What a good affinity model is for In practice, affinity models are most valuable not as oracles but as filters and guides inside an iterative design cycle. A medicinal chemistry team makes and tests compounds in rounds, each round informed by the last. A model that ranks a few thousand proposed analogues, so that the twenty actually synthesised are enriched for good binders, can save months even if its individual predictions are imprecise. Used this way, the model's calibration and its estimate of its own uncertainty matter more than its average error, because the team needs to know when to trust it and when to test the compound anyway. Approaches such as active learning, in which the model chooses the next compounds to test partly to reduce its own uncertainty, make this explicit. Binding, however, is only the beginning of what a drug must do. It must reach its target in the body, avoid binding to other proteins in harmful ways, be metabolised at a suitable rate, not be toxic, and above all act on a target whose modulation actually treats the disease. The next chapter turns from the model to the pipeline, and to what has happened when companies built around machine learning have tried to carry molecules from prediction to patients. CHAPTER 7 From Prediction to Pipeline: What AI Has Changed in Drug Discovery Drug development is a long funnel with a leaky bottom. A programme typically begins with a hypothesis that modulating some biological target will treat a disease. Chemists search for molecules that act on the target, optimise the most promising ones over years of iterative design, and test the best in animals. A small number enter human trials: phase I to establish safety and dosing, phase II to look for evidence of efficacy in patients, phase III to confirm it in large populations. From first human dose to approval commonly takes the better part of a decade. The Tufts Center for the Study of Drug Development estimated in 2016 that the average cost of developing a new approved drug, including the cost of the many failures and the cost of capital, was around 2.6 billion dollars. An analysis by Chi Heem Wong, Kien Wei Siah and Andrew Lo, published in Biostatistics in 2019 and drawing on tens of thousands of clinical trials, estimated that the overall probability that a drug entering phase I would eventually be approved was roughly one in seven, with large variation between therapeutic areas and particularly low success in oncology. The critical fact about this funnel, for anyone hoping that machine learning will transform it, is where the losses occur. The most expensive failures happen in phase II and phase III, and most of them happen because the drug does not work well enough in patients or because it causes unacceptable harm. The molecule usually binds its target perfectly well. What failed was the hypothesis that binding the target would help, or the expectation that the molecule would be safe in the complicated environment of a human body. Improvements in predicting binding affinity, the subject of the previous chapter, act on the stages of the funnel that are comparatively cheap and comparatively successful. That does not make them worthless, since faster and cheaper early discovery has real value, but it bounds how much they can change the overall economics of bringing medicines to patients. Where machine learning acts on the funnel Machine learning has been applied at nearly every stage of drug development, and it is useful to separate these applications by the question each answers. At the very beginning, target identification asks which protein, pathway or cell state to intervene on. Approaches here mine large collections of genomic, transcriptomic and proteomic data from patients and model systems, scientific literature and clinical records, looking for targets whose modulation is likely to alter disease. Knowledge graphs, which link genes, proteins, diseases, drugs and published findings into a single network, are a common substrate, and graph-based machine learning is used to predict missing links, such as an unrecognised relationship between a drug and a disease. Hit finding asks which molecules interact with the chosen target at all. Virtual screening of very large libraries, using docking, ligand-based models or newer co-folding methods, can enrich the set of compounds tested experimentally. Generative models propose entirely new molecules designed to satisfy several criteria at once. Lead optimisation, the long middle phase of medicinal chemistry, asks which modifications to a promising molecule will improve its potency, selectivity and drug-like properties. This is where predictive models of affinity, solubility, metabolic stability, permeability and toxicity, often graph neural networks trained on a company's historical data, are used most routinely and with the clearest benefit. Later stages bring different questions: predicting toxicity in humans from preclinical data, selecting patients for trials, choosing doses, designing trials more efficiently, and identifying existing drugs that might be repurposed for new diseases. Each has attracted machine learning research, though rigorous evidence of impact is sparse. A new antibiotic from a graph neural network One of the cleanest demonstrations of what molecular machine learning can do came from James Collins, Regina Barzilay and colleagues at MIT, published by Jonathan Stokes and co-authors in Cell in 2020. Antibiotic discovery had stalled for decades: the obvious places to look had been exhausted, and screening programmes repeatedly rediscovered known compounds. The team trained a directed message passing neural network, the architecture behind the Chemprop package, on a modest dataset of a little over two thousand molecules whose ability to inhibit the growth of Escherichia coli had been measured. They then used the model to score a collection of about six thousand molecules from a drug repurposing library and selected high-scoring compounds that were structurally unlike known antibiotics. One of them, a compound previously investigated as a potential diabetes treatment, which the team renamed halicin, killed a broad range of bacterial pathogens in laboratory tests, including strains resistant to existing antibiotics, and cleared infections in mouse models. It appeared to act by disrupting the electrochemical gradient across bacterial membranes, a mechanism against which resistance may be slow to develop. The team then screened more than a hundred million molecules from a large virtual library and identified further candidates. In 2023 a related study led by Gary Liu and Jonathan Stokes, published in Nature Chemical Biology, used the same approach to find abaucin, a compound with narrow activity against Acinetobacter baumannii, a hospital pathogen on the World Health Organization's list of critical priorities. These results are instructive because of what made them work. The prediction task, whether a molecule inhibits bacterial growth, was measured directly in a cheap, reliable laboratory assay, so the labels were clean. The model was used to prioritise compounds for experimental testing, not to replace it. Each prediction was checked at the bench, and the work proceeded through the ordinary stages of biological validation. And the authors were clear that a promising preclinical compound is a long way from an approved antibiotic, which must still pass through toxicology and clinical trials. Neither halicin nor abaucin has reached patients; the economic challenges of antibiotic development, in which new drugs are held in reserve and sell in small volumes, are not problems a model can solve. A repurposing hypothesis tested at scale In early February 2020, as COVID-19 spread from Wuhan, researchers at the company BenevolentAI used their biomedical knowledge graph to search for approved drugs that might reduce viral entry into cells while also dampening inflammation. In a short letter in The Lancet, Peter Richardson and colleagues proposed baricitinib, a Janus kinase inhibitor approved for rheumatoid arthritis, as a candidate. The hypothesis drew on known biology: baricitinib inhibits a kinase involved in the process by which viruses enter cells, and its anti-inflammatory effects might moderate the immune overreaction seen in severe disease. The hypothesis was then tested in large randomised trials. Baricitinib improved outcomes in hospitalised patients in trials including the US-led ACTT-2 and COV-BARRIER studies, and in the large British RECOVERY platform trial it reduced mortality among hospitalised patients. In 2022 the FDA approved it for the treatment of COVID-19 in hospitalised adults requiring oxygen or ventilation. It would be wrong to say that an algorithm discovered a COVID treatment. Several groups independently considered Janus kinase inhibitors for COVID-19, and the drug's value was established by conventional clinical trials involving thousands of patients. What the knowledge graph did was help generate and prioritise a plausible hypothesis quickly, at a moment when speed mattered. That is a realistic and valuable role for machine learning in drug discovery: not replacing the slow work of testing, but improving the quality of what enters it. Generating molecules, and the limits of generation Predictive models score molecules that someone has proposed. Generative models propose the molecules themselves. Since the late 2010s, a succession of methods, from recurrent networks that write out molecules as text strings, through variational autoencoders and reinforcement learning, to graph-based and diffusion models that build molecules atom by atom or directly in three dimensions inside a protein pocket, has been used to design compounds optimised for several properties at once: predicted potency, predicted selectivity, predicted solubility and so on. Generated molecules frequently look superb on the computer and disappoint at the bench, for reasons that echo earlier chapters. The properties they are optimised against are themselves predictions, from models with the limitations already described, and an optimiser pushing hard against an imperfect predictor will find exactly the regions of chemical space where the predictor is most wrong, much as a student who studies only past exam papers learns the examiner's habits rather than the subject. Many generated structures are difficult or impossible to synthesise, or contain chemical groups that experienced medicinal chemists would reject as unstable or reactive. In a 2020 commentary in Nature Biotechnology, W. Patrick Walters and Mark Murcko, both veterans of industrial drug discovery, examined a widely publicised claim of rapid AI-driven design and observed that the resulting molecule closely resembled a known drug, raising the question of how much the model had invented rather than rediscovered. Their broader point was that claims of generative design should be judged against what a competent chemist could have proposed in the same time with conventional tools. Generative methods are improving, and the most effective industrial uses now tie them closely to synthesis planning and to rapid automated testing, so that proposals are filtered for feasibility and checked experimentally within days. Used this way, generation is one more source of ideas in the design cycle, judged like any other by what happens when the compounds are made. The same capability has an uncomfortable mirror image. In 2022 Fabio Urbina, Filippa Lentzos, Cédric Invernizzi and Sean Ekins reported in Nature Machine Intelligence that they had taken a generative model built to design drugs while avoiding toxicity and simply reversed the toxicity objective. Within hours it proposed tens of thousands of molecules predicted to be highly toxic, including known chemical warfare agents and novel compounds predicted to be more toxic still. The authors did not synthesise any of them, and turning a structure on a screen into a weapon involves many further obstacles. But the experiment showed that the tools of molecular design are inherently dual-use, and it prompted discussion, still unresolved, about how access to such models and their training data should be governed. The first AI-designed molecules in the clinic By the early 2020s, several companies built around machine learning had begun taking molecules into human trials. In 2020 the British company Exscientia and its Japanese partner Sumitomo Dainippon Pharma announced that DSP-1181, a candidate for obsessive-compulsive disorder, had entered phase I, describing it as the first drug designed with artificial intelligence to do so. The programme was discontinued after phase I. Exscientia later merged with the American company Recursion, which itself has reshaped its pipeline repeatedly, discontinuing several programmes after clinical results. These outcomes are not unusual by the standards of drug development, where most programmes fail; they are a reminder that a faster route to a clinical candidate does not change the odds that the candidate is the right one. The most closely watched case to date is rentosertib, developed by Insilico Medicine for idiopathic pulmonary fibrosis, a progressive scarring of the lungs with limited treatment options. The company reported that both the target, a kinase called TNIK, and the molecule were identified with its machine learning platforms, the target through analysis of omics and literature data and the molecule through generative chemistry. In June 2025 the results of a phase IIa trial were published in Nature Medicine. The trial randomised 71 patients in China to one of three doses or placebo for twelve weeks. At the highest dose, 60 milligrams once daily, forced vital capacity, the standard measure of lung function in this disease, rose by a mean of 98.4 millilitres, compared with a decline of 20.3 millilitres on placebo. The result is encouraging, and the right way to read it is with exactly the care this book has urged throughout. The trial was small and short, with fewer than twenty patients per arm and a follow-up of twelve weeks in a disease that progresses over years. All participants were from a single country. Adverse events included liver injury and other side effects that led some patients to stop treatment, and fewer patients completed the trial in the high-dose arm than on placebo. A phase IIa result is a signal worth pursuing in larger trials, not evidence of benefit. That is no criticism of the trial, which was designed as an early test, but it places the achievement accurately: a molecule from an AI-driven pipeline has produced a plausible efficacy signal in patients, which is more than most programmes of any origin achieve, and it now faces the same long tests as any other candidate. Do AI-discovered drugs succeed more often? In 2024 Madura Jayatunga and colleagues at the Boston Consulting Group, with collaborators, published an analysis in Drug Discovery Today of molecules from companies describing themselves as AI-native that had entered clinical trials. They reported that such molecules had passed phase I at a rate of roughly 80 to 90 percent, substantially higher than historical averages, while phase II success, based on a much smaller number of completed trials, was about 40 percent, close to historical norms. The analysis has been widely cited as evidence that AI improves drug development. The finding is consistent with the argument of this chapter. Phase I primarily tests safety and pharmacokinetics, properties that molecular models are trained to optimise, so a higher pass rate there is plausible. Phase II tests whether the drug works, which depends mostly on whether the biological hypothesis is right, and that is where AI-discovered molecules look much like everyone else's. The sample sizes are small and the comparison groups imperfect, since companies that choose to label themselves as AI-native may differ from others in many ways, and the result should be held loosely. But it suggests what one would expect: machine learning has so far helped mainly with the parts of drug discovery that were already the most tractable. The harder problem: choosing the right target If the dominant cause of clinical failure is that the biological hypothesis is wrong, then the largest gains from any technology will come from choosing better targets. Here a different kind of evidence has proved powerful. Analyses of drug development pipelines have repeatedly found that drugs whose targets are supported by human genetic evidence, such as variants in the target's gene that are associated with the disease, are more likely to succeed. An analysis by Eric Minikel and colleagues, published in Nature in 2024 and building on earlier work by Matthew Nelson and colleagues in 2015, estimated that the probability of a drug programme progressing from phase I to approval was about 2.6 times greater when its target had genetic support. This is a striking result, and its implication for machine learning is subtle. Human genetics is valuable precisely because it provides something close to a natural experiment: people born with variants that alter a protein's function show what happens when that protein is modulated across a lifetime, which is causal evidence about the target, not merely correlation. Machine learning methods that integrate genetic data, functional genomics and large-scale perturbation experiments, in which genes are switched off or chemicals applied to cells and the consequences measured, aim to scale up this kind of causal inference. Companies such as Recursion have built very large automated experimental platforms that image cells under millions of perturbations, generating training data designed for this purpose. Whether these approaches will move the phase II success rate is the most important open question in the field, and it will take a decade of trials to answer. Reading the claims The commercial setting of drug discovery produces a steady stream of claims: that a model has designed a drug in a few weeks, that a platform has cut years from development, that a company has some number of AI-discovered candidates in its pipeline. The tools in this chapter suggest how to read them. Ask which stage of the funnel the technology acted on, and whether that stage was the bottleneck. Ask whether predictions were tested experimentally, and with what success rate, including the failures. Ask what the comparison is: faster than what, cheaper than what, better than what. And, above all, ask what has been shown in patients, in trials of what size and duration, against what control. The genuine achievements are substantial: models that make medicinal chemistry more efficient, structure predictions that open new targets to rational design, screening approaches that have found novel antibiotic scaffolds, and the first efficacy signals from molecules discovered with heavy computational input. The transformation, if it comes, will be measured in the success rate of phase II trials and in medicines reaching patients who need them. That measurement has barely begun. CHAPTER 8 Precision Medicine: From Who Is at Risk to Who Will Benefit Medicine has always been personal in aspiration and average in practice. Clinical trials estimate the effect of a treatment across a population, and guidelines translate those averages into recommendations. A treatment that helps the average patient may help some patients a great deal, others not at all, and harm a few, and for most of medicine's history there was no way to tell in advance which was which. Precision medicine is the attempt to change that: to use information about the individual, from genes and tumours to images and physiological signals, to choose the treatment most likely to help that particular person. Its successes before machine learning were real and specific. Breast cancers that overexpress the HER2 protein respond to drugs that target it, such as trastuzumab, and testing for HER2 has been standard for a quarter of a century. Lung cancers with particular mutations in the EGFR gene respond dramatically to EGFR inhibitors. Variants in genes encoding drug-metabolising enzymes predict who will suffer dangerous reactions to certain medicines. Each case links a measurable characteristic of the patient to a difference in how they respond to a treatment, through a known biological mechanism. Machine learning promises to extend this in two ways. It can extract new kinds of information from data that clinicians already collect, finding molecular signals in routine pathology slides or cardiac disease in an ordinary electrocardiogram. And it can, in principle, combine many weak signals into predictions about individual patients that no single biomarker could provide. The first promise has been partly realised. The second runs into a problem that runs through this whole book, and which in precision medicine becomes impossible to avoid: the difference between predicting what will happen to a patient and predicting what a treatment will do for them. Reading more from the same data Pathology offers the clearest example of the first promise. For more than a century, the diagnosis of cancer has rested on a pathologist examining thin slices of tissue, stained with haematoxylin and eosin, under a microscope. As laboratories have digitised their slides, producing gigapixel images, deep learning models have been trained on them. Some of these models do what pathologists do, such as grading prostate cancer or detecting metastases in lymph nodes, and do it consistently and quickly. In 2021 the FDA granted De Novo authorisation to Paige Prostate, software that flags areas suspicious for cancer on prostate biopsy slides, the first AI product authorised in digital pathology. Others do something pathologists cannot. In 2018 Nicolas Coudray and colleagues at New York University reported in Nature Medicine that a network trained on lung adenocarcinoma slides could predict the presence of mutations in several genes, including EGFR, from the stained tissue alone, with moderate accuracy. In 2019 Jakob Nikolas Kather and colleagues showed in the same journal that deep learning could predict microsatellite instability, a molecular feature of some colorectal and other cancers that predicts response to immunotherapy, directly from routine slides. The practical appeal is large. Molecular testing is expensive and slow and is not available everywhere, whereas stained slides are produced for nearly every cancer diagnosis. A model that identified which patients most needed molecular testing, or that could stand in for it where testing was unavailable, would widen access to targeted therapy. The caution is that these models predict molecular status from morphology with less than perfect accuracy, that their performance varies across laboratories with different staining and scanning, and that, for decisions about targeted therapy, a false prediction has direct consequences. In practice they are most promising as screening tools that decide which samples to send for confirmatory testing, rather than as replacements for it. Since 2023, foundation models for pathology, pre-trained on hundreds of thousands or millions of slides, have improved performance across many such tasks and reduced the need for labelled data, though they inherit the familiar questions about where their training slides came from and whom they represent. Cardiology offers a parallel. The twelve-lead electrocardiogram is cheap, quick and ubiquitous. Researchers at the Mayo Clinic, led by Zachi Attia and Paul Friedman, trained networks to detect conditions from it that clinicians cannot see on the tracing, including reduced pumping function of the left ventricle and a history of atrial fibrillation in a patient whose current rhythm is normal. What distinguishes this work is that it was taken to a pragmatic randomised trial. The EAGLE trial, published by Xiaoxi Yao and colleagues in Nature Medicine in 2021, randomised 120 primary care teams, caring for more than 22,000 patients who had an electrocardiogram for any reason, to receive or not receive the algorithm's report of possible low ejection fraction. In the intervention group, diagnoses of low ejection fraction within ninety days rose from 1.6 to 2.1 percent. The effect was modest, and it depended heavily on whether clinicians ordered the confirmatory echocardiogram the algorithm recommended. But the trial measured a real clinical consequence, in the setting of use, against usual care, which is what the earlier chapters argued all clinical AI should eventually do. Interpreting the genome The human genome is another domain in which machine learning extracts more from existing data. Sequencing a person's genome is now routine in many clinical contexts, especially for rare disease and cancer. The bottleneck is interpretation. Each person carries thousands of variants that differ from the reference genome, and for most of them nobody knows whether they matter. Missense variants, which change a single amino acid in a protein, are a central difficulty. Of the more than seventy million possible missense variants in the human genome, only a tiny fraction have been classified by clinical experts as pathogenic or benign. In 2023 Jun Cheng, Žiga Avsec and colleagues at Google DeepMind described AlphaMissense in Science. Building on the AlphaFold architecture and trained on patterns of variation across human and primate populations, which reveal which changes evolution has tolerated, it assigned a pathogenicity score to essentially all possible missense variants and classified the large majority as likely benign or likely pathogenic. Clinical genetics laboratories now use such predictions as one line of supporting evidence, alongside population frequency, family history and functional studies, under professional guidelines that limit how much weight computational predictions can carry on their own. That restraint is sensible. A prediction of pathogenicity is a prior, not a diagnosis, and it can be wrong in systematic ways for genes and populations underrepresented in training data. Polygenic risk scores combine the small effects of many common variants into a single estimate of inherited risk for conditions such as coronary artery disease or breast cancer. A 2018 study by Amit Khera, Sekar Kathiresan and colleagues in Nature Genetics showed that people in the top few percent of polygenic risk for several common diseases had risks comparable to those conferred by rare single-gene mutations. But these scores are derived mostly from genome-wide association studies in people of European ancestry, and their predictive accuracy is substantially lower in other populations; a 2019 analysis by Alicia Martin and colleagues in Nature Genetics found accuracy several-fold lower in people of African ancestry. Deploying such scores clinically without addressing that gap would give the most precise predictions to the populations already best served by medical research, a pattern that recurs wherever training data are unrepresentative. The central difficulty: effects, not risks Everything described so far in this chapter predicts something about the patient: a mutation, a diagnosis, a level of risk. Precision medicine's real promise is different. It is to predict, for this patient, how much better or worse their outcome would be under one treatment than under another. That quantity, the individual treatment effect, is what a clinician choosing between treatments needs to know. And it is fundamentally harder to estimate than risk. The difficulty is logical, not technical. For any patient, we observe the outcome under the treatment they actually received. We never observe what would have happened to the same patient, at the same time, under the alternative. Statisticians call this the fundamental problem of causal inference. Risk can be learned from observational data by watching what happens to patients; treatment effects require comparing what happens under different treatments in patients who are otherwise alike, which is what randomised trials are designed to do. Observational data, the electronic records and registries on which most clinical machine learning is trained, are deeply confounded for this purpose. Clinicians choose treatments based on their assessment of the patient. Sicker patients get more aggressive treatment, and a naive model will find that aggressive treatment is associated with worse outcomes. The asthma-and-pneumonia example from the first chapter is a case of exactly this: the model learned that asthma predicted good outcomes because asthmatic patients were treated more intensively. Methods exist to adjust for confounding, including propensity scores, instrumental variables and target trial emulation, in which an observational analysis is designed to mimic the protocol of a randomised trial. They can work well when the factors driving treatment decisions are measured, and fail silently when they are not. Machine learning has contributed new methods for estimating how treatment effects vary between patients. Causal forests, developed by Stefan Wager and Susan Athey and published in 2018, adapt random forests to search for subgroups in which the treatment effect differs. Meta-learners combine predictive models of outcomes under each treatment. These methods are most trustworthy when applied to randomised trial data, where confounding is controlled by design. But trials are usually powered to detect an average effect, not to estimate how it varies, and searches for heterogeneity within them are notoriously prone to false discoveries. The history of subgroup analyses in clinical trials is littered with effects that did not replicate. What works for now Two approaches have proved more robust. The first is risk-based heterogeneity analysis, recommended by the Predictive Approaches to Treatment effect Heterogeneity statement, published by David Kent and colleagues in the Annals of Internal Medicine in 2020. The idea is simple. Many treatments have a roughly constant relative effect, reducing risk by a similar proportion across patients. If so, patients at higher baseline risk gain more in absolute terms, because a given proportion of a larger risk is a larger number. A well-validated risk model, applied to the participants in a randomised trial, can therefore identify patients for whom the absolute benefit of treatment is large enough to justify its costs and harms, and those for whom it is too small. This approach uses prediction to inform treatment decisions without asking the model to estimate causal effects directly, and it is far more stable than searching for subgroups with different relative effects. A worked example shows why this matters. Imagine a preventive drug that reduces the risk of a heart attack over ten years by a quarter, a relative risk reduction of 25 percent, in every patient, and that causes a serious side effect in one patient in a hundred. For a patient whose ten-year risk is 4 percent, the drug lowers it to 3 percent: an absolute benefit of one percentage point, so a hundred such patients must be treated to prevent one heart attack, at the cost of about one serious side effect. Benefit and harm are roughly balanced, and a reasonable patient might decline. For a patient whose ten-year risk is 20 percent, the drug lowers it to 15 percent: an absolute benefit of five percentage points, so only twenty patients need to be treated to prevent one heart attack, while the same twenty patients would expect only a fifth of a serious side effect between them. The drug's relative effect is identical in both patients; what differs is the baseline risk, and with it the balance of benefit and harm. A good risk model, which machine learning can sometimes improve, is enough to separate these patients without any need to estimate individual causal effects. The decision still depends on a trial having established the relative effect in the first place. This logic also reveals a limitation. Where relative effects genuinely vary between patients, because the drug acts on a mechanism that is present in some and absent in others, risk stratification alone will miss it. A patient at high risk because of a disease process the drug does not touch will gain nothing, however high their risk. For those cases, the question is no longer how risky the patient is but what is driving the risk. The second approach is biological subphenotyping: using data to identify groups of patients whose disease is driven by different mechanisms and then testing whether they respond differently to treatments that target those mechanisms. The acute respiratory distress syndrome, a form of lung failure seen in intensive care, provides an example. Carolyn Calfee and colleagues identified, using latent class analysis of clinical and biomarker data from trial participants, two subphenotypes, one hyperinflammatory and one hypoinflammatory, with different outcomes. Secondary analyses of completed randomised trials suggested that the subphenotypes responded differently to some treatments, including different strategies of mechanical ventilation and fluid management. In 2019 Christopher Seymour and colleagues described four clinical phenotypes of sepsis in JAMA, derived from routine data, and simulations suggested that the results of past trials might have differed depending on the mix of phenotypes enrolled. These findings are hypotheses, not yet treatment rules. Their value lies in proposing mechanistically meaningful subgroups that can be tested prospectively in trials that enrol on the basis of phenotype. The shape of precision medicine with machine learning Drawing these strands together gives a clear picture of where machine learning is helping precision medicine and where it is not yet. It is good at extracting latent information from images, signals and sequences: inferring molecular status from tissue, cardiac function from an electrocardiogram, pathogenicity from a variant. Those capabilities extend the reach of precision medicine's existing logic, which links measurable characteristics to treatment through known biology, and their value is clearest when they widen access to diagnostics that are otherwise expensive or scarce. It is not yet good at the problem at the centre of the precision medicine vision, which is predicting from routine data how an individual will respond to a choice of treatments. That problem is not primarily a matter of better algorithms. It requires data in which treatments were assigned in ways that permit causal comparison, which means trials, ideally designed to measure heterogeneity, and it requires mechanistic understanding that can generate hypotheses worth testing. Machine learning can help design those trials, select patients for them, and analyse their results. It cannot substitute for them. This returns us, by another road, to the argument with which the book began. A model that predicts accurately is not yet a model that helps, and the gap between the two is bridged by evidence about what happens when its outputs are acted on. In precision medicine that gap has a precise name, the difference between risk and effect, and the tools for crossing it are the oldest in clinical research. The final chapter turns to the institutions that must decide when that gap has been crossed. CHAPTER 9 Evidence, Regulation and Trust By this point a pattern should be clear. Across imaging, health records, drug discovery and precision medicine, the systems that have helped patients are those whose developers and users asked, and answered with evidence, what happens when the model's output is acted upon in the setting where it will be used. The systems that disappointed were usually accurate on some benchmark and never tested on that question. This final chapter is about the institutions that are supposed to ensure the question gets asked: the standards for how evidence is reported, the regulators who decide what may be sold, and the hospitals and health systems that decide what to deploy. It ends with the class of models, large language models, that is putting all of those institutions under the greatest strain. What counts as evidence Clinical medicine has well-developed conventions for judging evidence about drugs, and they rest on a hierarchy: case reports and observational studies at the bottom, randomised controlled trials and systematic reviews of trials at the top. For clinical AI, the equivalent hierarchy follows the validation ladder described in the first chapter, from internal validation through external, prospective and finally interventional evaluation. The striking feature of the field is how little of its literature has climbed above the lower rungs. Systematic reviews have tried to count the randomised trials of machine learning interventions in clinical care. A review led by Deborah Plana, published in JAMA Network Open in 2022, identified 41 such trials, and found that many were at high risk of bias, were conducted at single centres, and did not report results by demographic subgroup. A later review in The Lancet Digital Health in 2024, led by Ryan Han, identified 86, a rapid increase but still a tiny number set against the thousands of models published and the more than fifteen hundred devices authorised in the United States alone. Most of the trials were in gastroenterology, where AI-assisted colonoscopy has been tested extensively for its effect on adenoma detection, and in radiology and cardiology. Many measured process outcomes such as detection rates rather than patient outcomes such as cancer incidence or mortality. Part of the reason is cost: randomised trials are expensive, slow and logistically complex, especially when the intervention is software embedded in a clinical workflow. Part is incentive: regulators in most jurisdictions do not require them for authorisation, and purchasers rarely demand them. And part is genuine methodological difficulty. Randomising individual patients to have their scans read with or without AI can contaminate the control arm, because the same radiologists read both and may learn from the AI's suggestions. Cluster randomisation, in which whole clinics or teams are randomised, as in the EAGLE trial of electrocardiogram screening, avoids this but requires larger samples. Stepped-wedge designs, in which sites switch to the intervention in random order over time, suit staged rollouts of the kind Kaiser Permanente used, though they are vulnerable to secular trends. None of these difficulties is insurmountable, and the trials that have been done show that it is feasible to evaluate clinical AI properly. MASAI randomised more than a hundred thousand women. The question is not whether such evidence can be obtained but who will require it. Reporting standards One concrete step the research community has taken is to develop reporting guidelines specific to AI. Reporting guidelines do not dictate how studies should be designed; they specify what a paper must disclose so that readers can judge its validity and reproduce its methods. Their value is in making the weaknesses discussed throughout this book visible: how labels were defined, how data were split, whether validation was external, how performance varied across groups, what happened to cases the model could not process. Several guidelines now cover the main stages of evaluation, and Table 5 summarises them. Table 5. Reporting guidelines for clinical AI studies by stage of evaluation. Guideline Stage What it covers TRIPOD+AI Model development and validation Data sources, outcome definition, performance, fairness STARD-AI Diagnostic accuracy Reference standard, case selection, reader comparisons DECIDE-AI Early live clinical evaluation Human factors, workflow, safety in first use SPIRIT-AI Trial protocols AI intervention version, inputs, handling of errors CONSORT-AI Randomised trial reports Integration into care, analysis of performance errors TRIPOD+AI, published by Gary Collins and colleagues in the BMJ in 2024, updated a long-established guideline for prediction models to cover machine learning, adding items on fairness and on the availability of code and data. SPIRIT-AI and CONSORT-AI, published together in 2020 by an international group led by Xiaoxuan Liu, Samantha Cruz Rivera, Alastair Denniston and Melanie Calvert, extended the standard guidelines for trial protocols and trial reports. DECIDE-AI, published by Baptiste Vasey and colleagues in Nature Medicine in 2022, fills a gap that the others missed: the stage between retrospective validation and a full trial, when a system is first used with real patients and the most important questions concern usability, how clinicians interact with it and whether unexpected harms appear. Journals increasingly require authors to follow these guidelines. Readers outside research can use them too, as checklists of the questions to ask of any study. How regulators are adapting Regulators face a problem that their frameworks were not built for. Medical device regulation assumed that a device, once authorised, stayed the same, and that its performance could be characterised by testing before it reached the market. Machine learning software is easily modified, may perform differently in different settings, and can degrade over time without any change to the software at all. The US Food and Drug Administration has responded with a series of incremental steps. In 2021, together with Health Canada and the United Kingdom's Medicines and Healthcare products Regulatory Agency, it published ten guiding principles of good machine learning practice, covering matters such as representative training data, independence of test data, attention to the performance of the human–AI team, and monitoring after deployment. In December 2024 it finalised guidance on predetermined change control plans, allowing manufacturers to obtain authorisation in advance for specified types of model updates, with specified validation, so that improvements need not each go through a new submission. In January 2025 it issued draft guidance on the full lifecycle of AI-enabled device software functions, describing what marketing submissions should contain, including information on data, performance across subgroups and plans for monitoring. Much clinical decision support software, however, is not regulated as a device at all. The 21st Century Cures Act of 2016 excluded from the device definition software that supports a clinician's decision when the clinician can independently review the basis for the recommendation. The FDA's interpretation, set out in guidance in 2022, narrowed that exclusion, but in practice many predictive models embedded in electronic health records, including deterioration and sepsis models of the kind discussed in earlier chapters, have reached patients without FDA review. A separate rule from the US Office of the National Coordinator for Health Information Technology, finalised in December 2023, began requiring developers of certified health record software to make information about predictive decision support tools available to their customers, including details on training data, intended use and validation. It is a transparency requirement rather than a performance standard, but it gives hospitals the information they need to ask better questions. In Europe, AI software that qualifies as a medical device must meet the requirements of the Medical Device Regulation, assessed by notified bodies. The Artificial Intelligence Act, which entered into force in August 2024, designates AI systems that are safety components of regulated medical devices as high-risk, adding obligations on risk management, data governance, technical documentation, transparency, human oversight, accuracy and robustness. The obligations for such systems were scheduled to apply from August 2027, and the European Commission proposed in late 2025 to adjust parts of the Act's timetable; the precise interplay between the AI Act and device regulation remains a subject of guidance and debate. The United Kingdom has taken a more flexible approach, including a regulatory sandbox launched by the MHRA in 2024 to test how AI medical devices can be assessed and monitored. Across these jurisdictions, regulators are converging on a model in which authorisation before market is necessary but not sufficient, and in which the manufacturer and the deploying institution share ongoing responsibility for monitoring performance in use. That model places considerable weight on hospitals and health systems, which were not built for the task. Governance inside the hospital The organisation that deploys a clinical AI system is the one best placed to know whether it is working. It has the data on its own patients, it sees the outcomes, and it controls the workflow into which the model's outputs are delivered. In the United States, and increasingly elsewhere, larger health systems have created governance structures for clinical AI: committees that review proposed deployments, require local validation, set monitoring requirements, assess equity, and decide when to retire a model. Industry and academic coalitions have developed frameworks for assurance, model cards describing a model's intended use and limitations, and shared approaches to testing. The questions such a committee should ask follow directly from the preceding chapters. What exactly does the model predict, and how was the label defined? Was it validated externally, and on data like ours? How does it perform across the groups we serve? What action will its output trigger, who will take that action, and do they have the capacity? What threshold will we use, and how many alerts will that generate? How will we know if performance degrades? What happens if the model is wrong? What is the comparison: is care with the model better than care without it, and how will we find out? The final question deserves emphasis because hospitals rarely ask it. Most deployments are not evaluated against outcomes at all; they are switched on and assumed to help. Yet hospitals are unusually well positioned to run pragmatic evaluations. A staggered rollout across wards or sites, randomised in order, costs little more than an unplanned one and yields evidence about effectiveness. Health systems that treat each deployment as an opportunity to learn, rather than as a procurement decision, will accumulate the evidence that the field as a whole lacks. Smaller hospitals, which lack data science teams and governance committees, face a harder problem. They depend on vendors' claims and on external validation done elsewhere, and they are least able to detect when a model performs poorly on their population. Proposals for shared assurance laboratories, regional evaluation networks and public registries of model performance aim to fill that gap. None is yet established at scale. Accountability and the human in the loop Much regulation of clinical AI relies on the idea that a human clinician remains in the loop, reviewing the model's output and taking responsibility for the decision. The arrangement is sensible in principle, and it is the reason most imaging and decision support tools are classed as assistive rather than autonomous. But the earlier chapters showed that humans in the loop are subject to automation bias, alert fatigue and time pressure, and that their oversight may be more nominal than real. The researcher Madeleine Clare Elish coined the term "moral crumple zone" for situations in which responsibility for a failure of an automated system falls on the human operator who had limited ability to control it, much as a crumple zone in a car absorbs the force of a crash. A radiologist who reads two hundred studies a day with AI assistance, and misses a finding the AI also missed, may bear the legal and professional consequences of an error that the system design made likely. Meaningful human oversight requires that the human has the time, information and authority to disagree with the model, that the system's design supports independent judgement rather than undermining it, and that responsibility is shared with those who designed, sold and deployed the system. Getting this right is as much a matter of professional norms, liability law and employment practice as of technology, and it is unsettled in every jurisdiction. Large language models Since late 2022, general-purpose large language models have changed the conversation about AI in medicine. Systems trained on vast amounts of text answer medical questions fluently, summarise records, draft letters and, in research settings, reason through clinical cases. Google's Med-PaLM 2, described in Nature Medicine in 2025 by Karan Singhal and colleagues, reached expert-level scores on questions in the style of the United States Medical Licensing Examination, and several general-purpose models now do as well or better. Ambient documentation tools, which listen to consultations and draft clinical notes, have been adopted rapidly by health systems eager to reduce clinicians' administrative burden. Every lesson of this book applies to these systems, with greater force. Examination performance is a benchmark, and benchmark performance has repeatedly failed to predict clinical utility. A 2024 study by Paul Hager, Daniel Rueckert and colleagues in Nature Medicine tested leading models on realistic diagnostic workflows constructed from real patient records, requiring them to request information, interpret results and reach a diagnosis, and found that they performed substantially worse than clinicians, failed to follow guidelines and were sensitive to the order and format in which information was presented. Language models can be confidently wrong in ways that are hard for users to detect, their outputs vary with small changes in phrasing, and they are updated by their developers on timescales that make conventional validation difficult. The most defensible current uses keep the model close to tasks where errors are visible and correctable, such as drafting text that a clinician reviews, and are evaluated against outcomes that matter, such as clinicians' time, the accuracy of documentation and patients' experience. Early evaluations of ambient scribes have generally found reductions in clinicians' documentation burden and improvements in their reported well-being, with more mixed and less well-studied effects on note quality and errors. Uses that place a language model's judgement directly into diagnostic or treatment decisions will need to climb the same validation ladder as any other clinical prediction, and the difficulty of characterising a general-purpose model's failure modes makes that climb steeper, not shallower. Trust that is earned Trust in clinical AI should not be a feeling, induced by impressive demonstrations or eroded by well-publicised failures. It should be a judgement, made model by model and setting by setting, on the basis of evidence about what happens when the model's outputs are used. The institutions described in this chapter, from reporting guidelines to regulators to hospital governance committees, exist to make that judgement possible. They are all younger than the technology they oversee and all still incomplete. Their maturity, more than the next improvement in model architecture, will decide how much good machine learning does in medicine. Conclusion: Where the Leverage Lies It is tempting to end a book about machine learning in medicine with a forecast: that within some number of years algorithms will read every scan, watch every patient and design most new drugs. The chapters here suggest a different kind of conclusion. The technical capability is already larger than its clinical impact, and the reasons for the gap are mostly not technical. What follows from that is a set of claims about where effort should go, what remains genuinely uncertain, and what different readers should do differently. The gap is the point Across the three domains this book has examined, the same structure keeps appearing. A model is trained on data produced by some process: radiologists writing reports, clinicians ordering tests, laboratories measuring binding, trials enrolling patients. It learns to reproduce the regularities in that data, including the ones that reflect the process rather than the biology. It is evaluated on more data produced by the same process, and it performs well. Then it is moved into a setting where the process is different, or where its own presence changes the process, or where the question that matters is not the one it was trained to answer, and the performance that looked so solid does not translate into benefit. The pneumonia network that learned which hospital an image came from, the sepsis model that fired after clinicians had already acted, the cost-prediction algorithm that inherited unequal spending, and the affinity models that memorised familiar proteins are not four unrelated failures. They are one failure in four settings, and recognising it as one is the most useful thing a reader can take from this book. The defence is also the same in every setting: define the prediction with care for what will be known when it is needed and what will be done with it; validate it on data produced by a different process; and measure, wherever possible, the outcome that the prediction was meant to improve. The successes share a structure too. The Swedish mammography trial, the autonomous retinal screening system, Kaiser Permanente's deterioration programme, the TREWS sepsis system, the electrocardiogram trial at the Mayo Clinic and the antibiotic screens at MIT all coupled a prediction to a defined action, compared care with the model against care as it would otherwise have been delivered, and checked the result in the real world. None of them depended on the most sophisticated algorithm available at the time. Several used architectures that would be considered unremarkable today. What follows for the people involved For clinicians, the implication is that the most important questions about a clinical AI tool are ones they are already equipped to ask. What is this predicting, exactly? What am I supposed to do when it fires? Was it tested on patients like mine? What happens when it is wrong, and how would I know? A clinician who asks these questions and gets vague answers has learned something important. Clinicians should also recognise automation bias in themselves. The evidence that experienced readers are swayed by incorrect suggestions is strong enough that forming an independent impression before looking at the model's output is a reasonable professional habit. For health system leaders, the implication is that buying a model is the cheap part. The expensive and decisive parts are designing the response, staffing it, setting thresholds to match capacity, monitoring performance over time, auditing equity, and evaluating outcomes. A budget for clinical AI that covers the licence and the integration but not those things is a budget for a tool that will probably not help. Leaders also have an opportunity that few others share: the ability to deploy in a way that generates evidence, by staggering rollouts, randomising where feasible, and measuring outcomes as a matter of course. For developers and data scientists, the implication is that the hardest and most valuable work lies at the edges of the modelling problem: in the definition of labels and prediction times, in the design of honest evaluation splits, in external validation, and in collaboration with the people who will act on the model's output. A one-point gain in AUROC on a leaky benchmark is worth less than an honest evaluation that shows a model's limits. The field's culture of leaderboards and benchmark competitions has produced real methodological progress, and it has also rewarded exactly the shortcut learning and data leakage that keep models from working in practice. For drug developers and investors, the implication is to look carefully at which stage of the pipeline a technology acts upon. Improvements in hit finding and lead optimisation are real and valuable, but the economics of drug development are dominated by failures in phase II, which depend mostly on whether the target was right. Methods that improve target selection, especially those grounded in human genetics and causal perturbation data, have the most leverage, and their claims should be judged by success rates in efficacy trials, which will take years to accumulate. For regulators and policymakers, the implication is that pre-market authorisation based on retrospective accuracy is not enough, and that the infrastructure for post-market evidence is the missing piece. Requirements for subgroup performance reporting, for monitoring plans, for transparency about training data and for outcome evaluation in at least some settings would change what developers build. So would payment policies that reward demonstrated benefit rather than use. The precedent of computer-aided detection in mammography, paid for for well over a decade before large studies showed it did not improve accuracy, should not be repeated. For patients, the implication is that it is reasonable to ask whether AI is involved in your care and what it is doing. The answer will often be reassuring: a second check on a mammogram, a faster alert to a stroke team, a tool that helps a pathologist. But patients have a legitimate interest in whether the tool was validated on people like them, and their questions are part of what will make health systems take the matter seriously. What remains unsettled Some questions this book cannot resolve because no one yet can. The first is whether machine learning will change the success rate of drug development in efficacy trials, which is the only measure that ultimately matters for the industry and for patients awaiting treatments. Early evidence suggests improvements in the early stages, and there are plausible routes to better target selection, but the decisive data will come from trials that are only now beginning. The second is whether general-purpose language models will follow the path of narrower predictive systems, with a long climb from impressive benchmark performance to demonstrated clinical benefit in specific uses, or whether their flexibility will allow faster adoption with acceptable risk. Their rapid uptake for documentation suggests the latter in low-risk tasks. For diagnostic and therapeutic decisions, the history recounted in this book counsels patience. The third is how health systems will handle the problems of drift and feedback over the long term. A model that works today may not work in three years, and a model that changes practice will eventually change the data on which it was built. Managing a portfolio of such models, each needing monitoring, recalibration and periodic re-evaluation, is a new kind of institutional responsibility, and no one yet knows what it costs at scale or how small and under-resourced hospitals can share in it. The fourth is equity. Machine learning can extend expert-level diagnostics to places without experts, as autonomous retinal screening has begun to do, and can make unequal care visible, as the analysis of the care-management algorithm did. It can also encode and amplify existing inequities, through unrepresentative training data, biased labels and devices that measure some patients less accurately than others. Which effect dominates will depend on choices about data, evaluation and deployment that are being made now. The modest revolution The transformation of medicine through machine learning is real. It is also slower, narrower and more dependent on ordinary institutional work than either its promoters or its critics usually suggest. It will not arrive as a single breakthrough. It will arrive one validated use at a time: a screening programme that finds more of the cancers that matter, a ward where deteriorating patients are noticed hours earlier by a team that knows what to do, a drug programme that chose a better target because the evidence for it was stronger. Each of those requires a good model. Each also requires a clear action, a responsible person, an honest evaluation, and someone who keeps checking that it still works. The leverage, in the end, lies not in the model but in the coupling of prediction to action and in the evidence that the coupling helps. That is a less glamorous conclusion than the forecasts of a decade ago. It is also a more hopeful one, because those things are within the control of the people who build, buy, regulate and use these systems, and they can start doing them now. Notes Introduction 1. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–2410. 2. US Food and Drug Administration. Artificial Intelligence-Enabled Medical Devices (public list, updated periodically; more than 1,500 entries by March 2026, about three quarters in radiology). 3. Gommers J, Lång K, et al. Primary results of the MASAI randomised trial of AI-supported mammography screening. The Lancet. 2026. 4. Xu Z, Ren F, Wang P, et al. A generative AI-discovered TNIK inhibitor for idiopathic pulmonary fibrosis: a randomized phase 2a trial. Nature Medicine. 2025. Chapter 1 1. Wang X, Peng Y, Lu L, et al. ChestX-ray8: hospital-scale chest X-ray database and benchmarks. Proceedings of IEEE CVPR. 2017. 2. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making. 2006;26(6):565–574. 3. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020;369:m1328. 4. Van Calster B, McLernon DJ, van Smeden M, et al. Calibration: the Achilles heel of predictive analytics. BMC Medicine. 2019;17:230. 5. Caruana R, Lou Y, Gehrke J, et al. Intelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission. Proceedings of ACM KDD. 2015. 6. Geirhos R, Jacobsen JH, Michaelis C, et al. Shortcut learning in deep neural networks. Nature Machine Intelligence. 2020;2:665–673. Chapter 2 1. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. MICCAI. 2015. 2. Rajpurkar P, Irvin J, Zhu K, et al. CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv:1711.05225. 2017. 3. Zech JR, Badgeley MA, Liu M, et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLOS Medicine. 2018;15(11):e1002683. 4. DeGrave AJ, Janizek JD, Lee SI. AI for radiographic COVID-19 detection selects shortcuts over signal. Nature Machine Intelligence. 2021;3:610–619. 5. Roberts M, Driggs D, Thorpe M, et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nature Machine Intelligence. 2021;3:199–217. 6. Gichoya JW, Banerjee I, Bhimireddy AR, et al. AI recognition of patient race in medical imaging: a modelling study. The Lancet Digital Health. 2022;4(6):e406–e414. 7. Saporta A, Gui X, Agrawal A, et al. Benchmarking saliency methods for chest X-ray interpretation. Nature Machine Intelligence. 2022;4:867–878. 8. McKinney SM, Sieniek M, Godbole V, et al. International evaluation of an AI system for breast cancer screening. Nature. 2020;577:89–94. 9. Lång K, Josefsson V, Larsson AM, et al. Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI). The Lancet Oncology. 2023;24(8):936–944. Chapter 3 1. Fenton JJ, Taplin SH, Carney PA, et al. Influence of computer-aided detection on performance of screening mammography. New England Journal of Medicine. 2007;356:1399–1409. 2. Lehman CD, Wellman RD, Buist DSM, et al. Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Internal Medicine. 2015;175(11):1828–1837. 3. Abràmoff MD, Lavin PT, Birch M, et al. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. npj Digital Medicine. 2018;1:39. 4. Dratsch T, Chen X, Rezazade Mehrizi M, et al. Automation bias in mammography: the impact of artificial intelligence BI-RADS suggestions on reader performance. Radiology. 2023;307(4):e222176. 5. Budzyń K, Romańczyk M, Kitala D, et al. Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study. The Lancet Gastroenterology and Hepatology. 2025. 6. US Food and Drug Administration. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions. Final guidance, December 2024. Chapter 4 1. Agniel D, Kohane IS, Weber GM. Biases in electronic health record data due to processes within the healthcare system: retrospective observational study. BMJ. 2018;361:k1479. 2. Royal College of Physicians. National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS. London: RCP, 2017. 3. Wong A, Otles E, Donnelly JP, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Internal Medicine. 2021;181(8):1065–1070. 4. Tomašev N, Glorot X, Rae JW, et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature. 2019;572:116–119. 5. Escobar GJ, Liu VX, Schuler A, et al. Automated identification of adults at risk for in-hospital clinical deterioration. New England Journal of Medicine. 2020;383:1951–1960. 6. Adams R, Henry KE, Sridharan A, et al. Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis. Nature Medicine. 2022;28:1455–1460. 7. Henry KE, Adams R, Parent C, et al. Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. Nature Medicine. 2022;28:1447–1454. Chapter 5 1. van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association. 2006;13(2):138–147. 2. Sendak MP, Ratliff W, Sarro D, et al. Real-world integration of a sepsis deep learning technology into routine clinical care: implementation study. JMIR Medical Informatics. 2020;8(7):e15182. 3. Elish MC, Watkins EA. Repairing Innovation: A Study of Integrating AI in Clinical Care. New York: Data & Society, 2020. 4. Romero-Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Critical Care. 2015;19:285. 5. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447–453. 6. Sjoding MW, Dickson RP, Iwashyna TJ, et al. Racial bias in pulse oximetry measurement. New England Journal of Medicine. 2020;383:2477–2478. 7. Lenert MC, Matheny ME, Walsh CG. Prognostic models will be victims of their own success, unless…. Journal of the American Medical Informatics Association. 2019;26(12):1645–1650. 8. Davis SE, Lasko TA, Chen G, et al. Calibration drift in regression and machine learning models for acute kidney injury. Journal of the American Medical Informatics Association. 2017;24(6):1052–1061. 9. Finlayson SG, Subbaswamy A, Singh K, et al. The clinician and dataset shift in artificial intelligence. New England Journal of Medicine. 2021;385:283–286. Chapter 6 1. Wang L, Wu Y, Deng Y, et al. Accurate and reliable prediction of relative ligand binding potency in prospective drug discovery by way of a modern free-energy calculation protocol and force field. Journal of the American Chemical Society. 2015;137(7):2695–2703. 2. Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE. Neural message passing for quantum chemistry. Proceedings of ICML. 2017. 3. Duvenaud D, Maclaurin D, Aguilera-Iparraguirre J, et al. Convolutional networks on graphs for learning molecular fingerprints. NeurIPS. 2015. 4. Yang K, Swanson K, Jin W, et al. Analyzing learned molecular representations for property prediction. Journal of Chemical Information and Modeling. 2019;59(8):3370–3388. 5. Öztürk H, Özgür A, Ozkirimli E. DeepDTA: deep drug–target binding affinity prediction. Bioinformatics. 2018;34(17):i821–i829. 6. Wang R, Fang X, Lu Y, Wang S. The PDBbind database: collection of binding affinities for protein–ligand complexes with known three-dimensional structures. Journal of Medicinal Chemistry. 2004;47(12):2977–2980. 7. Su M, Yang Q, Du Y, et al. Comparative assessment of scoring functions: the CASF-2016 update. Journal of Chemical Information and Modeling. 2019;59(2):895–913. 8. Volkov M, Turk JA, Drizard N, et al. On the frustration to predict binding affinities from protein–ligand structures with deep neural networks. Journal of Medicinal Chemistry. 2022;65(11):7946–7958. 9. Li J, Guan X, Zhang O, et al. Leak Proof PDBBind: a reorganized dataset of protein–ligand complexes for more generalizable binding affinity prediction. arXiv:2308.09639. 10. Corso G, Stärk H, Jing B, Barzilay R, Jaakkola T. DiffDock: diffusion steps, twists, and turns for molecular docking. ICLR. 2023. 11. Buttenschoen M, Morris GM, Deane CM. PoseBusters: AI-based docking methods fail to generate physically valid poses or generalise to novel sequences. Chemical Science. 2024;15:3130–3139. 12. Jumper J, Evans R, Pritzel A, et al. Highly accurate protein structure prediction with AlphaFold. Nature. 2021;596:583–589. 13. Abramson J, Adler J, Dunger J, et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature. 2024;630:493–500. 14. Passaro S, Corso G, Wohlwend J, et al. Boltz-2: towards accurate and efficient binding affinity prediction. bioRxiv. 2025. doi:10.1101/2025.06.14.659707. Chapter 7 1. DiMasi JA, Grabowski HG, Hansen RW. Innovation in the pharmaceutical industry: new estimates of R&D costs. Journal of Health Economics. 2016;47:20–33. 2. Wong CH, Siah KW, Lo AW. Estimation of clinical trial success rates and related parameters. Biostatistics. 2019;20(2):273–286. 3. Stokes JM, Yang K, Swanson K, et al. A deep learning approach to antibiotic discovery. Cell. 2020;180(4):688–702. 4. Liu G, Catacutan DB, Rathod K, et al. Deep learning-guided discovery of an antibiotic targeting Acinetobacter baumannii. Nature Chemical Biology. 2023;19:1342–1350. 5. Richardson P, Griffin I, Tucker C, et al. Baricitinib as potential treatment for 2019-nCoV acute respiratory disease. The Lancet. 2020;395(10223):e30–e31. 6. RECOVERY Collaborative Group. Baricitinib in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial and updated meta-analysis. The Lancet. 2022;400(10349):359–368. 7. Walters WP, Murcko M. Assessing the impact of generative AI on medicinal chemistry. Nature Biotechnology. 2020;38:143–145. 8. Urbina F, Lentzos F, Invernizzi C, Ekins S. Dual use of artificial-intelligence-powered drug discovery. Nature Machine Intelligence. 2022;4:189–191. 9. Xu Z, et al. A generative AI-discovered TNIK inhibitor for idiopathic pulmonary fibrosis: a randomized phase 2a trial. Nature Medicine. 2025. 10. Jayatunga MKP, Ayers M, Bruens L, Jayanth D, Meier C. How successful are AI-discovered drugs in clinical trials? A first analysis and emerging lessons. Drug Discovery Today. 2024;29(6):104009. 11. Minikel EV, Painter JL, Dong CC, Nelson MR. Refining the impact of genetic evidence on clinical success. Nature. 2024;629:624–629. 12. Nelson MR, Tipney H, Painter JL, et al. The support of human genetic evidence for approved drug indications. Nature Genetics. 2015;47:856–860. Chapter 8 1. Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nature Medicine. 2018;24:1559–1567. 2. Kather JN, Pearson AT, Halama N, et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nature Medicine. 2019;25:1054–1056. 3. Yao X, Rushlow DR, Inselman JW, et al. Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial. Nature Medicine. 2021;27:815–819. 4. Cheng J, Novati G, Pan J, et al. Accurate proteome-wide missense variant effect prediction with AlphaMissense. Science. 2023;381(6664):eadg7492. 5. Khera AV, Chaffin M, Aragam KG, et al. Genome-wide polygenic scores for common diseases identify individuals with risk equivalent to monogenic mutations. Nature Genetics. 2018;50:1219–1224. 6. Martin AR, Kanai M, Kamatani Y, et al. Clinical use of current polygenic risk scores may exacerbate health disparities. Nature Genetics. 2019;51:584–591. 7. Wager S, Athey S. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association. 2018;113(523):1228–1242. 8. Kent DM, Paulus JK, van Klaveren D, et al. The Predictive Approaches to Treatment effect Heterogeneity (PATH) statement. Annals of Internal Medicine. 2020;172(1):35–45. 9. Calfee CS, Delucchi K, Parsons PE, et al. Subphenotypes in acute respiratory distress syndrome: latent class analysis of data from two randomised controlled trials. The Lancet Respiratory Medicine. 2014;2(8):611–620. 10. Seymour CW, Kennedy JN, Wang S, et al. Derivation, validation, and potential treatment implications of novel clinical phenotypes for sepsis. JAMA. 2019;321(20):2003–2017. Chapter 9 1. Plana D, Shung DL, Grimshaw AA, et al. Randomized clinical trials of machine learning interventions in health care: a systematic review. JAMA Network Open. 2022;5(9):e2233946. 2. Han R, Acosta JN, Shakeri Z, et al. Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review. The Lancet Digital Health. 2024;6(5):e367–e373. 3. Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. 4. Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nature Medicine. 2020;26:1364–1374. 5. Cruz Rivera S, Liu X, Chan AW, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nature Medicine. 2020;26:1351–1363. 6. Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine. 2022;28:924–933. 7. US FDA, Health Canada, MHRA. Good Machine Learning Practice for Medical Device Development: Guiding Principles. October 2021. 8. Office of the National Coordinator for Health Information Technology. Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) Final Rule. December 2023. 9. Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union. 2024. 10. Elish MC. Moral crumple zones: cautionary tales in human–robot interaction. Engaging Science, Technology, and Society. 2019;5:40–60. 11. Singhal K, Tu T, Gottweis J, et al. Toward expert-level medical question answering with large language models. Nature Medicine. 2025;31:943–950. 12. Hager P, Jungmann F, Holland R, et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine. 2024;30:2613–2622. Further Reading Eric Topol, Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again (Basic Books, 2019).** A cardiologist's broad and optimistic survey of AI across medicine, valuable for its range of examples and its argument that automation should give clinicians more time with patients. Alvin Rajkomar, Jeffrey Dean and Isaac Kohane, "Machine Learning in Medicine," New England Journal of Medicine 380 (2019): 1347–1358.** A clear, authoritative review written for clinicians, covering how models are built, where they help and the pitfalls of clinical data. Ziad Obermeyer, Brian Powers, Christine Vogeli and Sendhil Mullainathan, "Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations," Science 366 (2019): 447–453.** Short, readable and essential: the clearest demonstration of how an accurate model can be harmful when its label is a poor proxy for what matters. Jenna Wiens, Suchi Saria, Mark Sendak and colleagues, "Do No Harm: A Roadmap for Responsible Machine Learning for Health Care," Nature Medicine 25 (2019): 1337–1340.** A concise framework, from problem selection through deployment and monitoring, written by researchers who built and deployed clinical systems. Ewout Steyerberg, Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating, 2nd edition (Springer, 2019). The standard methodological text on prediction models in medicine, especially strong on validation, calibration and model updating. William L. Hamilton, Graph Representation Learning (Morgan & Claypool, 2020).** A compact technical introduction to graph neural networks and message passing, for readers who want the mathematics behind molecular machine learning. Gary Collins and colleagues, "TRIPOD+AI Statement," BMJ 385 (2024): e078378.** The current reporting standard for clinical prediction models; useful as a checklist for anyone reading or commissioning a model evaluation. Madeleine Clare Elish and Elizabeth Anne Watkins, Repairing Innovation: A Study of Integrating AI in Clinical Care (Data & Society, 2020).** An ethnographic account of the Sepsis Watch deployment at Duke that shows the human work required to make a clinical model useful. Hashtags: #AIInHealthcare #ArtificialIntelligence #HealthcareAI #MedicalArtificialIntelligence #MachineLearning #DeepLearning #MedicalDiagnostics #AIDiagnostics #MedicalImaging #RadiologyAI #ComputerAidedDiagnosis #ClinicalDecisionSupport #ElectronicHealthRecords #PredictiveHealthcare #SepsisPrediction #PatientDeteriorationPrediction #DrugDiscovery #AIDrugDiscovery #DrugDevelopment #MolecularModeling #GraphNeuralNetworks #AlphaFold #ProteinStructurePrediction #PrecisionMedicine #PersonalizedMedicine #GenomicMedicine #TreatmentResponsePrediction #ClinicalAIValidation #HealthcareDataScience #ResponsibleHealthcareAI

Download the Book (PDF): Introduction On a summer evening in 2022, two yellow school buses sat in a depot in Beverly, Massachusetts, with their students long gone home. They were not idle. A signal arrived from the local utility's peak-reduction program, and the buses began pushing electricity out of their batteries, back through their chargers and into the distribution network that served the town. Over that summer they answered thirty-two such calls and exported more than seven megawatt-hours, enough to be noticed on a utility's books and too little to make headlines [13]. Nobody in Beverly saw anything happen. That is rather the point. The idea behind those buses is now more than twenty years old. In 2005 Willett Kempton and Jasna Tomić, then at the University of Delaware, published two papers that set out, with arithmetic rather than enthusiasm, what it would take for the batteries in parked cars to sell services to the power grid [1][2]. Their core observation has not dated. A private car spends the great majority of its life parked. If it carries a large battery and is plugged in while parked, it is, electrically speaking, a storage device that happens to have wheels. Multiply it by millions and you have something that looks, from the grid's side, like a fleet of power plants that never needed a site, a permit for a smokestack, or a new transmission line. For most of the two decades since, that picture stayed mostly on paper. The reasons were not mysterious. The first mass-market electric cars had small batteries. Bidirectional chargers were rare and expensive. Utilities had no way to pay a car for what it did. Carmakers worried about warranties. And, less often discussed, there was no common language in which a car, a charger, a charging network and a grid operator could agree on who would move how much energy, in which direction, at what moment, under whose authority, and with what guarantee that the driver would still be able to leave in the morning. That last problem is the subject of this book. Its controlling claim is simple to state: a parked electric vehicle becomes a dependable grid resource only when an unbroken chain of agreements holds from the grid operator down to the battery cell, and the battery is the easiest link in that chain. The hard links are the others. There is the electrical agreement, the interconnection rules that let a device inject power into a public network without endangering line workers or destabilising voltage. There is the digital agreement, the stack of protocols, with ISO 15118-20 at its base, that lets a vehicle and a charger negotiate power flow in both directions and lets the systems upstream of the charger direct that flow. And there is the commercial agreement, the tariffs, market rules and contracts that turn a kilowatt-hour exported at the right moment into money that reaches someone who cares. When all three hold, the results are no longer hypothetical. By late 2026 drivers in Britain can lease a car bundled with a bidirectional charger and an energy tariff that gives them free home charging in return for plugging in often and long enough [16]. A Dutch car-sharing service in Utrecht runs Renault hatchbacks that discharge into the city's grid between rentals [15]. In Germany, BMW and the utility E.ON launched the country's first consumer vehicle-to-grid offer in early 2026, shortly after the federal parliament removed a rule that had charged grid fees twice on electricity stored and returned [17][18]. In Texas, Tesla has begun paying some Cybertruck owners to discharge into the grid during stress events [23]. In California, an entire urban school district's bus fleet was bought bidirectional from the start [14]. When any one of the three agreements fails, projects stall. That pattern recurs throughout the record, and it explains most of what has happened in the field. What this book does and does not cover The book concentrates on vehicle-to-grid in the strict sense: vehicles discharging into the public electricity network, under external direction, as a service. It treats vehicle-to-home and vehicle-to-building, where the car supplies a house or a site behind the meter, as close relatives, because much of the hardware and protocol work is shared and because behind-the-meter use is often how bidirectional capability first earns its keep. It mostly sets aside vehicle-to-load, the ability to run a power tool or a kettle from a socket on the car, which needs no grid agreement at all. It does not try to survey every pilot. Hundreds have been run, and many say the same thing. Instead it follows the chain link by link. The first two chapters establish what a parked vehicle actually is in electrical terms and what the grid actually buys, because "peak demand" covers several very different services with very different requirements. The middle three chapters take the technical chain apart: the ISO 15118 conversation between vehicle and charger, the protocols that connect the charger to the systems above it, and the power electronics and interconnection rules that decide whether a discharge is legal and safe. The sixth chapter confronts the question every driver asks first, which is what this does to the battery. The seventh explains why fleets, and especially fleets with predictable schedules, are where the economics close first. The eighth assembles the pieces into the thing the title promises, the virtual power plant, and examines the market and regulatory rules that decide whether it gets paid. A note on the reader The reader this book has in mind is intelligent and curious but not an electrical engineer: a fleet manager weighing an electrification plan, a policymaker drafting a tariff, an investor looking at an aggregator, a utility planner from the generation side who has never had to think about a car, or simply someone who owns an electric vehicle and wants to know what it could do. Technical terms are explained when they first appear. Where a mechanism matters, it is walked through step by step in words. Where numbers are given, their source is named, and where a figure is uncertain or disputed, the book says so rather than smoothing it over. One caution belongs here at the start. Vehicle-to-grid attracts two kinds of exaggeration. The first is the promotional kind, which multiplies the number of electric cars on the road by the size of their batteries and announces that the grid's storage problem is solved. It is not, because most of those cars are not plugged in at the right time, most are not bidirectional, most drivers will not surrender most of their battery, and a kilowatt-hour that cannot be dispatched reliably is worth little to a grid operator. The second is the dismissive kind, which points at battery wear and charger costs and declares the whole idea a curiosity. That is also wrong, because the wear is smaller and more manageable than early studies suggested when discharge is planned well, charger costs are falling, and in specific settings the money already works. The truth sits in the details, and the details are mostly about the chain. Chapter 1: The Idle Battery Start with the car, not the grid. A modern battery-electric car carries somewhere between about 40 and 100 kilowatt-hours of usable energy, and larger pickups and SUVs carry more; Tesla's Cybertruck, for instance, has a pack of roughly 123 kilowatt-hours [23]. A typical household in the United States uses on the order of 30 kilowatt-hours of electricity a day; a typical British household uses much less. So a single mid-sized electric car, fully charged, holds something like one to three days of an average home's electricity. That comparison is the first fact on which everything else rests: the energy stored in a car is not trivial when set against the loads around it. The second fact is about time. Transport researchers have long observed that private cars spend the overwhelming majority of their lives stationary. The figure most often quoted, that a car is parked about 95 percent of the time, comes from back-of-envelope estimates popularised by the transport analyst Paul Barter and others, and it holds up as an order of magnitude across rich countries [3]. A car driven an hour a day is parked for twenty-three. The battery is idle for most of those hours, and for much of that time the car sits at a home, a workplace or a depot where a charger could be. The third fact is about need. Most daily driving uses a small fraction of a modern battery. A commuter who drives 50 kilometres a day in a car that covers about 6 kilometres per kilowatt-hour uses around 8 kilowatt-hours. If the car holds 60, roughly 50 kilowatt-hours sit unused at the end of the day, and much of it will still be unused the next morning. The owner paid for the whole battery because of the few days a year they drive far, not the many days they do not. Put those three facts together and the opportunity is obvious. The difficulty is that each of them hides a qualification, and the qualifications are where the engineering and the business live. From smart charging to bidirectional power It helps to be precise about the different things a car can do for the grid, because they are often blurred together. The simplest is unmanaged charging: plug in, and the car draws as much power as the charger allows until it is full. From the grid's point of view this is the worst case. In many places drivers arrive home in the early evening, which is exactly when household demand peaks, and an unmanaged fleet of chargers adds to that peak. The next step is managed or smart charging, often called V1G. Here the charging rate can be varied or shifted in time, by the driver, the charger, a utility or a third party. The car never sends power back; it simply charges when power is cheap or plentiful and holds off when the grid is stressed. This is not a lesser cousin of vehicle-to-grid to be skipped past. Most of the value that flexible vehicles offer the grid can be captured by smart charging alone, because the largest single benefit is not adding to the peak. A car that shifts its charging from six in the evening to two in the morning has, from the grid's perspective, removed several kilowatts from the peak at almost no cost to anyone. The British trial discussed later in this chapter found smart charging alone saved its participants about £120 a year [12]. The final step is bidirectional charging, in which energy can flow out of the battery as well as in. The terminology here has multiplied. Vehicle-to-load (V2L) means powering appliances from a socket on the car. Vehicle-to-home (V2H) means supplying a house, usually through a transfer switch or a home energy system that isolates the house from the grid during an outage or offsets household consumption when connected. Vehicle-to-building (V2B) is the same at the scale of an office, school or depot. Vehicle-to-grid (V2G) means exporting through the meter into the public network, under external direction, as a service someone pays for. The umbrella term V2X covers all of them. The difference between V2H and V2G is not mainly electrical. A car supplying a house during an evening peak and a car exporting to the grid at the same moment may push current through the same charger. The difference is that V2G crosses the meter. Once power crosses the meter, it enters a public system, and the rules of that system apply: interconnection standards, export permissions, metering, settlement, and the market or tariff that defines what the export is worth. A great deal of what follows is about those rules. What makes a vehicle valuable to the grid A grid operator does not value energy in the abstract. It values the ability to change supply or demand at a particular place, at a particular time, by a particular amount, with a particular certainty. A parked car offers each of these in a distinctive way. Power and energy are different. The size of the battery, in kilowatt-hours, is energy. The rate at which energy can move, in kilowatts, is power. For most grid services, the binding constraint on a parked car is power, not energy, because the charger limits it. A home charger in North America typically delivers 7 to 11 kilowatts; in Europe, 7 or 11 kilowatts from single- or three-phase supply. Bidirectional home units on the market by 2026, such as the Wallbox Quasar 2 offered to Kia EV9 owners, are rated around 11 to 12 kilowatts in both directions. At 11 kilowatts, emptying even 20 kilowatt-hours takes nearly two hours. So a home V2G car is best thought of as an 11-kilowatt resource with a few hours of duration, not as a 60-kilowatt-hour reservoir that can be tipped out at once. Depot chargers for buses and trucks are larger, often tens of kilowatts per vehicle or more, which is one reason fleets matter so much. Availability is probabilistic. Any single car may or may not be plugged in when it is needed. Its driver may have left early, forgotten to plug in, or set a high minimum state of charge for a long trip the next day. A grid operator cannot rely on one car. It can rely, statistically, on a large number of them. The British Sciurus trial, which ran around three hundred V2G chargers in homes from 2018 to 2021, found that participants plugged in far more often than ordinary electric car owners: about 57 percent of the time before the pandemic and 70 percent during the 2020 lockdowns, against 30 to 40 percent for drivers without V2G [12]. That is a striking behavioural finding. Paying people to plug in changes when they plug in. But it also shows the scale of the problem: even motivated participants were absent a large share of the time. Location matters. Electricity is not a single pool. A car in a suburb whose local transformer runs hot on summer evenings is worth more to the distribution company there than a car in a district with plenty of spare capacity. Some of the highest values for vehicle-to-grid come not from wholesale markets, which price energy across whole regions, but from relieving specific local constraints. This makes vehicle-to-grid partly a local business, and it means the same car can be worth very different amounts depending on where it is parked. Speed and precision are strengths. A battery inverter can change its output within a fraction of a second, far faster than a gas turbine. For the grid services that pay for speed, which are discussed in the next chapter, a well-controlled battery on wheels can be technically excellent, if the communication path to it is fast and reliable enough. That last condition is the crux of the middle chapters of this book. From headline capacity to dependable capacity These qualifications compound, and it is worth seeing by how much. The following calculation uses round, illustrative assumptions rather than data for any real region, but each step reflects one of the constraints just described. Imagine a region with one million electric cars averaging 60 kilowatt-hours of usable battery each. The headline figure is 60 gigawatt-hours of storage, a number large enough to appear in press releases as equivalent to many large grid batteries. Now apply the constraints in turn. Suppose one car in five is both bidirectional and enrolled in a program: 200,000 cars. Suppose half of them are plugged in at the evening peak, which is better than ordinary owners manage but in line with motivated trial participants: 100,000 cars. Each can export at about 11 kilowatts through a home charger, so the available power is about 1.1 gigawatts. Each owner is willing to release perhaps 15 kilowatt-hours before hitting the floor they have set for the next day's driving, so the available energy is about 1.5 gigawatt-hours, enough to sustain the 1.1 gigawatts for a little over an hour, or a smaller output across the full evening peak. And the aggregator, needing confidence, will commit somewhat less than its central forecast. The result is a resource of roughly a gigawatt for an hour or so, from a fleet whose headline storage was some forty times larger in energy terms. A gigawatt of dispatchable capacity is still significant; it is comparable to a large power station. But the gap between the headline and the dependable figure is the gap between the battery and the chain. Every step that narrowed it, from enrolment to plug-in rate to export power to owner limits, was a matter of hardware choice, behaviour, contract or rule, not of battery chemistry. Raising any one of them, by making bidirectional capability standard, by paying people to plug in, or by allowing higher export power where the local network can take it, raises the dependable figure directly. Twenty years of almost The history of vehicle-to-grid is a history of correct predictions arriving late. Kempton and Tomić's 2005 papers did the essential accounting [1][2]. They distinguished the services a vehicle might sell, from bulk energy to fast-acting reserves and frequency regulation, and showed that the most valuable early market was not selling energy at peak, where a car competes with power plants that have far lower costs per kilowatt-hour, but selling fast regulation, where the payment is largely for standing ready rather than for energy delivered. They calculated what a vehicle could earn, set it against the cost of the extra hardware and battery wear, and concluded that regulation could pay. The structure of that argument has survived intact. What has changed is the numbers on both sides. The Delaware group went on to build working demonstrations. By the end of the 2000s a small number of converted cars, including a Toyota Scion modified by the company AC Propulsion and known as the eBox, were responding to signals from PJM, the grid operator for the mid-Atlantic United States, and were described in the press as selling power into its regulation market. The work was real, but it was hand-built, and every connection between car, charger and grid was bespoke. Meanwhile, the first production vehicle capable of serious bidirectional operation came from Japan. After the earthquake and tsunami of March 2011 and the power shortages that followed, Nissan and the electronics maker Nichicon offered a system that let the original Nissan Leaf power a home. The Leaf charged through the Japanese CHAdeMO fast-charging connector, whose designers had built in a communication link over which the car and charger could coordinate discharging. For most of the 2010s, nearly every serious V2G experiment in the world, from Denmark to Britain to California, used Nissan Leafs or Nissan vans and CHAdeMO chargers, because they were the only mass-produced vehicles that could do it. That dependence turned out to be a trap. The rest of the global car industry, led by European and American carmakers, standardised on the Combined Charging System (CCS), whose communication was defined first by the German specification DIN SPEC 70121 and then by the international standard ISO 15118-2, published in 2014 [5]. Neither defined how a car should discharge into the grid. As CCS came to dominate outside Japan, and as later Nissan models moved away from CHAdeMO in many markets, the vehicles on which V2G had been proven became a shrinking share of the fleet. The industry needed a version of the CCS conversation that could handle power flowing both ways. That arrived as ISO 15118-20, published in 2022 [4], and the years since have largely been spent implementing it in vehicles, chargers and backend systems. What the demonstrations proved The Danish experience is a good illustration of both promise and limits. From September 2016, a Copenhagen-area utility, Frederiksberg Forsyning, ran ten Nissan e-NV200 vans on bidirectional CHAdeMO chargers rated at about 10 kilowatts each, controlled by the aggregator Nuvve with the Italian utility Enel supplying the chargers. The vans provided frequency containment reserve, a service in which the Danish grid operator pays resources to absorb or inject power automatically as grid frequency drifts. They were available on weekday evenings and nights and all weekend, about 6,150 hours a year, and earned roughly €1,400 per vehicle per year from the service, according to a presentation Nuvve gave to California regulators in 2017 [11]. Those numbers deserve to be read carefully. €1,400 a year per vehicle is real money, enough over a few years to pay for a bidirectional charger. But the vans were fleet vehicles with highly predictable parking, Denmark's frequency reserve market happened to be open to small resources and paid well at the time, and the prices in such markets fall as more batteries enter them. The result proved that the technology worked and that the service could pay. It did not prove that every car everywhere could earn the same. The Sciurus trial in Britain, run by OVO Energy with the consultancy Cenex, Nissan and the charger maker Indra, gave the domestic view [12]. Across around three hundred homes over three years, participants saved on average about £340 a year with V2G, compared with about £120 for smart charging alone. When the trial modelled stacking the frequency services that National Grid then offered, notably Dynamic Containment, the potential rose to around £725. The same report found that the V2G hardware and installation cost about £3,700 more than a smart charger, and concluded that costs had to fall before mass adoption made sense. That is the shape of the problem in miniature: the value exists, the behaviour can be shaped, and the bottleneck is everything around the battery. Why now Several things changed between those trials and the present. Batteries grew. A typical new electric car in 2026 carries roughly twice the usable energy of an early Leaf, which means far more headroom between what a driver needs and what the car holds. Bidirectional capability moved from aftermarket add-on to factory feature. Hyundai, Kia, Ford, General Motors, Volkswagen, BMW, Renault, BYD, Tesla and others ship vehicles that can at least power a home, and a growing number can export to the grid. The protocol gap closed on paper and is closing in practice. ISO 15118-20 defines bidirectional power transfer for both alternating- and direct-current charging. The backend protocol that most chargers speak, OCPP, gained a bidirectional functional block with version 2.1 in January 2025 [6]. Interconnection standards have caught up: in the United States, a certification pathway for AC bidirectional chargers, UL 1741 SC, was approved in May 2026 [8]. And the rules began to move. The European Union's alternative fuels infrastructure regulation requires new and renovated charging points to support ISO 15118-20 from January 2027 [19]. Germany stopped charging grid fees twice on stored and returned electricity in late 2025 [18]. California passed a law in 2024 giving its regulators the power to require bidirectional capability in classes of vehicle where a compelling use case exists [21]. In the United States, federal regulators had already, in 2020, ordered wholesale markets to open themselves to aggregations of small distributed resources [20]. None of these alone makes vehicle-to-grid inevitable. Together they move it from a demonstration to a product. The remaining chapters explain what the product has to do, and why each link in the chain matters. Chapter 2: What the Grid Actually Buys The phrase "discharging power back to the grid during peak demand" sounds like a single act. It is not. A power system buys many different things from the resources connected to it, on timescales that run from a fraction of a second to several years, and each has its own requirements for speed, duration, location, measurement and reliability. A vehicle that is excellent at one may be useless for another. Before looking at how a car and a charger talk to each other, it is worth understanding what they are being asked to do, because the protocols, the hardware and the contracts are all shaped by the service. The grid's balancing problem An alternating-current power system has one unforgiving property: at every instant, the power fed in must equal the power taken out plus losses. There is almost no storage inside the network itself. When demand rises faster than supply, the spinning generators connected to the grid slow very slightly, giving up some of their rotational energy, and the system frequency falls below its nominal value, 50 hertz in Europe and most of the world, 60 hertz in North America. When supply exceeds demand, frequency rises. Frequency is therefore a live, system-wide measurement of balance, and every grid operator works to keep it within a narrow band. They do this in layers. The first layer acts within seconds and is automatic: resources sense frequency themselves and push it back toward nominal. The next layers act over minutes and are directed by the operator's control systems, restoring frequency fully and freeing up the fast resources to act again. Over longer horizons, the operator schedules generation and demand hour by hour and day by day through energy markets, and over years it secures enough capacity to meet the highest demand expected. Separately, the companies that run local distribution networks have their own problems: transformers and cables that overheat on certain evenings, voltages that sag at the ends of long rural lines or rise on sunny afternoons when rooftop solar floods a street. Each of these layers is a product a vehicle could, in principle, provide. They are worth taking in turn, roughly from the fastest to the slowest. Fast services: frequency response and regulation Frequency containment and response. In continental Europe this is called frequency containment reserve (FCR); in Britain the National Grid's electricity system operator introduced a family of fast services, starting with Dynamic Containment in 2020; in the Nordic countries there are normal-operation and disturbance variants. The common principle is that a resource holds some headroom in reserve and adjusts its output automatically, in proportion to how far frequency has drifted, within seconds or less. A battery is technically ideal. It can respond in milliseconds and in either direction, absorbing power when frequency is high and injecting it when frequency is low. These services pay mainly for availability: a resource is paid per megawatt per hour for standing ready, whether or not it is called upon much. That suits vehicles well, because the energy actually moved over an hour is usually small and roughly balances out, since frequency wanders above and below nominal. It was this property that made Kempton and Tomić identify regulation as the first market vehicle-to-grid could win [1], and it was a frequency containment market that paid the Danish vans their €1,400 a year [11]. There is a catch, and it bears directly on the protocols. Fast frequency services usually require the resource to measure frequency locally and respond on its own, rather than wait for a remote instruction, because a round trip through a cloud server can take longer than the response the service demands. They also require high-resolution metering, often at one-second intervals or finer, so that the operator can verify that each resource did what it was paid to do. A car whose charger cannot measure frequency precisely, cannot act on it locally, or cannot report what it did at the required resolution, cannot be in this market however good its battery is. Much of the friction in early projects came from precisely this mismatch. Secondary regulation. In the American markets run by independent system operators such as PJM, regulation is a service in which the operator sends a continuously updated setpoint, every few seconds, and resources follow it. PJM's market separates a slow traditional signal from a fast, energy-neutral one designed for batteries. Here the instruction does come from outside, which means the communication path from the operator through the aggregator to the charger and the vehicle must be fast and reliable enough to track a signal changing every couple of seconds. The weakness of fast services as a long-term business is that they are small markets. A grid needs only so many megawatts of fast reserve, and stationary battery projects have flooded into them in many countries. Prices in Britain's Dynamic Containment market fell sharply after its early years as grid-scale batteries piled in. A vehicle-to-grid business built solely on frequency response risks being a business built on a shrinking margin. The fast services remain valuable to vehicles, but more as one revenue among several than as a foundation. What a service actually demands It is instructive to look at what entering one of these services involves in practice, because the gap between "a battery can do this" and "this battery is allowed to do this" is where many projects lose time. Take a fast frequency service of the kind European and British operators buy, and follow an aggregator trying to enter a portfolio of cars into it. The details differ by country, but the shape is common. First comes prequalification. The operator wants evidence that the resource behaves as promised before it will pay for it. The aggregator must typically demonstrate, in a supervised test, that its portfolio responds to a simulated frequency deviation with the right amount of power, in the right direction, within the required time, and sustains the response for the required duration. Every vehicle and charger type in the portfolio may need to be represented, and a change of firmware can raise the question of whether the test must be repeated. Second comes metering and telemetry. The operator will usually require the aggregator to report the portfolio's output, and often the frequency it measured, at high resolution and close to real time, and to keep detailed records for audit. Chargers that report energy every fifteen minutes cannot meet this; the aggregator must either choose hardware that can, or install additional meters. Third comes energy management. A symmetric frequency service asks the resource to stand ready to move up or down. A car offering 7 kilowatts of response must therefore have room both to absorb 7 kilowatts and to deliver 7 kilowatts for the required duration. If the car is nearly full, it cannot absorb; if it is nearly empty, it cannot deliver. The aggregator's software must keep each battery in a band that allows both, while still meeting the owner's departure needs, and must adjust the portfolio continually as cars arrive and leave. This is a demanding control problem, and solving it well is a large part of what distinguishes a competent aggregator. Fourth comes the bid. The aggregator must commit, usually a day or more ahead, to a quantity of capacity for each hour or block. Committing too much risks penalties if too few cars turn up; committing too little leaves money on the table. The commitment rests on the forecast of availability, which rests on data about how the owners behave. None of these steps is exotic. Stationary battery operators go through them routinely. But each depends on a link in the chain beyond the battery: test procedures that recognise aggregated mobile resources, chargers with suitable meters, software with access to each car's state of charge, and contracts with owners that permit the battery to be held in a band. The service is defined by the grid's needs, and the vehicle must be fitted to it, not the reverse. Slower, local and behind-the-meter services Operating reserves. Beyond the fastest layer, operators buy reserves that can be called within minutes and held for tens of minutes or hours. In Europe these are the automatic and manual frequency restoration reserves. A vehicle can provide these if it can be counted on for the required duration, which at 11 kilowatts from a home charger is not a problem for an hour but becomes one for several. Energy arbitrage. The simplest economic idea is to charge when electricity is cheap and discharge when it is expensive. In markets with strongly varying prices, such as those with high solar output that drives midday prices down and leaves evenings expensive, the spread can be large. The difficulty is that arbitrage pays per kilowatt-hour moved, which means it uses the battery hard. Every kilowatt-hour discharged must be recharged, losing perhaps 10 to 20 percent of it in round-trip conversion through the charger and battery, and each cycle consumes a little of the battery's life. Arbitrage works when the spread between cheap and expensive hours comfortably exceeds those losses plus the wear cost. On many days it does not. Peak reduction and demand response. This is the service most people imagine when they hear about vehicles discharging during peak demand. Grid operators and utilities run programs that pay participants to reduce consumption or export power during a limited number of high-stress hours each year: heat waves, cold snaps, unexpected plant outages. California's Emergency Load Reduction Program, in which participants in a 2025 General Motors and Pacific Gas and Electric pilot had to enrol, is one example [22]. The Massachusetts ConnectedSolutions program that paid the Beverly school buses is another [13]. These programs suit vehicles well because events are relatively rare and short, often a few hours on a few dozen days a year, so battery wear is limited, and because the payments can be large per event. Capacity. The slowest service is capacity: being available to meet the year's highest demand. In some markets this is bought through auctions years in advance; elsewhere it is handled through utility planning. Capacity is in principle the largest prize for vehicles, because the cost of building and keeping power plants and grid infrastructure that run only a few hours a year is enormous. In practice it is the hardest to win, because capacity buyers demand high certainty of performance, multi-year commitments, and penalties for failure, and an aggregation of privately owned cars has trouble making such promises. The local grid All of the above are system-wide services bought by transmission or market operators. The distribution network has separate needs, and they may in the end be the most important for vehicles. The reason is simple. When electric cars, heat pumps and other new electric loads arrive on a residential street, the first thing to run out of capacity is often not the region's generation but the street's transformer or the feeder cable. Upgrading these is expensive and slow. A distribution company that can call on vehicles in a specific neighbourhood to cut charging, or to export a little, during the few hours each year when the local transformer would otherwise overload, can defer or avoid the upgrade. In Britain, distribution network operators have run flexibility tenders for exactly this purpose, open to any resource that can deliver in the right place. The value is locational and seasonal, and the requirement is often modest: a few kilowatts from a few dozen homes for a few hours. Distribution networks also care about voltage. Long rural lines can see voltage fall too low when everyone draws power at once, and residential streets with many solar panels can see it rise too high at midday. Modern inverters, including the ones inside bidirectional chargers and vehicles, can help by adjusting the reactive power they exchange with the grid, a capability that interconnection standards such as IEEE 1547-2018 now require of new inverter-based resources [7]. Reactive power support costs the battery almost nothing, because it shifts the timing relationship between voltage and current rather than draining stored energy. It is one of the quietly valuable things a bidirectional charger can do, and one of the reasons the interconnection rules matter so much. Behind the meter Finally, some of the most reliable value for bidirectional vehicles comes before power ever reaches the grid. Bill management. Many commercial customers pay a demand charge, set by their highest fifteen- or thirty-minute power draw in a month. A school, office or recreation centre with a single afternoon spike pays for that spike all month. A bidirectional vehicle parked on site can discharge into the building during the spike and cut the bill. The city of Boulder, Colorado, reported that a vehicle-to-building installation by Fermata Energy at its North Boulder Recreation Center reduced energy costs by roughly $250 a month in its first year [24]. Under time-of-use tariffs, a household can do something similar, running the house from the car during expensive hours and recharging overnight. Resilience. The ability to keep a house or building running during an outage has no wholesale price, but people value it. Much of the early commercial push for bidirectional capability in North America, from Ford's F-150 Lightning with its home backup system to General Motors' vehicle-to-home kits, has been sold on resilience rather than grid revenue. The GM and PG&E pilot launched in March 2025 offered up to $4,500 toward the cost of a vehicle-to-home bundle, explicitly pairing backup power with enrolment in a peak-reduction program [22]. Behind-the-meter value matters to the vehicle-to-grid story because it often pays for the hardware. Once a bidirectional charger is installed for backup or bill savings, the marginal cost of also offering grid services drops sharply. Matching services to vehicles Table 1 sets these services side by side, with the response times and durations typical of each and a rough judgement of how well each suits a parked vehicle. The response times are indicative of common market designs rather than exact rules for any one country. Table 1. Grid services a parked vehicle can offer, and how well they fit. Service Typical response Typical duration Main buyer Fit for vehicles Fast frequency response Under 1 to 30 seconds Minutes, energy-neutral Transmission operator Strong technically; small market, prices falling Regulation / restoration reserve Seconds to 5 minutes 15 minutes to 1 hour Transmission operator Good if communication is fast Peak reduction events Minutes to day-ahead notice 1 to 4 hours, few days a year Utility or operator Very good; low wear, high value per event Energy arbitrage Day-ahead schedule Hours, daily Market or supplier Mixed; wear and losses eat margin Local network relief Scheduled or minutes 1 to 3 hours, seasonal Distribution operator Very good where constrained Demand charge / bill savings Real time on site Minutes to hours Site owner Very good; often pays for hardware Two patterns stand out. First, the best fits are services that ask for power at predictable times, for limited durations, with modest annual energy throughput. That is a description of peak events and local network relief more than of daily arbitrage. Second, the speed and reliability of the communication chain determine which services are reachable at all. A charger that can only accept a schedule a day in advance can do arbitrage and planned peak reduction. A charger that can accept a new setpoint every few seconds, report its output every second, and act on frequency locally can do almost everything. The difference between those two chargers is not in the battery. It is in the protocols, which is where the next chapter goes. Chapter 3: Inside the Handshake When a driver plugs an electric car into a charger, something like a short diplomatic negotiation takes place in the first few seconds. The two machines must discover each other, agree on a language, establish whether they trust each other, decide who will pay, exchange what each is physically capable of, check that the cable and connections are safe, and only then begin moving energy. If power is to flow out of the car as well as in, they must also agree on how much discharge is permitted, for how long, and under what conditions it must stop. The international standard that governs this negotiation for most of the world's cars is ISO 15118, and its 2022 part, ISO 15118-20, is the one that made bidirectional charging a first-class citizen of the conversation [4]. This chapter walks through that conversation from the copper wire upward. The detail matters for anyone trying to understand vehicle-to-grid, because a surprising share of the practical failures in early projects, the chargers that would not discharge, the cars that dropped out mid-session, the fleets that could not be enrolled, traced back to some step in this sequence. From the wire upward The control pilot Every conductive charging connector in the CCS family, and the North American NACS connector standardised as SAE J3400, carries a thin signalling wire called the control pilot alongside the heavy power conductors. The charger places a 1-kilohertz square wave on this wire, as defined in the IEC 61851 standard. The car alters the voltage of that wave through resistors, and the voltage level tells the charger the state of the connection: no vehicle, vehicle connected, vehicle ready to accept energy. In the simplest form of alternating-current charging, that is the whole conversation. The width of each pulse in the square wave, its duty cycle, tells the car the maximum current it may draw. There is no identity, no negotiation, no exchange of information about the battery, and no possibility of discharge. When the charger sets the duty cycle to 5 percent, however, it is signalling something different: digital communication is required, and current limits will be negotiated in messages rather than read from the pulse width. This is the doorway into ISO 15118. The layers of the conversation ISO 15118 is not a single document but a family. Part 1 sets out the general concepts and use cases. Part 3 defines the physical and data-link layers for wired charging. Part 2, published in 2014, defines the network and application protocol for the first generation of the conversation [5]. Part 20, published in 2022, defines the second generation, including bidirectional power transfer [4]. Other parts cover wireless charging and conformance testing. The car's side of the conversation is handled by what the standard calls the electric vehicle communication controller; the charger's side by the supply equipment communication controller, usually abbreviated SECC. The conversation is built in layers, like any modern network, and it is worth climbing them one at a time. The physical link. Rather than add wires, the designers chose to send data as a high-frequency signal superimposed on the control pilot and protective earth conductors, using a version of powerline communication called HomePlug Green PHY. In effect, the control pilot wire doubles as a small data network between car and charger. Finding the right partner. At a depot or a car park, many chargers may share cable trays and grounding, and powerline signals leak between them. A car plugged into charger 3 might hear charger 4 just as clearly. Part 3 of the standard solves this with a procedure called signal level attenuation characterization, or SLAC. The car broadcasts a series of sounding messages; every charger that hears them measures how strongly they arrived and reports back; the car picks the charger with the least attenuation, which is almost always the one it is physically plugged into, and the two form a private logical network with their own encryption keys. This step sounds obscure, but it is what lets fifty buses at a depot each talk to their own charger rather than to their neighbour's. Finding the address. The car now needs to know the charger's network address. It sends a short discovery request, using a protocol defined in ISO 15118 called SECC discovery, over a standard internet protocol. The charger replies with its address, its port and whether it requires an encrypted connection. Both sides use IPv6, the modern version of the internet's addressing scheme, so each device can form its own address on the link without any central configuration. Securing the channel. The car opens a standard TCP connection to the charger and, where required, wraps it in Transport Layer Security, the same cryptographic protocol that protects online banking. In ISO 15118-2, encryption was used mainly when the session involved automatic authentication; in ISO 15118-20 it became mandatory, using TLS 1.3, the current version. This was a deliberate decision. A conversation that can command a vehicle to export power into a public grid is a conversation that an attacker would very much like to join. Agreeing on a language. The first application-level message is a handshake in which the car lists the protocol versions it supports, for example the older German specification DIN SPEC 70121, ISO 15118-2 and ISO 15118-20, in order of preference, and the charger picks one. This backward compatibility is why a 2026 charger can still serve a 2016 car. It is also why bidirectional charging only happens when both sides support the -20 version: if either falls back to an older one, the session proceeds as ordinary one-way charging. Encoding. The messages themselves are defined as structured XML documents, but sending verbose XML over a low-bandwidth powerline link would be wasteful, so they are compressed into a compact binary form called Efficient XML Interchange, or EXI, before transmission. Engineers who build chargers spend a surprising amount of their time on EXI encoders and decoders, because a single mismatch in how the two sides interpret the schema can cause a session to fail without any obvious reason. The negotiation itself Once the channel is up, the application conversation in ISO 15118-20 proceeds through a defined sequence of request-and-response pairs. Each request comes from the car; each response from the charger. For a direct-current bidirectional session it runs roughly as follows. Session setup. The car identifies itself and the charger opens a session, assigning it an identifier. If a previous session was paused, it can be resumed here. Authorisation. The two agree on how the session will be authorised and paid for. The options include external identification, such as a card or an app, or Plug & Charge, in which the car proves its identity and the driver's contract with certificates. The charger checks authorisation, often by consulting its backend system, and says yes or no. Service discovery and selection. The charger lists the energy transfer services it offers. In ISO 15118-20 these include direct-current charging, alternating-current charging, and, crucially, their bidirectional counterparts, labelled DCBPT and ACBPT, where BPT stands for bidirectional power transfer. Further services, such as wireless or pantograph charging, are also defined. The car asks for details of the services it is interested in and then selects one, along with a control mode, described below. Parameter exchange. The car and charger exchange their physical limits: maximum and minimum voltage, maximum charge current and power, and, for a bidirectional session, maximum discharge current and power. The car also states its energy needs. Schedule exchange. The charger provides the information on which power flow will be decided: price tables, power limits over time, or both. The car responds with what it intends to do or accepts that the charger will direct it. Safety checks. For direct-current sessions, the charger performs a cable check to confirm insulation integrity, then a pre-charge step in which it raises its output voltage to match the car's battery before the car closes its contactors. This prevents a damaging inrush of current. Power delivery. The car announces that it is starting to draw or deliver power. The charge loop. For the rest of the session, the car and charger exchange status messages several times a second. The car reports its present voltage, current and state of charge; the charger reports what it is delivering or absorbing and, in some modes, sends new targets. Stop and welding detection. At the end, the car announces it is stopping, the charger ramps down, and for direct-current sessions the charger checks that the car's contactors have opened properly rather than welded shut. The session is then closed or paused. The sequence for an alternating-current session is shorter, because there is no external direct-current power stage to check, but the logic of authorisation, service selection and control is the same. Two ways to decide power: scheduled and dynamic The most consequential design choice in ISO 15118-20, for anyone interested in vehicle-to-grid, is the split between two control modes. In scheduled mode, the charger gives the car a set of constraints, such as maximum power allowed in each period, together with prices that may vary over time, including prices for energy discharged. The car's own software then works out a charging and discharging plan that respects the limits, meets the driver's needs and minimises cost, and tells the charger what it intends to do. The car is the decision-maker. This mode suits households and situations where the car's manufacturer wants to keep control of battery use, because the car can weigh price signals against its own model of battery health. In dynamic mode, the roles reverse. The car tells the charger what it needs: when the driver intends to leave, how much energy it must have by then, the most energy it can accept, and the minimum energy it must retain. The charger, or more often the backend system behind it, then sends power setpoints during the charge loop, and the car follows them within its stated limits. The charger, and through it an aggregator or fleet manager, is the decision-maker. This mode suits fleets and grid services, because a central system can coordinate many vehicles at once and respond to external signals within seconds. The energy limits in dynamic mode are what protect the driver. The standard lets the car declare a target energy for departure, a maximum it will accept, and, for bidirectional sessions, a minimum below which discharging is not allowed. If the battery is below that minimum when the car plugs in, the charger must bring it up before any discharging starts. These parameters are the technical form of a promise every vehicle-to-grid program must make: that the car will be ready when its owner needs it. The car can also renegotiate during the session, for instance if the driver changes the departure time in an app. The existence of two modes reflects a real tension in the industry. Carmakers generally prefer scheduled mode, or at least retaining a veto, because they carry the battery warranty and want their own software to decide how hard the battery is worked. Grid service providers generally prefer dynamic mode, because it lets them respond to fast signals and aggregate predictably. The standard accommodates both. Which one a given program uses depends on the commercial agreement between the carmaker and the aggregator, which is one more reason the commercial links in the chain cannot be separated from the technical ones. Trust: Plug & Charge and its certificates Plug & Charge is often described as a convenience feature: the driver plugs in and charging starts, with no card or app. For vehicle-to-grid it is more than that. It is the mechanism by which a charger can know, cryptographically, which car is connected and which contract governs it, and therefore who should be paid for what that car exports. It works through a public key infrastructure, a hierarchy of digital certificates similar to the one that secures websites. At the top sit root certificate authorities trusted by the industry; several operators, including the German company Hubject and the industry association CharIN, run such roots. Beneath them, charging station operators obtain certificates for their chargers, carmakers install provisioning certificates in their vehicles at the factory, and mobility service providers, the companies with which drivers hold charging contracts, issue contract certificates that are installed into the car. When a car plugs in, it and the charger authenticate each other through these certificates over the encrypted channel, and the charger can determine, without human input, which contract applies. ISO 15118-20 improved the arrangement in ways that matter for vehicle-to-grid. It allows a car to hold several contract certificates, so the same vehicle could have one contract for public charging and another with a home energy supplier that governs discharging. And it accommodates multiple root authorities, which matters because no single company runs the whole ecosystem. The weakness is administrative rather than mathematical: a certificate system works only if every party issues, renews and revokes certificates properly, and the commercial agreements between root operators, carmakers, charging operators and mobility providers took years to settle. How the alternatives compare The Japanese CHAdeMO standard reached bidirectional operation first. Its chargers and cars talk over a dedicated controller-area-network link on separate pins, a robust automotive bus, and its designers published guidelines for discharging well before ISO 15118-20 existed. Nearly every vehicle-to-grid pilot of the 2010s depended on it. But CHAdeMO handled less of the surrounding problem: its communication carried electrical parameters well but did not include a certificate-based identity system or the rich scheduling and pricing information that -20 carries. Grid integration in CHAdeMO projects was therefore done by each charger vendor's own software, which made projects work but made them hard to replicate. Table 2 sets the main vehicle-to-charger protocols side by side. Table 2. Vehicle-to-charger protocols compared. Protocol Published Physical link Bidirectional Built-in security CHAdeMO 2010 onward CAN bus on dedicated pins Yes, DC, via guidelines Limited DIN SPEC 70121 2012, revised 2014 Powerline on control pilot No None ISO 15118-2 2014 Powerline on control pilot No TLS 1.2, used for Plug & Charge ISO 15118-20 2022 Powerline on control pilot; wireless options Yes, AC and DC TLS 1.3 mandatory; multiple roots and contracts The table explains a good part of the recent history. When CHAdeMO lost ground, the industry's working bidirectional standard went with it, and the successor took several years to move from publication to products. Through 2023 and 2024 the engineering community ran repeated interoperability events, at which car and charger makers plug prototypes into each other to find where their implementations disagree. By 2025 and 2026, cars and chargers that complete a full -20 bidirectional session with each other were shipping in Europe, North America, Australia and elsewhere. In Australia, for instance, Hyundai and partners announced in 2025 what they described as the country's first vehicle-to-grid discharge using the -20 standard. What -20 does not do is equally important. It governs only the link between the car and the charger. It says nothing about how the charger learns what the grid needs, how an aggregator tells ten thousand chargers what to do, or how a utility verifies and pays for what happened. Those links run over other protocols, and the chain can break there just as easily. The next chapter follows the signal upward. Chapter 4: Beyond the Plug A grid operator does not talk to cars. It talks to a handful of large counterparties: generators, utilities, retailers, aggregators. Somewhere between a control room that sees the grid in megawatts and a single charger that moves eleven kilowatts, a signal has to be translated, divided, routed and acted upon, and evidence of what happened has to travel back the other way. ISO 15118-20 handles the last metre of that journey. This chapter follows the rest of it, because a vehicle-to-grid system is only as good as its weakest communication link, and the weakest link is rarely the one between the car and the charger. The chain in outline Consider what has to happen when a grid operator, facing a heat wave, decides it needs 20 megawatts of extra supply in a particular region between six and eight in the evening. The operator announces an event or dispatches resources through its market systems. An aggregator, a company that has contracted with thousands of vehicle owners or fleet operators and bid their combined flexibility into the market, receives the instruction. The aggregator's software works out which vehicles are plugged in, how much each can discharge given its owner's departure time and minimum charge, and how to divide the 20 megawatts among them. It sends instructions to the systems that control the chargers, which are usually cloud platforms run by charge point operators. Those platforms send commands to the individual chargers. Each charger passes the resulting limits or setpoints to its car through ISO 15118-20. The car discharges. Meters record what happened, and the data flows back up the chain for settlement, so that the operator pays the aggregator and the aggregator pays the owner. At every arrow in that description there is a protocol, a contract and a potential failure. Table 3 lists the main links and the protocols most commonly used on each. Different regions and companies make different choices, and some links can be served by more than one protocol. Table 3. The communication chain from grid to battery. Link Common protocols What travels Typical timescale Grid operator or utility to aggregator Market interfaces, OpenADR, IEEE 2030.5 Events, prices, dispatch, settlement data Seconds to day-ahead Aggregator to charger backend Proprietary APIs, OpenADR, IEEE 2030.5 Fleet-level targets, availability Seconds to minutes Charger backend to charger OCPP 2.0.1 or 2.1 Charging and discharging profiles, meter values Seconds Site energy manager to charger OCPP, EEBUS, Modbus Local limits, site load Sub-second to seconds Charger to vehicle ISO 15118-20, CHAdeMO Setpoints, limits, state of charge Fractions of a second The protocols at each link OCPP: the charger's link to the cloud Almost every networked charger in the world is managed through the Open Charge Point Protocol, OCPP, maintained by the Open Charge Alliance, an industry consortium. OCPP defines how a charger, called a charging station in the protocol, talks to its central management system: how it reports status and faults, starts and stops sessions, handles authorisation, uploads meter readings, receives firmware updates and accepts limits on how much power it may draw. OCPP came in several generations. Version 1.6, published in the mid-2010s, became the workhorse of the industry and is still widely deployed. It introduced smart charging profiles, which let the backend set a maximum charging power over time for a whole station or a single session. Version 2.0.1, released in 2020, was a substantial redesign that added better security, a richer device model and support for passing ISO 15118 information, including Plug & Charge certificates, between the car and the backend. Neither version, however, had a native way to tell a charger to make a car discharge. Version 2.1, released on 23 January 2025, filled the gap [6]. It added a functional block devoted to bidirectional power transfer, aligned with ISO 15118-20, and introduced the concept of operation modes that say who decides the power flow. In some modes, the central system sends a setpoint directly. In others, the charger follows an external energy management system on site, or responds to grid frequency measured locally or centrally, or simply stays idle. Charging profiles can now carry negative values, meaning discharge. And the protocol gained mechanisms for configuring a charger's grid-support behaviour, the settings required by interconnection codes, from the backend. The Open Charge Alliance stated that application logic built for 2.0.1 would continue to work in 2.1, which matters because charger makers had only recently completed their move from 1.6 to 2.0.1. A great deal of the practical vehicle-to-grid capacity in the world depends on how quickly this upgrade spreads. A charger that speaks ISO 15118-20 to the car but only OCPP 1.6 to the cloud can, at best, discharge under some proprietary extension specific to its manufacturer. That works within a single vendor's ecosystem but not across them, which is exactly the kind of fragmentation that kept vehicle-to-grid in pilots for a decade. Utility-facing protocols: OpenADR and IEEE 2030.5 Above the charger management systems, two protocols dominate the link between utilities and the resources they want to dispatch. OpenADR, short for Open Automated Demand Response, was developed in California in the 2000s to let utilities send demand response signals, such as price changes and event notifications, to buildings and devices automatically. Its 2.0 versions became widely used for commercial and industrial demand response, and version 3.0, published in 2023, rebuilt it as a modern web interface that is far simpler for software developers to implement. OpenADR is good at saying "here is an event, here is a price, here is when it starts and stops." It is less concerned with the detailed electrical behaviour of the device that responds. IEEE 2030.5, which grew out of an earlier specification called Smart Energy Profile 2.0, takes a different approach. It models the device as a distributed energy resource with specific electrical capabilities and settings, and it lets a utility read and change those settings: export limits, voltage and reactive power behaviour, schedules of active power. California made IEEE 2030.5 the default communication protocol for inverter-based resources connecting under its interconnection rule, known as Rule 21, and implementers follow a detailed profile of the standard called the Common Smart Inverter Profile. Because a bidirectional charger or vehicle exporting to the grid is an inverter-based resource, IEEE 2030.5 is in many North American programs the language in which the utility tells it how to behave. The two are not rivals so much as tools for different jobs. A program paying for peak reduction may use OpenADR to announce events; the same program, if the resources export, may need IEEE 2030.5 or an equivalent to manage their grid-support settings. Aggregators often speak both upward and translate into OCPP or proprietary interfaces downward. The other road: through the carmaker's cloud There is a second way to reach a car that bypasses the charger's backend entirely. Modern electric vehicles are permanently connected to their manufacturers' servers through cellular modems. The carmaker can tell the car to start or stop charging, set a charge limit, or schedule charging, through its own cloud and its own app. Many smart charging programs already work this way: an aggregator or utility connects to the carmaker's interface and adjusts the car's charging directly, whatever charger it is plugged into. For vehicle-to-grid, this route has attractions. The carmaker keeps control of the battery, which eases its warranty concerns. The car knows its own state of charge and its owner's plans better than any charger does. And the aggregator need not care what charger brand is on the wall. Some of the prominent consumer programs of 2025 and 2026 lean on this model: carmakers bundling vehicles, chargers and tariffs with a single energy partner, as BMW did with E.ON in Germany [17] and as Tesla has done by enrolling Cybertruck owners through its own retail electricity plan in Texas [23]. The disadvantages mirror the advantages. The aggregator must strike a separate deal with every carmaker, and each exposes different capabilities through different interfaces. Telematics links have latencies of seconds or more and can drop out in underground car parks. And when power is exported, it is the charger, or the car's inverter, that the grid's interconnection rules regulate, and the meter at the site that records it, so the charger side cannot be ignored. In practice, mature programs are likely to use both routes: the carmaker's cloud for planning and consent, the charger path for real-time control and verified metering. ISO 15118-20's scheduled and dynamic modes map onto this split surprisingly well, with the car deciding in the first and the charger's backend directing in the second. Local intelligence One more layer sits between the cloud and the charger: the site. A home with solar panels and a battery, an office building, or a bus depot has its own load, its own connection limit and often its own energy management system. That system needs to coordinate the chargers with everything else on site, for example by making sure the depot's total draw never exceeds the capacity of its grid connection, or that a house's car absorbs surplus solar rather than exporting it. Several protocols serve this layer. In Germany and parts of Europe, EEBUS is used to connect home energy managers with heat pumps, chargers and other appliances. Germany has given this layer regulatory teeth: since January 2024, under section 14a of its energy industry act, distribution network operators may reduce the power of newly installed controllable devices, including wallboxes, to a guaranteed minimum of 4.2 kilowatts when the local network is under strain, in return for reduced network charges. In commercial sites, simpler industrial protocols such as Modbus are common. OCPP 2.1 explicitly allows a charger to take its limits from such a local system rather than from the cloud. Local intelligence matters for vehicle-to-grid in two ways. It is often the only way to meet the fastest grid services, because acting on locally measured frequency within a second or so cannot wait for a round trip through several clouds. And it keeps the site safe if the cloud connection fails. A depot whose chargers lose contact with their backend must default to a sensible behaviour, usually stopping discharge and reverting to managed charging, rather than continue exporting on stale instructions. One event, traced end to end To see how these layers fit together, follow a single peak event through an illustrative system. The timings are representative of how such chains are typically configured, not measurements from a particular program. At ten in the morning, a utility forecasting a hot afternoon decides to call a peak event for six to eight in the evening. Its demand response system publishes the event to enrolled aggregators, perhaps as an OpenADR event specifying the start time, the end time and the region. Within minutes, the aggregator's platform receives the event and queries its view of the portfolio. For cars it reaches through carmakers' clouds, it requests current state of charge and owner settings. For chargers it reaches through charge point operators, it requests which are online and which have vehicles connected. It cannot yet know which cars will be plugged in at six, so it forecasts, using each owner's history and the day of the week, and decides how many kilowatts it can safely promise. It confirms participation to the utility. During the afternoon, the aggregator watches arrivals. As cars plug in after the commute, each charger opens an ISO 15118-20 session with its car. The car declares its departure time, its target energy and the minimum below which it will not discharge. The charger passes that information upward through OCPP to its backend, and the backend passes it to the aggregator. The aggregator's optimiser now has, for each car, a window of feasible behaviour: how much it can discharge between six and eight while still meeting its owner's needs by morning. At a few minutes before six, the aggregator issues instructions. For chargers directed through the charge point operator's backend, it sends discharge targets, which the backend converts into OCPP charging profiles with negative power values for the event period. Each charger, receiving its profile, passes the corresponding limits or setpoints to its car through the charge loop of ISO 15118-20. For cars reached through the carmaker's cloud, the aggregator sends equivalent requests through that route. At six, the cars begin to discharge. Each charger reports its meter readings upward at whatever interval it has been configured to use. The aggregator watches the portfolio's total output. If a car is unplugged early, or a charger drops offline, the aggregator shifts its share to other cars with spare capacity. At eight the profiles end, and the chargers revert to their default behaviour, usually smart charging toward each car's departure target overnight. Over the following days, the aggregator compiles the meter data, calculates its portfolio's performance against its commitment, submits it to the utility for settlement, and in due course pays each owner according to their contract. Every arrow in this sequence crosses an organisational boundary and depends on a protocol, a data agreement and a commercial contract. The utility, the aggregator, the charge point operator, the carmaker and the owner each hold a piece. The sequence works when each hand-off is defined in advance. It fails, often silently, when one is not. Where chains break Projects that have struggled with vehicle-to-grid tend to describe the same set of communication failures. Latency stacks up. Each hop adds delay. A dispatch that passes through a market system, an aggregator, a charge point operator's cloud, a cellular link to the charger and finally the ISO 15118 loop to the car can take many seconds end to end. That is fine for a peak event announced an hour ahead. It is not fine for regulation services that update every few seconds. Telemetry is too coarse. Grid operators verify performance from meter data. Many chargers report energy readings every few minutes, which is adequate for billing but useless for proving a one-second frequency response. Programs that pay for fast services must specify meters and reporting rates that many off-the-shelf chargers do not provide. Semantics do not line up. One system's "available capacity" is another's "maximum power," and a third's "flexible energy." Translating between OpenADR events, IEEE 2030.5 settings, OCPP profiles and ISO 15118 parameters involves design choices, and different implementations make different ones. Fallback behaviour is undefined. When communication fails, devices must do something. If the rule is not written down, different vendors pick different behaviours, and the aggregator cannot predict how its fleet will behave in exactly the moments that matter most. Consent and data rights are unclear. A charger backend may know the car is plugged in; the carmaker may know its state of charge; the aggregator needs both; the driver must agree to share them. Every one of these data flows needs a contractual basis, and in Europe a lawful basis under data protection law as well. None of these failures is a matter of physics. Each is a matter of agreement, and each is being worked through, in standards committees, industry testing events and the fine print of program contracts. The most successful deployments described later in this book reduced the problem by owning or tightly integrating several links at once: one company supplying car, charger, software and tariff, or one fleet operator controlling buses, chargers and the depot energy system. That integration is a practical response to an immature chain. As the open protocols mature, the hope is that the chain can be assembled from interchangeable parts. Whether that happens depends partly on the protocols, and partly on the electrical rules that the next chapter examines. Chapter 5: Where the Inverter Lives A battery stores direct current. The grid runs on alternating current. Every time energy moves between them, a power electronic converter has to change one into the other, and when energy flows from battery to grid, that converter becomes, legally and electrically, a generator connected to a public network. The single most important hardware question in vehicle-to-grid is where that converter sits: in the charger on the wall, or in the car itself. The answer shapes cost, certification, safety, and which companies hold the keys. This chapter explains both architectures, then turns to the electrical rules that any exporting device must obey. Two architectures Every electric car already contains a converter for charging from an ordinary alternating-current socket or wallbox. It is called the onboard charger, and it turns the grid's alternating current into direct current for the battery. Its power rating, commonly 7 to 11 kilowatts and in some models 22, sets the maximum rate of alternating-current charging. When a car uses a direct-current fast charger, the onboard charger is bypassed: the fast charger's own, much larger converter feeds direct current straight to the battery. That existing division gives two routes to bidirectional power. DC bidirectional. The converter lives in the charger. The charger draws direct current from the car's battery through the direct-current pins of the connector, converts it to alternating current, and feeds it to the house or the grid. The car's role is to open its contactors, report its battery state and follow the charger's instructions under ISO 15118-20 or CHAdeMO. From the grid's point of view, the charger is the generator, and it can be tested and certified like a stationary battery inverter. Nearly all early vehicle-to-grid projects used this route, including the CHAdeMO-based pilots in Denmark and Britain, because it required no change to the car's power electronics. AC bidirectional. The converter lives in the car. The onboard charger is designed to run in reverse, turning battery direct current into alternating current, and the wall unit is a relatively simple device that switches the alternating-current connection, communicates with the car and measures what passes. Renault, for example, built bidirectional alternating-current capability into the Renault 5 and its siblings, and the Utrecht car-sharing fleet runs on alternating-current bidirectional charge points [15]. The trade-off between these routes is one of the most argued-over questions in the field. Table 4 summarises it. Table 4. Where the converter sits: DC versus AC bidirectional charging. Attribute DC bidirectional AC bidirectional Converter location In the charger In the vehicle (onboard charger) Wall unit cost High; contains full power converter Lower; mostly switching, metering, communication Vehicle changes needed Contactor and software support Bidirectional onboard charger and grid-code software Grid-code certification Charger certified like a stationary inverter Vehicle and wall unit must be certified together Typical home power Around 10 to 12 kW Up to the onboard charger rating, often 7 to 11 kW Moving between grid codes Not an issue; charger stays put Car must adopt the settings of each location The cost argument The argument for alternating-current bidirectionality is mainly about cost. A direct-current wall unit duplicates power electronics that every car already carries. The Sciurus trial in Britain found that its direct-current vehicle-to-grid hardware and installation cost around £3,700 more than a smart charger [12], and similar premiums were reported in other markets through the early 2020s. If the onboard charger can run in reverse, the wall unit needs only to switch, meter and communicate, which is not much more than an ordinary smart wallbox already does. The additional cost of making the onboard charger bidirectional is modest when designed in from the start and spread across a production run of hundreds of thousands of vehicles. The counterargument is about control and certification. When the converter is in the charger, the charger maker designs, tests and certifies it for the local grid code once, and every compatible car simply supplies energy. When the converter is in the car, the car itself must behave as a grid-compliant generator wherever it is plugged in. A car sold in Germany may be driven to France and plugged in there, under a different network operator's settings. A car sold in California may be plugged into a utility in Arizona with different interconnection requirements. Somehow the car must know the rules where it is and apply them. This is not an unsolvable problem, and the protocols are being built to carry the necessary settings. But it took years for regulators and certification bodies to work out who would test what. In the United States the answer arrived in pieces. The established inverter safety standard, UL 1741, gained a supplement in 2021, known as SB, that tests inverters against the grid-support requirements of IEEE 1547-2018 [7]. For alternating-current vehicle-to-grid, the vehicle itself certifies to UL 1741 SB, and uses SAE J3072, a standard that lets a car declare its grid-support capabilities to the charger. The wall unit was the missing piece. A further supplement, UL 1741 SC, written specifically for bidirectional alternating-current supply equipment, was approved in May 2026 [8]. Under this arrangement, discharging is permitted only when a certified car and a certified wall unit recognise each other: the wall unit checks, through the communication link, that the car is a compliant generator before allowing it to export. Both architectures will persist. Direct-current bidirectional suits depots, where higher power per vehicle matters and a single certified charger serves many vehicles over its life, and suits homes where the owner wants maximum flexibility across car brands. Alternating-current bidirectional suits mass-market households, where every pound or dollar on the wall matters. The more interesting consequence is commercial. In the alternating-current model, the carmaker's power electronics and software determine whether the car can participate, which strengthens the carmaker's hand in any vehicle-to-grid program. What every exporting device must do Whichever architecture is used, once a device can push power into a public network it must obey the rules for distributed generation. These rules exist for reasons that are easy to forget until something goes wrong. Do not energise a dead line. When a fault or planned work disconnects part of the network, line workers assume the section is dead. A generator that keeps feeding power into the disconnected section, a condition called islanding, can kill them. Every grid-connected inverter must therefore detect loss of the grid and stop exporting quickly. IEEE 1547-2018 requires an unintentional island to be detected and the resource to cease energising within two seconds [7], and European grid codes impose comparable requirements. Bidirectional chargers and vehicles inherit this duty. For vehicle-to-home systems that are meant to keep a house running during an outage, a transfer switch must physically separate the house from the grid before the car supplies it, so that the house becomes a deliberate island rather than an accidental one. Do not drop off at the worst moment. Early rules required small generators to disconnect at the first sign of abnormal voltage or frequency, which seemed safe. As solar and batteries multiplied, regulators realised that thousands of devices disconnecting simultaneously during a brief disturbance could turn a minor event into a major one. Modern standards therefore require ride-through: devices must stay connected and keep operating through defined ranges of voltage and frequency excursions, only disconnecting outside them. The required ranges vary by region. Help with voltage and frequency. IEEE 1547-2018 and its counterparts in Europe, such as EN 50549 and national codes like Germany's VDE-AR-N 4105, also require inverters to support the grid actively: adjusting reactive power in response to local voltage, curtailing export when frequency rises, and accepting remotely set limits. These functions are the reason a bidirectional charger can do more for a distribution network than merely export energy, and the reason the communication protocols discussed in the previous chapter must carry grid-support settings. Get permission to export. Most networks require a generator above a certain size to apply for permission before connecting. In Great Britain, small generators up to 16 amperes per phase can generally connect under a notification process known as G98, while larger ones need prior approval under G99. A single-phase home bidirectional charger exporting at 7 kilowatts or more exceeds the smaller threshold, which is why the Octopus Energy and BYD vehicle-to-grid bundle launched in June 2025 required customers to obtain G99 export permission from their distribution network operator, a process the company warned could take up to twelve weeks and might be refused where the local network lacked capacity [16]. That condition illustrates how an electrical rule becomes a commercial constraint. The network operator is not being obstructive; a street where every house exports 7 kilowatts at once could push voltages and transformer loadings beyond their limits. But an approval process designed for occasional solar installations does not scale to millions of cars. When the grid goes down Much of the bidirectional hardware sold in North America so far has been bought not for the grid but against it: to keep a house running when the grid fails. Ford's F-150 Lightning, launched in 2022, was marketed with a home backup system built with the solar company Sunrun, which used a dedicated bidirectional charger and a home integration unit to power a house from the truck's battery during an outage. General Motors followed with its own vehicle-to-home kits, and Tesla's Powershare system extended the idea to the Cybertruck. These systems are worth understanding because they solve a harder electrical problem than vehicle-to-grid does, and because they are often the route by which bidirectional hardware first enters a home. When a car exports to a functioning grid, its inverter can follow the grid. The grid sets the voltage and frequency, and the inverter simply injects current in step with them. Engineers call this grid-following operation. It is relatively simple and is how almost all rooftop solar works. When the grid fails and the car must supply a house on its own, there is nothing to follow. The inverter must itself create a stable voltage at the right frequency and hold it steady while appliances switch on and off, motors start and refrigerators cycle. This is grid-forming operation. It demands an inverter that can respond instantly to sudden changes in load, supply surge currents when motors start, and do so without the stabilising support of a large network behind it. Grid-forming operation also demands isolation. Before the car can form a voltage in the house, the house must be physically disconnected from the dead grid outside, or the car would attempt to energise the whole street. That is the job of the transfer switch or home integration unit: it senses the outage, opens a contactor between the house and the utility supply, and only then allows the car to energise the house. When the grid returns, it reverses the sequence, waiting until the grid is stable before reconnecting. For the vehicle-to-grid argument, the significance of backup systems is twofold. First, they give owners a reason to pay for bidirectional hardware that has nothing to do with grid markets, which spreads the hardware across homes that may later join grid programs. Second, they show that the same equipment can play very different roles depending on the agreements around it. A car powering a house during an outage is outside the grid's rules altogether; the same car exporting through the same charger during a heat wave is fully within them. The GM and Pacific Gas and Electric pilot of 2025 made the connection explicit by requiring participants in its vehicle-to-home program to enrol in a peak-reduction program as well [22]. Metering and measurement A kilowatt-hour that crosses the meter has to be measured to be paid for, and vehicle-to-grid creates measurement problems that ordinary household metering was not designed to solve. The first is separation. A house with a bidirectional car may also have solar panels, a stationary battery and ordinary consumption. If the house exports power at six in the evening, how much came from the car? For a program that pays for vehicle exports, or that exempts stored and returned electricity from taxes and fees, the answer matters. Germany's November 2025 decision to exempt electricity that is drawn from the grid, stored, and returned to the same grid from network fees and electricity tax applies to vehicles as well as stationary storage [18], but making the exemption work in practice requires measuring the flows precisely, and the country's grid operators were reported to need nine to twelve months to adapt their systems. The Octopus and BYD bundle, for its part, excluded customers who already exported from solar or a home battery [16], almost certainly in part to keep the accounting clean. The second is resolution. Billing meters usually record energy in half-hour or fifteen-minute blocks. Verifying a fast frequency response needs data at one-second resolution or better. This is why many fast-service programs require their own metering inside the charger or at the circuit, certified to a standard the grid operator accepts. The third is trust. Meters used for billing in the European Union must comply with the Measuring Instruments Directive; in North America they must meet the requirements set by state or provincial authorities. A meter inside a charger may be accurate but not certified for billing, in which case its readings can be used for control but not for settlement. Several countries now accept certified sub-meters inside charging equipment for exactly this purpose, but the rules differ and are still settling. Power, heat and the street A final hardware consideration concerns the network beyond the house. A neighbourhood transformer is sized for the households it serves, on the assumption that they consume power. When many homes export at once, current flows backward through the transformer and voltages along the street rise. Solar panels raised this issue first. Vehicles raise it more acutely, because a bidirectional charger exports at a much higher rate than a typical rooftop array and because an aggregator may want every car on the street to export at once, during exactly the hours when a peak event is called. The answer is not to forbid vehicle export but to coordinate it with the distribution operator, so that local limits are respected while system-level needs are met. This is one of the strongest arguments for building distribution-level control into the communication chain from the start, and for treating local network relief, which asks cars to reduce load or export modestly within a constrained area, as a core service rather than an afterthought. It also explains why network operators, not just regulators and carmakers, have become central to vehicle-to-grid. The hardware can now do almost everything asked of it. Whether it is permitted to, at what power, and on which streets, is a question for the people who run the wires. Chapter 6: The Battery Question Ask any electric car owner about vehicle-to-grid and the first question is nearly always the same: what will it do to my battery? It is the right question. The battery is the most expensive component in the car, its health determines the car's range and resale value, and the owner bears the cost if it wears out early. It is also the question on which the most confused information circulates. Early laboratory studies produced alarming headlines; later modelling studies produced reassuring ones; carmakers have said little and written less into their warranties. This chapter sorts out what is actually known. The short version is that the battery is the easy link in the chain, but only if the other links are designed to respect it. Vehicle-to-grid that is planned around the battery's chemistry adds modest wear and can, in some circumstances, reduce it. Vehicle-to-grid that treats the car as a free storage device to be cycled whenever prices move can wear it significantly. The difference lies in the control software, the commercial agreement, and the parameters the car is allowed to set for itself in the ISO 15118 conversation. How lithium-ion batteries age A lithium-ion battery loses capacity through two broad mechanisms, and understanding them is the key to understanding everything else. Calendar ageing happens whether the battery is used or not. Inside every cell, a thin film called the solid electrolyte interphase forms on the surface of the graphite anode. It is necessary, because it protects the anode from further reaction with the electrolyte, but it grows slowly over time, consuming lithium that would otherwise store energy. The rate of growth depends mainly on two things: temperature and state of charge. A battery stored hot and full ages much faster than one stored cool and half full. A car that sits at 100 percent charge in a hot car park for days at a time is ageing its battery even though it never moves. Cycle ageing happens because the battery is used. Each charge and discharge moves lithium in and out of the electrode materials, which expand and contract slightly. Over many cycles, this mechanical stress cracks particles, exposes fresh surfaces on which the interphase layer grows further, and isolates bits of active material so they no longer participate. The rate of cycle ageing depends on how much energy passes through the battery, how deep each cycle goes, how fast the current flows, and the temperature during charging. Fast charging in the cold is especially hard on cells because lithium can plate as metal on the anode surface instead of entering it. For most private cars, calendar ageing accounts for a large share of total degradation over the vehicle's life, because cars sit far more than they drive and many are kept at high states of charge out of habit or anxiety. This is the starting point for the most important argument in this chapter. If vehicle-to-grid adds cycling but is also managed to reduce the time the battery spends full and hot, the net effect on battery life can be smaller than the added energy throughput alone would suggest. What the studies found Two widely cited studies from 2017 illustrate the range of findings and why they seem to contradict each other. At the University of Hawaii, Matthieu Dubarry and colleagues tested commercial lithium-ion cells of a type then used in electric vehicles under a range of simulated driving and grid-support patterns [9]. When they added vehicle-to-grid discharges on top of normal driving, capacity loss increased substantially. In the scenario with two extra vehicle-to-grid discharges a day, capacity fade accelerated by about 75 percent; with one a day, by about a third. Smart charging without discharge, by contrast, had negligible negative impact. The study was careful and is often quoted as proof that vehicle-to-grid damages batteries. At the University of Warwick in Britain, Kotub Uddin and colleagues built a detailed model of battery degradation, validated against laboratory tests, and combined it with a smart-grid control algorithm that decided when to discharge each vehicle with battery health as one of its objectives [10]. In their simulations, controlled vehicle-to-grid that took energy from the battery at times and in amounts chosen to minimise ageing could reduce capacity fade compared with a vehicle that was simply charged and driven, mainly by keeping the battery at lower states of charge for longer and so slowing calendar ageing. The study is often quoted as proof that vehicle-to-grid helps batteries. Both studies are right about what they tested. Dubarry's cells were cycled hard, with substantial extra discharges added daily, on top of a usage pattern that did not reduce time at high state of charge. Uddin's model assumed an optimiser whose job included protecting the battery and a usage pattern in which vehicle-to-grid was used selectively. The lesson is not that one is correct and the other wrong. It is that the outcome depends on how discharge is used. Uncontrolled daily cycling for arbitrage adds wear. Selective discharge, chosen with the battery in mind, need not. A worked example in throughput It helps to put numbers to the cycling side of the question. The following example uses round figures chosen for illustration, not measured data for any specific car. Take a car with a 60-kilowatt-hour usable battery driven 15,000 kilometres a year at an efficiency of about 6 kilometres per kilowatt-hour. Driving takes about 2,500 kilowatt-hours a year out of the battery, which is roughly 42 equivalent full cycles. Now add vehicle-to-grid under three different programs. Peak events only. The car takes part in, say, 40 peak events a year, each discharging 15 kilowatt-hours. That adds 600 kilowatt-hours a year, or about 10 equivalent full cycles. Total throughput rises by roughly a quarter. Daily evening support. The car discharges 10 kilowatt-hours every weekday evening and recharges overnight, about 2,600 kilowatt-hours a year. Throughput roughly doubles. Aggressive arbitrage. The car cycles 30 kilowatt-hours a day every day, about 11,000 kilowatt-hours a year. Throughput rises more than fivefold. Whether those increases matter depends on how many cycles the battery can tolerate. Modern automotive cells are typically designed for well over a thousand equivalent full cycles before falling to 70 or 80 percent of original capacity, and lithium iron phosphate chemistries, now common in many cars and especially in buses and vans, are often rated for several thousand. A battery good for 1,500 cycles that does 42 a year from driving will die of calendar ageing long before it runs out of cycles. Adding ten cycles a year from peak events makes little practical difference. Doubling throughput still leaves the cycle budget far from exhausted over a typical vehicle life, though it may shorten the time to a given level of fade somewhat. Aggressive arbitrage is a different matter: 200 or more equivalent cycles a year would consume a 1,500-cycle budget in well under a decade. It is also possible to put a rough price on the cycling. BloombergNEF's annual survey put the average lithium-ion pack price across all applications at $108 per kilowatt-hour in late 2025 [25]. A 60-kilowatt-hour pack at that price is worth about $6,500 to manufacture, though replacing one in a car costs considerably more. If such a pack could deliver 1,500 full cycles before replacement, its lifetime throughput would be 90,000 kilowatt-hours, and the cost of wear would average about 7 cents per kilowatt-hour cycled. With a chemistry good for 3,000 cycles, the figure halves. These are averages, and the true marginal cost of an extra cycle is lower when calendar ageing is the binding limit, but the order of magnitude is useful. A peak event paying many tens of cents or more per kilowatt-hour clears that bar easily. Arbitrage on a spread of a few cents does not, once round-trip losses are added. Designing discharge around the battery The practical rules that follow from all this are well understood by battery engineers and increasingly built into vehicle-to-grid software. Keep the battery in its comfortable middle. Discharging from 80 to 50 percent is gentler than discharging from 50 to 20. Limiting discharge to the middle of the state-of-charge range reduces stress, and keeping the car below full while parked slows calendar ageing. A vehicle-to-grid program can set its minimum and maximum limits accordingly, and ISO 15118-20 gives the car the means to declare them in every session. Charge late, not early. A car plugged in at six in the evening that will leave at eight in the morning does not need to be full at nine at night. Filling it just before departure minimises time at high state of charge. Smart charging does this already; vehicle-to-grid can extend it by drawing the battery down a little in the evening and filling it just in time. Prefer services with low throughput and high value. Frequency response and peak events move relatively little energy for the money they pay. Daily arbitrage moves a great deal. A program designed with the battery in mind favours the former. Discharge gently. Vehicle-to-grid power levels are modest relative to battery size. An 11-kilowatt discharge from a 60-kilowatt-hour pack is less than a fifth of its capacity per hour, far gentler than motorway driving or fast charging. Rate is rarely the problem; total throughput and state-of-charge window are. Watch the temperature. Discharging a battery that is already hot, on a summer evening after a long drive, adds heat. Good software takes battery temperature into account and limits discharge when necessary. All of these rules require that the battery's own state be visible to the system deciding when to discharge. That is one reason carmakers want to stay in the decision loop, and one reason ISO 15118-20 provides for the car to compute its own plan in scheduled mode, or to bound the charger's commands with its own limits in dynamic mode. The warranty problem For owners, the decisive issue is not the chemistry but the warranty. Electric vehicle batteries are typically warranted for a period of years and a distance driven, commonly eight years and 160,000 kilometres or 100,000 miles, with a guarantee that capacity will not fall below a stated level, often 70 percent. Vehicle-to-grid complicates this, because it adds throughput that does not show up on the odometer. A car might reach eight years with a modest distance driven but a large amount of energy cycled through the grid. Carmakers have responded in different ways. Some have simply not supported vehicle-to-grid, or have stated that unapproved discharging may affect the warranty. Others approve specific chargers and programs, and some have begun to express warranties partly in terms of energy throughput rather than distance alone. The most decisive answer so far has come from bundled offers in which the carmaker is itself a party to the vehicle-to-grid program. When BMW launched its vehicle-to-grid offer with E.ON in Germany in early 2026, the carmaker, the charger and the tariff were designed together [17]; when Renault's Mobilize brand offers vehicle-to-grid in France, the carmaker controls the program. In such bundles the carmaker sets the limits on discharge and so takes responsibility for the result. This points to a broader truth. Battery wear is a technical problem with known solutions, but the risk of wear has to be allocated by contract. Somebody must stand behind the battery: the carmaker through its warranty, the aggregator through a guarantee or insurance, or the owner through accepting the risk in exchange for payment. Programs that leave this unclear struggle to recruit participants, however good their engineering. Programs that settle it, whether by carmaker bundles, by throughput-based warranties, or by fleet operators who own both the vehicles and the grid revenue, find that the battery question largely answers itself. The driver's guarantee Behind the question about battery wear usually lies a second, more immediate one: what if I need the car and it is empty? A driver who returns home planning a quiet evening and is then called to a hospital at midnight does not want to discover that the car spent the evening discharging into the grid. The answer lies in the minimum limits discussed in Chapter 3, and in how programs present them. Every well-designed vehicle-to-grid program lets the owner set a floor below which the battery will not be discharged, and most set a sensible default. Many also guarantee a target by a stated departure time, so that whatever happens in the evening, the car will be at, say, 80 percent by seven in the morning. Some offer an override, a button in the app that stops discharging immediately and starts charging at full power. Programs differ in whether an override costs the owner anything; some simply forgo the payment for that event, others count overrides against a monthly allowance. The floor is a trade. Every kilowatt-hour held in reserve for emergencies is a kilowatt-hour that cannot be sold. A floor of 30 kilowatt-hours in a 60-kilowatt-hour car, combined with an evening plug-in at 45 kilowatt-hours, leaves only 15 to offer. Owners who set high floors earn less; owners who set low ones accept more risk. In practice, most daily driving needs are small enough that a modest floor, enough for perhaps 100 kilometres, covers the overwhelming majority of unexpected trips, and the remaining risk is similar to that of a driver who forgets to plug in at all. Fleet operators approach the same trade with more data and less anxiety. They know their routes and their vehicles' energy use, and they set floors from operational requirements rather than imagined emergencies. That is another reason, explored in the next chapter, why fleets can offer more of each battery than private owners typically will. The second life of a first-life battery One further point is worth making. A vehicle battery that has been cycled for grid services is not necessarily a waste. Many electric vehicle batteries reach the end of their useful automotive life with 70 to 80 percent of their original capacity, still enough for stationary storage where weight and volume matter less. Grid services can thus be seen as one stage in a longer life of the battery, in which the line between vehicle and stationary storage is blurred. Some fleet operators already plan on this basis, treating their bus or van batteries as assets whose total value includes driving, grid services in their first life, and stationary use in their second. The battery, then, is not the reason vehicle-to-grid has taken so long. It is the part of the problem that chemistry and engineering have largely solved. What remains is to write the rules, the software and the contracts so that the battery is used the way the chemistry prefers. The settings where that is easiest to do are fleets, and they are the subject of the next chapter. Chapter 7: Fleets First If the chain of agreements that vehicle-to-grid requires is long and fragile, the obvious strategy is to shorten it. The settings where that is easiest are fleets: groups of vehicles owned or controlled by a single organisation, parked in known places on known schedules, managed by people whose job is to optimise their cost. A fleet collapses several links of the chain into one decision-maker. The fleet operator owns the vehicles, chooses the chargers, controls the depot connection, sets the battery policy and signs the grid contract. Where a private car owner must be persuaded, a fleet operator must merely be shown a spreadsheet. This chapter explains why fleets, and especially fleets with predictable schedules, are where the economics of vehicle-to-grid close first, and examines the kinds of fleet that fit best. It also explains the ways in which fleets can disappoint, because the record shows those too. What makes a fleet a good grid resource Four properties matter more than any others. Predictable dwell. A grid operator values certainty. A private car may or may not be plugged in at six in the evening. A school bus is parked at the depot between the end of the afternoon route and the start of the morning one, every school day, and all day during holidays. A delivery van returns at a known time. A municipal pool car sits at the town hall overnight. When the schedule is known, availability can be forecast accurately, and an aggregator can commit a fleet's capacity to a market or a utility with confidence that it will be there. Concentrated power. A depot with forty buses and forty bidirectional chargers is a single site with a single grid connection, often rated in megawatts. It can be metered, telemetered and controlled as one resource. Its total power is large enough to qualify for markets with minimum sizes and to be worth the attention of the distribution company. Compare this with forty private cars scattered across a city, each behind its own meter, each requiring its own export permission. Central control. In a fleet, one organisation decides the battery policy, the minimum charge levels, the departure times and the trade-offs between operational readiness and grid revenue. There is no need to negotiate with forty owners. The fleet's energy management system can run in dynamic mode, directing every charger from one place, which makes fast services and precise dispatch feasible. A cost the fleet already cares about. Electrifying a depot usually forces a fleet operator to confront its electricity costs for the first time: the size of the grid connection, the demand charges, the tariff structure. Vehicle-to-grid and smart charging offer ways to reduce those costs that a fleet manager can evaluate against a fuel budget. The same capability that lets buses export to the grid can also keep the depot's peak demand down, shrinking demand charges and sometimes avoiding an expensive connection upgrade. These properties are not equally present in every fleet, and the differences explain why some fleet types have led and others lag. School buses: the textbook case The electric school bus is the closest thing vehicle-to-grid has to a natural fit. Its virtues are almost absurdly well matched to the grid's needs. School buses carry large batteries, commonly well over 100 kilowatt-hours. They are driven twice a day for an hour or two, morning and afternoon, and parked the rest of the time. They sit idle all summer, which in much of North America is precisely when the grid is most stressed by air conditioning load. And their owners, school districts and the contractors who serve them, are public or quasi-public bodies with an interest in local resilience and an obligation to spend money carefully. The Beverly, Massachusetts, project mentioned at the start of this book shows the model at small scale. Highland Electric Fleets, which supplies electric buses to school districts under a service contract, worked with the bus maker Thomas Built Buses and charger supplier Rhombus Energy Solutions to enrol two buses in National Grid's ConnectedSolutions program. In the summer of 2022 the buses responded to thirty-two events, providing grid support for more than eighty hours and exporting more than seven megawatt-hours [13]. Seven megawatt-hours from two buses over a summer is a considerable amount of energy for vehicles that would otherwise have sat unused. The Oakland Unified School District in California shows the model at fleet scale. In 2024 the bus operator Zum completed the conversion of the district's fleet to seventy-four electric buses, all bidirectional, making it the first large American district with a fully electric, vehicle-to-grid-capable bus fleet. Pacific Gas and Electric provided 2.7 megawatts of charging capacity at the depot, and the partners projected that the fleet could return around 2.1 gigawatt-hours a year to the grid [14]. The capital came in part from the federal Environmental Protection Agency's Clean School Bus Program and the California Air Resources Board. The figures from Oakland are projections, not measured outcomes, and should be read as such. But the structure of the arrangement is instructive. The bus operator owns the vehicles and the chargers and contracts with both the school district, for transport, and the utility and grid programs, for energy services. One party controls almost the whole chain. The school district gets clean buses without carrying the technology risk; the operator earns grid revenue that helps pay for them. A worked depot To see why the economics work, consider an illustrative depot. The numbers are round figures chosen to show the reasoning, not measurements from any particular site. The depot holds forty electric school buses, each with about 150 kilowatt-hours of usable battery. Each bus uses around 50 kilowatt-hours a day on its routes, and the operator insists that every bus leave in the morning with at least 120 kilowatt-hours, a generous margin for cold days and route changes. Each bus is plugged into a 60-kilowatt bidirectional direct-current charger. The depot's grid connection is rated at 2 megawatts. On a school-day evening, buses return from their afternoon routes by around half past four with perhaps 100 kilowatt-hours left. A peak event is called from five to eight. How much can the depot deliver? The power available is forty buses at 60 kilowatts each, 2.4 megawatts, but the depot connection caps export at 2 megawatts. The energy available is the gap between what each bus holds and a safe floor. If the operator allows discharge down to 60 kilowatt-hours, each bus can contribute about 40 kilowatt-hours, or 1.6 megawatt-hours across the fleet. Over a three-hour event, that supports a little over half a megawatt continuously, or a full 2 megawatts for about three quarters of an hour. After the event, every bus must be recharged by about 90 kilowatt-hours to reach its 120-kilowatt-hour morning target, around 3.6 megawatt-hours across the fleet, comfortably achievable overnight within the connection limit. In summer the picture changes. The buses sit at the depot all day, fully available. The operator can charge them in the morning, when solar output in many regions pushes prices down, and offer far more energy during the late afternoon and evening peak, constrained only by the connection, the chargers and the battery policy. The depot becomes, for three months of the year, a substantial battery plant. The financial value of all this depends on local programs and prices, which vary enormously, so a single dollar figure would mislead. The structure, however, is clear. The vehicles are bought for transport; the chargers would be needed anyway, with a bidirectional premium; the grid connection would be needed anyway; and the extra revenue from events, capacity payments or local network relief comes with modest added wear because peak events involve limited throughput. The marginal economics of adding bidirectionality to a depot that must be electrified regardless are far better than those of building a stationary battery from scratch. Car sharing and municipal fleets Shared cars are a different kind of fleet, with a different fit. The Utrecht Energized project in the Netherlands is the most prominent example. It began operating in June 2025 with fifty Renault 5 cars on fifty alternating-current bidirectional charge points, run by the car-sharing company MyWheels, with We Drive Solar supplying the charging infrastructure and Renault and its Mobilize brand providing the vehicles; the target is five hundred cars, which the partners estimate could provide around 5 megawatts [15]. Utrecht is a city whose grid faces serious congestion as electrification accelerates, which makes local flexibility valuable. Shared cars sit at fixed charge points between bookings, and the operator knows from its booking system when each car will be needed next. That is a powerful forecasting advantage: the aggregator need not guess when a car will leave, because the next booking says so. Early reports indicated typical use of 10 to 20 kilowatt-hours per car for bidirectional operation, and MyWheels reported that operating costs began to fall soon after launch. The weakness of car sharing as a grid resource is that a shared car's purpose is to be used, and the most popular hours for bookings, late afternoons and evenings, can coincide with grid peaks. The operator must balance revenue from rentals against revenue from the grid, and the former usually wins. Car-sharing vehicle-to-grid works best where rental demand and grid stress fall at different times, or where the grid value is high enough to justify holding some cars back. Municipal and utility fleets are often good candidates for a different reason: their owners are motivated by public objectives as well as costs, and their vehicles often sit unused overnight and at weekends. The Danish vans of Frederiksberg Forsyning, a municipal utility, provided frequency reserve on weekday evenings, nights and weekends, about 6,150 hours a year, and earned roughly €1,400 each from the service [11]. That was nearly a decade ago, in a market whose prices have since changed, but the logic of a fleet idle in predictable hours remains. Where fleets disappoint Fleets are not a guaranteed success, and some of the reasons are worth stating plainly. Operations come first. A fleet exists to move people or goods. If a bus is not ready for its route because it discharged too much, the cost to the operator in disruption and reputation dwarfs any grid revenue. Fleet operators are therefore conservative in setting minimum charge levels, sometimes more conservative than necessary, which reduces the energy they offer. Building trust takes time and a record of reliability. Some fleets are rarely idle at the right time. Delivery vans return in the evening, often during the peak, needing to charge for the next day. Ride-hailing cars are on the road precisely when people are travelling. Long-haul trucks spend little time at depots at all. These fleets may benefit greatly from smart charging, which keeps their charging off the peak, but have less to offer as exporters. Charger reliability matters more. A fleet relying on bidirectional chargers to deliver contracted capacity is exposed when chargers fail. Early bidirectional hardware was not always robust, and repair times for specialised equipment could be long. A fleet with twenty percent of its chargers out of service delivers twenty percent less than it promised. The aggregator business is hard. Companies that set out to build vehicle-to-grid around fleets have found the business demanding. Nuvve, the aggregator behind the Danish vans and many American school bus projects, has spent years as a small listed company working to reach scale and profitability. The fleets themselves can be good grid resources while the businesses that aggregate them remain fragile, because revenue arrives slowly, project by project, while the costs of building software, integrating hardware and winning market access arrive up front. What a fleet manager should ask For a fleet operator considering bidirectional capability, the argument of this book translates into a short set of practical questions, each corresponding to a link in the chain. Do the vehicles support bidirectional power transfer, and on which standard? A vehicle that supports only one-way ISO 15118-2 charging, or bidirectional operation only through a proprietary arrangement, constrains every later choice. Vehicles that support ISO 15118-20 bidirectional sessions, or CHAdeMO for older vehicles, keep options open. Will the vehicle manufacturer honour the battery warranty under grid use, and under what limits? This must be in writing, ideally specifying permitted throughput or approved programs. Do the chargers speak ISO 15118-20 to the vehicle and a recent version of OCPP to the backend, and are they certified for export under local interconnection rules? A charger that can discharge but cannot be directed through open protocols ties the fleet to one vendor's software. What does the depot's grid connection allow? Export capacity may be limited separately from import capacity, and the distribution company must agree to both. Which programs or markets are open to the fleet, and what do they require? Metering resolution, telemetry, minimum size, prequalification and penalties differ widely. The answer determines which services are within reach. Who operates the energy management system, and how does it balance operational readiness against grid revenue? The fleet's own staff must trust that buses will be ready, which means clear minimum charge rules and a record of meeting them. What happens when communications fail? The depot's default behaviour on loss of contact with the backend or aggregator should be defined and tested. None of these questions concerns battery chemistry, and none can be answered by the vehicle alone. They are questions about agreements: with the manufacturer, the charger supplier, the network operator, the program operator and the fleet's own staff. A fleet that has clear answers to all seven is ready for vehicle-to-grid. A fleet that has answers to only some will find that its revenue depends on the one it missed. Why fleets lead Even with these caveats, the case for fleets first is strong, and it follows directly from the argument of this book. Vehicle-to-grid needs an unbroken chain of electrical, digital and commercial agreements. A fleet operator controls more of that chain than any other participant. It can choose chargers and vehicles that speak the same protocols, standardise its software, set its own battery policy, apply for one large grid connection rather than hundreds of small ones, and sign one contract with a utility or aggregator. The difficulties of the chain do not vanish, but they are concentrated in one organisation with the means and motive to solve them. Fleets also teach the rest of the market. Every school bus depot that operates successfully produces data on availability, battery wear, charger reliability and grid performance. Every fleet program forces utilities and grid operators to write rules for vehicle participation. Those rules, once written, are available to the next participant, including the private car owner. The final chapter turns to how those rules, and the aggregators that work within them, turn many separate vehicles into something a grid operator can rely on. Chapter 8: Building the Virtual Power Plant A single bidirectional car is a curiosity to a grid operator. Ten thousand of them, coordinated, are a power plant. The phrase "virtual power plant" describes exactly this: a portfolio of small, distributed resources, controlled by software and contracted as a unit, that offers the grid the same services a conventional plant would. Vehicles are only one possible member of such a portfolio, alongside home batteries, smart thermostats, water heaters and commercial buildings, but they are an increasingly large one. This chapter explains how aggregation turns unreliable individual cars into a reliable collective resource, what market and regulatory rules decide whether that resource gets paid, and how the business models of 2025 and 2026 have begun to assemble the chain end to end. Why aggregation works The basic idea is statistical. Any single car may be unplugged when it is needed. But if each of ten thousand cars has, say, a 60 percent chance of being plugged in at seven in the evening, and their owners' choices are reasonably independent, the number actually plugged in will cluster tightly around six thousand. The uncertainty of each car mostly cancels out across the portfolio. An aggregator can therefore promise a grid operator a quantity of capacity far more reliably than any individual owner could. In practice it is more subtle. Owners' choices are not fully independent. A public holiday, a heat wave, a major sporting event or a school half-term shifts everyone's plans at once. So aggregators build forecasting models from historical plug-in data, calendars and weather, and they hold back a safety margin. The better their data, the smaller the margin and the more capacity they can sell. This is why the plug-in behaviour found in the British Sciurus trial, where participants with a financial incentive plugged in far more often than ordinary owners [12], matters so much commercially: every point of extra availability turns directly into sellable capacity. Aggregation also lets a portfolio stack services. The same fleet of cars can provide fast frequency response from a subset of vehicles, while others are held for a peak event later in the evening, and the rest simply charge smartly. Software decides, minute by minute, which vehicle does what, subject to each owner's limits. A well-run portfolio earns more per vehicle than any single service could provide, because it sells each slice of capacity to the buyer who values it most at that moment. Proving it happened A grid operator paying for a service needs proof that it was delivered. For a power plant, this is easy: meter its output. For a virtual power plant, it is harder, because many of its services involve doing something different from what would otherwise have happened. If a fleet of cars stops charging during a peak event, the "service" is the charging that did not happen, and measuring something that did not happen requires an estimate of what would have occurred. That estimate is called a baseline. Baselines are a perennial source of argument in demand response. If the baseline is set too high, aggregators are paid for reductions that would have happened anyway; too low, and they are underpaid. For exports, the problem is simpler, because power flowing out through a meter is directly measurable. This is one quiet advantage of vehicle-to-grid over smart charging alone: an export is unambiguous evidence. But it also introduces the separation problem discussed in Chapter 5. If a house also has solar panels, the meter alone cannot say how much of the export came from the car. The usual solution is to meter the charger itself, with accuracy and resolution the grid operator accepts, and to report those readings through the communication chain. This is why the telemetry capabilities of chargers, and the OCPP and utility protocols that carry their readings, are not technicalities. They determine which services a portfolio can prove it delivered, and therefore which services it can be paid for. Opening the markets For most of the history of electricity markets, small resources were simply not allowed in. Market rules were written for power plants, with minimum sizes of a megawatt or more, requirements for dedicated telemetry, and settlement processes that assumed a single large meter. Aggregations of small devices did not fit. That has changed substantially in the last decade, though unevenly. In the United States, the Federal Energy Regulatory Commission issued Order 2222 in September 2020, requiring the regional grid operators under its jurisdiction to allow aggregations of distributed energy resources to participate in wholesale markets, with a minimum aggregation size no larger than 100 kilowatts [20]. Implementation has been slow. Each regional operator had to design rules for registration, telemetry, coordination with distribution utilities and the avoidance of double payment, and the timelines stretched. PJM, the largest market, has had its full implementation date set for February 2028, while others are earlier or later. California's operator already had a framework for distributed aggregations before the order. In Europe, market access has been shaped by national rules and the gradual harmonisation of balancing markets. Several transmission operators have lowered minimum bid sizes and allowed pooling of small units for frequency and restoration reserves. Britain's system operator has repeatedly opened its services and its central balancing mechanism to smaller and aggregated units; in August 2022, in a trial with Octopus Energy, twenty electric vehicles charged and discharged in response to the system operator's balancing mechanism instructions, a test an Octopus executive called a "line in the sand" moment for the technology [26]. Market access, however, is only half of the problem. The other half is the retail and network layer. A household exporting from its car must have a tariff that pays for exports at a sensible rate, an export permission from its network operator, and protection from being charged twice for electricity it stores and returns. That last issue proved decisive in Germany. Until late 2025, electricity drawn from the grid to charge a car, then fed back, could incur network fees and taxes on the way in without any refund on the way out, which made most vehicle-to-grid business cases impossible. In November 2025 the Bundestag adopted an amendment exempting electricity that is taken from the grid, stored in any storage system, including vehicles, and fed back into the same grid, from network fees and electricity tax [18]. Within months BMW and E.ON had launched Germany's first consumer vehicle-to-grid offer [17]. The sequence is worth noting: the hardware and software were ready before the rule changed, and the product followed the rule within months. Policy as a link in the chain Governments have begun to treat bidirectional capability as infrastructure policy rather than an industrial curiosity, and three examples show different approaches. The European Union has acted through charging infrastructure standards. Its regulation on alternative fuels infrastructure, known as AFIR, together with a delegated regulation adopted in 2025, requires new and renovated charging points to support ISO 15118-20 from 1 January 2027 [19]. This does not force anyone to discharge cars, but it ensures that the communication capability for bidirectional power transfer will be present at a growing share of the continent's chargers, removing one link of the chain as an obstacle. California has acted through enabling authority. A 2024 law, SB 59, authored by state senator Nancy Skinner, gives the California Energy Commission, in consultation with the Air Resources Board and the Public Utilities Commission, the power to require bidirectional capability in specific weight classes of electric vehicle if it finds a compelling use case [21]. An earlier draft would have set a hard date; the enacted version leaves the decision to the regulators. It is a statement of intent backed by a credible threat, aimed squarely at carmakers. Germany, as already described, acted through tariffs and fees, removing a financial barrier that no standard could have fixed. Britain has acted mainly through its system operator's market reforms and through its smart charging regulations of 2021, which required chargers sold for home use to be capable of smart operation, laying a foundation that bidirectional charging builds on. These approaches are complementary. Standards ensure that the digital link exists; market and tariff rules ensure that the commercial link pays; interconnection rules, as Chapter 5 showed, ensure that the electrical link is permitted. A country that does only one of the three will see little vehicle-to-grid. A country that does all three will see it emerge as a product. The bundles of 2025 and 2026 The most important commercial development of the last two years has been the appearance of complete consumer offers, in which one or two companies assemble the whole chain and sell it as a single product. Table 5 sets out several real deployments and offers, from early pilots to current products, with the figures their operators or reporters have published. Table 5. Selected vehicle-to-grid deployments and offers, with reported figures. Project or offer Where and when Vehicles and hardware Service Reported figure Frederiksberg Forsyning with Nuvve Denmark, from 2016 10 Nissan vans, about 10 kW DC Frequency reserve About €1,400 per vehicle per year [11] Sciurus (OVO, Cenex, Nissan, Indra) Great Britain, 2018 to 2021 About 300 homes, DC Tariff and frequency services About £340 per year average saving [12] Highland Electric Fleets, Beverly Massachusetts, 2022 2 school buses, DC Peak events 7+ MWh exported, 32 events [13] Utrecht Energized (MyWheels, We Drive Solar, Renault) Netherlands, from 2025 50 Renault 5, AC, target 500 Local flexibility Target about 5 MW [15] Octopus Energy and BYD Power Pack Great Britain, from 2025 BYD Dolphin, bidirectional home charger Bundled tariff Free home charging if plugged in 20 times a month for 12 hours [16] BMW and E.ON Germany, from early 2026 BMW iX3, 11 kW bidirectional wallbox Bundled tariff Credits up to €720 per year [17] The bundles share a pattern. The Octopus Energy and BYD offer, launched in June 2025, packaged a leased BYD Dolphin, a bidirectional home charger and an energy tariff for a monthly payment of just under £300; customers who plugged in at least twenty times a month for at least twelve hours each time received free home charging within their lease mileage, and Octopus estimated savings of over £620 a year against a standard electric vehicle tariff [16]. The plug-in condition is the aggregator's availability forecast turned into a contract term. The BMW and E.ON offer, launched in Germany in early 2026, paired the new iX3 with an 11-kilowatt bidirectional wallbox and a dedicated tariff, crediting customers up to €720 a year for allowing the companies to use the car's battery while plugged in [17]. Renault's Mobilize brand has offered vehicle-to-grid to private customers in France, reporting savings of around €300 a year for a driver covering 10,000 kilometres, with a Dutch consumer offer planned [15]. And in February 2026 Tesla began inviting Cybertruck owners in parts of Texas, enrolled in its own retail electricity plan and fitted with its home gateway, to discharge their trucks during grid events in return for bill credits, with California named as the next market [23]. Each of these closes the chain by having a small number of companies own almost every link: the carmaker supplies a vehicle certified to discharge, the charger is specified or supplied, the energy company provides the tariff, the export permission, the metering and the market access, and the software platform of one partner directs the whole. The driver sees a simple bargain: plug in, and pay less. Risks of the virtual power plant Turning millions of cars into a coordinated resource creates risks that do not exist when each car acts alone, and the industry is still learning to manage them. Concentrated control. An aggregator able to command a gigawatt of vehicles is, in effect, operating a large power plant. If its software misbehaves, or if all its vehicles respond to the same price signal at the same moment, the result can be a sudden swing in demand or supply large enough to disturb the grid. Grid operators increasingly require aggregators to ramp their portfolios gradually and to coordinate with distribution companies. Cybersecurity. The same concentration makes aggregators and charger backends attractive targets. A compromised platform could, in principle, order thousands of chargers to start or stop at once. This is why ISO 15118-20 mandates encryption between car and charger, why OCPP has steadily strengthened its security profiles, and why regulators in several countries have begun to impose security requirements on charger software and aggregation platforms. The protection is only as strong as the weakest link, which may be a poorly maintained server rather than a protocol. Fairness and access. Early bundles are available mainly to people who can afford a new car, have off-street parking, and live where the network operator grants export permission. Those who rent, park on the street, or live in congested areas may be excluded from the savings while still paying for grid upgrades through their bills. How the benefits of vehicle-to-grid are shared is a policy question that the market alone will not answer. Lock-in. Bundles that tie a particular car to a particular charger and tariff solve the chain problem by closing it. They may also make it harder for a driver to switch energy supplier or for a new aggregator to compete. The long-term promise of open protocols such as ISO 15118-20 and OCPP 2.1 is that vehicles, chargers and service providers can be mixed freely. The near-term reality is that integrated bundles work best. Regulators will need to watch that the first does not foreclose the second. The shape of what is emerging Step back and the outline of a mature system is visible. Cars leave factories with bidirectional capability and certified grid-support behaviour. Chargers speak ISO 15118-20 to the car and OCPP 2.1 to the cloud, carry certified meters, and enforce local grid-code settings. Aggregators, whether energy companies, carmakers or independent specialists, combine thousands of vehicles with home batteries and flexible loads, forecast their availability, and sell their combined capacity into wholesale, balancing and local network markets that have opened to them. Owners set their limits in an app and receive a lower bill or a monthly credit. Distribution companies see the portfolio's behaviour on their networks and can constrain it where necessary. None of the components of that system is speculative in 2026. Each exists, somewhere, in working form. What is incomplete is the joining of them, at scale, across brands and borders. That is the work of the next several years, and it is the subject of the conclusion. Conclusion The argument of this book has been that a parked electric vehicle becomes a dependable grid resource only when an unbroken chain of agreements holds from the grid operator to the battery cell, and that the battery is the easiest link. The record of the last two decades bears this out with unusual clarity. The battery has been capable of grid service since the first Nissan Leafs. What delayed vehicle-to-grid was everything around it: a vehicle-to-charger protocol that could not discharge until 2022, a charger-to-cloud protocol that gained a bidirectional block only in 2025, a certification pathway for alternating-current bidirectional chargers in the United States that arrived in 2026, export permissions designed for occasional solar panels, metering not built to separate a car's export from a house's, and fee rules in Germany that taxed stored electricity twice until late 2025. Each time one of these links was repaired, products appeared within months. That pattern carries several consequences for the people who will decide how fast vehicle-to-grid grows. Six consequences Treat the chain as the product The first consequence is that anyone trying to accelerate vehicle-to-grid should spend less effort on the vehicle and more on the joins. The question to ask of any proposed program is not whether the cars can discharge, but whether each link can be named, tested and contracted: which protocol carries the signal at each hop, who certifies the exporting device, who grants export permission and how quickly, which meter settles the payment, who stands behind the battery. A program that cannot answer each question has not yet been designed. For regulators this means that the most useful interventions are often unglamorous. Making export permission for certified bidirectional chargers fast and predictable, perhaps through pre-approved settings that a network operator can push to the device rather than a twelve-week application, would unlock more capacity than any subsidy for hardware. Accepting certified meters inside chargers for settlement would remove a barrier that otherwise forces every program to install its own. Requiring that charging equipment support the open protocols, as the European Union has done for ISO 15118-20 from 2027, ensures that the digital link exists when the commercial one is ready. Price the services, not the energy The second consequence follows from Chapter 2 and Chapter 6 together. The services that suit vehicles best are those with high value per kilowatt-hour and low throughput: peak events, local network relief, fast frequency response in moderation. The service that suits them worst, as a foundation, is daily energy arbitrage on thin margins, which works the battery hard for little reward. Policies and tariffs that pay vehicles mainly for energy moved will tend to produce programs that wear batteries and disappoint owners. Policies that pay for availability, for performance at specific stressed hours, and for relief at specific congested places will produce programs that are gentle on batteries and valuable to the grid. This also means that distribution network operators, not just wholesale markets, should be among the most important customers for vehicle-to-grid. The greatest single cost that the electrification of transport and heating imposes on power systems is the reinforcement of local networks. A car that can reduce its draw, and export modestly, during the few hours a year when a particular street's transformer would otherwise overload is worth more to that network than almost anything it could sell in a regional market. Network operators that build mechanisms to buy this service, transparently and at scale, will capture value that the rest of the system cannot. Never lose sight of smart charging The third consequence is a corrective to enthusiasm. Bidirectional charging gets the attention, but managed one-way charging delivers most of the system benefit of flexible vehicles at a fraction of the cost and with no battery wear at all. The British trial results showed vehicle-to-grid roughly tripling a household's savings compared with smart charging, but smart charging was already worth having. Every program and policy aimed at vehicle-to-grid should first ensure that every charging vehicle is at least charging intelligently: off the peak, responsive to local constraints, available to shift. Vehicle-to-grid is the second step on a staircase whose first step is still not universally climbed. Let the fleets go first, on purpose The fourth consequence concerns sequence. Fleets with predictable dwell, school buses above all, collapse the chain into one decision-maker, deliver concentrated power at single sites, and generate data that teaches everyone else. Public bodies buying electric buses and vans with public money have a strong case for specifying bidirectional capability as standard, as Oakland did, and for enrolling those fleets in grid programs from the outset. The marginal cost of bidirectionality on a vehicle being electrified anyway is modest, and the experience gained, about availability, wear, charger reliability and program design, is a public good. Settle who carries the battery risk The fifth consequence concerns the owner's fear. The chemistry of battery wear under vehicle-to-grid is reasonably well understood, and with sensible limits the added wear is modest. What is not settled is who bears the risk. Warranties written in years and kilometres do not fit a product whose wear depends partly on energy cycled while parked. The bundles of 2025 and 2026 solved this by making the carmaker a party to the program. A broader market will need warranties expressed partly in throughput, aggregator guarantees, or insurance products that price the risk explicitly. Regulators considering mandates for bidirectional capability, as California's SB 59 allows, would do well to require clarity on this point at the same time, because a capability that owners are afraid to use delivers nothing. Keep the chain open The final consequence is the most important for the long run. The first successful consumer offers are closed chains: one carmaker, one charger, one energy company. They work because they sidestep the interoperability problems that kept vehicle-to-grid in pilots for years. But a grid that depends on millions of vehicles cannot rest on a few proprietary bundles. Drivers need to be able to change energy supplier without changing car; aggregators need to be able to compete for vehicles regardless of brand; network operators need to be able to call on every capable vehicle on a constrained street, not just those of one manufacturer. The open protocols exist for exactly this purpose. ISO 15118-20, OCPP 2.1, IEEE 2030.5 and OpenADR together describe a chain that anyone can join. The task for standards bodies, regulators and the industry over the coming years is to make that open chain work as smoothly as the closed bundles do now, through rigorous conformance testing, through rules that require interoperability rather than merely permit it, and through attention to the data and consent arrangements that let information flow where it must. If that happens, the parked car will become what Kempton and Tomić described in 2005: not a substitute for power plants and grid infrastructure, but a large, distributed, fast-acting resource that makes both cheaper and more resilient. The buses in Beverly and the hatchbacks in Utrecht will look in retrospect less like experiments than like the first visible signs of an ordinary feature of the power system. If it does not happen, vehicle-to-grid will remain what it has too often been, a technology that works in each demonstration and nowhere else. The batteries are ready either way. The question is whether the agreements around them will be. Notes Willett Kempton and Jasna Tomić, "Vehicle-to-grid power fundamentals: Calculating capacity and net revenue," Journal of Power Sources 144, no. 1 (2005): 268–279. Willett Kempton and Jasna Tomić, "Vehicle-to-grid power implementation: From stabilizing the grid to supporting large-scale renewable energy," Journal of Power Sources 144, no. 1 (2005): 280–294. Paul Barter, "Cars are parked 95% of the time. Let's check!", Reinventing Parking, February 2013. The figure is an order-of-magnitude estimate, not a precise measurement. International Organization for Standardization, ISO 15118-20:2022, Road vehicles — Vehicle to grid communication interface — Part 20: 2nd generation network layer and application layer requirements (2022). International Organization for Standardization, ISO 15118-2:2014, Network and application protocol requirements, and ISO 15118-3:2015, Physical and data link layer requirements. Open Charge Alliance, "OCPP 2.1 is now available," announcement of 23 January 2025. IEEE Std 1547-2018, Standard for Interconnection and Interoperability of Distributed Energy Resources with Associated Electric Power Systems Interfaces; IEEE Std 2030.5-2018, Standard for Smart Energy Profile Application Protocol. QualityLogic, "UL 1741 SC: What it takes to certify the bidirectional charger behind AC vehicle-to-grid," 2026, describing the approval of UL 1741 Supplement SC in May 2026 and its relationship to UL 1741 SB, IEEE 1547 and SAE J3072. Matthieu Dubarry, Arnaud Devie and Katherine McKenzie, "Durability and reliability of electric vehicle batteries under electric utility grid operations: Bidirectional charging impact analysis," Journal of Power Sources 358 (2017): 39–49. Kotub Uddin, Tim Jackson, Widanalage D. Widanage, Gael Chouchelamane, Paul A. Jennings and James Marco, "On the possibility of extending the lifetime of lithium-ion batteries through optimal V2G facilitated by an integrated vehicle and smart-grid system," Energy 133 (2017): 710–722. Nuvve, "The Value of V2G: Denmark," presentation to the California Public Utilities Commission, 28 November 2017, reporting results from the Frederiksberg Forsyning installation, September 2016 to September 2017. Project Sciurus final results (OVO Energy, Cenex, Nissan, Indra), as reported by Current± and EV Infrastructure News, "World's largest domestic V2G trial finds hardware costs still too high despite significant financial rewards," June 2021. School Bus Fleet, "Electric buses get summer job in Massachusetts," 2022, reporting results from Highland Electric Fleets' Beverly project with National Grid's ConnectedSolutions program. Zum, press release, "Zum launches nation's first school district with 100% electric, bidirectional V2G school bus fleet in Oakland," May 2024; Electrify News, "Oakland Unified is the first U.S. school district with 100% electric, bidirectional V2G school bus fleet," 28 May 2024. The 2.1 GWh annual figure is a projection by the partners. electrive, "Utrecht Energized launched: how bidirectional charging is set to make the breakthrough," 11 June 2025. electrive, "Octopus Energy and BYD launch UK's first V2G bundle," 24 June 2025. BMW Group, press release, "Germany's first bidirectional vehicle-to-grid (V2G) charging offer from BMW Group and E.ON," September 2025; EV Infrastructure News, "BMW launches 'Germany's first' consumer V2G bundle with E.ON," 2026. electrive, "Germany clears the way for bidirectional charging," 14 November 2025. Regulation (EU) 2023/1804 on the deployment of alternative fuels infrastructure (AFIR), and the supplementing Commission delegated regulation of 2025 on technical specifications, as summarised by Pionix, "AFIR Regulation Update – Spring 2025: ISO 15118-20, Displays, Smart Charging & More," 2025. Federal Energy Regulatory Commission, Order No. 2222, Participation of Distributed Energy Resource Aggregations in Markets Operated by Regional Transmission Organizations and Independent System Operators, 17 September 2020. Implementation status as summarised by Microgrid Knowledge and regional operator filings through 2025. California Senate Bill 59 (Skinner), chaptered 2024; Electrek, "Bidirectional charging may be required on EVs soon due to new CA law," 28 September 2024. Utility Dive, "GM Energy, PG&E launch bidirectional charging pilot," March 2025. Electrek, "Tesla launches Cybertruck V2G program in Texas," 11 February 2026. City of Boulder, Colorado, "Innovative electric vehicle charger shows financial promise in first year," news release on the Fermata Energy installation at the North Boulder Recreation Center. BloombergNEF, "Lithium-ion battery pack prices fall to $108 per kilowatt-hour despite rising metal prices," December 2025. EV Infrastructure News, "Octopus Energy and National Grid ESO hail 'line in the sand' moment for V2G tech," 2022. Glossary Aggregator. A company that contracts with many owners of small energy resources, such as vehicles or home batteries, and offers their combined flexibility to a grid operator, utility or market as a single resource. Bidirectional power transfer (BPT). The ISO 15118-20 term for an energy transfer service in which power can flow from the vehicle to the charger as well as from the charger to the vehicle, defined for both AC and DC charging. Calendar ageing. Loss of battery capacity that occurs over time regardless of use, driven mainly by temperature and state of charge. Control pilot. The signalling conductor in a charging connector over which the charger and vehicle exchange basic state information and, for ISO 15118, powerline data communication. Dynamic mode. An ISO 15118-20 control mode in which the charger or its backend sends power setpoints and the vehicle follows them within limits it has declared. Interconnection standard. A rule set, such as IEEE 1547 in North America or EN 50549 and national codes in Europe, that governs how a generating or exporting device must behave when connected to the public grid. Islanding. A condition in which a generator continues to energise a section of grid that has been disconnected from the wider network, a hazard to line workers that interconnection rules require devices to prevent. OCPP. The Open Charge Point Protocol, used between a charger and its central management system; version 2.1 (2025) added native support for bidirectional charging. Plug & Charge. Automatic authentication and billing through digital certificates exchanged between vehicle and charger under ISO 15118, without a card or app. Scheduled mode. An ISO 15118-20 control mode in which the charger supplies limits and prices and the vehicle computes and announces its own charging and discharging plan. Smart charging (V1G). Varying the timing or rate of one-way charging in response to prices, grid conditions or external signals, without discharging energy back. Virtual power plant. A portfolio of distributed resources, coordinated by software and contracted as a unit, that provides services to the grid comparable to those of a conventional power plant. Further Reading Willett Kempton and Jasna Tomić, "Vehicle-to-grid power fundamentals: Calculating capacity and net revenue," Journal of Power Sources, 2005. The founding analysis of the field. Its method for weighing a vehicle's revenue against its costs remains the right starting point, even though every number in it has since changed. Willett Kempton and Jasna Tomić, "Vehicle-to-grid power implementation: From stabilizing the grid to supporting large-scale renewable energy," Journal of Power Sources, 2005. The companion paper, setting out how vehicles might move from regulation markets toward supporting high shares of wind and solar. ISO 15118-20:2022, Road vehicles — Vehicle to grid communication interface — Part 20. The standard itself is long and expensive, but its introductory sections and use-case descriptions are the clearest statement of what the vehicle and charger are meant to agree. Readers wanting the practical view will find the explanatory material published by charging software firms and testing companies a gentler entry. Open Charge Alliance, OCPP 2.1 specification and accompanying documentation (2025). Freely available from the alliance. The sections on bidirectional power transfer and operation modes show how a charger's backend directs discharge. IEEE Std 1547-2018. The North American interconnection standard for distributed energy resources. Its requirements for ride-through, voltage support and island detection explain why exporting devices behave as they do. Matthieu Dubarry, Arnaud Devie and Katherine McKenzie, "Durability and reliability of electric vehicle batteries under electric utility grid operations: Bidirectional charging impact analysis," Journal of Power Sources, 2017. A careful laboratory study showing how added vehicle-to-grid cycling can accelerate wear. Kotub Uddin and colleagues, "On the possibility of extending the lifetime of lithium-ion batteries through optimal V2G facilitated by an integrated vehicle and smart-grid system," Energy, 2017. The counterpart study, showing how battery-aware control can limit or even reduce wear. Read together with Dubarry, it explains why the answer to the battery question depends on control strategy. Cenex and partners, Project Sciurus reports (2021). The largest domestic vehicle-to-grid trial of its era, with unusually frank findings on plug-in behaviour, savings and hardware cost. Federal Energy Regulatory Commission, Order No. 2222 (2020). The order that required American wholesale markets to admit aggregations of distributed resources, and the starting point for understanding how virtual power plants reach those markets. Hashtags: #VehicleToGrid #V2G #V2GIntegration #ElectricVehicles #EVTechnology #BidirectionalCharging #VehicleToEverything #V2X #VehicleToHome #V2H #VehicleToBuilding #V2B #DistributedEnergyResources #DER #SmartGrid #GridIntegration #EnergyStorage #BatteryEnergyStorage #VirtualPowerPlants #VPP #SmartCharging #EVChargingInfrastructure #ISO15118 #OCPP #DemandResponse #GridStability #FrequencyRegulation #PeakLoadManagement #RenewableEnergyIntegration #EnergyTransition

Download the Book (PDF): Introduction Somewhere in a pumping station, bolted to the housing of a motor that has run for eleven years, there is a sensor the size of a matchbox. It contains an accelerometer, a small radio, a battery, and a microcontroller with less working memory than a single photograph on a phone. Most of the time it is asleep. A few times a minute it wakes, samples the vibration of the motor for a second or two, and runs a neural network over what it heard. If the vibration looks like the motor's normal self, the sensor goes back to sleep and says nothing. If it does not, it sends a short message: something has changed. That sensor is the subject of this book, along with its cousins: the chip in a kitchen appliance that listens for a spoken command, the wearable that counts steps and recognises a fall, the trail camera that only wakes its image processor when something warm moves in front of it. Together they make up the field known as TinyML, which means running machine learning models on microcontrollers: processors that cost a few dollars or less, draw milliwatts or microwatts, and have memory measured in kilobytes rather than gigabytes. The usual way to introduce this field is through the model. Here is a neural network that recognises the word "yes"; here is how to shrink it from megabytes to kilobytes; here is how to make it run on a chip with no operating system. All of that is in this book, because none of it can be skipped. But the model is not where the field begins, and it is not where most TinyML projects succeed or fail. The argument The controlling idea of this book is that TinyML is an energy-budget discipline before it is a machine learning discipline. On a battery-powered sensor, every design decision is ultimately a decision about how many joules to spend and where. The reason to put a model on a microcontroller at all is that moving raw data off a device by radio costs far more energy than turning that data into a decision locally. A system that streams vibration waveforms to the cloud will exhaust its battery in weeks; a system that sends one byte when something is wrong can run for years. Machine learning is the tool that makes the second system possible, because it is the only general method we have for turning messy sensor signals into a small number of reliable judgements. Seen this way, the model is one component in a chain that also includes the sensor's sampling schedule, the signal processing that prepares features, the memory plan that lets everything fit, the thresholds that turn scores into alarms, and the update mechanism that keeps a fleet of devices honest over years. Getting the model's accuracy from 91 to 93 percent matters less than deciding correctly how often to wake up, how much audio to buffer, and what the device should do with a borderline score. Engineers who come to TinyML from cloud machine learning tend to over-invest in the model and under-invest in everything around it. Engineers who come from embedded systems tend to do the reverse. The work is to hold both in view. What the book covers The book follows that chain from the outside in and then back out again. Chapter 1 sets out the energy economics that justify the whole enterprise: why radio dominates the power budget of a wireless sensor, how duty cycling works, and what it means to design a device that must last five years on a pair of batteries. Chapter 2 describes the machine you are given: the Arm Cortex-M family that dominates the field, its memory hierarchy, the instruction-set extensions that matter for neural network arithmetic, and the new generation of microcontrollers with dedicated neural accelerators. Chapter 3 is about making a model fit, through quantization, pruning, distillation and the design of architectures that respect a budget of a few hundred kilobytes. Chapter 4 turns to the code itself. It explains how a model is actually executed in C++ on a device with no heap to speak of, no file system and often no operating system, using the LiteRT for Microcontrollers runtime (long known as TensorFlow Lite for Microcontrollers) as the main example and surveying the alternatives. The C++ appears in short inline fragments rather than long listings, because the point is to understand the structure of an embedded inference program, not to copy one. The next three chapters apply all of this to two problems that between them cover much of what TinyML is used for. Chapter 5 is about wake-word detection: the always-listening keyword spotter that sits in front of a voice interface, which must run continuously, react within a fraction of a second, and almost never fire by mistake. Chapter 6 is about predictive maintenance on industrial machinery: using vibration and sound to detect that a bearing, gearbox or pump is beginning to fail, framed as anomaly detection because real failure data is so scarce. Chapter 7 stays on the factory floor to deal with what happens after deployment, when machines change speed, sensors loosen and maintenance teams start to ignore alarms. Chapter 8 covers the problem of running a fleet: updating models over the air, securing devices, measuring power honestly, and benchmarking. The conclusion draws out what follows from all of this for anyone deciding whether, and how, to build a TinyML system. Who this is for The reader this book has in mind is technically literate but not necessarily a specialist in either half of the field. You might be a data scientist who has trained models in Python and wants to know what changes when the target has 256 kilobytes of RAM. You might be a firmware engineer who has written drivers for years and wants to understand what a quantized convolution is doing. You might be a reliability engineer or plant manager who has heard that sensors can now "detect failures with AI" and wants to know what that claim actually involves. Each chapter defines its terms as it goes, and the glossary at the back collects the ones that recur. A note on the state of the field TinyML as a named field is young. The term came into wide use around 2019, when Pete Warden and Daniel Situnayake published the first book on the subject and a research and industry community formed around an annual summit. Since then the field has matured and been absorbed into a broader conversation about "edge AI". The community organisation that began as the tinyML Foundation renamed itself the EDGE AI FOUNDATION in late 2024. Google renamed TensorFlow Lite as LiteRT the same year, and its microcontroller runtime followed. Qualcomm acquired Edge Impulse, one of the most widely used TinyML development platforms, in 2025. Microcontroller vendors now ship parts with on-chip neural processing units that would have seemed exotic five years ago. None of this changes the underlying physics. A coin cell still holds a fixed amount of energy, a radio still costs what a radio costs, and a model still has to fit in the memory the chip provides. Tools and product names will continue to change. The reasoning in this book is meant to outlast them. Chapter 1: The Battery Decides Every wireless sensor begins life with a fixed amount of energy and a promise about how long it will last. A coin cell of the kind found in a car key fob holds a little over two hundred milliamp-hours at three volts. A primary lithium cell the size of an AA battery, the kind industrial sensor makers favour because it tolerates heat and cold and loses little charge on the shelf, holds a few thousand. Those numbers are the whole budget. There is no recharging on a motor housing in a pumping station or a beehive in a field, and a sensor that has to be visited every few months to change its battery has usually cost more in labour than it saved in insight. The design question for a battery-powered sensor is therefore not "what can this device compute?" but "how should this device spend a fixed, small quantity of energy over five or ten years?" TinyML exists because of one particular answer to that question. To see why, it helps to look at where the energy in a wireless sensor actually goes. Where the joules go A typical battery-powered sensor node has four consumers of energy: the sensor itself, the microcontroller, the radio, and the leakage and housekeeping that never fully stop. Of these, the radio is almost always the most expensive per useful unit of work, and by a wide margin. The reason is physical. Computation on a modern microcontroller is the switching of very small transistors over very short distances on a chip. Radio transmission means pushing electromagnetic energy into free space, most of which is lost, so that a receiver tens or hundreds of metres away can pick out a signal from the background noise. The receiver also has to be powered, and the protocol overhead of establishing a link, synchronising, acknowledging and retransmitting adds further cost. Engineers in wireless sensor networks noticed this asymmetry early. In a much-cited article from 2000, Gregory Pottie and William Kaiser estimated that the energy needed to transmit a kilobit of data a hundred metres was roughly the same as the energy needed to execute three million instructions on a processor of the day. The exact ratio has shifted with every generation of radios and processors, but the direction has not. Processors have become more efficient per operation faster than radios have become more efficient per bit, so the gap has tended to widen. The practical consequence is stark. Consider a vibration sensor that samples an accelerometer at a few kilohertz. A single second of three-axis data at sixteen bits per sample is tens of kilobytes. Sending that once a minute over a low-power radio would keep the radio busy for a meaningful fraction of the time and drain a battery in weeks or months, depending on the radio. Sending it continuously is out of the question. The low-power wide-area networks used in industry, such as LoRaWAN, make the point even more sharply: they are designed for messages of a few dozen bytes, sent a handful of times an hour, and regional duty-cycle regulations on the unlicensed radio bands they use limit how much any device may transmit at all. A LoRaWAN sensor cannot stream a waveform even if it had the energy to. So the raw data must stay on the device. The only question is what the device does with it before deciding what, if anything, to send. The case for deciding locally There are three classical answers. The device can send summary statistics, such as the root-mean-square level of vibration over the last minute, and let a server decide. It can apply a fixed rule, such as an alarm if the level exceeds a threshold. Or it can run a model that turns the raw signal into a judgement: normal or abnormal, keyword or not, person or empty scene. Summary statistics and fixed rules have served industry for decades, and they remain the right answer more often than enthusiasts admit. A single overall vibration level, compared against a severity chart, catches many gross faults. But summaries throw away exactly the structure that distinguishes early, subtle problems from benign variation. A bearing defect in its early stages produces small, periodic impacts at characteristic frequencies that barely change the overall energy of the signal. A spoken keyword and a television in the background may have identical loudness. The information that matters is in the shape of the signal, and recognising shapes in noisy, high-dimensional data is precisely what machine learning is good at. TinyML, in this light, is a compression strategy. The model is the most aggressive compressor possible: it reduces kilobytes of raw signal to a single bit, or a handful of bytes, that carry the decision the operator actually needed. The energy spent running the model is the price of not running the radio. When that price is lower than the radio cost it replaces, local inference pays for itself in battery life, and the sensor can afford to look at its data far more often than it could afford to report it. There are secondary benefits that often matter as much. Data that never leaves the device cannot be intercepted or leaked, which is why on-device speech detection is attractive for privacy as well as power. A decision made locally is made in milliseconds, without waiting for a network round trip that may never complete in a basement, a mine or a moving vehicle. And a device that only reports exceptions generates far less data for someone to store, pay for and ignore. These benefits are real, but they are not the foundation. The foundation is energy. Duty cycling: the art of being asleep A microcontroller that ran its neural network continuously would undo most of the savings. The central technique of low-power design is duty cycling: keeping every part of the system in its lowest-power state for as much of the time as possible, and waking it only to do necessary work. Modern microcontrollers offer a ladder of power modes. At the top is run mode, with the processor core clocked and executing instructions; current draw is typically a few milliamps at tens of megahertz, and scales roughly with clock frequency. Below that are sleep modes in which the core stops but peripherals keep running, so that a timer or a direct memory access controller can keep collecting sensor samples into a buffer. At the bottom are deep sleep or standby modes in which almost everything is switched off except a real-time clock and a small amount of retained memory; current falls to around a microamp or less on the most efficient parts. The ratio between run and deep sleep can be a factor of several thousand. The average power of a duty-cycled device is dominated by two terms: the energy of each active burst multiplied by how often bursts happen, plus the floor current during sleep multiplied by all the time in between. Designers work this out on paper before anything is built. An illustrative calculation shows the shape of the reasoning. Suppose a vibration sensor wakes once a minute, spends one second sampling with the accelerometer and microcontroller drawing a combined two milliamps, then spends a hundred milliseconds running a model at five milliamps, then goes back to sleep at two microamps. Each minute it uses about 2 milliamp-seconds for sampling, 0.5 for inference and about 0.12 for sleep, a total of roughly 2.6 milliamp-seconds, or an average current of around 44 microamps. A cell of 2,600 milliamp-hours would, ignoring self-discharge, temperature and the radio, last for well over five years at that rate. The numbers are invented for the sake of illustration, but the lesson they teach is general: in this design the sampling, not the model, is the largest consumer, and the radio, if it were used to send raw data instead, would dwarf everything. Several design moves follow directly from this arithmetic. Shortening the sampling window or using a sensor with a lower-power acquisition mode saves more than optimising the model. Using the microcontroller's direct memory access controller to move samples from the sensor to memory while the core sleeps avoids paying for the core during acquisition. Running the model as fast as possible and then sleeping, a strategy known as "race to sleep", is usually better than running it slowly at a low clock, because fixed overheads such as leakage and the power of support circuitry are paid for every millisecond the chip is awake. And waking up less often is the most powerful lever of all, which is why the question of how often a machine actually needs to be checked is a design question, not a detail. The cascade The most important architectural idea in TinyML follows from duty cycling: the cascade. Rather than run one expensive model on everything, a system runs a sequence of increasingly expensive stages, each of which decides whether the next one needs to wake. The canonical example is voice. A smart speaker does not run a full speech recogniser on everything it hears. A first stage, sometimes implemented in analogue circuitry or a tiny always-on digital block, detects whether there is any sound energy at all. A second stage, a small keyword-spotting network on a low-power processor, listens for the wake word. Only when the keyword fires does the device wake its main processor, open a network connection and stream audio to a large recogniser in the cloud. The first stages run all the time at microwatts or milliwatts; the expensive stages run for a few seconds a day. The same structure appears everywhere once you look for it. A wildlife camera uses a passive infrared sensor to wake a small vision model that decides whether the warm object is an animal of interest before waking the image sensor at full resolution and the radio. A wearable uses an accelerometer's built-in motion detection to wake the microcontroller, which runs an activity classifier, which in turn decides whether a heart-rate sensor needs to be powered. An industrial sensor checks overall vibration energy, runs an anomaly detector only when the machine is running, and transmits a spectrum only when the detector is unsure. Cascades matter for TinyML in two ways. First, they change what each model needs to be good at. The early stages need high recall, because anything they miss is lost forever, but can tolerate false alarms, because the next stage will catch them. The late stages can afford to be more selective. A keyword spotter in a cascade can fire too often if a cloud recogniser will check its work; one that controls a device on its own cannot. Second, cascades mean that the most energy-critical model is often the smallest and simplest one in the system, the one that runs all the time. That is where TinyML techniques earn their keep. The hidden price of a battery change The energy budget has an economic twin that is easy to overlook. A battery cell costs very little. Replacing it does not. Someone has to know which device needs attention, travel to it, perhaps obtain a permit to work near running machinery, open an enclosure rated against dust and water, replace the cell, reseal the enclosure and confirm that the device has rejoined its network. In a factory, that visit can cost far more than the sensor itself. On a pipeline, a wind turbine or a remote agricultural site, it can cost more than the sensor many times over. Multiply by a fleet of thousands, and the difference between a device that lasts two years and one that lasts eight becomes the difference between a viable product and an unviable one. This is why industrial buyers ask about battery life before they ask about accuracy, and why manufacturers of wireless sensors quote lifetimes under stated assumptions about measurement frequency. Every additional measurement per day, every additional transmission, every extra hundred milliseconds of computation, is ultimately a cost charged to a maintenance budget years in the future. A TinyML designer who understands this will resist feature creep in a way that a cloud engineer, used to elastic resources, may find puzzling. Harvesting changes the shape, not the logic Some devices escape the fixed-battery constraint by harvesting energy from their surroundings. Small solar cells work outdoors and under bright indoor lighting. Thermoelectric generators extract power from temperature differences, such as between a hot pipe and the surrounding air. Piezoelectric and electromagnetic harvesters convert vibration into electricity, which has an obvious appeal for sensors mounted on vibrating machines. Radio-frequency harvesting can power the very smallest devices at short range from a dedicated transmitter. Harvesting does not abolish the energy budget; it changes its shape. A harvested supply is typically small, intermittent and dependent on conditions the device cannot control. A solar-powered sensor receives nothing at night and little on overcast days. A vibration harvester on a machine produces nothing when the machine is stopped, which is precisely when a monitoring sensor might be expected to report that it has stopped. Harvested energy is usually stored in a capacitor or small rechargeable cell, which buffers the supply but has its own leakage and limited capacity. Designing for harvesting therefore means designing for an energy income rather than an energy savings account. The device must adapt its behaviour to what is available: measure and infer more often when energy is plentiful, fall back to minimal operation when it is scarce, and never attempt an operation, such as a radio transmission, that would drain its store below the point at which it can recover. Some research systems go further and design computation to survive sudden power loss, saving progress so that a long inference can resume after the next burst of harvested energy. The logic of this chapter applies with even greater force: when every microjoule must be earned before it is spent, the case for reducing data to decisions locally, rather than transmitting it, becomes stronger still. When TinyML is the wrong answer Because this book is about deploying models on microcontrollers, it is worth saying plainly when not to. If the device has mains power and a wired network, the energy argument evaporates and the decision becomes one of latency, privacy and cost. A gateway or a small Linux computer may be a better place to run a model that serves many sensors. If the data rate is low enough that the radio cost is trivial, such as a temperature reading every ten minutes, there is nothing to compress. If the decision requires context from many sensors or from historical data, a single node cannot make it alone. And if a fixed threshold works, it should be used: it is cheaper to run, easier to explain and impossible to overfit. TinyML earns its place when four conditions hold together: the device runs on a constrained energy source, the raw signal is rich enough that transmitting it is expensive, the decision depends on patterns in that signal that simple rules miss, and the decision is needed locally or quickly. Wake-word detection and machine vibration monitoring, the two applications examined in detail later in this book, satisfy all four. Many of the products that have been marketed with "AI" on the label do not. Thinking in microjoules The habit this chapter argues for is to express every design choice in units of energy before anything else. How many microjoules does one inference cost? How many inferences per day does the use case require? What does one transmitted message cost, and how many messages will the device send? What is the sleep current, and what does it add up to over a year? These questions have concrete answers once a microcontroller and radio are chosen, and they can be measured on a bench with a current probe or a power analyser long before a model is finished. Teams that ask these questions first tend to make good decisions about everything else. They notice that a model twice as large but twice as accurate may reduce false alarms enough to save more radio energy than it costs to run. They notice that sampling one second instead of two halves the dominant cost. They notice when the radio's join procedure, not its transmissions, is quietly eating the battery. Teams that start from the model tend to discover these things late, on a prototype that lasts a tenth as long as the specification promised. The remaining chapters descend into the details: the processors, the compression techniques, the code, and the two case studies. Through all of them, the battery remains the final judge of whether a design is any good. Chapter 2: The Machine You Are Given A data scientist moving from cloud machine learning to microcontrollers experiences something like moving from a warehouse to a studio flat. Everything still has to happen, but there is no room for anything that does not earn its place, and the layout of the space determines what is possible. This chapter describes that space: the microcontroller, its memory, and the specific features of the Arm Cortex-M family that make neural network inference feasible on it. Arm is not the only architecture in TinyML. RISC-V cores are increasingly common, particularly in China and in custom silicon, and some vendors build their own digital signal processors. But Cortex-M cores ship in more microcontrollers than any other design, the software ecosystem is built around them, and their features illustrate the general principles well. What a microcontroller is A microcontroller is a complete small computer on a single chip: a processor core, non-volatile memory for program code, volatile memory for working data, and a collection of peripherals for talking to the outside world, including timers, analogue-to-digital converters, serial buses for sensors, and often a radio. It is designed to be cheap, to start instantly, to draw very little power and to behave predictably. It is not designed to run general-purpose software. The contrast with the processors in phones and servers is instructive. An application processor has a memory management unit that gives each program its own virtual address space, a cache hierarchy measured in megabytes, external dynamic memory measured in gigabytes, and an operating system that schedules many programs and manages files. A typical Cortex-M microcontroller has none of these. Memory is addressed physically; there is usually no cache, or a small one on high-end parts; the total on-chip memory is a few hundred kilobytes to a few megabytes; and the software is a single program, often running directly on the hardware ("bare metal") or under a small real-time operating system such as FreeRTOS or Zephyr that provides tasks and timers but no virtual memory or file system. Clock speeds range from a few tens of megahertz on the smallest parts to several hundred megahertz on high-performance ones, and in a few recent devices close to a gigahertz. That is an order of magnitude or two below a phone, but the gap in raw speed is not the main constraint. Memory is. Flash and SRAM: the two budgets Every microcontroller has two distinct kinds of memory, and a TinyML designer must budget both separately. Flash memory is non-volatile: it keeps its contents without power. It holds the program code and any constant data, including, crucially, the weights of a neural network. Flash is relatively plentiful, typically from 256 kilobytes to 2 megabytes on general-purpose parts and more on high-end ones. It is slow to write and wears out after many write cycles, so it is treated as read-only during normal operation. On most microcontrollers, code and constants can be read directly from flash by the processor, a property known as execute-in-place, which means model weights do not need to be copied into working memory. Static RAM, or SRAM, is volatile working memory. It holds everything that changes while the program runs: the stack, global variables, buffers of sensor samples, and the intermediate results of a neural network, known as activations. SRAM is fast but expensive in silicon area and in leakage power, so there is much less of it, typically from 64 kilobytes to 512 kilobytes on mainstream parts, and a few megabytes on the largest. On a battery-powered device, retaining SRAM contents during deep sleep costs current, so designers sometimes power down banks they do not need. This division shapes model design in a way that surprises newcomers. A model's weights live in flash, so a model with a million parameters, stored as eight-bit integers, needs about a megabyte of flash, which many parts have. But a model's activations live in SRAM, and the peak activation memory is determined not by the number of parameters but by the size of the largest intermediate tensors that must exist simultaneously during inference. A convolutional network processing a 96 by 96 pixel image with 32 channels in an early layer has an activation tensor of nearly 300,000 values at that layer, which as eight-bit integers is close to 300 kilobytes. The parameters of that layer might be only a few kilobytes. The result is that, on microcontrollers, memory problems are usually activation problems, and the early high-resolution layers of a vision network are where they arise. Researchers at MIT made this observation central to their MCUNet work, which showed that inference on small microcontrollers was limited far more by peak SRAM than by flash, and designed both the network architecture and the inference schedule to keep the peak low. The Cortex-M family Arm licenses its Cortex-M processor designs to chip vendors such as STMicroelectronics, NXP, Nordic Semiconductor, Infineon, Renesas, Microchip, Silicon Labs and many others, who combine them with memory and peripherals to make microcontrollers. The cores share a common programming model, so code written for one runs, with recompilation, on another, but they differ greatly in performance and in the arithmetic features that matter for machine learning. Table 1 summarises the cores most often encountered in TinyML work. Table 1. Arm Cortex-M cores commonly used for TinyML and the features relevant to inference. Core ISA Floating point Integer SIMD for ML Typical role Cortex-M0+ Armv6-M None None Ultra-low-power control, tiny models Cortex-M4 Armv7E-M Optional single precision DSP extension (32-bit SIMD) Mainstream sensor and audio processing Cortex-M7 Armv7E-M Optional single or double DSP extension, dual-issue pipeline High-performance control and vision Cortex-M33 Armv8-M Mainline Optional single precision Optional DSP extension Secure IoT with TrustZone Cortex-M55 Armv8.1-M Optional, includes half precision Helium vector extension (128-bit) ML-focused, often paired with Ethos-U NPU Cortex-M85 Armv8.1-M Optional, includes half precision Helium vector extension Highest-performance Cortex-M Source: Arm Cortex-M processor documentation. The Cortex-M0+ is the simplest core in common use: a short pipeline, no hardware floating point and no special arithmetic instructions. It can run small models, such as a gesture classifier on accelerometer data, but slowly. The Cortex-M4 is the workhorse of sensor processing. Its DSP extension adds single-instruction, multiple-data (SIMD) operations that work on several narrow integers packed into one 32-bit register, so that, for example, two 16-bit multiply-accumulate operations can be performed in one instruction. For neural networks, which consist overwhelmingly of multiply-accumulate operations, this roughly doubles throughput compared with plain integer code. The Cortex-M7 adds a longer, dual-issue pipeline and caches, and appears in high-performance parts running at several hundred megahertz. The Cortex-M33 brings Armv8-M's TrustZone security extension, which partitions the chip into secure and non-secure worlds, and is common in connected IoT devices. Helium and the vector turn The most significant change in the Cortex-M line for machine learning came with the Armv8.1-M architecture and its M-Profile Vector Extension, marketed as Helium. Helium gives a microcontroller core 128-bit vector registers and instructions that operate on all their lanes at once: sixteen 8-bit values, eight 16-bit values or four 32-bit values. It is conceptually similar to the Neon extension on Arm's application processors, but designed for the small area and power budgets of microcontrollers, with tricks such as overlapping the execution of successive vector instructions ("beats") so that a small datapath can still achieve useful throughput. For neural networks, Helium means that an eight-bit quantized convolution can process many more multiply-accumulates per clock cycle than on a Cortex-M4. Arm has reported large improvements in machine learning performance for the Cortex-M55 over earlier cores, and the Cortex-M85 and the smaller Cortex-M52 have since extended Helium across the performance range. The practical benefit for a developer is that the inference library, rather than the application code, takes advantage of these instructions. Arm's CMSIS-NN library provides optimised implementations of common neural network operators, such as convolution, depthwise convolution, fully connected layers and pooling, written to use the DSP extension or Helium where available and falling back to portable code where not. Runtimes such as LiteRT for Microcontrollers call CMSIS-NN automatically when compiled for an Arm target, so the same model runs faster on a better core with no change to the model. Neural processing units arrive The next step beyond vector instructions is a dedicated accelerator. Arm's Ethos-U family of micro neural processing units (microNPUs) are small blocks of silicon designed to sit next to a Cortex-M core and execute the bulk of a quantized network's operators, chiefly convolutions and fully connected layers, far more efficiently than any general-purpose core. The Ethos-U55 was announced alongside the Cortex-M55 in 2020; the Ethos-U65 targeted higher-performance systems; and the Ethos-U85 extended the family with support for transformer operators. A model destined for an Ethos-U is first passed through Arm's Vela compiler, which rewrites the supported parts of the network into a command stream for the NPU, leaving unsupported operators to run on the Cortex-M core. Chips combining these parts are now commercially available. Alif Semiconductor's Ensemble family pairs Cortex-M55 cores with Ethos-U55 NPUs; its second generation, announced in 2025, added the Ethos-U85. Other vendors have built their own accelerators. STMicroelectronics introduced the STM32N6, its first microcontroller with its in-house Neural-ART accelerator, at the end of 2024, aimed at vision workloads. NXP's MCX N series includes its own neural processing unit. These parts change the arithmetic of what is possible: models that would take seconds on a Cortex-M4 can run in milliseconds, and image-based tasks that were once marginal on microcontrollers become routine. It is worth being clear about what accelerators do not change. They do not create SRAM, so activation memory remains a constraint, though NPU-equipped parts tend to have more of it. They support a defined set of operators, and a model that uses an unsupported operator in the middle of the network will bounce back and forth between NPU and CPU, losing much of the benefit. They add cost and, when active, power; on a device that runs one small inference a minute, an accelerator may save little energy compared with a well-optimised Cortex-M4 that wakes briefly and sleeps. And they make the software stack more vendor-specific. The choice of hardware is therefore an energy and cost decision made against the actual workload, not a matter of buying the fastest chip. Peripherals that matter Two peripheral features deserve mention because they shape TinyML systems as much as the core does. The first is the direct memory access controller, which moves data between peripherals and memory without the processor's involvement. A microphone sampled at 16 kilohertz, or an accelerometer at a few kilohertz, delivers a steady stream of samples. Without DMA, the processor would have to wake for each one. With DMA, samples flow into a buffer in SRAM while the core sleeps or runs inference on the previous buffer, and the core is interrupted only when a buffer is full. Double buffering, in which DMA fills one buffer while the processor works on the other, is the standard pattern for continuous audio processing and appears again in the wake-word chapter. The second is the set of low-power, always-on features that let a system cascade without waking the main core. Many MEMS accelerometers and inertial measurement units now contain their own small processing logic: they can detect motion, free fall or taps, and some include programmable decision trees or small machine learning cores that classify simple activities inside the sensor itself. Some microphones and audio codecs include voice activity detection. Using these features to keep the microcontroller asleep until there is something worth processing is often the biggest single power saving in a design. Beyond Arm Although this book uses the Cortex-M family as its main example, three other kinds of processor appear regularly in TinyML work, and a designer should know where they fit. The first is RISC-V, an open instruction-set architecture that anyone may implement without a licence fee. RISC-V microcontroller cores are now produced by many companies, and the architecture includes an optional vector extension and a packed-SIMD proposal comparable in purpose to Arm's DSP and Helium extensions. Academic projects such as the PULP platform from ETH Zurich and the University of Bologna have built clusters of small RISC-V cores with instruction-set extensions specifically for low-precision neural network arithmetic, and some commercial chips derive from that work. The software ecosystem for RISC-V machine learning is less uniform than Arm's, because implementations vary more, but it is growing quickly. The second is the family of microcontrollers built around Cadence Tensilica Xtensa cores, most visibly Espressif's ESP32 series. These chips integrate Wi-Fi and Bluetooth at very low cost and are hugely popular with hobbyists and in consumer products. The ESP32-S3 added vector instructions aimed at signal processing and neural networks, and Espressif supplies its own optimised libraries. Their radios make them less suitable for the lowest-power battery applications, but for mains-powered or rechargeable devices they are a common TinyML platform. The third is the dedicated digital signal processor, frequently a Cadence HiFi or similar core, embedded in audio chips and in the always-on subsystems of larger processors. These are where many commercial keyword spotters run, sitting beside a main processor that stays asleep until the DSP detects something worth waking it for. The principles in this chapter carry over to all of them. Each has a small working memory, a larger read-only store for weights, some form of SIMD arithmetic that a vendor library exploits, and a ladder of power states. The vocabulary differs; the budget does not. A memory budget in practice It is worth making the flash and SRAM budgets concrete with the kind of calculation a designer does at the start of a project. Suppose the target is a mainstream microcontroller with 1 megabyte of flash and 256 kilobytes of SRAM, and the application is a vibration sensor with a Bluetooth Low Energy radio. On the flash side, the Bluetooth stack, the real-time operating system, sensor drivers, the bootloader and the application logic together might consume a few hundred kilobytes. If the device must support over-the-air updates by holding a complete second copy of its firmware, as Chapter 8 describes, the usable space is roughly halved. What remains for model weights may be a couple of hundred kilobytes, which at eight bits per weight allows a model of perhaps two hundred thousand parameters, generous for vibration but not for vision. On the SRAM side, the radio stack and operating system need their own memory, often tens of kilobytes. The buffer for raw sensor samples might be another few tens of kilobytes: two seconds of three-axis data at a few kilohertz and sixteen bits per sample is already in that range, and double buffering doubles it. Working memory for the fast Fourier transform and feature extraction adds more. The tensor arena for the model must fit in what is left, which may be well under a hundred kilobytes. The exact numbers vary by product, but the pattern is consistent. The model rarely gets the whole chip. It gets what remains after everything the device needs to function, and designers who budget for the model first and the rest of the system afterwards are routinely forced into painful compromises late in development. Choosing a part There is no universally right microcontroller for TinyML, but there is a sensible order in which to make the decision. Start with the energy budget and the sensors, which determine the acceptable sleep current, the peripherals required and the radio. Then estimate the model's needs: the number of multiply-accumulate operations per inference, the peak activation memory and the size of the weights. Then ask how often inference must run and how quickly it must finish. A keyword spotter running every few tens of milliseconds needs sustained throughput; a vibration classifier running once a minute needs very little. Only then compare cores, keeping in mind that a faster core finishing quickly and sleeping may use less energy than a slower one. Development boards make early experimentation cheap. Boards built around the Nordic nRF52840 (a Cortex-M4 with Bluetooth), the STM32 families, Arduino's Nano 33 BLE Sense and its successors, and various Cortex-M55 evaluation kits are widely used in teaching and prototyping. The constraint to bear in mind is that a development board's power consumption, with its regulators, LEDs and debug circuitry, tells you little about the power of a finished product. Measurements that matter are made on custom hardware, or at least on a board where the microcontroller's supply can be isolated and measured directly. The microcontroller is the fixed frame of the problem. The next chapter takes up the variable part: how to make a model small enough to live inside it. Chapter 3: Shrinking the Network The networks that dominate public discussion of artificial intelligence have billions of parameters and run on accelerators that draw hundreds of watts. A model destined for a microcontroller might have twenty thousand parameters and run in a few milliwatts. These are not points on one continuum that can be reached by taking a big model and making it smaller by degrees. They are different engineering problems, and the techniques for the second have their own logic. This chapter covers the four that matter most: choosing an architecture that fits, quantization, pruning and distillation. It then describes how automated search has begun to combine them. Start with the right architecture The single most effective way to get a small model is to design a small model, rather than compress a large one. Most TinyML models are built from a handful of architectural patterns that give good accuracy per operation and per byte. The most important of these is the depthwise separable convolution, popularised by the MobileNet family of networks from Google. A standard convolutional layer combines spatial filtering and channel mixing in a single operation: each output channel is computed by sliding a filter across every input channel. A depthwise separable convolution splits this into two steps. First, a depthwise convolution filters each input channel independently with its own small spatial kernel. Then a pointwise convolution, a one-by-one convolution, mixes channels. For typical layer sizes, this factorisation reduces both parameters and multiply-accumulate operations by close to an order of magnitude, at a modest cost in accuracy. Almost every successful small vision and audio network uses it. For audio keyword spotting, a 2017 study by Yundong Zhang and colleagues at Arm, titled "Hello Edge", compared several neural network architectures for keyword spotting under microcontroller-sized memory and compute budgets. Their depthwise separable convolutional network, usually called DS-CNN, achieved the best accuracy for a given budget among the families they tested, and it became the reference keyword-spotting architecture in later benchmarks. For time-series data from accelerometers and vibration sensors, one-dimensional convolutions over time, small recurrent networks and simple fully connected networks on hand-engineered features are all common; the right choice depends heavily on what preprocessing has been done, a theme Chapter 6 returns to. The arithmetic behind the depthwise separable saving is simple enough to work through. Take a layer with 64 input channels and 64 output channels and a 3 by 3 kernel. A standard convolution needs one 3 by 3 filter for every pair of input and output channels: 3 times 3 times 64 times 64, or 36,864 weights. The depthwise separable version needs one 3 by 3 filter per input channel, 576 weights, followed by a one-by-one mixing layer of 64 times 64, or 4,096 weights, for a total of 4,672. That is a reduction of nearly eightfold. Because each weight is used once at every spatial position, the number of multiply-accumulate operations falls by the same factor. If the layer operates on a 25 by 5 grid of positions, as it might on a small audio spectrogram after some downsampling, the standard convolution needs about 4.6 million multiply-accumulates and the separable one fewer than 600,000. On a Cortex-M4 that distinction can be the difference between an inference that fits within a 20-millisecond audio frame and one that does not. Two design principles follow from the memory structure described in the previous chapter. First, reduce resolution early. Because peak activation memory is set by the largest intermediate tensor, a network should downsample its input quickly, with strided convolutions or pooling, before expanding the number of channels. Second, prefer operators that the target runtime and hardware support efficiently. A clever layer that the inference library implements only in slow reference code, or that an NPU cannot run, may cost more than it saves. Designing for TinyML means reading the runtime's operator list before choosing layers. Quantization: from floats to bytes Neural networks are trained using 32-bit floating-point numbers. Almost all TinyML deployment uses 8-bit integers instead, for weights and activations alike. This conversion, called quantization, is the most important single step in getting a model onto a microcontroller. The benefits are immediate. Weights stored as 8-bit integers occupy a quarter of the space of 32-bit floats, so a model's flash footprint shrinks fourfold. Activations stored as 8-bit integers occupy a quarter of the SRAM. Integer arithmetic is far cheaper than floating point on microcontrollers, especially on cores without a floating-point unit, and the SIMD instructions of the DSP extension and Helium can process four or sixteen 8-bit values at once. Many NPUs support only integer arithmetic. The scheme most commonly used, described in a 2018 paper by Benoit Jacob and colleagues at Google, is affine quantization. Each tensor is associated with two numbers: a scale, which is a positive real number, and a zero point, which is an integer. A real value is represented by an 8-bit integer q such that the real value equals the scale multiplied by the difference between q and the zero point. The scale sets the step size of the representation; the zero point ensures that the real value zero is exactly representable, which matters because zero appears constantly in neural networks through padding and through activation functions such as ReLU that clip negative values to zero. During inference, a layer multiplies 8-bit inputs by 8-bit weights, accumulates the results in 32-bit integers to avoid overflow, adds a 32-bit bias, and then rescales the accumulated value to the output tensor's 8-bit representation using a fixed-point multiplier and a shift. No floating-point arithmetic is needed at all. Two refinements matter in practice. Weights are usually quantized per channel: each output channel of a convolution gets its own scale, because the ranges of different filters can differ greatly and a single scale for the whole tensor would waste resolution on the narrow ones. Activations are usually quantized per tensor, with the scale and zero point chosen from the range of values observed on representative data. Post-training quantization and its limits The simplest way to quantize is post-training quantization. A model is trained in floating point as usual. Then a converter, such as the LiteRT converter, runs a few hundred representative input samples through the model to record the range of each activation tensor, chooses scales and zero points, and writes out a fully integer model. For many networks the loss of accuracy is small, often a fraction of a percentage point. When the loss is larger, the usual cause is a tensor with an awkward distribution: a long tail of rare large values forces a wide range, leaving little resolution for the common small values. Remedies include clipping activations during training (ReLU6, which caps values at six, was adopted in MobileNets partly for this reason), choosing a better representative dataset, and, when necessary, quantization-aware training. In quantization-aware training, the model is fine-tuned with simulated quantization inserted into the forward pass, so that the network learns weights that are robust to rounding. It adds complexity to the training pipeline but typically recovers most of the lost accuracy. The representative dataset deserves more attention than it usually gets. If it does not cover the conditions the device will encounter, such as loud environments for a keyword spotter or high-speed operation for a vibration sensor, the chosen ranges will be wrong in the field and values will saturate. A quantized model that tests well on a laptop and behaves erratically on a device is very often suffering from calibration data that did not resemble reality. Below eight bits Research and some commercial tools go further, to 4-bit, 2-bit or even binary weights. Lower precision saves more memory, and some accelerators support it natively. But accuracy losses grow, training becomes more delicate, and support in general-purpose runtimes is limited. For most TinyML work in 2026, 8-bit integer quantization is the practical standard, with 16-bit activations used selectively where audio front ends or sensitive layers need more dynamic range. Some runtimes support a mixed mode of 8-bit weights with 16-bit activations for exactly this reason. Pruning: removing what is not needed Trained neural networks are usually over-parameterised: many weights are close to zero and contribute little. Pruning removes them. In unstructured pruning, individual weights are set to zero, often by repeatedly removing the smallest-magnitude weights and fine-tuning. Networks can often lose a large fraction of their weights this way with little accuracy loss. The difficulty is that unstructured sparsity rarely helps on a microcontroller. A sparse weight matrix with zeros scattered at random still has to be stored in some form, and the inference kernels in common runtimes are written for dense tensors; skipping individual zeros costs more in bookkeeping than it saves in arithmetic. Unless a runtime or accelerator explicitly supports a compressed sparse format, unstructured pruning reduces nothing but the number of nonzero values in a file that is the same size as before. Some accelerators, including Arm's Ethos-U with its Vela compiler, do apply weight compression that benefits from zeros, so the picture depends on the target. Structured pruning is more useful on general-purpose cores. Instead of individual weights, it removes whole filters, channels or layers, producing a smaller dense network that every runtime can execute efficiently. Removing channels from early layers also reduces activation memory, which is often the binding constraint. In practice, structured pruning blurs into architecture design: the outcome is simply a narrower network, and many practitioners skip pruning entirely and train a narrower network from the start. Distillation: learning from a larger teacher Knowledge distillation, introduced in its modern form by Geoffrey Hinton, Oriol Vinyals and Jeff Dean in 2015, trains a small "student" network to mimic the outputs of a larger "teacher" network, rather than learning only from the hard labels in the training data. The teacher's output probabilities carry information about which wrong answers are nearly right, which helps the student learn a better decision boundary than it would find alone. In TinyML, distillation is useful when a large model can be trained to high accuracy on a server and the deployed model must be a fraction of its size. Its benefits are real but usually incremental, a point or two of accuracy rather than a transformation. It is best thought of as a final polish on a well-chosen small architecture rather than a substitute for one. How the techniques compare The four approaches differ in what they save, what they cost and how well they are supported on real devices, as Table 2 sets out. Table 2. Model compression techniques for microcontroller deployment. Technique Main saving Typical accuracy cost Runtime support on Cortex-M Main pitfall Efficient architecture Compute, flash and SRAM Depends on design Universal Missing operators 8-bit quantization 4x flash and SRAM, faster integer maths Small, often under 1 point Standard Poor calibration data Unstructured pruning Nonzero weights Small Rarely exploited No real memory saving Structured pruning Compute and activations Small to moderate Universal (dense result) Needs retraining Distillation Indirect, enables smaller student Recovers accuracy Not applicable Extra training cost The pattern in the table is that the techniques producing smaller dense integer networks are the ones that pay off on general-purpose microcontrollers. Exotic sparsity and very low precision pay off mainly where specific hardware supports them. Searching for the network Choosing channel counts, kernel sizes and layer depths by hand under several simultaneous constraints is tedious, and neural architecture search has been adapted to automate it. Two research efforts defined much of the approach. MCUNet, from Ji Lin, Song Han and colleagues at MIT, presented at NeurIPS in 2020, jointly designed a network search procedure (TinyNAS) and an inference engine (TinyEngine), first narrowing the search space to fit a given microcontroller's memory and then searching within it. It demonstrated ImageNet-scale image classification on off-the-shelf microcontrollers, which had previously been considered out of reach. MicroNets, from Colby Banbury and colleagues at Arm and Harvard, presented at MLSys in 2021, used differentiable architecture search with explicit models of microcontroller latency and memory, and reported state-of-the-art results on keyword spotting, visual wake words and anomaly detection tasks under tight constraints. A practical insight from that work was that, on microcontrollers, model latency is close to linear in the number of operations, which makes latency easy to predict during search. Commercial platforms have built these ideas into their tooling. Edge Impulse, STMicroelectronics' ST Edge AI tools, NXP's eIQ and others offer automated searches over model and preprocessing choices with the target device's memory and latency as constraints. These tools are valuable, particularly for teams without deep machine learning experience. But they search within the space they are given. If the input representation is poor, the labels are noisy, or the evaluation does not reflect field conditions, the search will efficiently find a small network that solves the wrong problem. The cheapest compression is better input One more lever deserves a place beside the four techniques above, because it is often more powerful than all of them: the choice of what the network sees. A model given raw sensor samples must learn, from data, whatever transformations reveal the patterns that matter. A model given well-chosen features starts with much of that work done, and can be dramatically smaller as a result. For audio, the spectrogram front end discussed in Chapter 5 does exactly this: it discards phase information and fine temporal detail that are irrelevant to recognising a word, and compresses a second of audio from 16,000 samples to a few hundred or a couple of thousand values. For vibration, as Chapter 6 shows, spectra, envelope spectra and a few statistical descriptors encode decades of engineering knowledge about how machines fail. A classifier operating on forty well-chosen features can be a fully connected network of a few thousand parameters, or even a decision tree or a small gradient-boosted ensemble that needs no neural network runtime at all. Some of the most effective TinyML deployments use no deep learning whatsoever, and are no worse for it. The trade-off is that features are a fixed hypothesis about what matters. If the hypothesis is wrong, information the model needed has been thrown away before it ever arrives. End-to-end learning on raw signals avoids this risk at the cost of larger models and much more training data. The practical middle course, which most successful systems take, is to use a cheap, well-understood transformation such as a spectrogram that preserves most of the useful information, and to let a small network learn the rest. Feature computation also costs time and energy on the device, so it belongs in the same budget as the network: a model that is tiny because its preprocessing is enormous has not saved anything. The budget is the specification The lesson of this chapter is that model compression for microcontrollers is not a post-processing step applied to whatever network the data scientists produced. It is a design activity that begins with the device's flash, SRAM, latency and energy budget and works backward to a network that respects them. A useful discipline is to write the budget down before training anything: this much flash for weights, this much SRAM for activations after the application's own buffers are subtracted, this many milliseconds per inference, this many microjoules. Every candidate model can then be measured against those numbers, on the actual device, as early as possible. Those measurements require running the model on the device. The next chapter turns to how that is done in C++. Chapter 4: C++ Without the Heap A trained, quantized model is a file of numbers and a description of the operations that connect them. To do anything, it has to be executed by code running on the microcontroller, and that code has to coexist with everything else the firmware does: reading sensors, managing power, talking to the radio, handling faults. This chapter explains how that works, with LiteRT for Microcontrollers as the main example. It is the most widely used open-source runtime for the purpose, and its design choices illustrate the constraints that every microcontroller inference engine must respect. The project was known for most of its life as TensorFlow Lite for Microcontrollers, often abbreviated TFLM; Google renamed TensorFlow Lite as LiteRT in 2024, and much documentation and code still uses the old names. Why C++, and which C++ Firmware for microcontrollers has traditionally been written in C, and much of it still is. LiteRT for Microcontrollers is written in C++, as are several other inference engines, because C++ offers abstractions, such as classes, templates and namespaces, that make a large library easier to organise without imposing runtime cost when used carefully. The phrase "when used carefully" is doing a lot of work. Embedded C++ is a deliberately restricted dialect. The restrictions follow from the hardware. There is little SRAM and no virtual memory, so a memory allocation failure cannot be papered over by swapping to disk; it simply crashes the device, possibly months after deployment. There is often no operating system to clean up after a failure. Timing must be predictable, because a keyword spotter that occasionally pauses for an unpredictable interval will drop audio. And code size matters, because flash is shared between the application, the runtime and the model's weights. So embedded C++ projects commonly compile without exceptions and without run-time type information, using compiler flags such as -fno-exceptions and -fno-rtti, because both add code size and unpredictable behaviour. They avoid the standard library's containers and strings, which allocate memory dynamically. They avoid virtual functions in performance-critical paths, or use them sparingly. Templates are used, but with an eye on code bloat. And they avoid the general-purpose heap, the pool of memory managed by new, delete, malloc and free, after initialisation, if not entirely. The last restriction is the most important. On a desktop, dynamic memory allocation is routine. On a microcontroller, a heap that is allocated and freed repeatedly fragments over time, so that a request can fail even when the total free memory is ample, because no single contiguous block is large enough. A device that runs for years cannot tolerate this. The robust approach is to decide at compile time or at start-up exactly how much memory every component needs, allocate it once, and never free it. The tensor arena LiteRT for Microcontrollers embodies this approach in a single idea: the tensor arena. The application declares one contiguous block of memory, typically as a statically allocated byte array, for example alignas(16) static uint8_t tensor_arena[kArenaSize];. The alignment matters because SIMD instructions and some accelerators require data aligned to particular boundaries. Everything the interpreter needs at run time, including input, output and intermediate activation tensors, the interpreter's own bookkeeping structures, and any scratch buffers required by optimised kernels, is carved out of this arena. There is no other allocation. When the interpreter is initialised, it examines the model to determine the lifetime of each intermediate tensor: the point in the sequence of operations at which it is first written and the point at which it is last read. Two tensors whose lifetimes do not overlap can share the same memory. A memory planner, by default a greedy algorithm, assigns each tensor an offset within the arena so that the total footprint is as small as possible. For a simple chain of layers, the peak is roughly the size of the largest adjacent pair of input and output tensors; for networks with branches or skip connections, tensors must be kept alive longer and the peak rises. This is one reason that residual connections, which are ubiquitous in large networks, are used more cautiously in TinyML. Choosing the arena size is an empirical step. Developers typically start with a generous guess, run the model once, ask the interpreter how much of the arena it actually used through a method such as arena_used_bytes(), and then set the size to that figure plus a margin. If the arena is too small, allocation fails at initialisation with an error message rather than at some unpredictable point later, which is exactly the property a long-lived device needs. The anatomy of an inference program An inference program built on LiteRT for Microcontrollers has a small number of parts, and understanding their roles is more useful than memorising any particular listing. The model itself is compiled into the firmware as a constant byte array. The converter produces a flatbuffer file, a compact serialised format that can be read in place without parsing into separate data structures. A tool such as xxd -i turns the file into a C source file containing an array, which the linker places in flash. At run time, a call to tflite::GetModel() returns a lightweight view onto that array; nothing is copied. Next comes the operator resolver, which maps each operation type in the model, such as a convolution or a softmax, to the code that implements it. LiteRT for Microcontrollers offers a resolver that includes every supported operator, but this links all their code into the firmware and wastes flash. The usual practice is to declare a resolver with a fixed capacity, such as tflite::MicroMutableOpResolver<6> resolver;, and register only the operators the model actually uses, with calls such as resolver.AddConv2D();, resolver.AddDepthwiseConv2D(); and resolver.AddFullyConnected();. The linker then discards everything else. If a model is retrained with a different architecture and uses a new operator, the resolver must be updated, and forgetting to do so produces a clear error at initialisation. Then the interpreter is constructed from the model, the resolver and the arena, as an object of type tflite::MicroInterpreter. A call to AllocateTensors() runs the memory planner and prepares each operator. This happens once, at start-up. After that, the steady-state loop is short. The application obtains a pointer to the input tensor with interpreter.input(0), fills its data with quantized features, calls interpreter.Invoke(), which runs every operator in sequence, and reads the result from interpreter.output(0). Filling the input involves applying the input tensor's scale and zero point to convert real-valued features into 8-bit integers, unless the preprocessing already produces integers in the right representation. Reading the output similarly involves converting quantized scores back to real values, or simply comparing the integer scores directly, since the conversion is monotonic. Everything interesting about the application happens outside this loop: where the input data comes from, how often the loop runs, and what is done with the output. The runtime is deliberately boring. Optimised kernels and where the time goes The runtime ships with reference implementations of each operator, written in portable C++ that runs on any processor. They are correct and easy to read, and slow. For production, the build system substitutes optimised kernels for the target. On Arm cores, these come from CMSIS-NN, which uses the DSP extension or Helium. Other vendors supply their own optimised libraries for their cores and digital signal processors, and NPU vendors supply delegates that hand supported operators to the accelerator. The speed difference between reference and optimised kernels is often large, commonly several-fold, so one of the first checks when a model is unexpectedly slow is whether the optimised kernels were actually linked. The runtime includes a profiler that reports the time spent in each operator. On almost every model, the result shows that a small number of layers dominate, usually the convolutions with the most multiply-accumulates or the early layers with the largest activations. Those are the layers where architectural changes, such as fewer channels or an earlier stride, pay off. The same profiling discipline applies to the code around the model. In audio applications in particular, the feature extraction stage, computing spectrograms or filterbank energies from raw samples, can take as long as the network itself if implemented naively. Using the fixed-point fast Fourier transform and filterbank routines from the CMSIS-DSP library, or the audio front end supplied with the runtime's examples, is usually necessary to keep the total within budget. Testing on the host and on the target A practical strength of this style of runtime is that the same C++ code compiles for a desktop computer as well as for the microcontroller. Developers can run the full inference program, including feature extraction, on a laptop, feed it recorded test data, and compare its outputs bit for bit with the outputs of the quantized model running in Python. This catches a large class of errors, such as wrong input scaling, transposed tensor layouts and mismatched preprocessing, before any hardware is involved. The most common source of failure in TinyML deployments is precisely such a mismatch between training-time preprocessing, usually written in Python with floating-point libraries, and deployment-time preprocessing, written in fixed-point C++. A spectrogram computed with slightly different window functions, frame lengths, scaling or logarithm bases looks plausible but presents the network with inputs it never saw during training. The model's accuracy collapses with no error message. The defence is to treat the feature extraction code as part of the model, to use the same implementation for training and deployment wherever possible, and to test the two against each other on identical inputs. On the target, testing adds measurement of time, memory and energy. Cycle counters built into Cortex-M cores give precise timings. The interpreter reports arena usage, and the linker map file reports flash usage. Energy has to be measured externally, with a source-measure unit or a dedicated power analyser in series with the device's supply, and should be measured over complete duty cycles rather than inference alone. When things go wrong on the device Debugging a model on a microcontroller is a different experience from debugging one in Python. There is no interactive console, often no screen, and when something goes badly wrong the processor does not print a stack trace; it enters a fault handler and stops, or the watchdog timer resets it and it starts again as if nothing had happened. The failures that recur in TinyML projects fall into a few families. The first is memory corruption. If the stack, which grows downward through SRAM as functions call one another, collides with the tensor arena or another buffer, it silently overwrites data. The symptom may be a model that produces nonsense intermittently, or a crash far from the cause. Defences include sizing the stack generously, placing a known pattern at its limit and checking periodically whether the pattern has been overwritten, and using the memory protection unit present on many Cortex-M cores to turn an overflow into an immediate, diagnosable fault rather than a silent corruption. The second is alignment and type confusion. Optimised kernels assume that tensors are aligned to particular boundaries, and that their data types match what the model declares. Reading an 8-bit model's input as if it were 32-bit floats, or filling a signed 8-bit tensor with unsigned values, produces outputs that are wrong but look like numbers, so no error is raised. The most effective defence is the host-side comparison described earlier, repeated whenever the model or the preprocessing changes. The third is timing. An inference that takes slightly longer than expected can cause a DMA buffer to overflow, dropping samples. The model still produces outputs, but they are computed on audio or vibration data with gaps in it, and accuracy degrades in ways that are hard to attribute. Toggling a spare output pin at the start and end of inference and observing it with an oscilloscope or logic analyser is an old embedded technique that remains one of the quickest ways to see exactly where time goes, and how close the system is to its deadline. The fourth is power state interaction. Peripherals that are switched off to save power must be switched on again, and clocks that are slowed must be restored, before the code that depends on them runs. A model that works perfectly on a development board connected to a debugger, which often keeps the chip from entering its deepest sleep, may fail on the same board running from a battery. Testing the complete power cycle without a debugger attached is an essential step that tutorials rarely mention. Experienced teams build diagnostic hooks into the firmware from the start: a way to dump the input features and output scores over a serial link, a counter of watchdog resets reported in telemetry, a record of the last fault address stored in memory that survives a reset. These cost a few hundred bytes and save weeks. Fitting into the rest of the firmware The inference engine is a guest in a larger program. In a bare-metal design, it runs in the main loop between sleeps, triggered when a DMA interrupt signals that a buffer of sensor data is ready. In a design using an RTOS such as FreeRTOS or Zephyr, inference usually runs in a dedicated task with its own stack, at a priority below the time-critical tasks that service sensors and the radio, so that a long inference cannot cause samples to be dropped. Zephyr in particular has integrated support for LiteRT for Microcontrollers, and many silicon vendors provide example projects that put the pieces together. Two pieces of coordination need care. First, memory: the tensor arena, the sensor buffers, the radio stack and the RTOS all share the same SRAM, and the arena is usually the largest single block. Laying out memory deliberately, sometimes placing the arena in a particular bank of SRAM that can be powered down between inferences, is part of the design. Second, timing: if the device must process audio continuously, the time to compute features and run the model on one window must be reliably shorter than the time to acquire the next, with a margin for everything else the processor does. Alternatives to an interpreter LiteRT for Microcontrollers interprets a model at run time: the flatbuffer is read and each operator is dispatched in sequence. This costs a little flash for the interpreter and a little time for dispatch, which is usually negligible compared with the arithmetic. Several alternatives instead compile the model ahead of time into plain C or C++ code, with the memory plan fixed at compile time. Examples include Edge Impulse's EON compiler, STMicroelectronics' tools for its STM32 parts, the TinyEngine work from MIT's MCUNet project, and Apache TVM's microTVM, which applies a general deep learning compiler to microcontrollers. Ahead-of-time compilation can reduce memory overhead and enable optimisations across layers, at the cost of flexibility: changing the model means recompiling and reflashing the firmware, rather than replacing a model file. PyTorch's ExecuTorch project has also added microcontroller targets, including Arm's Ethos-U, giving teams who train in PyTorch a more direct path to embedded deployment. Table 3 summarises how the main approaches compare on the dimensions that matter in practice. Table 3. Approaches to running models on microcontrollers. Approach Examples Model update without rebuild Memory overhead Portability Interpreter runtime LiteRT for Microcontrollers Possible, if operators unchanged Small interpreter plus arena High, across vendors Ahead-of-time code generation Edge Impulse EON, microTVM, TinyEngine No, model becomes code Lowest Moderate Vendor toolchain ST Edge AI, NXP eIQ Varies Low, tuned to vendor parts Tied to vendor NPU compiler plus host runtime Arm Vela for Ethos-U Possible Depends on NPU memory use Tied to accelerator The choice among these depends on the product. A device that will receive new models over the air for years benefits from an interpreter whose model can be swapped as data. A cost-sensitive product with a fixed model may prefer the smallest possible compiled footprint. A product built on a chip with an NPU will use the vendor's or Arm's toolchain regardless. A short, boring program The striking thing about a well-built TinyML firmware, when examined closely, is how little of it concerns machine learning. The inference call is one line. Around it are sensor drivers, DMA configuration, power-mode transitions, feature extraction, a ring buffer, a decision rule, a radio driver and an update mechanism. This proportion is correct. It reflects the argument of the first chapter: the model is one component in an energy-constrained system, and its value depends on everything around it. The next two chapters examine that "everything around it" in two specific applications, starting with the one most people meet first: a device that listens for a word. Chapter 5: Listening for a Word The wake word is the most familiar TinyML application in the world, though few of the people who use it every day know it is there. When someone says the name of a voice assistant to a smart speaker, a phone or a car, a small neural network running on a low-power processor has been listening to every fraction of a second of sound in the room, waiting for that one word. Everything else the device does, from waking its main processor to opening a network connection and streaming speech to a large recogniser, depends on that small network getting the decision right. If it fires when no one said the word, the device intrudes, wastes energy and erodes trust. If it fails to fire when someone did, the device seems deaf. Keyword spotting, as the general problem is called, is a near-perfect illustration of the argument of this book. The model is small and its accuracy matters, but the success of a keyword spotter depends just as much on the audio front end, the streaming logic, the decision rule and the energy budget of always-on listening. The problem, precisely A keyword spotter takes a continuous stream of audio and must decide, continuously, whether one of a small set of target words has just been spoken. Three features make this harder than it sounds. First, the stream never stops. Unlike an image classifier presented with one photograph at a time, a keyword spotter has no natural boundaries to work with. It does not know when a word begins or ends, and must evaluate overlapping windows of audio many times a second. Second, almost everything it hears is not the keyword. Over a day, a device in a home hears hours of conversation, television, music, cooking, traffic and silence, and perhaps a few dozen genuine utterances of the wake word. The classes are wildly imbalanced, and the cost of errors is asymmetric in a way that depends on the product. Third, the budget is severe. Because the spotter runs all the time, its energy consumption is multiplied by every second of the device's life. On a battery-powered device such as earbuds or a remote control, the always-on stage must typically run in a small fraction of a milliwatt to a few milliwatts. Even on mains-powered devices, the keyword spotter is often placed on a separate low-power core so the main processor can sleep, both for energy regulation reasons and for heat. From sound to features The microphones used in small devices are usually MEMS microphones with a digital output, sampled at 16 kilohertz, which captures the frequency range that matters for speech intelligibility. One second of audio at this rate and 16 bits is 32 kilobytes, too much to feed directly to a tiny network and unnecessarily detailed, since much of the waveform's fine structure is irrelevant to which word was said. Almost all keyword spotters therefore begin with a front end that converts audio into a time-frequency representation, closely modelled on the processing that has served speech recognition for decades. The audio is divided into short overlapping frames, commonly around 30 to 40 milliseconds long with a step of 20 milliseconds between their start times, so that one second of audio yields about fifty frames. Each frame is multiplied by a window function to reduce edge artefacts and passed through a fast Fourier transform to obtain its spectrum. The spectrum is then summarised by a bank of overlapping triangular filters spaced according to the mel scale, which approximates human pitch perception by giving finer resolution at low frequencies than at high. The energy in each filter is compressed with a logarithm, because perceived loudness is roughly logarithmic and because the logarithm makes the features less sensitive to overall volume. The result, a log-mel spectrogram, is a small two-dimensional array: for example fifty time steps by forty frequency channels. A further step, the discrete cosine transform, can convert each frame's filterbank energies into mel-frequency cepstral coefficients, MFCCs, keeping perhaps ten or so of them. MFCCs were the standard in classical speech recognition because they decorrelate the features; neural networks need this less, and many modern keyword spotters use the filterbank energies directly. The Arm "Hello Edge" study used ten MFCCs per frame over a one-second window, giving an input of 49 frames by 10 coefficients. The micro speech example that ships with LiteRT for Microcontrollers uses a front end that produces 40 filterbank channels per frame, with additional noise-reduction and automatic gain control stages designed to make the features robust to background noise and varying volume. The front end is not a detail. As Chapter 4 noted, its implementation on the device must match the one used in training precisely, down to the window function, the frame alignment, the scaling and the logarithm. It must also run in fixed point within the time budget of each frame, while the microphone's DMA continues to fill the next buffer. A common engineering pattern computes features incrementally: each new 20-millisecond step of audio produces one new frame of features, which is appended to a rolling buffer of the most recent second, so that the cost of the front end is spread evenly over time. The network Given a spectrogram of about one second, the network's job is classification: which of the target keywords, if any, does this window contain? The output classes usually include each keyword, a "silence" class and an "unknown" class for speech that is not a keyword. The public benchmark that shaped this field is Google's Speech Commands dataset, released by Pete Warden in 2017 and described in a 2018 paper. Its second version contains around 105,000 one-second utterances of 35 short words, spoken by thousands of different people through whatever microphones they had to hand, under an open licence. The standard task derived from it asks a model to recognise ten words, such as "yes", "no", "up", "down", "left" and "right", plus silence and unknown. Because the clips were crowdsourced, they have the varied quality of real-world audio, which made the dataset far more useful than earlier studio-recorded collections. The MLPerf Tiny benchmark adopted this task, with a DS-CNN as its reference model, as one of its four standard workloads. Architectures for this input are now well understood. The DS-CNN from "Hello Edge" treats the spectrogram as a small image and applies a standard convolution followed by several depthwise separable convolution blocks, then average pooling and a fully connected classifier. Variants with tens of thousands of parameters reach accuracy in the mid-nineties on the twelve-class Speech Commands task, and fit comfortably in the memory of a Cortex-M4. Other effective families include temporal convolutional networks, which treat frequency channels as input channels and convolve only along the time axis, reducing activation size, and small recurrent networks such as gated recurrent units. Some production systems use streaming architectures that process one new frame at a time and carry state forward, avoiding recomputation of the overlapping portion of each window. Streaming and the decision rule A network that classifies one-second windows does not, by itself, make a keyword spotter. The device must decide how often to run it, how to combine successive outputs and when to declare a detection. The usual approach runs the network on a sliding window at regular intervals, for example every few tens of milliseconds to a few hundred, each time on the most recent second of features. A spoken keyword therefore appears in several successive windows, shifted in position. Individual outputs are noisy: a single window may score a keyword highly because of a coincidental sound. So the raw scores are smoothed, typically by averaging each class's score over the last several inferences, and a detection is declared only when the smoothed score for a keyword exceeds a threshold. After a detection, a suppression period prevents the same utterance from triggering again as it slides through subsequent windows. The micro speech example implements exactly this logic in a small component that keeps a history of recent results, averages them, applies a detection threshold and enforces a minimum interval between detections. The logic is simple, but its parameters, namely the averaging window, the threshold and the suppression time, have a large effect on behaviour and are tuned on realistic recordings, not on the clean one-second clips used to train the network. Measuring the right thing Accuracy on a held-out set of one-second clips is the number most often quoted for keyword-spotting models, and it is the least informative number for a product. What matters in deployment are two rates measured on continuous audio. The false rejection rate is the fraction of genuine keyword utterances that the system misses. It is measured by playing many recorded utterances, in realistic conditions, to the complete system. The false acceptance rate is the frequency with which the system fires when the keyword was not spoken, and is conventionally expressed as false accepts per hour of non-keyword audio, measured by running the system over long recordings of speech, television, music and household noise. The two trade off against each other through the detection threshold: lowering it catches more genuine utterances and also more impostors. Plotting one against the other as the threshold varies gives a detection error trade-off curve, and choosing an operating point on that curve is a product decision. The right operating point depends on the cascade around the spotter. If a keyword detection only wakes a larger, more accurate model that verifies the keyword before anything visible happens, the spotter can be tuned for very low false rejection and tolerate frequent false accepts, because they cost only a little energy and are caught downstream. If the detection directly triggers an action, such as starting a recording, lighting a display or unmuting a microphone that streams to the cloud, false accepts are costly in privacy and annoyance and must be rare, perhaps less than one per several hours or days of audio. The model can be identical in both cases; the system is not. Two further measures matter for users. Latency, the delay between the end of the keyword and the detection, should be short enough that the device seems to respond immediately, which constrains how much smoothing can be applied. And robustness across speakers, accents, ages, distances and acoustic conditions determines whether the product works for everyone or only for people who resemble the training data. Speech Commands, for all its value, is dominated by adult speakers of English recorded close to a microphone, and a device for a broader population needs broader data. Data is the hard part The custom wake word, a brand name or a product's own trigger phrase, is the normal case in industry, and no public dataset contains it. Collecting thousands of recordings of a new phrase from a diverse range of speakers is slow and expensive, which is why companies that build voice products guard their wake-word data closely. Several techniques reduce the burden. Data augmentation multiplies each recording into many variants by mixing in background noise at varied levels, shifting it in time, changing its speed and pitch slightly, and convolving it with recorded room impulse responses that simulate the reverberation of different spaces. Augmentation with realistic noise is particularly important, because a model trained on clean speech learns features that disappear in a noisy kitchen. Synthetic speech from modern text-to-speech systems, with many voices and speaking styles, has become a practical source of positive examples; open projects such as openWakeWord train models largely on synthetic utterances generated this way. Negative examples, by contrast, are abundant: any large corpus of speech and everyday audio that does not contain the phrase will do, and including words that sound similar to the keyword is essential for teaching the model what not to fire on. Even with these techniques, the final tuning of a wake-word system depends on recordings from the actual device, in the actual acoustic environments where it will be used, through the actual microphone and enclosure. Microphone placement, enclosure resonances and the noise of the device itself all change the signal. Teams that tune thresholds on laptop recordings and ship to a different enclosure are routinely surprised. Beyond a single word The wake word is the simplest case of a broader family. Many devices recognise a small vocabulary of commands entirely on the device, such as "lights on", "volume up" or "next track", without ever contacting a server. This is attractive for appliances, toys, industrial equipment and vehicles, where a network connection may be absent and where privacy or latency matters. Technically, a command recogniser is a keyword spotter with more classes, but the growth in classes changes the problem. Each added command is another way to confuse one word for another, the unknown class must absorb a wider variety of near-misses, and the data collection burden multiplies. Practical command vocabularies on microcontrollers tend to be small, typically a few dozen phrases at most, and are often arranged hierarchically: a wake word first, then a short window in which commands are accepted, which drastically reduces the opportunity for false triggers. A second extension is user-defined keywords, where the device learns a new trigger phrase from a few examples spoken by its owner. This is a few-shot learning problem. The usual approach uses a network pretrained on large amounts of speech to produce embeddings, compact vectors summarising a stretch of audio, and enrolls a new keyword by storing the embeddings of the user's examples. Detection then becomes a matter of comparing incoming embeddings with the stored ones. The same idea underlies speaker verification, which some devices use to respond only to an enrolled voice. Both techniques move some learning onto the device, which raises the same questions about adaptation and reliability that Chapter 7 discusses for industrial baselines. A third consideration is privacy, which is often the reason a keyword spotter exists on the device in the first place. A device that streams all audio to a server can claim to discard it, but its owner must trust the claim. A device whose microphone feeds only a local keyword spotter, and which connects to anything only after the keyword is detected, offers a structural guarantee that is easier to explain and to audit. That guarantee is only as good as the false acceptance rate: every false trigger sends a few seconds of unintended audio off the device. Here the threshold choice discussed above becomes a privacy decision as well as a usability one. The energy picture It is worth returning to energy, because always-on listening is the most demanding duty-cycling problem in TinyML. The microphone cannot sleep between inferences, because a keyword could begin at any moment. The front end must process every frame. The network runs many times a second. Designers attack this from several directions. The first stage of the cascade is often a voice activity detector that runs on the microphone, the audio codec or a tiny always-on block, keeping the keyword spotter itself asleep during silence, which in many environments is most of the time. When the spotter runs, it uses the smallest model that meets the accuracy target, on the most efficient core available, with features computed in fixed point and DMA moving the audio. The inference interval is set as long as the latency and accuracy targets allow, since halving the frequency of inference halves its energy. On some chips, the keyword spotter runs on a dedicated low-power core or a small NPU while the main application processor stays in deep sleep, and only a detection wakes it. Specialised silicon has pushed the limits further, with analogue and mixed-signal neural processors marketed for always-on keyword spotting at power levels well under a milliwatt. These remain niche compared with general-purpose microcontrollers, but they show how far the cascade logic can be taken: the cheapest possible detector, running all the time, guarding a series of progressively more capable and power-hungry stages. What the wake word teaches The wake word illustrates every theme of this book in a compact form. The network is small, well understood and close to a solved problem on public benchmarks. Yet building a wake-word system that works well in homes, cars and pockets is hard, and the difficulty lies almost entirely outside the network: in the match between training and deployment front ends, in the streaming and smoothing logic, in the choice of operating point given the cascade, in data that reflects real speakers and real rooms, and in the energy design of a system that can never stop listening. The next chapter takes the same reasoning into a very different setting, where the signal is vibration rather than speech, the events of interest are rarer still, and nobody can say in advance what a failure will sound like. Chapter 6: The Machine That Hums Industrial plants are full of rotating machines: motors, pumps, fans, compressors, gearboxes, conveyors and spindles. Most of them run for years without trouble, and when they fail, they rarely do so without warning. A bearing that is beginning to spall, a shaft that has drifted out of alignment, an impeller that has lost a fragment or a belt that has started to slip each changes the way the machine vibrates and sounds, often weeks or months before anything breaks. Experienced maintenance technicians have always known this; some could diagnose a pump by resting a screwdriver against its housing and listening through the handle. Condition monitoring is the discipline of making that judgement systematically, and predictive maintenance is the practice of scheduling repairs according to the machine's actual condition rather than the calendar. The economic case is simple. Running a machine until it breaks risks unplanned downtime, secondary damage and safety incidents. Replacing parts on a fixed schedule wastes components that had years of life left and introduces new failures through unnecessary intervention. Repairing a machine shortly before it would have failed, at a convenient time, captures most of the value of both approaches. Reliability engineers often describe this with the P-F interval: the time between the point at which a developing failure becomes detectable (P, for potential failure) and the point at which the machine actually fails (F). The longer that interval and the earlier a monitoring system can detect the onset of a fault, the more time there is to plan. Why TinyML fits Vibration monitoring is not new. For decades, large critical machines such as turbines and main compressors have been fitted with permanently wired sensors feeding dedicated monitoring systems, and analysts have walked routes through plants with handheld data collectors, measuring smaller machines monthly or quarterly. The gap has always been the long tail of the "balance of plant": the thousands of modest pumps, fans and motors that are individually not worth a wired monitoring system but whose failures, collectively, cause much of the unplanned downtime. A monthly walk-around catches slow faults but misses ones that develop in days, and it is labour-intensive. Wireless battery-powered vibration sensors were meant to fill this gap, and they ran directly into the energy problem described in Chapter 1. A meaningful vibration measurement samples at several kilohertz or more for a second or more, producing tens of kilobytes of data. Transmitting that from every sensor several times a day, over a low-power network shared by hundreds of devices, is impractical on battery power. Early wireless sensors compromised by sending only overall levels, a few numbers per measurement, which lose much of the diagnostic information, or by sending full waveforms rarely, which loses timeliness. On-device machine learning resolves the compromise. The sensor captures the full waveform, analyses it locally, and transmits only a verdict, a score, or a compact set of features, sending the full waveform only when something interesting has been found and an analyst may want to look. This is the energy logic of TinyML applied to an industrial problem, and it explains the rapid growth of on-device analytics in industrial wireless sensors. What failing machines sound like Before choosing a model, it helps to understand what the signal contains, because the physics of rotating machines gives strong prior knowledge that a pure data-driven approach would waste. A rotating machine vibrates at its running speed and at frequencies related to it. Mass imbalance, the most common fault, produces a strong component at exactly once per revolution, called 1X, mainly in the radial direction. Misalignment between coupled shafts tends to produce strong 2X components and significant axial vibration. Mechanical looseness, such as a cracked mount or loose bolts, generates many harmonics of running speed and sometimes components at half-multiples. Gear problems appear at the gear mesh frequency, the number of teeth multiplied by shaft speed, and as sidebands around it spaced at the speed of the faulty gear. Problems with fans and pumps appear at the blade or vane pass frequency, the number of blades multiplied by running speed. Rolling-element bearings, the most common source of failure in small machines, have their own characteristic frequencies. When a rolling element passes over a localised defect, such as a pit or crack on the outer race, it produces a brief impact. Those impacts repeat at a rate set by the bearing's geometry and the shaft speed, and they have names: the ball pass frequency of the outer race, the ball pass frequency of the inner race, the ball spin frequency and the fundamental train frequency of the cage. Each can be calculated from the number of rolling elements, their diameter, the pitch diameter of the bearing and the contact angle. In the early stages of a bearing fault, the impacts are tiny and their energy is negligible in the overall vibration spectrum. But each impact excites high-frequency resonances in the bearing and housing, so the defect shows up as a high-frequency carrier whose amplitude is modulated at the defect frequency. The classical technique for finding it is envelope analysis: band-pass filter the signal around a resonance, extract its envelope, and examine the spectrum of the envelope for peaks at the bearing defect frequencies. This body of knowledge matters for TinyML in two ways. First, it tells the designer what sampling rate and measurement duration are needed. Detecting early bearing faults through their high-frequency resonances requires sampling well into the kilohertz range, while diagnosing imbalance on a slow machine requires a measurement long enough to resolve its running speed precisely. Second, it suggests features. A model given the spectrum, the envelope spectrum, and a few statistical descriptors of the waveform, such as its root-mean-square level, peak value, crest factor (the ratio of peak to root-mean-square) and kurtosis (a measure of how impulsive the signal is), starts from a representation in which most faults are visible. A model given raw samples must learn all of this from data, which needs much more data and a larger network. Accelerometer or microphone A design choice that comes before any model is which physical quantity to sense. Vibration and sound are two views of the same mechanical events, and each has advantages. Accelerometers mounted on the machine measure vibration at the source, with little interference from neighbouring equipment, and their signals map directly onto the diagnostic tradition of vibration analysis. Industrial piezoelectric accelerometers have wide frequency ranges and excellent noise performance but draw more power and cost more. MEMS accelerometers, the kind found in phones, are cheap and frugal; for years their limited bandwidth and higher noise restricted them to low-frequency faults, but newer parts designed for condition monitoring extend usefully into the kilohertz range. The choice of accelerometer sets an upper limit on which faults a system can see early, regardless of how good the model is. Microphones capture airborne sound and need no contact with the machine, which makes them easier to install and lets one sensor hear several machines. That same property is their weakness: in a noisy plant, the sound of a failing pump mixes with the sound of everything else, and separating them is hard. The public machine-sound datasets discussed below are deliberately mixed with factory noise to reflect this. Acoustic monitoring is valuable for faults that are more audible than tangible, such as leaks, valve chatter and some electrical faults, and for machines where mounting a sensor is impractical. For most rotating equipment, contact vibration remains the primary signal, and TinyML systems for predictive maintenance are mostly built around accelerometers. Several of the techniques in this chapter apply equally to both. The scarcity of failure The obvious approach to machine learning for fault detection would be supervised classification: collect vibration data from healthy machines and from machines with each type of fault, label it, and train a classifier. In practice this rarely works, for a reason that shapes the whole field. Failures are rare, varied and specific. A given pump in a given plant may fail once in several years, and when it does, the failure will be one of a dozen possible kinds, at one of many possible stages of development, on a machine whose normal vibration depends on its mounting, its load, its speed and the machines next to it. No plant has enough examples of each fault on each machine type to train a reliable classifier, and data collected from test rigs with seeded faults, while valuable for research, often does not transfer to machines in the field. Moreover, the most valuable faults to detect are often the ones nobody anticipated. The data that every plant does have in abundance is data from healthy machines. Most of the time, most machines are fine. This turns the problem on its head. Instead of learning what each kind of failure looks like, the model learns what normal looks like for this machine, and flags anything sufficiently different. This is anomaly detection, and it is the dominant framing for TinyML in predictive maintenance. The research community adopted the same framing. In 2019, two public datasets of machine sounds were released for this purpose: ToyADMOS from NTT, recorded from miniature machines such as a toy car and a toy conveyor with deliberately induced faults, and MIMII from Hitachi, recorded from industrial fans, pumps, valves and slide rails under normal and abnormal conditions with factory noise added. Since 2020, the DCASE challenge, an annual competition in the detection and classification of acoustic scenes and events, has included a task on unsupervised anomalous sound detection for machine condition monitoring, in which systems are trained only on normal sounds and evaluated on their ability to rank anomalous sounds above normal ones. Later editions added harder conditions, including changes in operating conditions between training and test, and settings in which only a few examples of a new machine type are available. The MLPerf Tiny benchmark adopted a ToyADMOS task, with a small fully connected autoencoder as its reference model, as its anomaly detection workload. Anomaly detection on a microcontroller An anomaly detector, in this setting, is any method that assigns a score to a new measurement such that measurements resembling the training data of a healthy machine score low and unusual measurements score high. A threshold on the score then turns it into an alarm. Several families of methods are practical on microcontrollers, and they differ in what they can learn, how much they cost and whether they can be trained on the device itself, as Table 4 sets out. Table 4. Anomaly detection approaches for vibration and sound on microcontrollers. Approach What it models On-device cost Training on device Typical use Feature thresholds Fixed limits on RMS, kurtosis, band energies Negligible Trivial (set limits) Baseline and gross faults Statistical distance Mean and covariance of feature vectors Very low Feasible Per-machine baselines Clustering Centres of normal operating states Low Feasible Machines with several regimes Autoencoder Reconstruction of normal inputs Moderate Rarely, usually trained off device Complex spectra and sounds Classifier embedding Distance in a learned feature space Moderate Embedding fixed, statistics on device Fleets of similar machines The simplest approach is statistical. Compute a feature vector from each measurement, such as band energies of the spectrum together with statistical descriptors, and estimate the mean and covariance of those vectors over a learning period during which the machine is known to be healthy. Each new measurement is scored by its Mahalanobis distance from the mean, a distance that accounts for the natural variance and correlation of the features. This requires storing only a mean vector and a covariance matrix, can be computed and updated on the device itself, and is easy to explain: the score says how many standard deviations, in a multivariate sense, the measurement is from normal. Its limitation is that it assumes normal behaviour forms a single roughly elliptical cloud, which fails for machines that operate in several distinct states. Clustering methods relax this assumption. A k-means model learned on normal feature vectors represents each operating state, such as idle, half load and full load, by its own centre, and a new measurement is scored by its distance to the nearest centre. Like statistical methods, clustering can be learned on the device, which matters for the reasons discussed in the next chapter. Autoencoders are the main neural approach. An autoencoder is a network trained to reproduce its input at its output after squeezing it through a narrow intermediate layer, the bottleneck. Trained only on normal data, it learns to reconstruct normal patterns well, and it reconstructs unfamiliar patterns poorly, because the bottleneck forces it to capture only the regularities of what it has seen. The reconstruction error, the difference between input and output, serves as the anomaly score. The MLPerf Tiny reference model applies a small fully connected autoencoder to stacked frames of log-mel spectrogram features from machine sound. Convolutional autoencoders on spectrograms, and variational autoencoders, are common in research. Autoencoders can learn subtle structure that hand-built features miss, but they are usually trained off the device on a computer, then quantized and deployed, which means the notion of "normal" is fixed at training time unless the device can be updated. A further approach, prominent in recent DCASE results, trains a classifier on an auxiliary task, such as recognising which machine or which operating condition a recording came from, and uses the classifier's internal representation as an embedding. Anomalies are measured as distances in that embedding space. The insight is that a network forced to distinguish between many normal machines learns features that are sensitive to the kinds of differences that also reveal faults. On a microcontroller, the embedding network is fixed and the statistics of normal embeddings for each individual machine can be learned locally, combining the representational power of a trained network with on-device adaptation. A pipeline for one sensor Putting these pieces together, a practical on-device pipeline for a vibration sensor on a motor-driven pump might look like this. Every few minutes, the sensor wakes, checks a simple indicator such as overall vibration level to confirm the machine is running, and skips the measurement if it is not, since a stopped machine says nothing about its health and would distort a learned baseline. If the machine is running, the sensor samples a triaxial accelerometer for a second or two at a rate high enough to capture bearing resonances. It computes a spectrum with a fixed-point fast Fourier transform, summarises it into band energies, computes an envelope spectrum around a resonance band, and adds statistical descriptors. During an initial learning period, which might last days to a few weeks so as to cover the machine's normal range of operation, it accumulates statistics of these features. Afterwards, it scores each new measurement against the learned baseline, possibly also running a small autoencoder or classifier on the spectrum, and smooths the scores over several measurements so that a single odd reading does not trigger an alarm. It transmits a heartbeat with a few summary numbers at intervals, an alert when the smoothed score crosses a threshold, and on request, or after an alert, a full spectrum or waveform for an analyst. Every element of this pipeline is shaped by the energy budget. The machine-running check avoids wasting measurements. The feature computation compresses kilobytes to tens of bytes. The baseline statistics fit in a few kilobytes of memory. The radio transmits a few bytes routinely and kilobytes only when needed. The model, whether statistical or neural, is one component among many. The limits of anomaly Anomaly detection has an obvious weakness, which is that it detects change, not failure. A machine whose load has increased, whose speed has been changed by a variable-frequency drive, which has been re-mounted after maintenance or whose neighbour has started up will look anomalous, though it is perfectly healthy. Conversely, a fault that develops slowly from the first day of the learning period may be absorbed into the definition of normal. And an anomaly score says that something is different, without saying what, which leaves a maintenance team uncertain whether to act. These problems are not reasons to abandon the approach; they are the central challenges of deploying it. They are the subject of the next chapter, which follows a sensor from the laboratory onto the factory floor. Chapter 7: Thresholds, Drift and the Truth on the Floor A predictive maintenance model that works on a test rig has passed the easy exam. The rig runs at a fixed speed, under a fixed load, on a rigid bench, with faults that were deliberately seeded and are therefore known. A real plant offers none of these comforts. Machines change speed and load through the day, are bolted to structures that resonate, sit next to other machines that shake them, are repaired and reassembled, and fail in ways nobody seeded. And at the end of the chain is a maintenance team with limited time, who will decide within a few weeks whether the new sensors are useful or just another source of noise. This chapter is about the distance between a model that detects anomalies and a system that people trust. Installation is part of the model The first thing that goes wrong in the field happens before any data is collected. The way an accelerometer is attached to a machine determines what frequencies it can measure. A sensor screwed onto a stud on a flat, machined surface transmits vibration faithfully to high frequencies. An adhesive mount is nearly as good if done properly. A magnetic mount, convenient for portable measurement, has a lower mounted resonance and attenuates high frequencies, which are exactly where early bearing defects appear. A sensor fixed to a thin sheet-metal cover measures the cover's vibration rather than the bearing's. And a sensor placed far from the bearing, on the other side of a joint, receives a weakened, filtered version of the signal it is meant to detect. The orientation matters too. Imbalance shows mainly in the radial directions, misalignment often in the axial direction, and a triaxial sensor helps only if its axes are known relative to the machine. A model trained on data from sensors mounted one way will see different features from sensors mounted another way. The lesson for TinyML is that installation conditions are part of the input distribution. A fleet of sensors installed carelessly is, from the model's point of view, a fleet of different instruments. Good deployments treat installation as a controlled procedure, with prepared mounting points, recorded orientations and a check of the first measurements, and they prefer per-machine learning of normal behaviour precisely because no two installations are identical. Knowing what the machine is doing The most frequent source of false alarms in vibration anomaly detection is not sensor noise but changes in operating conditions. A pump driven through a variable-frequency drive runs at different speeds through the day; its vibration spectrum shifts with speed, all its characteristic frequencies scale proportionally, and its amplitudes change with flow and pressure. A fan's vibration depends on its damper setting. A conveyor's depends on its load. If the learning period happened to cover only one operating condition, every other condition will look anomalous. There are three ways to handle this. The first is to measure the operating condition directly. Running speed can often be read from the spectrum itself, by locating the dominant 1X peak, or obtained from the drive controller or a tachometer. With speed known, features can be normalised, for instance by expressing frequencies as multiples of running speed rather than in hertz, a technique called order analysis, so that a machine at two different speeds produces comparable features. The second is to learn several normal states, as the clustering approach in the previous chapter does, and score each measurement against the nearest one. The third is to condition the measurement schedule: measure only when the machine is in a defined state, such as full speed, and skip measurements otherwise. This sacrifices coverage for consistency and is often the pragmatic choice on battery-powered sensors, where each measurement has a cost anyway. Whichever approach is used, the learning period must cover the machine's genuine range of normal operation. A baseline learned over a quiet weekend will flag Monday's production. Practitioners often extend the learning period to cover at least one full operating cycle, such as a production week, and design the system so that the baseline can be extended or relearned when the operating pattern changes. The baseline problem Learning normal behaviour on each device solves the problem of installation and machine variation, and introduces a new one: what happens when normal changes? Some changes are legitimate. After a bearing is replaced, a coupling realigned or a motor rebuilt, the machine's vibration signature changes, usually for the better, and the old baseline is wrong. Seasonal temperature swings change lubricant viscosity and structural stiffness. Process changes alter loads. A system that never updates its baseline will gradually accumulate false alarms as the real machine drifts away from its learned self. Other changes are the faults themselves. A bearing defect grows over weeks; if the baseline continuously adapts to recent data, it will follow the fault as it develops and never alarm. This is the central dilemma of adaptive anomaly detection: adaptation is needed to track legitimate change but defeats detection of slow degradation. Practical systems resolve this with structure rather than cleverness. The baseline is frozen after the learning period and relearned only on an explicit event, such as a maintenance work order closing, a command from an operator or an engineer's decision after reviewing a change. Long-term trends of key features, such as overall velocity and high-frequency energy, are tracked separately from the anomaly score, so that a slow rise is visible even if each individual measurement is close to its predecessor. Some systems keep two baselines, a fixed reference from commissioning and a slowly adapting one, and report deviations from both. The ability to learn on the device makes this workable. A statistical or clustering model can be relearned on the sensor itself in response to a short downlink command, without retraining anything in the cloud or sending a new model over the air. This is one of the strongest arguments for keeping the on-device learner simple, even when a neural network provides the features. Research on full on-device training of neural networks for microcontrollers has advanced, including MIT's work on training within very small memory budgets, but in industrial deployments the common pattern remains a fixed network with locally learned statistics on top. From scores to alarms An anomaly score is a number. A maintenance team needs a decision: do nothing, watch, plan an intervention, or act now. Turning one into the other is where many deployments lose their audience. The simplest rule, an alarm whenever the score exceeds a threshold, generates alarms on single odd readings: a forklift hitting the machine's frame, a pressure transient, a measurement taken during start-up. Persistence rules help. An alarm is raised only if the score exceeds the threshold in several of the last several measurements, or if a smoothed score exceeds it. Graded levels help further: a lower threshold for "watch" and a higher one for "alert", so that the team sees a developing situation before it becomes urgent. Setting the thresholds themselves is a judgement that must be made with incomplete information, because there are usually no failure examples to calibrate against. A common approach sets them from the distribution of scores during the learning period, for example at a level exceeded by only a small fraction of normal measurements, and then adjusts them with experience. The target is not a statistical ideal but a false alarm rate that the team can absorb. A plant with a thousand sensors and a false alarm rate of one per sensor per month will see more than thirty false alarms a day, which no team will investigate for long. Alarm fatigue is a well-documented problem in process industries generally, and an anomaly detection system that contributes to it will be switched off, formally or informally. It also helps to anchor anomaly detection in the vibration standards maintenance teams already use. The ISO 20816 series, which superseded the older ISO 10816 series, defines evaluation zones for the broadband vibration velocity of various classes of machines, from newly commissioned condition to levels at which damage may occur. These zones are blunt instruments, sensitive mainly to gross faults, but they are familiar and defensible. A system that reports both a standards-based severity and an anomaly score gives the team a reference point and an early warning at the same time, and lets them calibrate their trust in the novel signal against the familiar one. Saying what changed The weakness of a bare anomaly score is that it says nothing about cause. A maintenance planner presented with "pump 14 is anomalous" has to send someone to investigate, perhaps with a handheld analyser, to find out whether anything is actually wrong. If that happens too often without finding anything, the system loses credibility. Systems that are trusted add diagnostic context to their alerts, and the physics of the previous chapter makes this possible at little cost. If the features that drove the anomaly score are spectral bands, the sensor can report which bands changed. A rise concentrated at 1X suggests imbalance. Rising 2X and axial energy suggest misalignment. Rising energy at high frequencies, especially with peaks in the envelope spectrum near a bearing's calculated defect frequencies, suggests a bearing problem. A broad rise in harmonics suggests looseness. None of these inferences is certain, but they turn an anomaly into a hypothesis that a technician can check. When the sensor transmits a full spectrum or waveform after an alert, an analyst can confirm the hypothesis remotely before anyone walks to the machine. This hybrid of learned anomaly detection and engineered diagnostic features is less elegant than an end-to-end neural network, and in practice it works better. It also helps with something the next section depends on: getting feedback. Closing the loop A deployed predictive maintenance system generates a stream of alerts. Each alert eventually has an outcome: a fault was found and repaired, nothing was found, the machine had been modified, or nobody looked. These outcomes are the only ground truth the system will ever get about its own performance, and they are almost always lost. They are lost because they live in a different system. Maintenance work is recorded in a computerised maintenance management system, often with free-text notes, and the link between a work order and the sensor alert that prompted it is rarely recorded. Without that link, the team cannot measure the system's real false alarm rate, its detection rate or the lead time it provides before failure, and cannot improve thresholds or models. Deployments that improve over time make the feedback cheap. When an alert is raised, the technician who investigates it records an outcome in a few taps, ideally inside the workflow they already use. Confirmed faults, with their waveforms, become labelled examples that are otherwise almost impossible to obtain. Over a fleet of hundreds or thousands of machines, these examples accumulate into a dataset of real failures that can train better diagnostic models, which can then be deployed back to the sensors. This is the point at which the anomaly detection framing begins to give way, gradually and for common fault types, to supervised diagnosis. The plant as an environment Factory floors are hostile to electronics in ways that shape the hardware and, through it, everything else. Motor housings run hot, sometimes too hot for the comfortable operating range of common batteries. Outdoor equipment sees frost and direct sun. Wash-down areas in food processing subject enclosures to high-pressure water and cleaning chemicals. Oil mist, dust and vibration itself attack connectors and seals. A sensor built for a laboratory bench will not last a year in many of these places. Some environments carry regulatory weight. In refineries, chemical plants, mines and grain handling facilities, areas where flammable gases or dusts may be present are classified as hazardous, and equipment installed there must be certified as unable to ignite the atmosphere, under schemes such as ATEX in Europe and IECEx internationally. For a battery-powered sensor, certification constrains the choice and size of the battery, the energy that can be stored in capacitors and the design of the circuitry, because a fault that dumps stored energy as a spark is precisely what the rules exist to prevent. These constraints tighten the energy budget further, and they make design changes expensive, since a modified product may need recertification. Connectivity is another environmental factor. Plants are full of steel that reflects and blocks radio signals, and their layouts change as equipment is added and moved. Low-power wide-area networks such as LoRaWAN offer long range through obstacles at very low data rates. Bluetooth Low Energy and mesh networks offer higher data rates over shorter ranges. Industrial wireless standards such as WirelessHART and ISA100.11a, designed for process instrumentation, offer the reliability and integration with control systems that some plants require. Each choice sets limits on how much data can leave the sensor and how often, which feeds directly back into how much analysis must happen on the device. A system that can send a full spectrum after every alert over Bluetooth may have to make do with a few summary bytes over LoRaWAN, and its on-device diagnostics must be designed accordingly. Finally, a TinyML monitoring system rarely arrives in a plant with no history. Larger sites often already employ vibration analysts, run route-based data collection and operate online monitoring on their critical machines. A new wireless system that ignores these people and tools will be treated as a rival; one that feeds its alerts and waveforms into the analysts' existing software, and uses their judgement to confirm and label events, gains allies and a source of expert feedback that no algorithm can replace. Measuring what matters The evaluation metrics used in research, such as the area under the receiver operating characteristic curve on a held-out test set, are useful for comparing methods but say little about operational value. In the field, the questions are different. Of the failures that occurred on monitored machines, how many were preceded by an alert, and how long before? How many alerts per machine per year turned out to be false? How much did the system cost to install and run, per machine? How often did batteries need replacement, compared with the specification? And did the maintenance team change what it did as a result? These numbers are slow to gather, because failures are rare, and they are seldom published, because they are commercially sensitive and often unflattering. Anyone evaluating a vendor's claims should ask for them anyway, and should be wary of figures derived only from test rigs or public datasets. The plant is the dataset This chapter has argued that the decisive work in industrial TinyML happens after the model is trained: in mounting sensors properly, accounting for operating conditions, managing baselines, setting thresholds that a team can live with, explaining alerts and capturing outcomes. Each of these depends on the energy and memory constraints of the device. A sensor that can relearn its baseline locally, report which features changed and send a waveform on demand is a sensor whose firmware and power budget were designed for those functions from the start. The model, once again, is the smallest part of the problem. The largest part, once there are hundreds or thousands of these devices in the field, is keeping them all working, secure and up to date. That is the subject of the final chapter. Chapter 8: Shipping and Sustaining a Fleet A single TinyML prototype on a lab bench is a demonstration. A thousand devices in the field, running for years, is a product, and it brings a different set of problems. Models must be improved and replaced without visiting each device. Firmware must be patched when vulnerabilities are found. Power consumption must hold up over temperature extremes and years of operation, not just a week on the bench. The behaviour of the fleet must be monitored so that a model that is quietly failing is noticed. And all of this must be done within the same energy and memory budgets that shaped the model in the first place. This chapter covers what it takes to keep a fleet of tiny devices working, and how to judge claims about their performance. Updating a device you cannot touch The defining capability of a maintainable fleet is the over-the-air update: replacing the firmware, the model or both on a device that is installed on a machine, in a ceiling or in a field, using only its radio. Without it, every improvement and every security fix requires a site visit, and the economics of most deployments collapse. On a microcontroller, an update has to be designed with care, because a failed update can render a device permanently useless. The standard approach uses a small bootloader that runs at power-up, checks which firmware image is valid, and starts it. Flash memory is divided into at least two slots. The running firmware occupies one; a new image is downloaded into the other in the background, possibly over hours or days on a slow network. When the download is complete and its integrity and authenticity have been verified, the bootloader swaps to the new image on the next restart. If the new image fails to start properly, or fails to confirm that it is working within some time limit, the bootloader reverts to the old one. Open-source bootloaders such as MCUboot, widely used with Zephyr and other platforms, implement this pattern, including cryptographic verification of images. This pattern costs flash. A device that must hold two complete firmware images needs roughly twice the flash of one that does not, and on a microcontroller where flash is shared with model weights, that cost shapes the design. Several techniques reduce it. Delta updates transmit only the difference between old and new images, which shrinks the download dramatically when changes are small. Some designs store the model in a separate flash partition from the application code, so that a model update replaces only the model. This is one of the real advantages of an interpreted runtime over compiled models: a new set of weights with the same operators can be delivered as data, without touching the code. External flash chips, cheap and low-power, can hold download buffers and spare images. The radio constraints of low-power networks make updates slow. A LoRaWAN device, for example, receives only small messages and is limited by duty-cycle regulations. The LoRa Alliance has defined specifications for firmware updates over the air that use multicast sessions and fragmentation with forward error correction, so that one transmission can update many devices and lost fragments can be reconstructed without retransmission. Even so, delivering a few hundred kilobytes to a fleet over LoRaWAN is a process measured in hours, not seconds, and it costs each device a noticeable amount of battery. Teams that plan to update models frequently should account for that energy in the original budget. Keeping model and firmware in step A model is not self-contained. It depends on the preprocessing code that produces its inputs, the operator implementations that execute it, the decision logic that interprets its outputs, and the thresholds and baselines that turn scores into actions. Updating any one of these without the others can break a device in ways that produce no error message. Fleets that are managed well treat these as a versioned bundle. The firmware records which model versions it is compatible with, the model file carries metadata describing the feature extraction it expects and the operators it needs, and the update system refuses to install combinations that have not been tested together. A model retrained on spectrograms computed with a new frame length must be deployed with firmware that computes the same frame length. A model that introduces a new operator must be deployed with firmware whose operator resolver includes it. A model that produces scores on a different scale must be deployed with new thresholds or with a recalibration step. Baselines learned on the device complicate this further. If a new anomaly detection model changes the feature space, every device's learned baseline is invalid and must be relearned, with a learning period during which the device provides no protection. A cautious rollout runs the new model in shadow mode alongside the old one, learning its baseline while the old model continues to raise alerts, and switches over only when the new baseline is ready. Rolling out safely Even with careful versioning, a new model can behave differently in the field than in testing. The standard practice from cloud software applies with adjustments: roll out to a small fraction of the fleet first, watch what happens, and expand only if the results are good. What to watch is the hard part, because a TinyML device by design reports very little. A keyword spotter sends nothing when it does not hear its keyword; an anomaly detector sends nothing when the machine is normal. To know whether a model is behaving well, the fleet needs a small amount of telemetry about its own operation: for instance, a daily summary of the distribution of scores, the number of detections, the device's battery voltage and temperature, and its uptime. Comparing these summaries between devices running the old and new models, and between the same devices before and after the update, reveals models that fire too often, never fire, or drain batteries faster than expected. The telemetry itself costs energy, so it is kept small and infrequent, but a fleet with no telemetry is a fleet whose failures are discovered by customers. The same telemetry reveals drift that has nothing to do with model updates: a keyword spotter deployed in a market with accents underrepresented in its training data, a cluster of vibration sensors whose scores rose after a plant changed suppliers, a batch of devices whose microphones degraded. Monitoring the distributions of inputs and outputs, not just the alerts, is how these problems become visible. Security on a small budget Connected microcontrollers have long been an attractive target for attackers, because they are numerous, often poorly secured and rarely updated. Adding machine learning does not change this, but it adds a few specific concerns, and regulation has begun to raise the floor. The basics are the same as for any connected device. The device should boot only firmware signed by its manufacturer, which requires a secure boot chain anchored in immutable code or hardware. Cryptographic keys should be stored where the application cannot read them, which on Armv8-M parts such as the Cortex-M33 and later can mean a secure partition protected by TrustZone, or a separate secure element. Communication should be encrypted and authenticated. Debug ports should be locked in production units. Industry schemes such as PSA Certified, which grew out of Arm's Platform Security Architecture, define levels of assurance that chip and device makers can be evaluated against. Machine learning adds three concerns. The model itself may be valuable intellectual property, the product of expensive data collection and training, and a device that stores it unencrypted in readable flash invites extraction. The model can also be tampered with: an attacker who can replace weights can make a sensor blind to faults or make a voice device respond to the wrong trigger, which is one reason model updates must be signed just as firmware updates are. And models can be fooled by crafted inputs, adversarial examples, which for audio means sounds designed to trigger or suppress a keyword. For most TinyML deployments, the practical risk from adversarial inputs is lower than the risk from basic weaknesses such as unsigned updates and open debug ports, and effort should go to those first. Regulation is making these basics mandatory in some markets. In the European Union, a delegated act under the Radio Equipment Directive applying from August 2025 introduced cybersecurity requirements for many internet-connected radio devices. The Cyber Resilience Act, which entered into force in December 2024, will impose broad security obligations on products with digital elements, including vulnerability handling and security updates over a support period. Its incident reporting obligations apply from September 2026 and its main requirements from December 2027. For makers of TinyML devices sold in Europe, the ability to deliver secure updates for years is becoming a legal requirement as well as an engineering one. Measuring power honestly The battery life printed on a datasheet is a calculation, and the assumptions behind it are where most power surprises come from. Several effects routinely make real battery life shorter than a bench measurement predicts. Battery capacity depends on conditions. A cell's rated capacity is measured at a particular temperature and discharge rate, and capacity falls at low temperatures and under high pulse loads. A radio transmission draws a brief pulse of current many times higher than the average, and some battery chemistries deliver such pulses poorly, especially when cold. Primary lithium thionyl chloride cells, popular in industrial sensors for their energy density and long shelf life, can develop a passivation layer during long idle periods that causes a voltage dip when a pulse is drawn, and designs often add a capacitor or a hybrid cell to supply the pulses. Self-discharge, though low for good primary cells, adds up over a decade. The device's own consumption varies too. Sleep current rises with temperature, sometimes severalfold between room temperature and the heat of a motor housing. Radio retries in poor coverage multiply the cost of each message. Network join procedures, repeated after a gateway outage, can cost more than weeks of normal transmissions. And firmware bugs that keep a peripheral powered or prevent the device from entering its deepest sleep state are common and devastating; a device whose sleep current is a few hundred microamps instead of a few microamps loses most of its battery life. The defence is to measure complete duty cycles on production-representative hardware over realistic temperature ranges, with an instrument that captures both the microamp sleep floor and the milliamp pulses, and to include worst-case radio conditions in the budget. Fleet telemetry of battery voltage, compared against the expected discharge curve, then catches devices whose consumption departs from the plan. Benchmarks and what they tell you Comparing hardware and software for TinyML was, until recently, nearly impossible, because every vendor reported results on different models, datasets and measurement methods. MLPerf Tiny, developed by a consortium of academic and industry researchers under MLCommons and described in a 2021 paper led by Colby Banbury, established a common basis. It defines four tasks with reference models: keyword spotting on Speech Commands, visual wake words (detecting whether a person is present in a small image), image classification on the CIFAR-10 dataset, and anomaly detection on machine sounds from ToyADMOS. Submitters report latency and, optionally, energy per inference, measured with a defined harness, and must meet accuracy thresholds. Results are published in rounds, and they give a reasonably fair comparison of how fast and efficiently different chips and software stacks run these particular workloads. Benchmarks answer a narrow question well. They do not say how a system will perform on a different model, with a different preprocessing pipeline, at a different duty cycle, with a radio, in a cold enclosure. Energy per inference, the headline number for many submissions, is only one term in the duty-cycle equation of Chapter 1, and often not the largest. A chip that wins on inference energy but has a high sleep current or slow wake-up may lose in a real product that infers once a minute. Benchmarks are a good starting point for shortlisting hardware and a poor substitute for measuring the actual application. The end of a device's life A fleet is not only deployed and maintained; eventually it is retired. Planning for that moment is part of responsible design, and it is increasingly a regulatory and commercial expectation as well. The most tangible issue is the battery. A deployment of tens of thousands of sensors is also a deployment of tens of thousands of primary lithium cells, which must be collected and recycled at the end of their lives rather than left on machines or thrown away. The European Union's Batteries Regulation, adopted in 2023, tightens obligations on collection, labelling and, over time, the removability and replaceability of batteries in many products. Designs that make the battery easy to replace safely, without discarding the rest of the device, extend the useful life of the electronics and reduce waste. Energy harvesting, where it is viable, reduces the problem further. The second issue is data and access. A retired device may still contain cryptographic credentials, configuration data and, in the case of an anomaly detector, a learned baseline describing a specific machine. Decommissioning procedures should revoke the device's credentials on the network side and, where possible, erase sensitive data on the device. Devices that are resold or reinstalled elsewhere need a reliable factory reset that clears learned state as well as configuration, or they will begin their new lives judging a different machine against the old one's idea of normal. The third issue is support. A manufacturer that stops issuing updates for a fleet still in service leaves its customers with devices that will not receive security fixes. Regulations such as the Cyber Resilience Act now require manufacturers to declare a support period and honour it. For products intended to last as long as their batteries, that period may be many years, and the engineering organisation, build tools and signing keys needed to produce updates must survive as long as the devices do. Taken together, these considerations argue for treating lifetime as a design parameter on a par with accuracy and battery life. The total cost and impact of a TinyML deployment includes its installation, its years of operation, its updates and its retirement, and the cheapest device to buy is not always the cheapest to own. The long life of small things A TinyML device is expected to work for years with little attention, in conditions its designers did not fully foresee. The practices in this chapter, namely safe updates, versioned bundles, staged rollouts, minimal telemetry, basic security and honest power measurement, are what make that expectation reasonable. They are not glamorous, and they rarely appear in tutorials, which end when the model first runs on a development board. But they are where much of the engineering effort in a successful product goes, and they share a common feature with everything else in this book: each one costs energy and memory, and each one has to be planned for from the beginning rather than added at the end. Conclusion The two applications at the centre of this book look very different. A wake-word detector lives in a consumer device, listens to human speech and must respond within a fraction of a second. A vibration anomaly detector lives on a pump in a plant, listens to steel and bearings and can take hours to make up its mind. Yet the chapters on each converged on the same finding: the neural network was the most tractable part of the problem, and the parts that decided success lay around it. The front end that turned raw signal into features, the schedule that decided when to sample, the rule that turned scores into actions, the baseline that defined normal, the update mechanism that kept everything consistent, and the battery that paid for all of it. That convergence is not a coincidence, and it has consequences for how TinyML systems should be designed, bought and judged. Design backwards from the decision The usual sequence in a machine learning project runs forward: gather data, train a model, optimise it, deploy it, and then work out how to integrate it. For a battery-powered device, this sequence puts the most consequential decisions last. By the time the team discovers that sampling dominates the power budget, that the radio cannot carry the alerts the model produces, or that the arena does not fit alongside the radio stack, the model has already been shaped by assumptions that do not hold. The chapters of this book point to the reverse order. Begin with the decision the device must make and who will act on it: a voice assistant that must wake, a technician who must be sent. From that, derive the acceptable error rates, which are always asymmetric, and the acceptable latency. From the deployment, derive the energy budget, and from the energy budget the duty cycle, the sampling window, the radio traffic and the number of inferences per day. Only then derive the model's budget in flash, SRAM, time and microjoules, and only then train. Every chapter's practical advice, from using a cascade to choosing a part to setting a threshold, follows naturally from this order and awkwardly from the other. This has an organisational implication. TinyML projects that succeed tend to have a single person or small team who understands both the firmware and the model well enough to trade one against the other. Projects that split the work, with data scientists handing a model file over a wall to embedded engineers, reproduce at the level of the organisation the same mismatch between training-time and deployment-time preprocessing that Chapter 4 identified as the commonest technical failure. Silence is the product A second conclusion follows from the energy argument of Chapter 1 and runs against most intuitions about artificial intelligence. The value of a TinyML device lies largely in what it does not say. A sensor that streams data is a data source; a sensor that reports only exceptions is a filter on reality, and the quality of that filter is the whole of its usefulness. A wake-word detector is judged mainly by the hours it stays quiet when it should. A vibration sensor is judged by whether its rare alerts are worth a technician's walk. This reframes what to measure. Accuracy on a balanced test set, the number most often quoted, measures how well a model distinguishes classes that it encounters in equal proportions. Deployed devices encounter the negative class almost always, so their usefulness is governed by false alarm rates per hour or per machine-year and by the cost each false alarm imposes. A buyer evaluating a TinyML product should ask for those numbers, measured in the field, and treat their absence as informative. A team building one should design its evaluation around them from the start, which means collecting long recordings of ordinary conditions as carefully as it collects examples of the event of interest. It also reframes the role of the threshold. The threshold is not a technical parameter to be tuned by the model's author. It is a statement about how much the system's users are willing to be disturbed in exchange for how much protection, and it belongs in the product specification. Chapters 5 and 7 showed that the same model, behind different thresholds and different cascades, makes for entirely different products. Where the budget is moving The constraints that define TinyML are loosening in one dimension and holding firm in another. Vector extensions and microNPUs have multiplied the arithmetic a microcontroller can perform per microjoule, and new parts carry more SRAM. Models that were impractical on a microcontroller five years ago, including real-time vision and small transformer-based audio models, are now routine on the more capable chips. The temptation is to read this as the end of the constraint, and to expect microcontrollers soon to run whatever larger devices run. The analysis of this book suggests caution. The cost of sampling sensors, the cost of radio transmission and the size of a battery that fits in a matchbox have not changed at anything like the pace of compute efficiency. As inference becomes cheaper, it shrinks as a share of the energy budget, and the other terms, already dominant in many designs, become more so. Faster accelerators will be used most profitably not to run larger models more often, but to run good-enough models more quickly and return to sleep sooner, or to move a stage of the cascade from a power-hungry processor onto the microcontroller. The economics that made the case for TinyML in the first place continue to govern how its new capacity is spent. What remains unsettled Some of the most important questions in the field are still open. On-device learning, in which a device adapts its model to its own environment, is clearly valuable, as the per-machine baselines of Chapter 7 show, but the tension between adapting to legitimate change and failing to detect slow degradation has no general solution, only careful engineering choices in each deployment. The feedback loop that would let anomaly detectors grow into diagnostic systems depends on capturing outcomes from maintenance teams, which is an organisational problem more than a technical one, and few deployments have solved it at scale. The data behind wake-word systems still underrepresents many speakers, and synthetic speech, while helpful, has not been shown to close that gap fully. And the security of fleets of small devices, newly regulated in Europe, will test whether manufacturers can support products for the years their batteries last. What to do differently For a reader about to start a TinyML project, the argument of this book reduces to a handful of questions to answer before training a model: 1. What decision will the device make, who acts on it, and what does each kind of error cost them? 2. What is the energy budget over the device's life, and what share of it goes to sensing, computing, transmitting and sleeping? 3. What cascade of stages, from cheapest to most capable, will guard the expensive ones? 4. What are the flash, SRAM, latency and energy budgets for the model, after everything else in the firmware is accounted for? 5. How will the preprocessing used in training be guaranteed identical to the preprocessing on the device? 6. How will the device be updated, secured and monitored, and what will that cost in flash and battery? 7. How will real-world outcomes be captured so that the system can be judged and improved? None of these is a machine learning question in the narrow sense, and all of them determine whether the machine learning is worth doing. The models in this book are small, clever and increasingly easy to build. The systems that make them useful are where the discipline lies. Notes Introduction 1. Pete Warden and Daniel Situnayake, TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers (Sebastopol, CA: O'Reilly Media, 2019). 2. On the renaming of the tinyML Foundation as the EDGE AI FOUNDATION, see the foundation's announcement of November 2024, reported by Hackster.io, "The tinyML Foundation Undergoes an Evolution: Meet the EDGE AI FOUNDATION." 3. Google Developers Blog, "TensorFlow Lite is now LiteRT," September 2024. 4. Qualcomm Technologies announced its acquisition of Edge Impulse in March 2025. Chapter 1 1. Gregory J. Pottie and William J. Kaiser, "Wireless Integrated Network Sensors," Communications of the ACM 43, no. 5 (2000): 51–58. 2. The duty-cycle calculation in this chapter is illustrative; the currents used are representative orders of magnitude, not measurements of any specific product. Chapter 2 1. Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan and Song Han, "MCUNet: Tiny Deep Learning on IoT Devices," Advances in Neural Information Processing Systems 33 (NeurIPS 2020). 2. Liangzhen Lai, Naveen Suda and Vikas Chandra, "CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs," arXiv:1801.06601 (2018). 3. Arm, Cortex-M55 and Ethos-U55 product announcements (February 2020) and Ethos-U85 announcement (2024); Arm Developer documentation for the Cortex-M processor family and the M-Profile Vector Extension. 4. Alif Semiconductor, announcement of second-generation Ensemble microcontrollers (2025); STMicroelectronics, launch of the STM32N6 series (December 2024). Chapter 3 1. Andrew G. Howard et al., "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications," arXiv:1704.04861 (2017). 2. Yundong Zhang, Naveen Suda, Liangzhen Lai and Vikas Chandra, "Hello Edge: Keyword Spotting on Microcontrollers," arXiv:1711.07128 (2017). 3. Benoit Jacob et al., "Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2018). 4. Geoffrey Hinton, Oriol Vinyals and Jeff Dean, "Distilling the Knowledge in a Neural Network," arXiv:1503.02531 (2015). 5. Colby Banbury et al., "MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers," Proceedings of Machine Learning and Systems 3 (MLSys 2021). Chapter 4 1. Robert David et al., "TensorFlow Lite Micro: Embedded Machine Learning for TinyML Systems," Proceedings of Machine Learning and Systems 3 (MLSys 2021). 2. The structure of the inference program described follows the examples distributed with the LiteRT for Microcontrollers (tflite-micro) repository, including the micro speech example. Chapter 5 1. Pete Warden, "Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition," arXiv:1804.03209 (2018). 2. Zhang et al., "Hello Edge," as in Chapter 3, note 2. 3. openWakeWord, an open-source wake-word framework trained largely on synthetic speech, is maintained on GitHub by David Scripka. Chapter 6 1. Yuma Koizumi, Shoichiro Saito, Hisashi Uematsu, Noboru Harada and Keisuke Imoto, "ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detection," IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA 2019). 2. Harsh Purohit et al., "MIMII Dataset: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection," Proceedings of the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE 2019). 3. Yuma Koizumi et al., "Description and Discussion on DCASE2020 Challenge Task2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoring," Proceedings of the DCASE 2020 Workshop. 4. Robert B. Randall and Jérôme Antoni, "Rolling Element Bearing Diagnostics: A Tutorial," Mechanical Systems and Signal Processing 25, no. 2 (2011): 485–520. Chapter 7 1. International Organization for Standardization, ISO 20816-1:2016, Mechanical vibration: Measurement and evaluation of machine vibration, Part 1: General guidelines, and ISO 20816-3:2022 for industrial machines. 2. Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan and Song Han, "On-Device Training Under 256KB Memory," Advances in Neural Information Processing Systems 35 (NeurIPS 2022). Chapter 8 1. Colby Banbury et al., "MLPerf Tiny Benchmark," Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (2021), arXiv:2106.07597. 2. Regulation (EU) 2024/2847 of the European Parliament and of the Council (Cyber Resilience Act); Commission Delegated Regulation (EU) 2022/30 supplementing the Radio Equipment Directive. 3. LoRa Alliance, specifications for remote multicast setup and fragmented data block transport used for firmware updates over the air. 4. MCUboot, open-source secure bootloader for 32-bit microcontrollers, documentation at the project's website. Further Reading Pete Warden and Daniel Situnayake, TinyML (O'Reilly, 2019).** The book that named the field. Its examples use earlier versions of the tools, but its step-by-step treatment of a keyword spotter, a person detector and a gesture recogniser remains the clearest introduction to how the pieces fit together. Daniel Situnayake and Jenny Plunkett, AI at the Edge (O'Reilly, 2023).** A broader, more workflow-oriented guide to designing edge machine learning products, strong on problem framing, data collection and evaluation rather than code. Vijay Janapa Reddi and contributors, Machine Learning Systems (open textbook, mlsysbook.ai).** A freely available textbook originating in Harvard's TinyML teaching, covering the full stack from data engineering to deployment and benchmarking, with a strong embedded emphasis. Robert David et al., "TensorFlow Lite Micro: Embedded Machine Learning for TinyML Systems" (MLSys 2021). The design paper for the runtime used as the main example in Chapter 4, explaining the interpreter, the arena and the trade-offs behind them. Ji Lin et al., "MCUNet: Tiny Deep Learning on IoT Devices" (NeurIPS 2020), and the follow-up "On-Device Training Under 256KB Memory" (NeurIPS 2022). The clearest demonstrations of how far co-design of network and inference engine can push small microcontrollers, and of what on-device training requires. Colby Banbury et al., "MLPerf Tiny Benchmark" (NeurIPS Datasets and Benchmarks, 2021). Describes the four benchmark tasks, reference models and measurement methodology; essential for interpreting published hardware comparisons. Yundong Zhang et al., "Hello Edge: Keyword Spotting on Microcontrollers" (2017). A short, practical study comparing network families for keyword spotting under microcontroller budgets, and the origin of the DS-CNN reference model. Robert B. Randall, Vibration-based Condition Monitoring (Wiley, 2nd edition 2021).** The standard engineering text on machine vibration analysis, including the bearing and gear diagnostics that underlie the features discussed in Chapters 6 and 7. Robert B. Randall and Jérôme Antoni, "Rolling Element Bearing Diagnostics: A Tutorial" (Mechanical Systems and Signal Processing, 2011). The most widely cited tutorial on envelope analysis and related techniques for finding bearing faults. DCASE Challenge, Task 2 reports (2020 onward). The annual task descriptions and result summaries for unsupervised anomalous sound detection are the best way to follow the state of the art in machine-sound anomaly detection, including domain shift and few-shot settings. Glossary Activation. An intermediate result produced by one layer of a neural network and consumed by the next. On microcontrollers, activations live in SRAM and usually determine peak memory use. Anomaly detection. Learning what normal data looks like and flagging inputs that differ from it, used when examples of the events of interest are too rare to train a classifier. Autoencoder. A neural network trained to reproduce its input through a narrow bottleneck; its reconstruction error serves as an anomaly score. Cascade. A chain of processing stages of increasing cost, each of which decides whether to wake the next, so that expensive stages run only when needed. DMA (direct memory access). A hardware controller that moves data between peripherals and memory without involving the processor core, allowing the core to sleep while sensors are sampled. Duty cycling. Keeping a device in a low-power sleep state most of the time and waking it briefly to do work, so that average power is far below active power. Envelope analysis. A vibration technique that extracts the amplitude modulation of a high-frequency resonance to reveal repetitive impacts, such as those caused by bearing defects. Flash. Non-volatile memory on a microcontroller that holds program code and constant data, including model weights. Helium. Arm's M-Profile Vector Extension for Cortex-M processors, providing 128-bit vector instructions that accelerate signal processing and neural network arithmetic. Log-mel spectrogram. A time-frequency representation of audio in which spectral energy is summarised by filters spaced on the mel scale and compressed with a logarithm; the standard input for keyword spotting. Quantization. Representing weights and activations with low-precision integers, typically eight bits, using a scale and zero point to map them to real values. SRAM. Fast, volatile working memory on a microcontroller, holding the stack, buffers and neural network activations; usually much scarcer than flash. Tensor arena. In LiteRT for Microcontrollers, a single statically allocated block of memory from which all tensors and runtime structures are allocated, avoiding dynamic allocation. Hashtags: #TinyML #MachineLearningOnMicrocontrollers #EmbeddedMachineLearning #EdgeAI #ArtificialIntelligence #MachineLearning #Microcontrollers #EmbeddedSystems #OnDeviceAI #LowPowerAI #EnergyEfficientAI #BatteryPoweredDevices #NeuralNetworkOptimization #ModelQuantization #ModelCompression #KnowledgeDistillation #TensorFlowLiteMicro #LiteRT #ARMCortexM #CMSISNN #EmbeddedInference #TinyNeuralNetworks #KeywordSpotting #WakeWordDetection #PredictiveMaintenance #AnomalyDetection #IndustrialIoT #SensorDataProcessing #EdgeImpulse #MicrocontrollerAI

Download the Book (PDF): Introduction Every engineering leader eventually inherits a system that everyone is afraid of. It usually has a name that once meant something, an acronym from a product line that no longer exists or the codename of a project whose sponsors left years ago. It runs the part of the business that makes money. It was written by people who were, for the most part, competent and under pressure, and it has been changed thousands of times since by people who were also competent and also under pressure. Nobody understands all of it. Deployments happen on a schedule that has slowly stretched from weekly to monthly to "when we can get a window." The most experienced engineers spend a growing share of their week explaining to newer colleagues why a small change is not small. And every planning cycle, someone proposes that the only real answer is to throw it away and start again. This booklet is about what to do instead, and about how to tell the difference between the debt that is quietly bankrupting a team and the debt that is merely ugly. Its argument can be stated in one sentence. Technical debt is worth paying down only where it charges interest on work the business actually needs to do next, and the way to pay it down is to find that interest precisely, retire it incrementally while the system keeps shipping, and fund the work by expressing the interest in the business's own currency rather than in engineering's. Each part of that sentence rules something out. It rules out the idea that all old code is debt, or that debt is measured by how much a codebase offends its current maintainers. A module can be badly written, poorly named and twenty years old and still cost almost nothing, because nobody needs to change it. It rules out the big-bang rewrite, which trades a known and serviceable liability for an unknown one and suspends delivery for the duration. It rules out treating microservices as a destination in themselves, since a migration justified by fashion rather than by interest tends to create a new and more expensive kind of debt. And it rules out the familiar ritual of asking for a "refactoring sprint" as though it were a favour the business grants to engineers, a request that is usually refused, and that deserves to be refused when it is framed that way. Who this is for The intended reader is a person responsible for engineering outcomes rather than for writing every line: a head of engineering, a director, a principal engineer who has started to spend more time in planning meetings than in code reviews, a chief technology officer at a company of a few hundred people, or a product or finance leader who wants to understand what the engineering organisation is actually asking for when it talks about debt. Some passages describe code-level techniques, because a leader who cannot follow the mechanics will not be able to judge a plan or notice when one is going wrong. But the emphasis throughout is on decisions: what to measure, what to fund, what to sequence first, what to leave alone, and how to explain all of this to people who will never read the code. The setting is the legacy monolith, because that is where these questions bite hardest. A monolith is not a defect. Many of the most successful software products in the world began as a single deployable application, and several well-run companies have deliberately chosen to keep one. The problem arises when a large monolith has accumulated enough tangled dependency, enough untested behaviour and enough organisational contention that changing it has become slow, risky and expensive, while the business still depends on it for revenue. At that point leadership faces a set of decisions that are partly technical, partly financial and partly political, and that are routinely made badly in all three dimensions at once. What the book argues, chapter by chapter The first chapter returns to the original meaning of the debt metaphor, which is sharper and more useful than its popular reputation. Ward Cunningham, who introduced it in 1992, was not describing sloppy code; he was describing the gap between what a team has learned about its problem and what its code still reflects. The distinction between principal and interest is the key idea of the whole booklet, because interest is paid only when someone touches the code. The second chapter is about finding that interest. Version control history records where change actually happens, and when it is combined with simple complexity measures it reveals a small number of hotspots that account for a disproportionate share of cost. The same history exposes hidden coupling between parts of the system and concentrations of knowledge in a handful of people, which are forms of debt that no static analysis tool reports. The third chapter turns a list of problems into a portfolio. It sets out a way of pricing debt items against the roadmap, assigning each one a disposition (pay down, contain, accept, or retire with the feature it supports), and deciding between refactoring in place, extracting a component, or replacing it. The fourth chapter examines why large rewrites so often fail and why the goal of a migration should be stated in terms of outcomes rather than architecture. It takes seriously the evidence that microservices impose real costs and that some organisations have been right to move back towards a monolith, or towards a modular monolith that gets most of the benefit at a fraction of the price. The fifth and sixth chapters are the practical core. The fifth describes the Strangler Fig pattern, named by Martin Fowler after the rainforest vines that grow around a host tree until they can stand on their own, and the supporting techniques that make it work: routing facades, branch by abstraction, feature toggles, and the choice of the first seam. The sixth deals with data, the part of every migration that takes longest and goes wrong most expensively, covering shared databases, change data capture, parallel runs and reconciliation. The seventh chapter is about persuasion, and it is placed late deliberately. A case for refactoring that is built before the interest has been found and the migration path designed is a case built on opinion. Once both exist, the argument to a chief financial officer or a product leader can be made in terms they already use: lead time, risk exposure, cost of delay, and the capacity that is currently being consumed without anyone having decided to spend it. The eighth chapter covers governance: how team structure shapes architecture, which few measures tell a leader whether a modernisation programme is working, how to stop the debt from re-accumulating, and how to recognise when a migration is finished, which is often earlier than its original plan said. A note on evidence Much of what is written about technical debt is opinion presented as fact, and some of the most widely repeated numbers have uncertain origins. Where this booklet cites figures, they come from named sources: industry surveys such as Stripe's 2018 Developer Coefficient report and McKinsey's 2020 work on technology debt, peer-reviewed studies such as Adam Tornhill and Markus Borg's 2022 analysis of code quality in proprietary codebases, and regulatory findings on well-documented failures. Survey figures are treated as what they are, which is the self-reported perception of developers or executives, and are useful as indications of scale rather than as precise measurements. Where the honest answer is that nobody has measured something rigorously, the text says so. The examples of failure are drawn from public record because they are instructive, not because the organisations involved were unusually careless. Most large software estates contain the same kinds of risk. The difference between the companies that appear in incident reports and those that do not is often a matter of which risks happened to be triggered, and when. A note on tone Technical debt arguments generate a great deal of moral language. Engineers describe legacy code as rotten or shameful; executives describe engineers as perfectionists who want to polish things that work. Neither framing helps. The code in a legacy monolith was, in most cases, a reasonable answer to the questions its authors faced. Some of those answers have aged badly because the business changed, because the team learned more, or because the volume of change exceeded what the original structure could absorb. Treating this as a management problem, with costs, options and trade-offs, rather than as a morality tale, is the first step towards making good decisions about it. The same applies to the people who will fund the work. A finance leader who refuses an unexplained request for three months of engineering time with no visible output is behaving rationally. The task is not to win an argument against them but to give them what they need to make a decision they can defend: a clear statement of what the current system is costing, what the proposed work will change, how risk will be controlled while it happens, and how everyone will know whether it is working. The rest of this booklet is about how to build that statement honestly and then deliver on it. Chapter 1: What the Metaphor Actually Says The phrase "technical debt" has become so common that it has nearly lost its meaning. In many organisations it now refers to anything engineers dislike: old frameworks, inconsistent naming, missing documentation, a test suite that takes forty minutes to run, a vendor product nobody chose, a module written by a contractor who left in a hurry. When one phrase covers everything, it stops helping anyone decide anything. A leader who wants to manage technical debt has to start by recovering a precise version of the idea, because the precise version contains the decision rule that the loose version lacks. Cunningham's original point Ward Cunningham introduced the metaphor in an experience report at the OOPSLA conference in 1992, describing the WyCash portfolio management system he had worked on. The passage is short and worth reading in full: Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite. Objects make the cost of this transaction tolerable. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise. Three features of this passage are routinely lost in later retellings. First, the debt Cunningham describes is not the result of carelessness. It is the natural consequence of writing software before the problem is fully understood, which is always. The first version of any system encodes the team's understanding at the time it was written. As the team learns more about the domain, the gap widens between what they now know and what the code expresses. Cunningham later clarified, in a widely circulated 2009 video, that he never meant the metaphor to excuse writing poor code with the intention of fixing it later; he meant that code written with an honest but incomplete understanding should be brought into line with the improved understanding as it arrives. The debt is the mismatch between the model in the code and the model in the heads of the people who now know better. Second, the debt can be a good decision. "A little debt speeds development." Shipping a version that reflects partial understanding allows the team to learn from real use, which is how understanding improves. A team that refused to ship until it fully understood the domain would never ship. Borrowing, in this sense, is an investment in learning. Third, and most important for everything that follows, the cost of the debt is described as interest: "every minute spent on not-quite-right code." The interest is not an abstract property of the code. It is paid in time, by people, when they work with that code. Code that nobody works with charges no interest at all, however bad it is. Principal and interest Taking the financial metaphor seriously yields two quantities that a leader needs to keep distinct. The principal is the cost of fixing the problem: the engineering effort required to restructure the code so that it reflects current understanding, add the tests that make it safe to change, split the module that has grown too large, or migrate away from the framework that is no longer supported. Principal is what engineers usually estimate when they talk about debt, and it is usually large, which is why the conversation so often stalls. The interest is the ongoing cost of not fixing it: the extra time each change takes because the code is hard to understand, the defects that escape because behaviour is untested, the incidents caused by unexpected interactions, the onboarding time for each new engineer, the features that are not attempted because they would be too risky. Interest is what the business actually experiences, though it rarely sees it labelled as such. It appears instead as slow delivery, missed estimates, production incidents and an engineering team that seems to be busy without producing much. The decision rule follows directly. Paying down debt makes sense when the interest you would stop paying, over the period that matters to you, exceeds the principal you would have to spend. If a module is changed every week and each change costs an extra day because of its structure, the interest is large and recurring, and a restructuring that costs two weeks pays for itself within a quarter. If a module is changed twice a year, the same restructuring might take a decade to pay back, and the money is better spent elsewhere. The badness of the code is identical in both cases. The decision is opposite. A worked comparison makes the rule concrete. Imagine two modules in the same system, both rated by the engineers as among the worst in the codebase. The first handles the calculation of shipping charges. It is changed most weeks, because the business keeps adding carriers, regions and promotional rules, and engineers estimate that each change takes about two days longer than it would in a well-structured module, with a meaningful chance of breaking an existing rule. Over a quarter of roughly twelve working weeks, that is something like twenty-four engineer-days of interest, before counting the incidents. A restructuring that isolates the rules, adds characterisation tests and separates carrier integrations from pricing logic is estimated at six engineer-weeks, or about thirty days. It pays for itself in a little over a quarter and keeps paying afterwards. The second module generates an annual regulatory report. Its code is, if anything, worse: long functions, global state, no tests. But it changes once a year, when the regulator adjusts the format, and the change typically takes a week. A full restructuring would take a similar six weeks. Even if it halved the annual effort, it would take more than a decade to pay back. The right answer is to leave it alone, perhaps with a set of characterisation tests to make the annual change safer, and to spend the six weeks on the shipping module. The numbers in such an exercise will always be rough. That does not matter much, because the differences between items in a real codebase are usually large, often an order of magnitude or more. Rough estimates are enough to separate the shipping module from the regulatory report; they do not need to be precise to be decisive. This is why a list of everything wrong with a codebase is not a debt register. It is a list of principal. Without the corresponding interest, it cannot be prioritised, and a team that tries to work through it in order of offensiveness will spend most of its effort on code that was costing almost nothing. There is a further subtlety. Interest is not fixed; it depends on what the business plans to do next. A billing module that has been stable for three years carries low interest today. If the company decides to introduce usage-based pricing next year, that same module is about to be changed heavily, and its interest rate is about to rise sharply. Debt management is therefore always relative to a roadmap. The same codebase carries different debt for a company that plans to expand into new markets than for one that plans to hold its product steady and cut costs. Kinds of debt, and kinds of things that are not debt Martin Fowler offered a useful classification in a 2009 essay on what he called the technical debt quadrant. He crossed two distinctions: whether the debt was taken on deliberately or inadvertently, and whether it was prudent or reckless. Deliberate and reckless debt sounds like "we don't have time for design." Deliberate and prudent debt sounds like "we must ship now and deal with the consequences." Inadvertent and reckless debt arises when a team does not know good design practice and produces a tangle without realising it. Inadvertent and prudent debt is the most interesting category, and corresponds most closely to Cunningham's original point: a skilled team builds something sensibly, and only afterwards realises, from what it has learned, how it should have been built. The quadrant is useful mainly because it separates two leadership problems. Reckless debt, deliberate or not, is a symptom of how the team is working: insufficient skill, insufficient review, or a delivery culture that rewards speed today over sustainability. Paying it down without addressing the cause simply recreates it. Prudent debt is a normal by-product of learning and should be expected, budgeted for and repaid as part of ordinary work. Treating all debt as a failure produces defensiveness; treating all debt as natural excuses practices that need to change. Several other things are commonly called technical debt but behave differently enough that they deserve their own names. Obsolescence is the decay of the platform underneath the code rather than of the code itself: an operating system reaching end of support, a language version no longer patched, a database vendor ending a product line, a library with known security vulnerabilities and no maintainer. Obsolescence does not charge interest in the same way, since the code may be perfectly easy to change. Instead it accumulates risk, which arrives suddenly when a vulnerability is disclosed or a compliance audit fails. It also has a deadline, which design debt usually does not. This makes obsolescence easier to fund, because the date is external and non-negotiable, and it is often wise to bundle design improvements into the work an upgrade requires. Missing capability is sometimes described as debt when it is really just unbuilt features: no self-service onboarding, no multi-currency support, no audit trail. These may be important, but they are product decisions, and labelling them technical debt muddies both conversations. Operational debt sits in the processes around the code: manual deployment steps, environments that differ from production in ways nobody has documented, alerts that fire so often they are ignored, runbooks that describe a system as it was three years ago. It charges heavy interest, often heavier than design debt, and it is frequently cheaper to fix. Knowledge debt is the gap between what the system does and what anyone currently employed understands about why. Code can be clean and still carry heavy knowledge debt if the people who wrote it have left. It can be messy and carry little if the people who wrote it are still there and still remember. This form of debt is invisible to every tool that analyses source code, and it becomes visible only when someone leaves or when an incident requires an explanation nobody can give. Michael Feathers, in Working Effectively with Legacy Code (2004), offered a definition that remains the most practically useful in the field: "To me, legacy code is simply code without tests." The definition is deliberately provocative, since it implies that code written last week can be legacy. But its point is precise. What makes code dangerous to change is not its age or its style but the absence of any fast, reliable way to know whether a change has broken something. Without that, every modification is a gamble, and rational engineers respond to gambles by making as few changes as possible, adding new code alongside old code rather than modifying it, and copying rather than refactoring. Each of these defensive habits makes the code harder to change next time. The absence of tests is therefore not one kind of debt among many; it is the condition that makes all other debt compound. This matters for leaders because it changes where the first investment usually goes. Before any significant restructuring of a legacy module, someone needs to establish what it currently does. Feathers called the technique characterisation testing: writing tests that record the system's actual present behaviour, including behaviour that may be wrong, so that any change in that behaviour during restructuring is detected. These tests do not assert what the code should do. They assert what it does, which is the only safe baseline when nobody is certain what it should do. The work is unglamorous and produces no visible feature, but it converts a gamble into an engineering task, and no modernisation plan for a large monolith succeeds without a great deal of it. Why debt compounds Financial debt compounds because unpaid interest is added to the principal. Technical debt compounds through a different but analogous mechanism, and understanding it explains why legacy systems tend to degrade faster as they age rather than at a steady rate. Meir Lehman, studying the evolution of large software systems at IBM and elsewhere from the 1970s onwards, formulated a set of observations that became known as Lehman's laws. Two are relevant here. A system that is used in the real world must be continually changed or it becomes progressively less useful. And as such a system is changed, its complexity increases unless work is specifically done to maintain or reduce it. The second observation is the one that matters. Complexity does not stay constant by default. Every change made without attention to structure adds a little, and the additions interact. The interaction is the compounding. When a module is hard to understand, engineers changing it make more mistakes and take more defensive shortcuts. They add a conditional rather than restructuring a branch, duplicate a function rather than generalising it, wrap a call in a try-catch rather than understanding why it fails. Each shortcut is rational in isolation and makes the next change harder. Meanwhile the engineers who understand the module best are pulled into every change and every incident involving it, which leaves them less time to improve it and concentrates knowledge further. New engineers avoid it. The module becomes a place where only certain people work, which slows everything that depends on it. There is also an organisational compounding. As a monolith becomes harder to change, coordination costs rise: more teams need to agree before a release, merge conflicts increase, the test suite grows slower, and release cadence falls. Slower releases mean larger releases, which mean riskier releases, which lead to more change control, which slows releases further. Many organisations arrive at a monthly or quarterly release train not because anyone chose it but because each step towards it was a sensible response to the previous step's failures. Three practical consequences follow from taking the metaphor seriously. The first is that debt cannot be managed in aggregate. Statements such as "we have too much technical debt" or "our debt ratio is high" are not actionable, because the decision to pay down any particular piece of debt depends on the interest it charges, and interest varies enormously across a codebase. A leader needs a map, not a total. The second is that interest must be measured where it is paid: in the time and risk associated with changing specific parts of the system. That measurement is possible, more precisely than most organisations realise, using data they already have. The next chapter shows how. The third is that the case for paying down debt is a case about the future, not the past. The question is never "how bad is this code?" but "what will it cost us to do what we plan to do next, with this code as it is, compared with the cost of fixing it first?" Kent Beck compressed this into a single line in 2012: "for each desired change, make the change easy (warning: this may be hard), then make the easy change." Debt management at the scale of an organisation is the same principle applied to a roadmap. The work worth doing is the work that makes the next important changes easy. Everything else, however tempting, is decoration. Chapter 2: Finding the Interest Most organisations try to locate technical debt by asking engineers where it is, or by running a static analysis tool over the codebase. Both methods produce useful information and both share a fundamental flaw: they measure principal, not interest. Engineers remember the code that annoyed them most recently or most memorably, which correlates only loosely with the code that costs most. Static analysis reports code that is complex, duplicated or poorly structured, which describes how bad the code is, not how much its badness costs. A ten-thousand-line class that nobody has touched in four years will top every complexity report and charge almost no interest. To find interest, a leader needs to look at where work actually happens. Fortunately, every organisation that uses version control already has a detailed, timestamped record of exactly that. Version control as a behavioural record A version control repository records every change made to every file, when it was made, by whom, and usually alongside a message that links it to a ticket or describes its purpose. Most teams use this history only to find out who broke something or to roll back a change. Read in aggregate, it is something much more valuable: a record of where engineering effort has gone over the life of the system. Adam Tornhill developed this idea in two books, Your Code as a Crime Scene (first published in 2015, with a second edition in 2024) and Software Design X-Rays (2018), and in the tooling built around them. His central technique is the hotspot analysis. For each file or module, compute two numbers: how often it changes (change frequency, taken from version control history over a period such as the last year) and how complex it is (using any reasonable proxy, even lines of code or indentation depth). Files that score high on both are hotspots: complex code that the organisation keeps having to change. These are the places where interest is being paid. The striking empirical finding, which shows up again and again when the analysis is run on real systems, is how concentrated change is. In most mature codebases, the great majority of files are changed rarely. A small fraction, often a few percent of the total, absorbs a large share of all changes. When those frequently changed files are also complex, they become the dominant source of delivery cost and defects. The remaining bulk of the codebase may be just as poorly structured, but its cost is mostly dormant. This concentration is the most useful fact a leader can know about a legacy system, because it converts an overwhelming problem into a tractable one. Nobody can refactor a two-million-line monolith. A team can absolutely restructure the dozen modules where most of the change is concentrated, and that work will deliver most of the available benefit. The link between code quality and delivery cost has historically been asserted more often than measured. One of the more careful attempts is a 2022 study by Adam Tornhill and Markus Borg, "Code Red: The Business Impact of Code Quality," presented at the International Conference on Technical Debt. They analysed 30,737 files across 39 proprietary production codebases, classified files by a code health measure, and linked them to issue-tracker data on defects and development time. Their findings were large. Low-quality code contained 15 times more defects than high-quality code. Resolving issues in low-quality code took on average 124 percent more development time. And the uncertainty was greater: the maximum cycle times for issues in low-quality code were 9 times longer. That last result deserves attention from anyone who manages delivery commitments. It is not only that work in poor code is slower on average; it is that it is far less predictable, because a change that looks simple may turn out to touch some unexpected dependency and take weeks. Unpredictability is in many ways more damaging than slowness, since it undermines every plan built on top of an estimate. Survey evidence points in the same direction, with the caveat that it measures perception. Stripe's Developer Coefficient report, published in September 2018 from a Harris Poll survey of more than a thousand developers and a similar number of executives across five countries, found that developers reported spending an average of 17.3 hours a week on maintenance work such as debugging, refactoring and modifying existing code. Stripe estimated the global opportunity cost of bad code at nearly 85 billion dollars a year. The precise numbers matter less than their scale: in the view of the people doing the work, a large share of engineering capacity goes to servicing existing systems rather than building new capability. These figures are useful context when making the case for investment, a subject taken up later. For locating debt, the important lesson from the Code Red study is that code quality and its cost can be measured file by file, and that the cost is concentrated where low quality coincides with high activity. Hidden coupling and concentrated knowledge Hotspots reveal where effort is spent. A second analysis of the same history reveals why some changes are so much more expensive than they appear. Change coupling, sometimes called temporal or logical coupling, measures which files tend to change together. If every time someone modifies the pricing module they also have to modify the invoice renderer and a particular database migration script, those three files are coupled, whether or not the code makes the dependency explicit. The analysis is straightforward: for each pair of files, count how often they appear in the same commit, or in commits linked to the same ticket, relative to how often each changes at all. Some change coupling is expected and harmless. A class and its unit tests should change together. But coupling between parts of the system that are supposed to be independent is a direct measurement of hidden debt. It reveals that a business concept has been smeared across modules, that a supposedly separate component depends on the internals of another, or that copy-and-paste has created parallel implementations that must be kept in step by hand. For anyone planning a migration from a monolith to independent services, change coupling is the single most important analysis to run, and it should be run before any decomposition is designed. A proposed service boundary that cuts through a cluster of strongly coupled files will produce two services that must always be changed and deployed together. That is worse than the monolith, because the coupling is now across a network, with versioning, latency and partial failure added to the original problem. Conversely, clusters of files that change together but rarely change with anything else are natural candidates for extraction, because the boundary already exists in how the system is actually maintained. The third analysis uses the authorship information in version control. For each module, it asks who has written most of the code, and whether those people are still in the organisation. Two patterns matter. The first is concentration: a module where one person has written most of the recent changes. This is sometimes called a low bus factor, after the grim thought experiment of how many people would need to be hit by a bus before the project stalled. Concentration is not always bad, since a module may be small and its owner diligent, but when it coincides with a hotspot, the organisation has a single point of failure in the most active part of its system. The second pattern is abandonment: a module whose principal authors have left. That code may still run perfectly well, but every change to it is now made by someone reasoning from the code alone, without the context that explains its stranger decisions. In practice, abandoned code that sits in a hotspot is often where the worst incidents originate. Knowledge risk is invisible to static analysis and often invisible to management, because the people who hold the knowledge are usually the strongest engineers and appear, from the outside, to be performing well. The interest they are absorbing shows up as their own overload: constant interruptions, involvement in every incident, inability to take leave without something going wrong. A leader who sees this pattern in a senior engineer is often looking at a hotspot from the human side. These analyses are not difficult to run, and both commercial tools and open-source scripts exist for them, but a few practical choices determine whether the results are trustworthy. The first is the time window. History from ten years ago reflects a system and a team that no longer exist. A window of the most recent twelve months is usually a good default, long enough to smooth out the effects of a single project and short enough to reflect current patterns. Comparing two windows, such as this year against last, is also revealing, because it shows whether a hotspot is cooling or heating. The second is noise. Generated files, vendored third-party code, lock files, translations and large automated reformatting commits all distort the counts, and should be excluded. File renames and moves need to be followed, or a module that was reorganised last year will appear to have no history. Very large commits that touch hundreds of files, such as a dependency upgrade or a licence header change, should usually be filtered out of change coupling analysis, since they create spurious links between unrelated files. The third is granularity. File-level hotspots are a good start, but in a legacy monolith the worst offenders are often enormous files, several thousand lines long, where change concentrates in a few functions. Tools that analyse history at the level of functions or methods can show that a two-thousand-line class has three methods that account for most of its changes, which shrinks the estimated cost of fixing it considerably. The fourth is the unit of work. Commits are an imperfect proxy for tasks, since engineers differ in how often they commit. Grouping commits by the ticket identifier in their messages, where the team follows that convention, gives a better picture of which files are involved in each piece of work and how long that work takes from first commit to deployment. None of these adjustments requires specialist expertise, but someone needs to own them. A common pattern is for a senior engineer to run the first analysis in a few days, review the output with the teams, and then set it up to run regularly so that trends can be followed over time. Delivery metrics as symptoms Version control analysis locates debt within the code. Delivery metrics show its aggregate effect on the organisation, and they are the measures that will eventually be used to judge whether paying down debt has worked. The DevOps Research and Assessment programme, whose findings were summarised by Nicole Forsgren, Jez Humble and Gene Kim in Accelerate (2018) and in a series of annual State of DevOps reports, identified four measures of software delivery performance that correlate with organisational outcomes: deployment frequency, lead time for changes (from commit to production), change failure rate (the proportion of deployments that cause a failure in production) and the time taken to restore service after a failure. A central and initially surprising finding of that research was that speed and stability are not opposed. The organisations that deploy most often also tend to have the lowest failure rates and fastest recovery, because small, frequent changes are easier to test, review and roll back than large, infrequent ones. For a leader managing a legacy monolith, these four measures serve as a diagnostic and as a baseline. A monolith with heavy debt typically shows long lead times, infrequent deployments and a high change failure rate, often alongside a slow recovery time because diagnosis is difficult. If a modernisation programme does not move these numbers over time, it is not succeeding, regardless of how many services have been extracted or how much the codebase has been tidied. The measures can be broken down further to connect symptoms back to specific parts of the system. Lead time measured per team or per module, rather than for the organisation as a whole, often shows that most of the delay comes from changes that touch a few specific areas. Incident post-mortems, tagged by the component at fault, show where failures originate. When these breakdowns agree with the hotspot analysis, the case for investment in those areas is very strong. These sources of evidence, taken together, give a far more reliable view of where debt charges interest than any single method. Table 1 summarises the main forms of interest, the evidence that reveals each one, and what a leader should look for. Table 1. Forms of technical debt interest and how to detect them. Form of interest Primary evidence Signal to look for Typical owner of the fix Slow, unpredictable change Hotspot analysis; per-module lead time Complex files with high change frequency Owning product team Escaped defects Defect tickets linked to files; incident tags Defect density far above codebase average Owning team with testing support Hidden coupling Change coupling in version history Files in different modules changing together Architecture or platform group Knowledge concentration Authorship history; on-call records One author or departed authors in a hotspot Engineering management Operational toil Deployment logs; incident timelines Manual steps, long recovery, noisy alerts Platform or operations team Platform risk Dependency inventory; vendor support dates Unsupported runtimes, unpatched libraries Platform team with security A note of caution belongs here. Any measure that becomes a target can be gamed, and code metrics are particularly easy to game. A team told to reduce cyclomatic complexity can do so by splitting functions mechanically without making the code any easier to understand. A team told to raise test coverage can write tests that execute code without asserting anything meaningful. The analyses in this chapter are for finding where to look, not for setting goals. The goals should be expressed in outcomes the business cares about: faster, more predictable delivery and fewer incidents in the areas that matter. Code-level measures are evidence about whether the work is on track, not the purpose of the work. Engineers' judgement still matters None of this replaces the knowledge of experienced engineers. Data from version control shows where change has happened, not where it is about to happen; it cannot know that the roadmap includes a major pricing overhaul next year that will put a dormant module under heavy load. It cannot distinguish code that is complex because the domain is complex from code that is complex because it is badly designed. And it cannot see debt in places that are not in version control: configuration held in production databases, scripts on a shared server, knowledge in people's heads. The most effective approach combines the two. Run the analyses, then take the results to the engineers who work in the system and ask them to explain what they see. Why does this file change so often? Why do these two modules always change together? Who knows how this part works? The answers turn a statistical picture into an understanding of mechanism, and they almost always surface a few things the data missed. They also have an important side effect. Engineers who have spent years trying to explain why a particular area is painful usually find it validating to see the pain measured, and they become strong allies in making the case for investment. The output of this work is not yet a plan. It is a short list of places where the system is charging significant interest, each supported by evidence. The next step is to set those places against what the business intends to do, price them, and decide what to do about each, which is the subject of the following chapter. Chapter 3: Pricing the Debt and Choosing What to Do A list of hotspots is evidence, not a decision. Turning it into a programme of work requires three further steps: setting each item against what the business plans to do, estimating roughly what the interest and principal are, and assigning each item a disposition. The last step is the one most often skipped. Teams tend to assume that every identified debt item should eventually be paid down, which turns the debt register into an ever-growing backlog that nobody believes will ever be cleared. In practice, most debt should not be paid down at all. Some should be contained, some accepted, and some retired along with the feature it supports. Deciding which is which is the core of debt management. The roadmap overlay The single most useful exercise at this stage is to place the evidence from the previous chapter alongside the next twelve to eighteen months of the product roadmap. For each significant roadmap item, ask which parts of the system it will require changing, and how heavily. For each hotspot, ask which roadmap items will touch it. This overlay sorts debt quickly. Some hotspots will sit directly in the path of several major initiatives; these are the urgent ones, because their interest is about to rise sharply. Some will sit in areas the roadmap does not touch; their interest may already be significant but will not grow, and they may be candidates for containment rather than repayment. And some roadmap items will turn out to depend on modules that the hotspot analysis did not flag, because they have been stable; these are latent debts whose interest has been dormant and is about to be called in. The overlay also changes the conversation with product leadership. A list of engineering complaints is easy to deprioritise. A statement that three of the next five planned initiatives run through the same tangled module, which currently doubles the time of every change made to it, is a planning fact that affects product's own commitments. Product managers who have repeatedly seen their estimates blown by "unexpected complexity" in a particular area are often the quickest to see the value of fixing it, once the connection is made explicit. Estimating interest and the cost of delay Leaders are often reluctant to put numbers on technical debt because any figure will be uncertain. But a decision is being made either way, and a decision made with a rough estimate is better than one made with none. The aim is to be approximately right rather than precisely wrong. Interest can be estimated from several angles, which should be triangulated rather than relied on individually. The first is delivery time: compare lead time or cycle time for changes in the hotspot with changes elsewhere in the system of similar scope. The ratio indicates the drag. If changes in the order module take on average twice as long as comparable changes elsewhere, and the team spends a known amount of time in that module each quarter, half of that time is a reasonable first estimate of the interest. The second is defect cost: count incidents and defects originating in the area, and estimate the engineering time, customer impact and any direct financial cost they caused. The third is opportunity cost: identify work that has been deferred, descoped or rejected because it would have required changing the area, which is often the largest cost and the hardest to quantify. The fourth is people cost: the time senior engineers spend on support, explanation and incident response for the area, and any attrition attributable to working in it. Principal is estimated in the usual way that engineering work is estimated, with the usual unreliability. Two adjustments help. First, estimate the principal for the smallest change that would materially reduce the interest, not for a complete clean-up. Restructuring the three functions through which most changes flow may deliver most of the benefit of rewriting the whole module at a fraction of the cost. Second, include the cost of establishing safety: the characterisation tests, monitoring and rollback mechanisms needed before restructuring can proceed. This work is frequently omitted from estimates and is frequently the bulk of the effort. With rough figures for interest per quarter and for principal, the payback period follows by simple division. Items that pay back within two or three quarters are almost always worth doing. Items that pay back over several years are rarely worth doing for their own sake, though they may be worth doing for other reasons, such as risk. Interest measured as engineering time understates the cost of debt in one important respect. When debt slows the delivery of a feature that generates revenue or saves cost, the business loses not only the extra engineering time but also the value the feature would have produced during the delay. Donald Reinertsen made this concept central to his book The Principles of Product Development Flow (2009), arguing that cost of delay is the single most important economic variable in product development and the one that organisations most often fail to quantify. Applied to technical debt, it changes the arithmetic considerably. Suppose a new product line is expected to generate a certain revenue per month once launched, and the debt in the catalogue module adds two months to the launch. The cost of that debt, for this initiative alone, includes two months of that revenue, which may dwarf the engineering cost of fixing the module. Cost of delay is also a way to rank debt items against features on the same scale, which is essential if debt work is to compete fairly for capacity. Methods such as weighted shortest job first, which divides cost of delay by job duration to set priority, can be applied to debt reduction as readily as to features, provided that the cost of delay of a debt item is expressed as the acceleration it gives to the features that depend on it. Product leaders who already use such methods will find this framing natural. Those who do not may still find it more persuasive than an engineering argument, because it is expressed in the business's own terms. Dispositions and the shape of the work Once interest, principal and roadmap relevance are roughly understood, each significant debt item should be assigned one of four dispositions. Table 2 summarises them; the discussion below explains when each applies. Table 2. Dispositions for an identified technical debt item. Disposition When it fits What the team does Main risk Pay down High interest, roadmap runs through it, payback within a few quarters Restructure, add tests, or extract, as part of planned work Scope expands into a rewrite Contain Moderate interest, little roadmap pressure, high principal Wrap behind a stable interface; stop further spread Wrapper becomes a second legacy layer Accept Low interest, rarely changed, stable behaviour Record it, monitor it, leave it alone Roadmap changes and interest rises unnoticed Retire Supports a feature with low business value Remove the feature and its code Hidden users or dependencies surface late To pay down a debt is to restructure the code so that its interest falls. This is the disposition people usually have in mind, and it should be reserved for items where the interest is high and the roadmap ensures it will remain high. The best way to pay down debt is usually to do it as part of the feature work that needs it, following Kent Beck's principle of making the change easy before making the change. Paying down debt in isolation, without a specific near-term change it enables, is harder to justify and easier to over-scope. To contain a debt is to stop it from spreading and from leaking into new work, without fixing it. The usual technique is to put a clean interface in front of the messy code and require all new callers to use the interface. The internals remain as bad as before, but their badness is confined, and new code is written against a model that reflects current understanding. Containment is often the right first step for a large, complex module that is too expensive to restructure now but is accumulating new dependents. It is also the first step of the Strangler Fig pattern, discussed in later chapters, since a well-placed interface becomes the seam at which the old implementation can eventually be replaced. The risk is that the interface itself becomes a layer that nobody maintains, adding a translation step without ever leading to replacement. To accept a debt is to make a deliberate, recorded decision to leave it alone. This sounds like inaction, and in effect it is, but the deliberateness matters. Accepted debt should be written down, with the reasoning, so that the decision can be revisited when circumstances change, and so that the team stops spending energy arguing about it. Most code in a legacy monolith falls into this category, and acknowledging that openly is liberating for teams who have felt obliged to apologise for every ugly corner of the system. To retire a debt is to delete the code along with the functionality it supports. This is the most underused disposition and often the most effective. Legacy systems accumulate features that were built for a customer who left, a market that was abandoned, a regulation that was repealed, or an experiment that was never cleaned up. These features still impose costs: they must be kept working when surrounding code changes, they complicate testing, and they create paths through the system that nobody fully understands. Usage data, gathered through instrumentation if it does not already exist, usually shows that a meaningful fraction of a legacy system's features are rarely or never used. Removing them reduces the surface area of every subsequent change. Retirement also carries a particular lesson about dead code. In August 2012, Knight Capital, then one of the largest equity traders in the United States, lost more than 460 million dollars in roughly 45 minutes after markets opened. The Securities and Exchange Commission's subsequent findings described how Knight had left unused, defective code in its order router for years; a new deployment for a fresh trading programme was not copied to all of its servers, and the dormant function was reactivated on one of them, sending more than four million orders into the market while attempting to fill 212 customer orders. The SEC fined Knight 12 million dollars; the firm itself did not survive as an independent company. Unused code is not harmless simply because it is unused. Code that is still deployed but not needed is a latent liability, and removing it is among the cheapest risk reductions available. For items assigned to pay down, a further choice determines the shape of the work: whether to restructure the code in place, extract it into a separate component, or replace it with something new. Refactoring in place is the least disruptive and usually the right starting point. It changes the internal structure of code without changing its behaviour, in small, verifiable steps, while the code remains in the same deployable unit. It is appropriate when the main problem is the internal design of the module and when there is no strong reason for the module to be deployed, scaled or owned independently. Done well, it is invisible to everyone outside the team. Extraction moves a part of the system out of the monolith into a separately deployable component, usually a service with its own data. It is appropriate when the module has a reason to be independent: a different rate of change from the rest of the system, a different scaling profile, a different team that needs to own it without coordinating with others, or a need for a different technology. Extraction is substantially more expensive and risky than refactoring, because it introduces network boundaries, data separation and operational overhead. It should be chosen for those reasons, not because extraction is the fashionable form of modernisation. Replacement substitutes a different implementation, often a commercial product or a new build, for an existing part of the system. It is appropriate when the existing implementation encodes a capability that is not distinctive to the business and that the market now provides well, such as authentication, payments processing, search or notification delivery. It is also, at its extreme, the rewrite, and it carries the rewrite's risks, which are the subject of the next chapter. Many organisations default to extraction because they have decided in advance that the end state is microservices. A better order of preference is to refactor where refactoring suffices, extract where there is a clear reason for independence, and replace where the capability is commodity. This order minimises the risk taken on for each unit of interest removed, and it keeps the organisation's attention on the purpose of the work, which is reducing interest on the roadmap, rather than on achieving a particular architecture. One practical test helps when the disposition of an item is contested. Ask what would have to be true for the item to move from accept to pay down, and write that condition into the register. It might be a roadmap initiative that touches the area, a defect rate above a threshold, or the departure of the last person who understands it. A condition of this kind turns an argument about whether code is bad enough to fix into an agreement about what evidence would change the decision, and it means that when the condition is met, nobody needs to reopen the debate from the beginning. Keeping the register alive The output of this chapter's process is a debt register: a short document, ideally no more than a page or two for the items that matter, recording each significant item, its evidence, its estimated interest and principal, its disposition and its owner. It should be reviewed whenever the roadmap changes substantially and at least quarterly otherwise, because both the interest and the right disposition shift as plans shift. A useful register entry is short. Consider, as an illustration, an entry for the shipping charges module described in Chapter 1. Its evidence section would note that the module is the second most frequently changed in the codebase over the past year, that its cycle time is about double the system median, and that it was the source of four production incidents in that period. Its roadmap section would note that two planned initiatives, the addition of international carriers and a new subscription delivery offer, run directly through it. Its estimate section would give the interest as roughly two engineer-days per change plus incident costs, and the principal as about six engineer-weeks for isolating the rules and adding tests. Its disposition would be pay down, bundled with the first of the two initiatives, owned by the fulfilment team, with success defined as cycle time in the module falling to the system median within two quarters. An accepted item next to it, such as the regulatory report generator, would carry a single paragraph explaining why it is being left alone and what would cause that decision to be reopened. That level of detail is enough to support a decision and to check later whether the decision was right. More detail rarely helps, and a register that takes days to update will not be updated. The register should be visible to product and business leadership, not hidden in an engineering wiki. Its value comes partly from the decisions it records and partly from the shared understanding it creates. When everyone can see that a particular module has been deliberately accepted as it is, nobody is surprised when work in that module takes longer than elsewhere. When everyone can see that another module has been scheduled for repayment ahead of a major initiative, the investment has a clear and visible purpose. The register is, in effect, the balance sheet of the system's technical liabilities, and like any balance sheet it is useful only if the people making decisions actually read it. Chapter 4: Why Rewrites Fail and What the Destination Should Be When a legacy system has become painful enough, someone will propose replacing it entirely. The proposal is attractive for understandable reasons. Engineers would rather build something new than repair something old. Executives like the clarity of a single project with a start, an end and a new system at the finish. Vendors and consultancies are happy to sell both. And the existing system's flaws are visible in detail, while the new system's flaws do not yet exist, which makes the comparison look lopsided in the new system's favour. The record of large rewrites is poor enough that the burden of proof should sit heavily on anyone proposing one. Understanding why they fail is also the best preparation for designing a migration that does not, and for setting a destination that is justified by outcomes rather than by fashion. Why rewrites fail Joel Spolsky's essay "Things You Should Never Do, Part I," published in April 2000, is still the most widely cited argument against rewrites. Writing about Netscape's decision to rewrite its browser from scratch, he called it "the single worst strategic mistake that any software company can make." The rewrite took years, during which Netscape shipped no major new version while Microsoft's Internet Explorer gained ground. Spolsky's central argument was that old code, however ugly, embodies accumulated knowledge. Each odd-looking conditional or special case is, more often than not, a fix for a bug someone encountered in the real world: a particular operating system version, a malformed input from a partner, a race condition that appeared only under load. When the code is thrown away, that knowledge is thrown away with it, and the new system must rediscover each case the hard way, usually in production. Fred Brooks identified a related danger decades earlier in The Mythical Man-Month (1975), which he called the second-system effect. A team building the successor to a system it knows well tends to over-design it, adding every feature and generalisation that was left out of the first version. The second system becomes larger, more ambitious and later than planned. Rewrites are especially prone to this, because the team has a long list of everything wrong with the old system and is tempted to fix all of it at once. Beyond these classic observations, rewrites of large business systems tend to fail for structural reasons that a leader can recognise in advance. The first is that nobody can write down what the old system does. Specifications, if they ever existed, are years out of date. The real specification is the code itself, plus the behaviour that users and downstream systems have come to depend on, including behaviour that was never intended. A rewrite team therefore spends much of its time reverse-engineering requirements from a system it is simultaneously trying to replace, and it discovers the important ones late, often when the new system is put in front of real users or real data. The second is that the target moves. A large rewrite takes long enough that the business keeps changing during it. New features are needed. The organisation faces a painful choice: freeze the old system, which harms the business, or keep adding features to both systems, which doubles the cost and means the new system is chasing a moving target. Most rewrites end up doing the latter, and the gap between the two systems closes more slowly than planned, or not at all. The third is that value arrives only at the end. A rewrite produces nothing usable until it is complete enough to replace the old system, which may be years away. All the risk is concentrated at the cutover, and all the benefit is deferred until after it. If the project is cancelled partway, as large rewrites frequently are when budgets tighten or sponsors change, the organisation has spent heavily and received nothing. The fourth is the cutover itself. Moving every user and every record from one system to another in a single event is extremely risky. In April 2018, the British bank TSB migrated its customers' accounts from the systems of its former parent to a new platform. Many customers were unable to access their accounts or saw incorrect information, and the disruption lasted for weeks. In December 2022, the Financial Conduct Authority and the Prudential Regulation Authority fined TSB a combined total of 48.65 million pounds over failures in managing operational risk around the programme. The details of any particular migration are always specific, but the general lesson is not: a single, large cutover concentrates every unknown into one moment, with no way to learn from a small exposure first. There is also a human cost that rarely appears in rewrite proposals. While a separate team builds the new system, the engineers maintaining the old one are told, in effect, that their work is temporary and their expertise soon obsolete. Some of the people who understand the old system best leave during the rewrite, which removes the knowledge the rewrite most needs. When the rewrite slips, as it usually does, the organisation finds itself depending on an old system whose most capable maintainers have gone. When replacement is right None of this means that replacement is never appropriate. There are cases where it is the right choice, and leaders should be able to recognise them. Replacement makes sense when the capability is a commodity that the market now provides well. Few organisations should maintain their own email delivery, identity management or general ledger if a reliable product exists that fits their needs. In such cases, the replacement is not a rewrite but a substitution, and its main risk is integration and data migration rather than rediscovering business logic. Replacement can also make sense when the old system is small enough that its behaviour can be specified and tested exhaustively, or when the platform it runs on is so obsolete that incremental change is impossible. Even then, the replacement should be delivered incrementally wherever possible, with old and new systems running side by side and users moved in stages. Finally, replacement may be justified when the business itself is changing so fundamentally that the old system's model of the domain no longer applies. A company moving from selling products to selling subscriptions, for example, may find that its order-centric system cannot be bent into the new shape. Even here, though, the safest route is usually to build the new capability alongside the old and move business across gradually, which is the essence of the Strangler Fig approach. The test a leader should apply to any replacement proposal is simple. Can it be delivered in slices, each of which moves some real users or transactions onto the new system and can be reversed if it goes wrong? If yes, it is not really a rewrite in the dangerous sense, and the risks can be managed. If no, if it requires everything to be finished before anything can be used, the proposal deserves extreme scepticism. Engineers who propose a rewrite are rarely naive, and their arguments deserve a fair hearing before they are set aside. Three versions of the argument come up often, and each contains something true. The first is that the existing code is so tangled that incremental change is impossible: every attempt to restructure one part breaks three others, and the team has tried and failed. This is sometimes accurate as a description of the present. But it is usually a description of code without tests and without seams, and the remedy is to create them, starting with characterisation tests at the boundaries of the system where behaviour can be observed from outside. Testing a tangle from the outside is slow and unglamorous work, but it is almost always possible, and it is far less risky than replacing the tangle wholesale. The second is that the technology is obsolete and the organisation cannot hire people to work on it. This can be true, particularly for systems built on languages or platforms that have fallen out of wide use. It is a real cost and should be priced as such. But it argues for moving away from the platform, not for a single replacement event. Incremental migration can move functionality to a modern platform piece by piece, and many organisations find that the pool of engineers willing to work on an old platform is larger than feared once the work is framed as a migration with a clear end. The third argument is newer. Since around 2023, AI coding assistants have become capable of reading, explaining and translating large amounts of legacy code, and some engineers and vendors argue that this makes rewrites cheap enough to change the calculation. The tools are genuinely useful for understanding unfamiliar code, generating characterisation tests, and translating well-bounded components. But they do not remove the structural problems described above. Translating code faster does not tell anyone which of its behaviours are relied upon, does not stop the business from changing during the project, and does not make a single cutover any less risky. If anything, the ability to produce a large volume of new code quickly increases the importance of the verification techniques, such as parallel runs and reconciliation, that establish whether the new code behaves like the old. Used within an incremental approach, these tools can make each step cheaper. Used to justify a big-bang replacement, they speed the arrival at the same cliff edge. Microservices are a means, not an end Many modernisation programmes are framed from the outset as migrations "from monolith to microservices." This framing deserves examination, because microservices are an architecture with substantial costs, and adopting them without a clear reason can create more debt than it removes. The benefits of a microservices architecture are real but specific. Independently deployable services allow teams to release without coordinating with each other. Services can be scaled independently according to their load. Each service can use technology suited to its job. Failures can be isolated so that one component's problem does not take down the whole system. And clear service boundaries can enforce modularity that a monolith's internal boundaries often fail to maintain. The costs are equally specific. Every call between services is a network call, which can fail, time out or be slow in ways that an in-process call cannot. Data that was in one database is now spread across many, which makes queries that span services difficult and transactions that span services much harder. Operational overhead multiplies: each service needs its own deployment pipeline, monitoring, logging, alerting and on-call coverage. Debugging a problem that spans several services requires distributed tracing and considerable skill. And versioning becomes a permanent concern, since services must keep their interfaces compatible as they evolve independently. Two public cases illustrate that these costs can outweigh the benefits. In 2018, the engineering team at Segment, a customer data company, published an account titled "Goodbye Microservices," describing how they had split their system for delivering data to third-party destinations into more than a hundred separate services, one per destination, and then consolidated them back into a single service because the operational burden had become unmanageable for the size of team. In 2023, Amazon's Prime Video engineering team described how they had rebuilt a video quality monitoring tool from a distributed, serverless design into a single application, reducing the infrastructure cost of that component by over 90 percent. Neither case shows that microservices are a mistake in general. Both show that the decision depends on the specifics, and that the specifics sometimes favour a single deployable unit. Between a tangled monolith and a fleet of microservices lies an option that is often overlooked: the modular monolith. This is a single deployable application whose internal structure is divided into well-defined modules with explicit boundaries and controlled dependencies. Modules communicate through defined interfaces rather than by reaching into each other's internals or sharing database tables freely. The application is deployed as one unit, so there are no network calls between modules and no distributed data problems, but the code is organised so that teams can work on separate modules with limited interference. Shopify, which runs one of the largest Ruby on Rails applications in existence, publicly described this approach in a 2020 engineering post titled "Deconstructing the Monolith." Rather than splitting into microservices, it reorganised its codebase into components with enforced boundaries and released an open-source tool, Packwerk, to detect violations of those boundaries automatically. The rationale was that most of the benefit it wanted, namely teams able to work independently with clear ownership, could be achieved through modularity without paying the operational costs of distribution. For many organisations, a modular monolith is the right destination, or at least the right intermediate step. It addresses the most common source of monolith pain, which is tangled internal dependencies, without introducing network boundaries. And it makes a later extraction to services much easier if one becomes justified, because the module boundaries have already been established and tested. Extracting a well-bounded module from a modular monolith is a far smaller undertaking than carving a service out of a tangle. The choice between these destinations is also reversible in a way that is easy to forget. A module that has been cleanly bounded inside a monolith can be extracted later if independent deployment or scaling becomes necessary. A service that has been extracted prematurely can be folded back in, as the Segment and Prime Video teams showed, but only at the cost of a second migration. When in doubt, the cheaper error is to keep a well-bounded module inside the monolith and wait for evidence that it needs to leave. Evidence of that kind tends to be specific and hard to miss: a team repeatedly blocked by other teams' release schedules, a component whose load profile forces the whole application to be scaled for its sake, or a regulatory requirement to isolate certain data. Without such evidence, the case for extraction is usually aesthetic. Stating the destination in outcomes The most important point of this chapter is that a modernisation programme's destination should be stated in terms of outcomes, not architecture. "Migrate to microservices" is an architectural goal; it can be achieved while every outcome the business cares about gets worse. "Reduce lead time for changes to the checkout flow from six weeks to one week, while keeping the change failure rate below its current level" is an outcome goal; it can be achieved with or without microservices, and it is the thing that actually matters. Outcome goals have several advantages. They make it possible to tell whether the programme is succeeding, since progress is measured by the outcome rather than by the number of services created. They allow the architecture to be chosen part by part according to what achieves the outcome, so that one area might be extracted as a service while another is simply refactored in place. They make it natural to stop when the outcome is achieved, rather than continuing until some architectural vision is complete. And they connect the programme directly to the business case, which makes it far easier to sustain funding and support. A useful discipline is to require that any architectural decision in the programme be justified by its contribution to an outcome. Extracting a service should be justified by a specific need for independent deployment, scaling, ownership or technology, tied to the outcome the programme is pursuing. If no such justification exists, the module should stay in the monolith, refactored as necessary. This discipline prevents the programme from drifting into architecture for its own sake, which is one of the most common ways that modernisation programmes run over budget and under-deliver. With the destination stated in outcomes, and with the dangers of a big-bang replacement understood, the question becomes how to move from the current system to a better one without stopping the business. That is the work of the Strangler Fig pattern. Chapter 5: The Strangler Fig Pattern in Practice In 2001, on holiday in the rainforests of Queensland, Martin Fowler noticed the strangler figs. These plants germinate in a nook high in a host tree, send roots down to the ground, and gradually grow around the host until they can stand on their own; in time the host may die and decay, leaving the fig standing in its place. A couple of years later Fowler wrote a short essay using the image as a metaphor for replacing legacy software, calling it the Strangler Application. He later retitled it "Strangler Fig Application," partly to emphasise the botanical origin and to soften the violent connotations of the word "strangler." The idea is simple. Rather than replacing a legacy system in one step, build new functionality around the edges of the old system, gradually route more of the work to the new parts, and eventually retire the old system when nothing depends on it any longer. At every stage, the combined system works and is in production. Fowler's essay contrasts this with the simple replacement plan, which he says his colleagues had seen "go down in flames most of the time." The pattern is easy to describe and demanding to execute. This chapter covers the mechanics: how to intercept traffic, how to choose what to move first, and the supporting techniques that make each step safe and reversible. The routing facade and the first seam The core mechanism of the Strangler Fig pattern is interception. Something must sit in the path of requests to the legacy system and decide, for each request, whether the legacy system or a new component should handle it. Initially, everything goes to the legacy system. As new components are built, the facade routes the relevant requests to them. Eventually, nothing goes to the legacy system, and it can be turned off. For systems accessed over HTTP, the facade is usually a reverse proxy or API gateway placed in front of the monolith. Routing rules can be based on the URL path, the HTTP method, headers or other attributes of the request. A rule might send all requests for product search to a new search service while everything else continues to go to the monolith. Because the facade is a separate, simple component, routing changes can be made and reversed quickly, without deploying the monolith or the new service. For systems that communicate through messages or events, the equivalent is to intercept at the message level. A new component can subscribe to the same queue or topic as the legacy system and gradually take over processing for certain message types, or a router can be inserted that forwards messages to old or new handlers according to rules. For user interfaces, interception can happen in the front end. A page or a section of a page can be served by a new component while the rest continues to come from the legacy application. Sam Newman, in Monolith to Microservices (2019), describes this as UI composition and discusses several variants, from page-level routing to embedding new widgets in old pages. The facade is also the first place to invest in observability. Because every request passes through it, it is the ideal point to record which routes are used, how often, by whom, with what latency and with what error rate. This data shows which parts of the legacy system are heavily used and which are rarely touched, which directly informs what to migrate first and what may be retired outright. It also provides a baseline against which to compare the behaviour of new components as traffic shifts to them. Michael Feathers introduced the term "seam" for a place in a program where behaviour can be changed without editing the code at that place. In migration, a seam is a boundary at which part of the system can be redirected to a new implementation. Choosing the first seam is one of the most consequential decisions in a Strangler Fig programme, because the first extraction establishes the patterns, tooling and confidence on which everything else will depend. The ideal first candidate has several properties. It should be reasonably self-contained, with few dependencies on the rest of the monolith and few parts of the monolith depending on it; the change coupling analysis from Chapter 2 is the best guide here. It should be valuable enough that moving it produces a visible benefit, so that the programme earns credibility early. It should not be the most critical or most complex part of the system, because the team will be learning as it goes and will make mistakes. And it should have a clear interface, ideally one that already exists in the form of a set of URLs, a message type, or a well-defined function boundary. These properties pull in different directions. The most valuable candidates are often the most entangled, and the least entangled are often the least valuable. A common and sensible compromise is to choose a first candidate that is modest in value but low in risk, with the explicit purpose of building the delivery pipeline, monitoring, deployment patterns and team skills that later extractions will need. The second or third candidate can then be one that delivers real business value, extracted with tooling and experience already in place. A frequent mistake is to start with a cross-cutting capability such as authentication, logging or user management, on the reasoning that everything else depends on it and so it should come first. Precisely because everything depends on it, it is one of the hardest things to extract, and getting it wrong affects the whole system. It is usually better to leave such capabilities in the monolith, accessed by new services through it, until the team has experience with simpler extractions. Domain boundaries and internal seams Beyond the first seam, the overall decomposition of the monolith determines whether the resulting system is easier or harder to change than the one it replaced. The most widely used guide to drawing these boundaries comes from Eric Evans's Domain-Driven Design (2003), and in particular from the concept of the bounded context. A bounded context is a part of the business domain within which a particular model and vocabulary apply consistently. The same word may mean different things in different contexts: a "customer" in the sales context may be a prospect with contact details and a pipeline stage, while a "customer" in the billing context is an account with payment methods and invoices. A legacy monolith often has a single, overloaded customer table trying to serve every context, with columns and logic for all of them tangled together. Recognising that these are different concepts, with different owners and different rates of change, is the first step towards separating them. Bounded contexts are good candidates for service boundaries because, when drawn well, they minimise the need for communication across boundaries. Most changes to business rules stay within one context. The contexts communicate through well-defined interactions, often events such as "order placed" or "payment received," rather than through shared data structures. In practice, the bounded contexts of a legacy system are rarely obvious from its code, which may have been structured around technical layers rather than business concepts. Workshops with domain experts, product managers and experienced engineers are usually needed to identify them, and the change coupling analysis provides a useful check: if the proposed contexts correspond to clusters of files that change together, the boundaries are probably in the right place. The routing facade works well when the part being migrated is reached from outside the system, through an API or a user interface. But many important parts of a monolith are called from inside it. The pricing logic, for example, may be invoked from dozens of places within the order flow. No external proxy can intercept those calls. The technique for this situation is called branch by abstraction, a name associated with Paul Hammant and described by Fowler on his website. It proceeds in steps. First, introduce an abstraction, such as an interface, in front of the existing implementation, and change all callers to use the abstraction rather than the implementation directly. This step changes no behaviour. Second, build a new implementation of the same abstraction, which may call out to a new service. Third, switch callers from the old implementation to the new one, gradually and reversibly. Fourth, once all callers use the new implementation and it has proved itself, remove the old implementation, and if it no longer adds value, the abstraction too. The essential feature of branch by abstraction is that every step leaves the system working and releasable. There is no long-lived branch in version control where a new implementation is developed in isolation while the main line moves on. Instead, both implementations coexist in the main line, and which one is used is controlled at run time. This avoids the painful merges that long-lived branches cause and allows the migration to proceed in parallel with ordinary feature work. Run-time control over which implementation handles a request is usually provided by feature toggles, also called feature flags. Pete Hodgson's article "Feature Toggles (aka Feature Flags)," published on Fowler's website in 2017, gives a thorough account of the different categories of toggle and their management. In a migration, toggles allow traffic to be moved to the new implementation progressively. A toggle might first route only internal users to the new path, then one percent of customers, then ten percent, then everyone. At each stage, the team monitors error rates, latency and business metrics, and if anything looks wrong, flips the toggle back. This turns each migration step from a high-stakes event into a controlled experiment with a fast rollback. Toggles carry their own debt. Each toggle adds a branch to the code and to the testing matrix. Toggles that are never removed accumulate into a confusing tangle of conditional behaviour, and the Knight Capital incident discussed in Chapter 3 is a reminder of what can happen when an old flag's meaning is repurposed. A disciplined migration treats every migration toggle as temporary, records when it should be removed, and removes it promptly once the migration step is complete. The techniques described in this chapter, together with the data-oriented techniques discussed in the next, form a toolkit from which the right combination is chosen for each extraction. Table 3 summarises the main ones. Table 3. Core techniques for incremental migration from a monolith. Technique What it intercepts Best suited to Key prerequisite Routing facade External requests (HTTP, API) Functions reached from outside the system Clear URL or API boundaries UI composition Pages or page fragments User-facing features Front end that can mix sources Event interception Messages on queues or topics Asynchronous processing Existing message-based integration Branch by abstraction Internal function calls Logic called from many places inside the monolith Ability to introduce an interface Feature toggles Run-time path selection Progressive rollout and fast rollback Monitoring to compare old and new A sequence, step by step Seeing the techniques in sequence makes the pattern easier to judge. Consider an illustrative online retailer whose monolith handles catalogue, search, cart, checkout, payments, fulfilment and customer accounts, and whose hotspot analysis shows that search and the product catalogue are changed constantly as merchandising teams ask for new filters and ranking rules. The first step is to put a reverse proxy in front of the monolith and route everything through it unchanged. Nothing visible happens, but the team now has a single place to observe traffic and change routing, and it has proved that the proxy adds no meaningful latency or failure. The second step is to build a search service that reads product data through change data capture from the monolith's catalogue tables and maintains its own index. The service runs in production but receives no traffic. The team compares its results with the monolith's for a sample of real queries and investigates differences. The third step is to route search requests from internal staff to the new service through the proxy, then a small percentage of customer traffic, then all of it, watching conversion and error rates at each stage. The monolith's search code is still present, so rolling back is a configuration change. The fourth step, once the new service has run stably for some weeks, is to delete the monolith's search code and the toggle that selected it. The proxy rule becomes permanent. The fifth step tackles the catalogue itself, which is harder because many parts of the monolith read catalogue data internally. The team introduces an abstraction for catalogue access inside the monolith, moves every internal caller onto it, and then gives the abstraction a second implementation that calls a new catalogue service. Callers are switched in groups, and the catalogue tables are eventually owned by the new service, with the monolith reading through its interface. At no point in this sequence was the retailer unable to ship features, and at every point a mistake could be reversed. Merchandising teams began to see faster changes to search within the first few months, long before the catalogue work was finished, which secured support for the harder steps that followed. The sequence also shows where judgement was exercised. The team chose search before catalogue because search had a clear external interface and could be fed data without changing the monolith, which made it a safe first seam. It used change data capture rather than modifying the monolith to publish events, because the catalogue code was a hotspot that nobody wanted to touch more than necessary. And it deleted the old search code promptly, rather than leaving it as a fallback indefinitely, because a fallback that is never exercised quietly stops working and creates false confidence. Each of these choices could reasonably have gone the other way in a different system. What matters is that they were made deliberately, with the reversibility of each step in mind. Keeping the monolith alive while it shrinks A Strangler Fig programme can take years, and for all of that time the monolith remains in production, supporting the business. This raises a practical question that is easy to neglect: how much should the team invest in the monolith while it is being replaced? The temptation is to freeze it, accepting no changes on the grounds that it will soon be gone. This is almost always a mistake. The business will continue to need changes, and a frozen monolith either blocks them or forces them into new services prematurely, before the boundaries are understood. A better rule is to keep the monolith healthy enough to change safely in the areas still being changed, while avoiding investment in areas scheduled for early extraction. Characterisation tests, monitoring and deployment improvements are rarely wasted, because they make both continued maintenance and eventual extraction safer. A related discipline is to stop adding new capabilities to the monolith once a clear alternative exists. If a new search service has been established, new search features should be built there, not in the monolith, even if it would be quicker to add them to the old code in the short term. Each new feature built in the monolith extends the work needed to retire it. Finally, the end of the pattern needs attention. The aim is not merely to route traffic away from the monolith but to switch it off. Systems that have been "almost replaced" for years, with a remnant of the monolith still running because a few functions were never migrated, carry the costs of both architectures: the operational overhead of the old system and the complexity of the new one. Planning explicitly for the last stage, including deciding which rarely used functions can be retired rather than migrated, is what distinguishes a successful Strangler Fig programme from one that leaves two systems where there used to be one. Every one of these steps is complicated by the question that the routing facade cannot answer on its own: where the data lives, and how it moves. That is the subject of the next chapter. Chapter 6: Data, the Hardest Part of Leaving Code can be moved one function at a time. Data resists this. In most legacy monoliths, a single relational database sits at the centre of everything, and its tables are read and written by code from every part of the application. The order table is joined to the customer table, which is joined to the address table, which is updated by a nightly batch job, which also writes to a reporting table that the finance team queries directly with spreadsheets nobody in engineering knows about. The database is the real integration layer of the system, and it is usually where the tightest and least visible coupling lives. This is why data migration takes the longest and goes wrong most expensively in almost every modernisation programme. Extracting the code for a service is often a matter of weeks. Untangling the data that service depends on, so that it can own its data rather than reaching into a shared database, can take many months. A leader who plans a migration based on the code alone will underestimate it badly. Why shared databases defeat independence The promise of extracting services from a monolith is independence: teams can change and deploy their services without coordinating with others. A shared database quietly breaks this promise. If two services read and write the same tables, a change to the structure of those tables requires both services to change in step. A slow query from one service can degrade the performance of the other. A bug in one service can corrupt data the other relies on. The services are separately deployed but not independent, and they combine the coordination costs of a monolith with the operational costs of distribution. For this reason, the general principle in service architecture is that each service should own its data, and other services should access that data only through the owning service's interface. Sam Newman argues this throughout Building Microservices and Monolith to Microservices, and it is one of the few points on which practitioners broadly agree. Applying the principle to a legacy monolith, however, is hard. The data for a given business capability is rarely confined to a set of tables that only that capability uses. It is shared, joined and referenced across the schema. Extracting it requires identifying every piece of code that reads or writes it, changing that code to go through an interface instead, and only then moving the data itself. There are two broad ways to sequence the extraction of a service and its data. The first is to extract the code first while leaving the data in the shared database. The new service is deployed separately but continues to read and write the monolith's tables. This gets the code out quickly and allows the team to gain experience operating the new service. But it leaves the service coupled to the monolith through the database, and the data extraction still has to be done later. It is a reasonable first step if it is clearly understood as temporary. The second is to separate the data first while leaving the code in the monolith. The monolith's code is reorganised so that all access to the relevant data goes through a single module, and that module is given its own schema or database. Only once the data is isolated is the code extracted into a service. This takes longer to show visible progress but tackles the harder problem first and avoids a period in which a separately deployed service shares a database with the monolith. Neither approach is always right. The choice depends on how entangled the data is, how urgently the team needs to show progress, and how comfortable it is operating services that share a database. What matters is that the sequence is chosen deliberately and that the temporary states are recognised as temporary. The worst outcome is a programme that extracts code first, declares success, and never extracts the data. Techniques for moving data safely Several techniques allow data to be moved incrementally, without a single cutover. They are usually combined. Expand and contract, sometimes called parallel change, is the basic technique for changing a database schema without breaking the code that uses it. Danilo Sato described it on Fowler's website in 2014. Rather than renaming a column or moving a table in one step, which would break every reader at once, the change is split into phases. First, expand: add the new structure alongside the old one and write to both. Second, migrate: move existing data to the new structure and switch readers to it, one at a time. Third, contract: once nothing reads from the old structure, stop writing to it and remove it. Each phase is independently deployable and reversible. The technique is slower than a direct change but avoids the coordinated big-bang deployment that a direct change would require. Change data capture is a way of propagating changes from one database to another by reading the source database's transaction log, rather than by modifying the application code to publish changes. Tools such as the open-source Debezium project read the log of databases like PostgreSQL or MySQL and emit an event for each insert, update or delete. A new service can subscribe to these events and maintain its own copy of the data it needs, kept in sync with the monolith's database in near real time. This is especially useful when the monolith's code is hard to modify, since it allows data to flow out of the monolith without changing it. The cost is that the events reflect the database's structure, not the business's concepts, so the new service must translate low-level row changes into meaningful business events, and it becomes coupled to the monolith's schema until the monolith is retired. Dual writes, in which the application writes to both the old and new data stores, are a tempting alternative and a notorious source of inconsistency. If one write succeeds and the other fails, the stores disagree, and without careful design there is no reliable way to reconcile them. Where dual writes are used, they should be accompanied by a reconciliation process that detects and repairs discrepancies, and the old store should remain the source of truth until the new one has been proven. A more robust variant is the transactional outbox. Instead of writing to two stores directly, the application writes its data change and a record of the corresponding event to the same database, in the same transaction. A separate process reads the outbox and publishes the events. Because the data change and the event record are committed together, they cannot disagree. Chris Richardson describes this pattern, along with related patterns for data consistency in distributed systems, in Microservices Patterns (2018). The most powerful technique for gaining confidence in a migration is to run the old and new implementations side by side on real traffic and compare their results. Newman calls this a parallel run. In a parallel run, every request is processed by both the old and the new implementation. The old implementation's result is returned to the user, so the new one has no effect on behaviour. But the two results are compared, and every discrepancy is logged. Over days or weeks of real traffic, the team builds a detailed picture of where the new implementation differs from the old. Some differences will be bugs in the new code. Some will be bugs in the old code that the new code has inadvertently fixed, which raises the question of whether anything depends on the old, buggy behaviour. And some will be undocumented behaviour of the old system that nobody knew about, which is exactly the hidden knowledge that Spolsky warned rewrites destroy. GitHub released an open-source library called Scientist to support this kind of comparison for code paths within a Ruby application, and similar libraries exist for other languages. The idea is the same at any scale: run both, return the trusted result, record the differences, and migrate only when the differences are understood and acceptable. For data migrations specifically, the equivalent is reconciliation. After data has been copied or synchronised to a new store, an automated process compares the two, record by record or by aggregate checks such as counts and sums, and reports discrepancies. Reconciliation should run continuously for as long as both stores exist, not just once after the initial migration, because synchronisation processes can drift or fail silently. Parallel runs are expensive. They double the processing load for the migrated path, require infrastructure for comparison and logging, and need someone to investigate discrepancies. But for critical paths such as pricing, payments, entitlements or anything with regulatory consequences, the cost is small compared with the cost of discovering a discrepancy in production after the old system has been switched off. Parallel runs also have a limit worth knowing in advance. They work naturally for operations that compute a result, such as a price, an eligibility decision or a search ranking, because both implementations can compute it and the results can be compared. They are harder for operations with side effects, such as sending an email, charging a card or calling a partner. For these, the new implementation must run in a mode where it records what it would have done without doing it, and the comparison is made between the intended actions. This takes extra design effort, and it is worth planning for at the start rather than discovering partway through. Consistency, hidden consumers and other traps Once data is split across services, operations that used to be a single database transaction become operations across several services, each with its own data. The classic example is placing an order, which might involve reserving inventory, charging a payment method and creating a shipment. In the monolith, these could happen in one transaction that either fully succeeds or fully rolls back. Across services, there is no shared transaction. Distributed transaction protocols such as two-phase commit exist but are generally avoided in service architectures because they couple services tightly and perform poorly. The usual alternative is the saga: a sequence of local transactions, each in one service, coordinated either by a central orchestrator or by services reacting to each other's events. If a step fails, previously completed steps are undone by compensating actions, such as releasing the reserved inventory or refunding the payment. Sagas work, but they are more complex than transactions and they change the semantics of the system. Between steps, the system is in an intermediate state that other operations can observe. A customer might see an order as placed while payment is still being processed, or inventory might appear reserved for an order that is about to fail. Business stakeholders need to understand and accept these intermediate states, which often requires changes to processes and user interfaces, not just to code. This is one of the strongest reasons for caution about extraction. If a business capability depends heavily on transactional consistency with other capabilities, extracting it into a separate service will introduce real complexity and real changes in behaviour. Keeping it in the same deployable unit, perhaps as a well-bounded module in a modular monolith, may be the better choice. One category of data dependency deserves special mention because it is so often discovered late. Legacy databases typically have consumers outside the application: reporting tools, data warehouse extracts, finance spreadsheets, regulatory reports, partner integrations and scripts written by analysts years ago. These consumers read directly from the database, bypassing the application entirely, and they are often unknown to the engineering team. When data moves out of the monolith's database, these consumers break, frequently without anyone noticing until a month-end report is wrong or a regulatory submission fails. Every data migration plan should therefore begin with an inventory of all database consumers, gathered from database access logs, network monitoring and conversations with finance, operations and analytics teams. Each consumer needs a plan: either it is migrated to read from the new service or a data platform, or it is provided with an equivalent feed, or it is retired. A common and effective pattern is to establish a dedicated analytics or reporting store, fed by events or change data capture from both the monolith and the new services, early in the migration. Reporting consumers are moved to this store, which decouples them from the operational databases and makes subsequent data moves invisible to them. This is extra work at the start, but it removes one of the most frequent causes of unpleasant surprises later. Three smaller data problems recur in almost every migration and are worth anticipating. The first is identifiers. Monoliths commonly use database-generated sequential keys, and those keys leak into URLs, emails, partner integrations, customer service scripts and printed documents. A new service that generates its own identifiers must either continue the old sequence, maintain a mapping between old and new identifiers indefinitely, or persuade every consumer to change. The cheapest choice is almost always to preserve existing identifiers for existing records and to decide carefully, early, how new identifiers will be generated so that they cannot collide with old ones while both systems are live. The second is reference data: lists of countries, currencies, tax codes, product categories, status values and the like. In a monolith these live in shared tables that every module reads. When modules become services, each one needs this data, and it is tempting either to copy it into every service, where it drifts, or to create a shared reference data service that every other service calls, which becomes a single point of failure. A common compromise is to give reference data a single owner that publishes changes as events, with each consuming service keeping a read-only local copy. The third is history. Operational systems often hold years of records that are rarely accessed but must be retained for legal, financial or customer service reasons. Migrating all of this history into a new service is expensive and often unnecessary. An alternative is to migrate only active and recent records, and to move older history to an archive store that remains queryable for the purposes that need it. This decision should be made with legal and finance teams, not by engineers alone, since retention obligations vary by jurisdiction and record type. All three problems share a feature: they are discovered by talking to people outside engineering at least as often as by reading code. Customer service teams know which identifiers customers quote on the telephone. Finance teams know which reference values drive their reports. Legal and compliance teams know which records must be kept and for how long. A data migration plan drawn up without these conversations will meet each of these problems at the worst possible moment, usually close to a planned cutover. Implications for planning The practical lesson for leaders is to plan around the data rather than around the code. When estimating the effort for an extraction, start with the data: which tables does the capability own, which does it read from other capabilities, who else reads and writes its tables, and what transactional guarantees does it rely on. The answers determine most of the cost and risk. A capability with cleanly separated data can be extracted quickly. One whose data is entangled with the core of the system may take a year, and may be better left in place. It also follows that data migration needs its own explicit milestones in the programme plan, with their own measures of progress. "Service X is in production" is not a meaningful milestone if service X still reads and writes the monolith's tables. "Service X owns its data, all other access goes through its interface, and reconciliation shows no discrepancies for thirty days" is a milestone that reflects real independence. Plans that track only code extraction consistently report progress that turns out not to exist when the time comes to switch off the monolith. With the mechanics of incremental migration in place, from routing through to data, the remaining questions are organisational: how to secure the time and money to do this work, and how to govern it so that it delivers. The next chapter takes up the first of these. Chapter 7: Making the Case to the Business Engineering leaders often describe the funding of technical debt work as a battle with the business, and the language reveals the problem. When the request is framed as engineering wanting time for engineering's own purposes, against the business's wish for features, it is set up to lose. Executives who are accountable for revenue and cost will, reasonably, prioritise visible outcomes over invisible ones unless they are shown that the invisible work produces outcomes they care about. The way to win this argument is not to argue harder but to change what is being argued about. By the time a leader asks for investment, the work in the previous chapters should already have produced most of what is needed: evidence of where interest is being paid, an estimate of its cost, a disposition for each significant item, and a migration approach that delivers value incrementally and controls risk. What remains is to present this in the business's own terms. Why "refactoring sprints" fail as a request The most common form of the request, and the weakest, is for a dedicated period of time, often called a refactoring sprint, a hardening sprint or a tech debt quarter, during which the team stops feature work and cleans up the codebase. It fails as a request for several reasons, and understanding them helps to construct a better one. It asks for a cost without describing a return. The business is told what it will give up (a period of feature delivery) but not what it will gain, beyond vague promises that things will be "better" or "faster" afterwards. It has no clear measure of success. When the sprint ends, how will anyone know whether it worked? Without a defined outcome, the business has no way to judge the investment, and the next request will be harder. It treats debt as a one-time clean-up rather than an ongoing condition. If the debt accumulated through the normal course of development, it will accumulate again, and the business correctly suspects that it will be asked for another sprint before long. And it often fails on its own terms. A team given a few weeks to "clean up" a large legacy system, without a prioritised list of what to clean and why, tends to spread its effort across whatever its members personally find most irritating, which as Chapter 2 showed is a poor proxy for interest. The results are hard to see, and the business's scepticism is confirmed. None of this means that dedicated time for debt reduction is always wrong. Sometimes a concentrated effort is the right way to tackle a particular problem, especially obsolescence with a fixed deadline. But the request should be for a specific piece of work with a specific outcome, not for a block of time. A last difficulty is that the people who most need to hear the case are often not in the room when it is made. A chief financial officer may never attend an engineering planning meeting, and a product leader may hear about debt only as a reason for a slipped date. Leaders who succeed in securing investment usually bring the case to these people directly, in their own forums and formats, rather than waiting for it to filter upwards through engineering channels. Speaking in outcomes the business already uses Every business already has a vocabulary for evaluating investments: revenue, cost, risk, time to market, customer satisfaction, regulatory exposure. The task is to express the case for debt reduction in these terms, using the evidence assembled earlier. Table 4 shows how common engineering signals translate into terms a business leader already uses. Table 4. Translating engineering signals into business terms. Engineering signal What it means for the business Example of how to express it Long lead time in a hotspot Slower time to market for features in that area "Pricing changes take six weeks; competitors' take days." High change failure rate Customer-facing incidents and lost revenue "One in four releases to checkout causes an incident." Unpredictable cycle time Missed commitments and unreliable plans "Estimates in this area have been wrong by months." Knowledge concentration Key-person risk; continuity exposure "Two people understand billing; one is leaving." Unsupported platform Security and compliance exposure "The runtime loses vendor support next year." Senior time on support Hidden cost; reduced capacity for new work "Our most expensive engineers spend half their time on this." The aim is to move from statements about the code to statements about what the business experiences and what it is trying to achieve. "The order module is badly designed" is a statement about the code. "The three largest initiatives on next year's roadmap all depend on the order module, where changes currently take twice as long and fail three times as often as elsewhere" is a statement about the business. The second invites a decision; the first invites a shrug. External evidence can help establish that the problem is real and widespread, though it should never substitute for evidence about the organisation's own systems. McKinsey's October 2020 article "Tech debt: Reclaiming tech equity," by Vishal Dalal and colleagues, reported on a survey of chief information officers. Respondents estimated that technology debt amounted to 20 to 40 percent of the value of their entire technology estate before depreciation, and that 10 to 20 percent of the technology budget dedicated to new products was diverted to resolving issues related to technology debt. Sixty percent felt their organisation's technology debt had risen perceptibly over the previous three years. These are perceptions rather than measurements, but they come from senior technology executives rather than from engineers, which gives them weight with business audiences. The Stripe Developer Coefficient figures and the Code Red study discussed in Chapter 2 provide further context: developers report spending a large share of their time on maintenance, and low-quality code has been measured to produce many times more defects and to take substantially longer to change. A leader can present these as an indication that the organisation's experience is typical rather than a sign of unusual incompetence, which helps to shift the conversation from blame to management. Public failures also have a role, used carefully. They are not evidence that the same thing will happen in any particular organisation, and presenting them as threats tends to provoke scepticism. But they illustrate the kind of risk that accumulates in legacy systems. In late December 2022, Southwest Airlines suffered a breakdown during winter storms in which it cancelled close to seventeen thousand flights; its crew scheduling technology, which could not keep up with the scale of disruption, was widely identified as a central factor, and in December 2023 the US Department of Transportation imposed a penalty of 140 million dollars as part of a settlement that the department described as a record for consumer protection violations by an airline. The lesson for a business audience is not that legacy systems always fail, but that the risk they carry is often invisible until conditions change, and that when it materialises, the cost is measured in a currency far larger than the engineering budget. Risk is often the most persuasive argument for debt reduction, and the most often abused. Engineers who have failed to secure investment on productivity grounds sometimes turn to warnings of catastrophe, which may be sincere but which, if not grounded in evidence, damage credibility when the catastrophe does not arrive. A better approach is to present risk in the form the business already uses for other risks: a risk register entry with a description, a likelihood, an impact and a mitigation. "Our billing system runs on a database version that loses vendor security support in March. After that date, vulnerabilities will not be patched. Our auditors have flagged unsupported software as a finding in previous years. The mitigation is an upgrade, estimated at eight weeks of work for one team, which we propose to combine with restructuring the invoice generation code that the roadmap requires us to change in the same period." This is concrete, verifiable and actionable, and it does not rely on dramatic language. Knowledge risk deserves particular attention in this framing, because business leaders understand key-person dependency well from other contexts. A system that only one or two people understand is a continuity risk, and the mitigation (documentation, characterisation tests, pairing, and restructuring the code so that it can be understood by others) is directly connected to debt reduction. Risk framing has one further advantage: it gives the business a choice it is used to making. Executives routinely decide whether to accept, mitigate, transfer or avoid risks, and they understand that accepting a risk is a legitimate decision if it is made knowingly. Presenting technical risk this way allows a business leader to decide, explicitly and on the record, to accept a risk the engineering organisation has identified. Sometimes that is the right decision, and the engineering leader should be willing to accept it. What matters is that the decision has been made by someone with the authority to make it, with the information needed to make it well, rather than being made by default because nobody raised the issue in terms the business recognised. Structures for ongoing funding Because debt accumulates continuously, the most sustainable funding arrangement is a continuous one rather than a series of special requests. Several structures are common, and each suits different circumstances. A fixed capacity allocation reserves a proportion of each team's time for debt reduction and other engineering-led improvements. Many organisations use a figure in the range of 15 to 25 percent, though there is no rigorous evidence for any particular number; the right proportion depends on how much interest the system is charging. The advantage is predictability: teams know they have the time, and they can plan debt work alongside features without negotiating each item. The risk is that the allocation becomes a budget to be spent rather than an investment to be justified, and that the work done within it is not tied to outcomes. The debt register helps prevent this, by ensuring that the allocation is spent on the highest-interest items. Bundling debt reduction with feature work is often the most effective approach for high-interest items in the roadmap's path. Rather than presenting debt reduction as a separate item, the estimate for a feature includes the work needed to make the feature easy to build, following Beck's principle. The business sees one estimate for one outcome. This is honest, as long as the debt work genuinely serves the feature, and it avoids the artificial competition between features and debt reduction. Its limitation is that it works only for debt in the path of planned features; it does not address risk-driven or platform work. A dedicated programme with its own budget and outcome goals suits large migrations, such as a Strangler Fig programme for a major monolith. The programme is funded like any other significant investment, with a business case, milestones and a governance structure, and it is judged by whether it delivers its stated outcomes. This makes the investment visible and accountable, which can be uncomfortable but is also what secures sustained support. In practice, most organisations use a combination: a modest standing allocation for routine improvement, bundling for debt in the roadmap's path, and a dedicated programme for large migrations. The combination should be explicit and visible to business leadership, so that everyone understands how much capacity is going where and why. Answering hard questions and delivering A leader who presents a case for debt reduction to a finance or executive audience should expect a few difficult questions, and should prepare answers that are honest rather than defensive. "Why did you let this happen?" is often the first, sometimes spoken and sometimes implied. The honest answer is that some of the debt was a deliberate and reasonable trade-off made to ship earlier, some reflects what the team has learned since the code was written, and some reflects practices that should have been better. A leader who can distinguish these, and who can describe what is changing to prevent the reckless kind from recurring, is far more credible than one who either blames predecessors or claims that all debt was inevitable. "How do we know it will not come back?" is a fair question, and the answer is that some of it will, because debt is a by-product of learning. What can be promised is that it will be measured, that it will be managed through the ordinary allocation of capacity rather than through periodic emergencies, and that the architectural protections described in the next chapter will make the most damaging forms of erosion harder. "Could we not just hire more engineers?" is tempting for organisations with budget to spare. But adding people to a system that is slow to change because of its structure rarely speeds it up proportionately. More engineers working in the same hotspots create more coordination, more merge conflicts and more contention for the few people who understand the code. Brooks's observation in The Mythical Man-Month that adding people to a late software project makes it later applies with particular force to a tangled codebase. Reducing the interest often increases effective capacity more than adding people does. "What happens if we do nothing?" deserves a straightforward answer, not a threat. In most cases, the honest answer is that nothing dramatic happens immediately. Delivery in the affected areas continues to slow, estimates continue to be unreliable, senior engineers continue to absorb support load and some of them leave, and the risk items on the register continue to age. The cost is gradual, which is exactly why it tends to be ignored, and why it is worth putting a number on it. A chart of lead time for the affected area over the past two years, extended forward at its current trend, is often more persuasive than any argument. "Why should this come before the features customers are asking for?" is the most important question, and the answer is that it should not come before them; it should come with them. The strongest cases for debt reduction are those that show how the work accelerates the delivery of the features customers want, which is why bundling and roadmap alignment matter so much. The final element of making the case is keeping it made. A business case for debt reduction that is approved and then never reported on will be the last one approved. Leaders should commit, at the outset, to reporting progress in the same terms the case was made in: lead time in the affected areas, change failure rate, incident frequency, delivery of the roadmap items the work was meant to accelerate, and any change in risk exposure. Early, visible wins matter disproportionately. If the first piece of debt work is chosen so that it produces a measurable improvement within a quarter, ideally one that a product leader notices without being told, the case for subsequent work becomes much easier. This is another reason to choose initial targets with care: the first extraction or restructuring is not only a technical step but also the proof of concept for the whole approach. When results are disappointing, they should be reported honestly, with an explanation and a revised plan. Business leaders are accustomed to investments that do not go to plan; what destroys trust is discovering that problems were hidden. A leader who reports a setback early, explains what was learned and adjusts the approach is more likely to retain support than one who reports steady progress until the programme collapses. Done this way, the conversation about technical debt stops being a negotiation between engineering and the business and becomes a shared discussion about how to invest in the system the business depends on. That shift does not happen through a single presentation. It happens through a track record of evidence, honest reporting and delivered outcomes, which is built through the governance described in the next chapter. Chapter 8: Governing the Programme A modernisation programme for a legacy monolith is a long undertaking. Even a well-run Strangler Fig migration of a large system typically takes years, and during that time the people, priorities and circumstances of the organisation will change. The technical approach described in earlier chapters is necessary but not sufficient. What determines whether the programme succeeds is mostly governance: how teams are organised, how progress is measured, how decisions are made as conditions change, how the debt is kept from re-accumulating, and how the programme knows when it is finished. Conway's law and team design In 1968, Melvin Conway published an article in Datamation titled "How Do Committees Invent?" Its central observation, which became known as Conway's law, was that "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations." A system built by four teams will tend to have four major components, with interfaces that mirror how those teams talk to each other. For a modernisation programme, Conway's law has a direct and often uncomfortable implication. If the target architecture does not match the organisation's team structure, the organisation will tend to pull the architecture back towards its own shape. A system designed as a set of independent services, but built and maintained by teams organised around technical layers such as front end, back end and database, will tend to develop tight coupling between services, because every change requires coordination across the layer teams. Conversely, teams organised around business capabilities will tend to produce services aligned with those capabilities. This observation has given rise to what is sometimes called the inverse Conway manoeuvre: deliberately structuring teams to match the desired architecture, so that the organisation's communication patterns reinforce it rather than erode it. Matthew Skelton and Manuel Pais developed this idea in Team Topologies (2019), which describes four fundamental team types. Stream-aligned teams own a flow of work aligned to a business capability. Platform teams provide internal services that stream-aligned teams use. Enabling teams help other teams adopt new skills or practices. And complicated-subsystem teams own parts of the system that require specialist knowledge. They also emphasise limiting the cognitive load on each team, so that no team is responsible for more of the system than it can reasonably understand. For a monolith migration, this suggests several practical arrangements. Teams should be aligned with the bounded contexts identified for extraction, so that each team owns a coherent part of the domain in both the monolith and the new services. A platform team should provide the shared infrastructure that new services need, such as deployment pipelines, observability and the routing facade, so that each stream-aligned team does not have to build these from scratch. And an enabling team, or a small group of experienced engineers, can help teams learn the migration techniques, then step back once the teams are self-sufficient. A common anti-pattern is the separate migration team: a group of engineers, often the most capable, tasked with building the new system while the rest of the organisation maintains the old one. This seems efficient but tends to fail. The migration team lacks the detailed domain knowledge held by the teams working in the monolith. The teams working in the monolith feel excluded and demoralised, and have little incentive to help. And when the migration team hands over its services, the receiving teams do not understand them. It is usually better for the teams that own a capability to also own its migration, with support from a platform team and enabling engineers. Ownership after migration deserves equal attention. Every new service needs a named owning team that is responsible for its operation, its on-call rota and its evolution. Services without clear owners decay quickly, becoming the legacy systems of the next decade. A useful rule is that no extraction is considered finished until the owning team has run the service in production, including on-call, for a defined period and has confirmed it is willing to keep it. Measures that tell the truth A long programme needs a small set of measures that show whether it is working. These should be chosen at the start, measured from a baseline before work begins, and reported consistently throughout. The most useful measures are the outcome measures discussed in earlier chapters: lead time for changes, deployment frequency, change failure rate and recovery time, measured specifically for the areas affected by the programme. These show whether the work is achieving its purpose. If lead time in the area being migrated is not falling, something is wrong, regardless of how much has been extracted. A second set of measures tracks the migration's progress directly. Useful ones include the proportion of traffic or transactions handled by new components rather than the monolith, the number of database tables or data domains that have been separated, the number of consumers still depending directly on the monolith's database, and the size of the monolith measured in active code. These show whether the programme is moving towards its end state, and they are useful for planning. But they are not measures of value, and a programme that reports only these is at risk of optimising for migration rather than outcomes. A third set tracks health and risk: incident frequency and severity in migrated and unmigrated areas, the results of reconciliation and parallel runs, and indicators of team health such as on-call burden and attrition. These provide early warning that the programme is creating new problems while solving old ones. Measures should be few. A dashboard with fifty metrics will be ignored. A handful of measures that everyone understands, reported in the same form every month or quarter, will become part of how the organisation thinks about the programme. Protecting the architecture A monolith did not become a tangle in a single decision. It became one through thousands of small changes, each reasonable, that gradually eroded its structure. Unless something changes in how the organisation works, the same process will erode the new architecture. Services will begin to depend on each other's internals, shared libraries will accumulate business logic, and boundaries will blur. A few years later, the organisation will have a distributed monolith, which combines the problems of both architectures. Neal Ford, Rebecca Parsons and Patrick Kua, in Building Evolutionary Architectures (2017), proposed a defence they called fitness functions: automated checks that verify the architecture continues to have the characteristics it was designed for. A fitness function might check that no service reads another service's database directly, that dependencies between modules flow only in permitted directions, that response times stay within limits, or that no new code is added to a module scheduled for retirement. These checks run in the build pipeline, so that a change violating an architectural rule fails the build just as a change that fails a unit test would. Shopify's Packwerk, mentioned in Chapter 4, is an example of this approach applied to a modular monolith: it checks that code in one component does not reference private code in another, and reports violations. Similar tools exist for many languages, and simpler checks can often be written with a few lines of script. The principle is that architectural rules that are only written in documents will be broken, while rules that are enforced automatically will mostly be kept. Fitness functions are not the whole answer. Some architectural qualities cannot easily be checked automatically, and some rules need to be broken occasionally for good reasons, which requires a lightweight process for granting exceptions. But they shift the default from erosion to preservation, which over a period of years makes an enormous difference. The other defence against re-accumulation is cultural. The practices that prevent debt from building up in the first place, such as small and frequent changes, continuous refactoring as part of feature work, code review that attends to design as well as correctness, and tests written alongside code, need to be expected and supported. Leaders signal what matters through what they ask about and what they reward. If the only question in planning meetings is when a feature will ship, teams will learn that design does not matter. If leaders also ask whether the change made the area easier or harder to work in, teams will learn otherwise. A migration programme involves a continuous stream of decisions about boundaries, sequencing, technology and trade-offs. Who makes these decisions, and how, affects both the quality of the decisions and the speed of the programme. Two failure modes are common. In the first, all significant decisions go through a central architecture board, which becomes a bottleneck, slows the programme and leaves teams feeling that they do not own their work. In the second, every team makes its own decisions without coordination, producing inconsistent approaches, duplicated infrastructure and services that do not fit together. A middle path that works in many organisations is to define a small number of decisions that require central agreement, such as the boundaries between bounded contexts, the standards for inter-service communication, the choice of shared platform components and the order of major extractions, and to leave everything else to the teams. Decisions within a team's own context are the team's. Decisions that affect other teams are discussed in a forum, often called an architecture forum or guild, where engineers from different teams review proposals and raise concerns, but where the aim is advice and coordination rather than approval. Architecture decision records, short documents recording what was decided, why, and what alternatives were considered, are a lightweight and effective way to make decisions visible and to preserve the reasoning for those who come later. They also help prevent the same debates from recurring, which saves a great deal of time in a long programme. Decision records also help with a problem that long programmes face constantly, which is turnover. Over several years, many of the people who designed the migration will move on, and their successors will inherit decisions whose reasoning is not obvious. Without a record, successors tend either to follow earlier decisions rigidly, without understanding when they no longer apply, or to reverse them casually, without understanding why they were made. A short written record of each significant decision, its context and its expected consequences allows a new leader or engineer to judge whether the reasoning still holds. In this sense, the programme's own decision history is a form of knowledge that must be managed as deliberately as the knowledge embedded in the legacy code it is replacing. Warning signs and knowing when to stop Long programmes drift, and the drift is usually visible well before it becomes a crisis to those who know what to look for. Several warning signs recur. The first is progress reported only in migration terms. If status reports describe services created, lines moved or percentage of the monolith decomposed, but say nothing about lead time, incidents or roadmap delivery in the affected areas, the programme has lost sight of its purpose. Sometimes the outcome measures are omitted because they are not moving, which is the most important thing the leader needs to know. The second is a growing number of half-finished extractions. A programme that starts new extractions before finishing old ones accumulates temporary states: services that share the monolith's database, toggles that are never removed, routes that send some traffic to old code and some to new. Each temporary state carries its own cost, and together they can leave the system harder to work with than before the programme began. A simple rule, that no new extraction begins until the previous one has reached a defined finished state, prevents most of this. The third is the appearance of a distributed monolith: services that must be deployed together, that call each other synchronously in long chains, or that fail together. This usually indicates that boundaries were drawn in the wrong places, often because change coupling was not examined, or that data was not separated. It is better to recognise this early and merge services back together, or redraw the boundaries, than to continue building on a flawed decomposition. The fourth is fatigue. Engineers who have spent years on a migration without seeing it finish, or who feel stuck maintaining the old system while others build the new, become disengaged and leave. Rotating people between old and new work, celebrating completed milestones and keeping the end in sight all help. When these signs appear, the response is usually to narrow rather than to push harder. Finish the extractions already under way, consolidate the gains, re-examine the remaining plan against the current roadmap and the debt register, and restart with a smaller scope and clearer outcome goals. This can feel like retreat, but a programme that delivers a few completed improvements is worth far more than one with many unfinished ones. Every modernisation programme should have an end, and recognising it is harder than it sounds. Programmes tend to acquire a life of their own: teams and budgets are organised around them, and there is always more that could be improved. The clearest end condition is achievement of the outcome goals set at the start. If the aim was to reduce lead time for changes in the order flow to a week, and it has been reduced to a week, the programme has succeeded, even if parts of the monolith remain. The remaining parts may be acceptable debt, in the terms of Chapter 3: stable, rarely changed and not worth the cost of migrating. Continuing to migrate them for the sake of architectural completeness is spending principal to remove debt that charges almost no interest. There is an important exception. If the remaining monolith imposes significant fixed costs, such as infrastructure, licensing, operational overhead or the need to maintain skills in an obsolete technology, then retiring it completely may be worthwhile even if the remaining functionality is rarely changed. In this case, the end condition is the decommissioning of the monolith, and the final stage of the programme should focus on retiring remaining functions, whether by migrating them, replacing them with commercial products or removing them altogether. Usage data from the routing facade is invaluable here, since it often shows that the remaining functions are used far less than assumed, and that some can simply be switched off. Whichever end condition applies, it should be decided explicitly. A programme with a clear end is easier to fund, easier to staff and easier to celebrate when it finishes. One without a clear end tends to drift, losing support gradually until it is quietly cancelled with the work half done, which is the worst outcome of all, leaving the organisation with both the old system and a partial new one. Through all of this, the leader's role shifts as the programme matures. At the start, the leader's main task is to establish the evidence, set the outcome goals, secure the investment and design the organisation. In the middle, the task is to protect the programme from distraction, maintain the measures, keep the business informed and make the hard calls about sequencing and scope. Towards the end, the task is to recognise when the goals have been met, resist the temptation to keep going for its own sake, and ensure that the practices that prevent re-accumulation are embedded deeply enough to outlast the programme. Throughout, the most important contribution a leader makes is keeping attention on the purpose of the work. The purpose is not a new architecture, a cleaner codebase or the retirement of an old system for its own sake. It is an organisation that can change its software as fast as its business needs to change, safely and predictably. Every decision in the programme should be tested against that purpose, and every measure should show whether it is being achieved. Conclusion The argument of this booklet has been that technical debt is a management problem with a precise structure, and that the structure tells a leader what to do. Debt charges interest only where the business needs to make changes. That interest can be found with evidence the organisation already holds. It can be retired incrementally, while the system keeps serving the business. And the work can be funded by describing it in the terms the business already uses to make every other investment decision. What follows from this is not a recap of the method but a set of consequences for how engineering leadership should think about its job. Debt is a property of the plan, not the code The most important consequence is that technical debt cannot be assessed by looking at code alone. The same module is a serious liability for a company that plans to change its pricing model next year and a harmless curiosity for one that does not. A codebase does not have a fixed quantity of debt; it has a quantity relative to what the organisation intends to do with it. This means that debt management belongs in the planning process, not beside it. A roadmap that is drawn up without asking which parts of the system each initiative will touch, and what condition those parts are in, is a roadmap built on hidden assumptions. When those assumptions fail, the result appears as missed estimates and "unexpected complexity," and the organisation learns the wrong lesson: that engineering is unreliable, rather than that the plan ignored the cost of changing the system it depends on. A leader who brings the debt register into roadmap planning is not adding an engineering concern to a business process. They are correcting a business process that was mispricing its own work. It also means that the engineering organisation's view of debt must change whenever the business's plans change. A merger, a new market, a regulatory change or a shift in strategy can transform which parts of a legacy system matter. The register should be revisited at those moments, and investment redirected accordingly. Organisations that treat debt reduction as a fixed programme, planned once and executed regardless of changing plans, will find themselves paying down debt that no longer charges interest while ignoring the debt that suddenly does. Incrementalism is a risk discipline, not a compromise The second consequence concerns how change is made. The Strangler Fig pattern, branch by abstraction, parallel runs and progressive rollout are sometimes presented as cautious alternatives to a bolder rewrite, as though incrementalism were a concession to timid stakeholders. That framing has it backwards. These techniques are how an organisation learns about its own system while changing it. Each small step reveals something: a hidden dependency, an undocumented behaviour, a consumer nobody knew about, an assumption that turns out to be wrong. The big-bang approach defers all of that learning to a single moment at which it is too late to act on it. Incrementalism also changes the economics of the work. A migration delivered in slices begins to return value with the first slice. It can be paused, redirected or stopped at any point with the benefits already gained intact. It never puts the whole business at risk at once. These properties make it far easier to fund, because the business is not asked to make a single large bet with a payoff years away; it is asked to make a series of small bets, each of which can be judged on its results before the next is made. For leaders, this means that the ability to change a system incrementally is itself an asset worth investing in. Routing layers, feature toggle infrastructure, observability, characterisation tests and deployment pipelines do not produce features, but they are what make every subsequent change safe and reversible. An organisation that has them can modernise continuously, in the ordinary course of work. One that lacks them is forced, sooner or later, into the large and dangerous projects that incrementalism exists to avoid. Architecture should follow outcomes The third consequence is that the destination of a modernisation programme should never be an architecture. Microservices, modular monoliths, event-driven designs and serverless platforms are all tools, each with costs as well as benefits, and the public record contains examples of organisations moving in every direction between them for good reasons. A programme that sets out to "move to microservices" has chosen its answer before asking its question. The question is what the business needs to be able to do, and what is preventing it. If the answer is that teams cannot release independently because of coordination costs, independent deployment may help, and that may or may not require separate services. If the answer is that the code is tangled and poorly tested, restructuring within the monolith may achieve most of the benefit at a fraction of the cost. If the answer is that a commodity capability is consuming effort that should go elsewhere, replacement with a product may be best. The right architecture for each part of the system follows from the outcome sought for that part, and different parts may end up in different places. Leaders who hold this line protect their organisations from one of the most expensive forms of new debt: an architecture adopted because it was fashionable, imposing operational and cognitive costs that the organisation did not need to pay. Credibility is the scarce resource The fourth consequence concerns the relationship between engineering and the rest of the business. Technical debt work competes for capacity with everything else the organisation wants to do, and it will be funded only to the extent that the people controlling that capacity trust the engineering organisation's judgement. That trust is built slowly and spent quickly. It is built by making claims that can be checked and then showing that they were right: this module slows every change by half, restructuring it will take six weeks, and afterwards changes will be faster, and here is the evidence that they are. It is spent by making claims that cannot be checked, by asking for time without describing outcomes, by warning of disasters that do not come, and by reporting progress that later turns out not to have existed. The practices described in this booklet, from hotspot analysis to outcome measures to honest reporting of setbacks, are as much about building credibility as about managing code. This places a particular responsibility on engineering leaders. They stand between two groups that often misunderstand each other: engineers who experience the cost of debt directly and cannot understand why the business will not fund its reduction, and business leaders who see engineering consuming resources without producing much and cannot understand why. The leader's job is to translate in both directions, giving engineers a way to express their concerns that will be heard, and giving the business a way to see costs that are otherwise invisible. Done well, the question stops being whether to invest in the system and becomes how much, and where, which is the question it should have been all along. The work never ends, but programmes should Finally, it follows from everything above that technical debt is a permanent condition of any software that is used and changed. Every team learns as it builds, and every lesson creates a gap between what the code says and what the team now knows. Lehman's observation holds: complexity grows unless work is done to contain it. No migration, however successful, produces a system that will stay clean on its own. This does not mean that modernisation programmes should be permanent. They should have clear goals, and they should end when those goals are met. What should be permanent is the ordinary discipline of paying down debt as it is incurred, in the course of normal work, so that it never again accumulates to the point where a special programme is needed. The measure of a successful modernisation is not only that the old system has been retired or tamed. It is that the organisation has learned to keep the new one healthy, so that the next leader to inherit it does not find a system everyone is afraid of. Notes 1. Ward Cunningham, "The WyCash Portfolio Management System," Addendum to the Proceedings of OOPSLA 1992 (Object-Oriented Programming Systems, Languages, and Applications), ACM, 1992. Source of the original technical debt passage quoted in Chapter 1. 1. Ward Cunningham, "Debt Metaphor," video, 2009. Cunningham's later clarification that the metaphor was not intended to justify writing poor code. 1. Martin Fowler, "Technical Debt Quadrant," martinfowler.com, October 2009. 1. Michael C. Feathers, Working Effectively with Legacy Code (Prentice Hall, 2004). Source of the definition of legacy code as code without tests, and of characterisation tests and seams. 1. M. M. Lehman, "Programs, Life Cycles, and Laws of Software Evolution," Proceedings of the IEEE 68, no. 9 (1980): 1060-1076. 1. Kent Beck, post on Twitter, 25 September 2012: "for each desired change, make the change easy (warning: this may be hard), then make the easy change." 1. Adam Tornhill and Markus Borg, "Code Red: The Business Impact of Code Quality - A Quantitative Study of 39 Proprietary Production Codebases," Proceedings of the International Conference on Technical Debt (TechDebt 2022), ACM/IEEE, 2022. Also available as arXiv:2203.04374. 1. Stripe, The Developer Coefficient, September 2018, based on a survey conducted with The Harris Poll of developers and C-level executives in the United States, United Kingdom, France, Germany and Singapore. 1. Nicole Forsgren, Jez Humble and Gene Kim, Accelerate: The Science of Lean Software and DevOps (IT Revolution, 2018). 1. Donald G. Reinertsen, The Principles of Product Development Flow: Second Generation Lean Product Development (Celeritas, 2009). 1. US Securities and Exchange Commission, "SEC Charges Knight Capital With Violations of Market Access Rule," press release 2013-222, 16 October 2013. 1. Joel Spolsky, "Things You Should Never Do, Part I," Joel on Software, 6 April 2000. 1. Frederick P. Brooks Jr., The Mythical Man-Month: Essays on Software Engineering (Addison-Wesley, 1975; anniversary edition 1995). 1. Financial Conduct Authority and Prudential Regulation Authority, final notices and joint announcement on fines imposed on TSB Bank plc in connection with its 2018 IT migration, December 2022. 1. Alexandra Noonan, "Goodbye Microservices: From 100s of problem children to 1 superstar," Segment engineering blog, July 2018. 1. Marcin Kolny, "Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%," Prime Video Tech blog, March 2023. 1. Kirsten Westeinde, "Deconstructing the Monolith: Designing Software that Maximizes Developer Productivity," Shopify Engineering blog, February 2020; Shopify, Packwerk (open-source tool), 2020. 1. Martin Fowler, "Strangler Fig Application," martinfowler.com (originally published as "Strangler Application" in 2004, later retitled). 1. Sam Newman, Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith (O'Reilly, 2019). 1. Eric Evans, Domain-Driven Design: Tackling Complexity in the Heart of Software (Addison-Wesley, 2003). 1. Martin Fowler, "Branch By Abstraction," martinfowler.com, January 2014, crediting the term to Paul Hammant. 1. Pete Hodgson, "Feature Toggles (aka Feature Flags)," martinfowler.com, October 2017. 1. Danilo Sato, "Parallel Change," martinfowler.com, May 2014. 1. Chris Richardson, Microservices Patterns: With Examples in Java (Manning, 2018). Source for the transactional outbox and saga patterns. 1. GitHub, Scientist: a Ruby library for carefully refactoring critical paths, open-source project, first released 2016. 1. Vishal Dalal, Krish Krishnakanthan, Björn Münstermann and Rob Patenge, "Tech debt: Reclaiming tech equity," McKinsey Digital, 6 October 2020. 1. US Department of Transportation, announcement of consent order and 140 million dollar civil penalty against Southwest Airlines over the December 2022 holiday disruption, 18 December 2023. 1. Melvin E. Conway, "How Do Committees Invent?" Datamation 14, no. 4 (April 1968): 28-31. 1. Matthew Skelton and Manuel Pais, Team Topologies: Organizing Business and Technology Teams for Fast Flow (IT Revolution, 2019). 1. Neal Ford, Rebecca Parsons and Patrick Kua, Building Evolutionary Architectures: Support Constant Change (O'Reilly, 2017; second edition with Pramod Sadalage, 2022). Further Reading Michael C. Feathers, Working Effectively with Legacy Code (Prentice Hall, 2004).** The standard practical guide to getting untested code under test so that it can be changed safely. Its catalogue of dependency-breaking techniques remains relevant to any language, and its definition of legacy code reframes the whole problem. Sam Newman, Monolith to Microservices (O'Reilly, 2019).** The most thorough treatment of incremental migration patterns, including the Strangler Fig, branch by abstraction, parallel run and the many patterns for splitting a shared database. Notably even-handed about when not to migrate at all. Adam Tornhill, Your Code as a Crime Scene, second edition (Pragmatic Bookshelf, 2024), and Software Design X-Rays (Pragmatic Bookshelf, 2018).** The foundation of behavioural code analysis: hotspots, change coupling and knowledge maps drawn from version control history. The second book is aimed more directly at technical leaders deciding where to invest. Nicole Forsgren, Jez Humble and Gene Kim, Accelerate (IT Revolution, 2018).** The research behind the four key delivery metrics and the finding that speed and stability reinforce each other. Essential for choosing outcome measures for a modernisation programme. Matthew Skelton and Manuel Pais, Team Topologies (IT Revolution, 2019).** A practical account of how team structure shapes software architecture, with a vocabulary for designing teams around the system one wants to build. Eric Evans, Domain-Driven Design (Addison-Wesley, 2003).** Dense but foundational. The chapters on bounded contexts and context mapping are the most useful for anyone deciding where to draw service boundaries. Donald G. Reinertsen, The Principles of Product Development Flow (Celeritas, 2009).** The clearest argument for cost of delay as the central economic variable in product development, and a source of tools for prioritising debt work alongside features. Neal Ford, Rebecca Parsons, Patrick Kua and Pramod Sadalage, Building Evolutionary Architectures, second edition (O'Reilly, 2022). Introduces architectural fitness functions and the practice of protecting architectural characteristics through automated checks. Martin Fowler, martinfowler.com. Short essays on the Strangler Fig Application, the Technical Debt Quadrant, Branch by Abstraction, Parallel Change and Feature Toggles, each freely available and widely referenced. Adam Tornhill and Markus Borg, "Code Red: The Business Impact of Code Quality" (TechDebt 2022). A short, readable empirical study linking code health to defects and development time, useful when making the quantitative case to business stakeholders. Hashtags: #TechnicalDebt #TechDebtManagement #LegacyMonoliths #LegacyCode #LegacySystemModernization #SoftwareRefactoring #RefactoringStrategies #SoftwareArchitecture #SoftwareEngineering #CodeQuality #CodeMaintainability #TechnicalDebtReduction #IncrementalModernization #StranglerFigPattern #ModularMonolith #MicroservicesMigration #LegacyCodeRefactoring #CharacterizationTesting #CodeHotspotAnalysis #ChangeCoupling #DomainDrivenDesign #BoundedContexts #BranchByAbstraction #FeatureToggles #DatabaseMigration #ChangeDataCapture #ContinuousDelivery #SoftwareDeliveryPerformance #EngineeringLeadership #TechnicalDebtPrioritization

Download the Book (PDF): Introduction A woman in Texas slips on the ice, breaks her wrist, and is taken to the nearest emergency room. The hospital is in her insurer's network. The emergency physician who sets the bone is not, because the physician group that staffs the department has never signed a contract with her plan. Before 2022, the bill for that physician's work could arrive weeks later for thousands of dollars, and whether she had to pay it depended on a question she had never thought to ask: was her employer's health plan "insured" or "self-funded"? If the plan was insured, Texas law might protect her. If it was self-funded, as most large employers' plans are, a federal statute written in 1974 to protect pension savings probably stopped Texas from protecting her at all. That single question, which almost no patient knows the answer to, is a good place to begin, because it exposes the structure of American health insurance law more clearly than any organizational chart. Health coverage in the United States is not governed by one body of law. It is governed by at least three, layered on top of one another, each built at a different time for a different purpose, and each with its own theory of who should decide what a health plan must cover, what a provider may charge, and what a patient may do when things go wrong. The first layer is state insurance regulation, the oldest of the three. Since the McCarran-Ferguson Act of 1945, Congress has left the business of insurance chiefly to the states, which license insurers, review their rates and forms, mandate benefits, and police claims practices. The second layer is the Employee Retirement Income Security Act of 1974, known as ERISA, which governs benefit plans that private employers sponsor and which contains one of the broadest preemption clauses in federal law. The third layer is the direct federal regulation of coverage itself, which began modestly with the continuation-coverage rules of COBRA in 1985 and the portability rules of HIPAA in 1996, and which became comprehensive with the Patient Protection and Affordable Care Act of 2010. Alongside all three sit the great public programs, Medicare and Medicaid, the second of which is itself a federal-state partnership with its own fault lines. The argument of this book The argument of this book is that the major legal battles in American health insurance over the past fifteen years are best understood not as separate controversies but as recurring collisions along the seams between these layers. The fight over Medicaid expansion is, at bottom, a fight about how far the federal government may push the states with money. The long series of ERISA preemption cases is a fight about how far the states may reach into the employer plans that cover most working Americans. The surprise-billing wars were caused by a gap that ERISA preemption created, and they were resolved, to the extent they have been resolved, by a federal statute that tried to fill the gap and immediately generated a new round of litigation over who, exactly, gets to set a fair price. The price-transparency rules are an attempt to let markets do what regulators and arbitrators have struggled to do, and they are contested on the same terrain. Seen this way, the subject has a shape. Each chapter examines a place where two layers meet and asks the same practical questions: who holds the power to decide, what institutions actually decide in practice, and what the patient, the employer, the insurer, or the state can do about it. The answers change more often than most readers would expect. Since the start of 2025 alone, the Supreme Court has upheld the machinery behind the ACA's free preventive-care guarantee, has narrowed the ability of Medicaid beneficiaries to sue states in federal court, and has made it easier to plead certain claims against the fiduciaries of employee benefit plans. Congress has rewritten large parts of Medicaid in a single reconciliation law signed on July 4, 2025, allowed the enhanced premium subsidies that had supported record marketplace enrollment to expire at the end of that year, and in early 2026 enacted a set of reforms aimed at pharmacy benefit managers. Federal agencies have strengthened hospital price-transparency enforcement, pulled back from enforcing parts of a mental health parity rule, and continued to defend the surprise-billing arbitration system against challenges from physicians. A reader who learned health insurance law even five years ago would find much of the ground has moved. Who this book is for This book is written for readers who need to understand health insurance law without becoming specialists in it: benefits managers and human resources leaders, hospital and physician-practice administrators, state legislative staff, journalists, students, patient advocates, and lawyers whose practice brushes against the subject. It assumes intelligence and curiosity but no prior training in insurance or benefits law. Technical terms are defined where they first matter, and a short glossary at the end collects the ones that recur. The book is an educational account of the law and not legal advice on any particular situation. It is also, deliberately, a book about the United States. Most wealthy democracies organize health coverage through either a national health service or tightly regulated, mandatory social insurance, and comparative notes appear where they sharpen the American picture. But the particular difficulties this book examines, from ERISA preemption to the arbitration of out-of-network bills, are products of the American choice to build a mostly private, mostly employment-based system and then regulate it in pieces. How the book is organized Chapter 1 lays out the architecture: the three regulatory systems, how a given person's coverage falls under one or another of them, and why the difference between an insured plan and a self-funded plan is the most consequential distinction most people have never heard of. Chapter 2 follows the Affordable Care Act through the courts, from the Supreme Court's 2012 decision in National Federation of Independent Business v. Sebelius through King v. Burwell, the risk-corridor litigation, California v. Texas, and the 2025 decision in Kennedy v. Braidwood Management. The purpose is not to recount every lawsuit but to show what the pattern of litigation reveals about the statute's durability and its weak points. Chapter 3 takes up Medicaid expansion. The 2012 decision that made expansion optional turned a national program into a state-by-state negotiation, conducted through legislatures, ballot initiatives, federal waivers, and lawsuits over work requirements. The chapter follows that negotiation into the 2025 reconciliation law, which imposes federal work requirements on expansion adults for the first time, and it examines the shrinking set of tools beneficiaries and providers have to enforce Medicaid's promises in court. Chapters 4 and 5 turn to ERISA. Chapter 4 explains the preemption clause and the Supreme Court's long struggle to give it sensible limits, through the modern cases on all-payer claims databases and pharmacy benefit managers. Chapter 5 examines managed care from the patient's side: how ERISA's remedial scheme channels disputes over denied care into federal court, limits what a harmed patient can recover, and interacts with the external-review rights the ACA added, the mental health parity law, and the new generation of disputes over prior authorization and algorithmic claim denials. Chapter 6 is devoted to surprise medical billing: why it happened, how the states tried and partly failed to stop it, how the No Surprises Act works, and why its arbitration process became one of the most heavily litigated regulatory programs in recent memory. Chapter 7 addresses price transparency, the government's attempt to make prices visible through disclosure mandates on hospitals and health plans, and asks what disclosure can and cannot accomplish. Chapter 8 examines the employer's emerging role as a fiduciary for health-plan costs, a role that recent statutes, lawsuits, and the 2026 reforms to pharmacy benefit management have made newly prominent. The conclusion draws the threads together and considers what the pattern of the past fifteen years suggests about the next several: a period in which coverage is likely to contract, costs to keep rising, and the old question of which level of government is responsible for what to become sharper, not softer. A note on reading legal change Health insurance law moves on several clocks at once. Statutes change rarely but massively. Regulations change with each administration, and since the Supreme Court's 2024 decision in Loper Bright Enterprises v. Raimondo ended the practice of deferring to agencies' reasonable readings of ambiguous statutes, those regulations stand on less secure footing in court than they once did. Litigation moves slowly, and a district court ruling can govern the entire country for years before an appeals court speaks. Sub-regulatory guidance, enforcement priorities, and non-enforcement announcements can change practice overnight without changing a word of law. The reader who keeps these clocks in mind will be better equipped to judge what in this book is durable and what is provisional. The architecture described in the first chapter has been stable for half a century. The specific arbitration rules described in Chapter 6 may not survive the next few years of litigation. The underlying tension that produces both, between a federal government that wants national rules and a system built on state regulation and private employment, is not going anywhere. Chapter 1: Three Systems Stacked: The Architecture of Coverage Law Ask an American where their health insurance comes from and the most common answer is "my job." That answer is accurate, but it conceals the legal fact that matters most. More than 150 million people get coverage through an employer, yet the law that governs that coverage differs sharply depending on how the employer has chosen to pay for it. A second large group buys coverage individually, most often through the marketplaces the Affordable Care Act created. Tens of millions more are covered by Medicaid, and most of them are enrolled in private managed care plans that contract with the state. Older Americans are covered by Medicare, and roughly half of them have chosen Medicare Advantage plans run by private insurers. Each of these arrangements sits under a different combination of state and federal law, and a person can move among them several times in a working life without ever being told that the rules have changed. This chapter lays out that architecture. It is the foundation for everything that follows, because nearly every major legal fight in the field turns on a question of jurisdiction: whose law applies to this plan, this claim, this bill. The state layer: insurance as a creature of state law Insurance regulation in the United States began with the states and, to a remarkable degree, remains with them. In 1944 the Supreme Court held in United States v. South-Eastern Underwriters Association that insurance was interstate commerce subject to federal antitrust law, which threatened the existing system of state regulation. Congress responded the next year with the McCarran-Ferguson Act, which declared that the continued regulation and taxation of insurance by the states was in the public interest and provided that no federal statute would be read to invalidate a state insurance law unless the federal statute specifically related to the business of insurance. The result is fifty-odd insurance codes, administered by elected or appointed insurance commissioners who coordinate through the National Association of Insurance Commissioners. The NAIC has no lawmaking power of its own, but its model acts, on subjects from utilization review to network adequacy to claims handling, are adopted in whole or in part by many states and give the state layer more uniformity than its fragmentation suggests. State insurance law does several distinct things to health coverage. It licenses the insurers that sell it and requires them to hold reserves adequate to pay claims. It reviews policy forms and, in many states, premium rates. It imposes benefit mandates, requiring insured policies to cover particular services or conditions, from newborn care and mammograms to, in some states, infertility treatment or autism therapies. It regulates the relationship between insurers and the providers in their networks, through laws on prompt payment, network adequacy, and in some states "any willing provider" requirements. And it regulates claims practices, prohibiting unfair denials and, in most states, giving policyholders a right to external review of medical-necessity decisions by an independent reviewer. What state law regulates, in each case, is an insurer, and specifically the insurance contract that the insurer issues. That turns out to be a critical limitation. The ERISA layer: the employer plan as a federal creature The Employee Retirement Income Security Act of 1974 was written primarily in response to the collapse of private pension plans, most famously the Studebaker plan failure in the 1960s, which left thousands of autoworkers without the pensions they had been promised. ERISA's central achievements were in the pension field: funding rules, vesting rules, and federal insurance of defined-benefit pensions. But the statute applies to all "employee benefit plans," a term that includes "employee welfare benefit plans" providing medical, surgical, or hospital care. Health plans came along for the ride. ERISA does comparatively little to regulate the substance of health coverage. As originally enacted, it required no particular benefits at all. What it did impose were procedural and fiduciary duties: plans must be established under a written instrument, must disclose their terms to participants, must follow reasonable claims procedures, and must be administered by fiduciaries who act solely in the interest of participants and beneficiaries. It also created a federal civil enforcement scheme, in section 502(a), that allows participants to sue to recover benefits due under the plan and to obtain certain equitable relief. And it contained a preemption clause of unusual reach. Section 514(a) provides that ERISA supersedes "any and all State laws insofar as they may now or hereafter relate to any employee benefit plan." The clause is then qualified by a savings clause, which preserves state laws "which regulate insurance, banking, or securities," and qualified again by the so-called deemer clause, which provides that an employee benefit plan shall not be "deemed" to be an insurance company for purposes of any state law purporting to regulate insurance. Read together, these clauses produce the division that organizes the whole field. When an employer buys a group insurance policy from an insurer, the plan is "insured" or "fully insured." The employer's plan is governed by ERISA, but the insurance policy the plan purchases is an insurance contract, and state laws regulating that contract are saved from preemption. State benefit mandates, external review laws, and claims-handling rules therefore reach the insured plan indirectly, through the policy. When an employer instead pays claims out of its own assets, typically hiring an insurer or a third-party administrator to process them and often buying "stop-loss" coverage to protect against catastrophic losses, the plan is "self-funded" or "self-insured." Because of the deemer clause, a self-funded plan cannot be treated as an insurer, and state insurance law cannot reach it. The Supreme Court confirmed this reading in FMC Corp. v. Holliday in 1990, holding that a Pennsylvania law barring subrogation in auto-accident cases could not be applied to a self-funded plan. The practical consequences are large. Self-funding has grown steadily for decades, and today roughly three in five workers with employer coverage are in self-funded plans, including nearly all workers at large firms. For those workers, the state insurance code is largely irrelevant. Their plan's benefits are what the plan document says they are, subject to whatever federal law requires, and their remedies are those ERISA provides. The federal coverage layer: from COBRA to the ACA Until the 1980s, federal law had little to say about what health plans must cover or to whom they must offer it. That began to change with the Consolidated Omnibus Budget Reconciliation Act of 1985, which required employer plans to offer departing employees the right to continue coverage at their own expense. The Health Insurance Portability and Accountability Act of 1996 limited preexisting-condition exclusions in group plans, barred group plans from discriminating in eligibility based on health status, and guaranteed that small employers could buy coverage. The Mental Health Parity Act of 1996, the Newborns' and Mothers' Health Protection Act of 1996, and the Women's Health and Cancer Rights Act of 1998 added targeted requirements. The Mental Health Parity and Addiction Equity Act of 2008 extended parity requirements to substance use disorder benefits and to treatment limitations. These statutes had a distinctive design. Rather than choosing between state and federal regulation, Congress wrote the same requirements into three parallel places: the Public Health Service Act, enforced chiefly by the Department of Health and Human Services and the states against insurers; ERISA, enforced by the Department of Labor against private employer plans; and the Internal Revenue Code, enforced by the Treasury through excise taxes. The three departments issue joint regulations, and for insured products the states retain primary enforcement, with HHS stepping in only where a state fails to act. This "tri-agency" structure remains the frame for most federal health coverage rules, including the No Surprises Act and the transparency requirements examined later in this book. The Affordable Care Act, signed on March 23, 2010, used the same machinery on a much larger scale. Its insurance reforms, most of which took effect in 2014, did several things at once. They required insurers in the individual and small-group markets to accept all applicants (guaranteed issue) and to price coverage without regard to health status, varying premiums only by age within a three-to-one band, by tobacco use, by family size, and by geography. They required those plans to cover ten categories of "essential health benefits." They banned annual and lifetime dollar limits on essential benefits across nearly all plans, required coverage of dependent children to age twenty-six, required coverage of recommended preventive services without cost-sharing, capped out-of-pocket spending, and imposed medical loss ratio rules that require insurers to spend at least 80 percent of premiums in the individual and small-group markets, and 85 percent in the large-group market, on care and quality improvement or else pay rebates. They created marketplaces, or exchanges, through which individuals could buy coverage, and income-based premium tax credits and cost-sharing reductions to make that coverage affordable. They imposed an individual mandate to maintain coverage and, on larger employers, a requirement to offer affordable coverage to full-time employees or pay an assessment. And they expanded Medicaid to adults with incomes up to 138 percent of the federal poverty level. The ACA's reach, however, varies by market segment, and this is often misunderstood. The essential health benefits requirement applies only to non-grandfathered individual and small-group insured coverage; it does not apply to large-group insured plans or to self-funded plans of any size. The rating rules likewise apply only to the individual and small-group markets. The preventive-services, dependent-coverage, annual and lifetime limit, and out-of-pocket maximum rules apply more broadly, including to self-funded plans. The structure of the ACA thus preserves, rather than erases, the older division between insured and self-funded coverage. Public programs and their private operators The public programs add further layers. Medicare, enacted in 1965, is a federal program administered by the Centers for Medicare and Medicaid Services. Its Medicare Advantage option allows beneficiaries to enroll in private plans that receive capitated payments from the government. The Medicare statute preempts state law with respect to Medicare Advantage plans, other than state laws on licensing and plan solvency, so those plans answer mainly to CMS. Disputes over denied care follow Medicare's own administrative appeals process, not ERISA and not state external review. Medicaid, enacted at the same time, is jointly financed and jointly administered. The federal government sets baseline requirements and pays a share of costs, the Federal Medical Assistance Percentage, that ranges from 50 percent to over 75 percent depending on state income, and a special 90 percent share for the ACA expansion population. States design and run their programs within those requirements, and they may seek waivers of many requirements under section 1115 of the Social Security Act. Most Medicaid enrollees are now in managed care plans run by private insurers under contracts with the state, which means that a Medicaid enrollee's coverage dispute may involve federal Medicaid law, state administrative law, the state's contract with the plan, and federal regulations on Medicaid managed care, all at once. Beneficiaries retain a right to a state fair hearing. The differences among these arrangements can be summarized, with some simplification, as Table 1 sets out. Table 1. Principal legal regime by type of coverage. Type of coverage Primary regulator State benefit mandates apply? ACA essential health benefits apply? Main route for disputing a denial Individual market (including marketplace) State insurance department; HHS Yes Yes (non-grandfathered) Insurer appeal, then state or federal external review; state-law claims Small-group, fully insured State insurance department; DOL for the plan Yes, through the policy Yes (non-grandfathered) Plan appeal, external review, then ERISA suit Large-group, fully insured State insurance department; DOL for the plan Yes, through the policy No Plan appeal, external review, then ERISA suit Self-funded employer plan Department of Labor No No Plan appeal, federal external review, then ERISA suit Medicaid managed care State Medicaid agency; CMS Program rules govern Benchmark rules for expansion adults Plan appeal, then state fair hearing Medicare Advantage CMS No (largely preempted) No (Medicare benefits govern) Medicare administrative appeals Why the seams matter This layered structure has three consequences that recur throughout the chapters that follow. The first is that reform efforts tend to leak. A state that wants to protect its residents from some practice, whether surprise bills, aggressive pharmacy benefit management, or narrow networks, can regulate only insured products and must accept that a majority of its employer-covered residents are beyond its reach. A federal agency that wants to regulate the same practice must fit its rule into a statute that often preserves the state role for insured coverage. The result is a constant search for workarounds: state laws framed as regulating providers rather than plans, federal laws written to operate alongside "specified state laws," and litigation over whether a given state law "relates to" employee benefit plans. The second is that the individual's legal position depends on facts the individual rarely knows. The Texas patient in the introduction did not choose whether her employer self-funded. A family moving from marketplace coverage to an employer plan may lose state protections without noticing. A retiree who moves from traditional Medicare into Medicare Advantage enters a different appeals system. Legal literacy in health insurance begins with identifying the regime. The third is that the fights are rarely about coverage alone. They are fights about power: whether Washington can require states to expand Medicaid, whether states can require self-funded plans to report their claims, whether an arbitrator or a court or a market sets the price of an out-of-network emergency visit. Comparative health systems make this vivid. In countries such as Germany, the Netherlands, and Switzerland, which also rely on private or quasi-private insurers, a single national framework sets the rules for all insurers and pools risk among them through formal risk-equalization schemes. The American system, by contrast, never made that choice. It inherited a state insurance system, overlaid a federal benefits statute written for pensions, and then added a federal coverage regime that works through both. Each subsequent chapter examines one of the places where that inheritance produces conflict. Chapter 2: The Affordable Care Act in Court No American statute of the past half-century has been litigated as persistently as the Affordable Care Act. Within minutes of the President's signature in March 2010, state attorneys general filed suit. Since then the law has reached the Supreme Court in some form more than half a dozen times, and the lower courts have heard hundreds of challenges to its provisions and implementing rules. Reading this litigation as a whole, rather than case by case, reveals a pattern. The early cases were existential: they asked whether the law could stand at all. Each time, the Court found a way to preserve it, but often by reshaping it. The later cases have been narrower and more technical, aimed at particular mechanisms, agencies, and payments. The ACA has proven very hard to kill in court, but it has proven quite easy to alter, and the alterations have accumulated. The foundational case: NFIB v. Sebelius The first great test came in National Federation of Independent Business v. Sebelius, decided on June 28, 2012. Twenty-six states and a small-business association challenged two provisions. The first was the individual mandate, which required most Americans to maintain minimum essential coverage or pay a penalty. The second was the Medicaid expansion, which, as written, required states to extend Medicaid to nearly all adults with incomes up to 138 percent of the poverty line or risk losing all their federal Medicaid funds. On the mandate, five justices concluded that Congress lacked power under the Commerce Clause to compel individuals to buy a product. Chief Justice Roberts, writing for himself, reasoned that the commerce power allows Congress to regulate existing economic activity but not to compel people to enter commerce, and the four joint dissenters agreed. But the Chief Justice then joined the four more liberal justices to uphold the mandate as an exercise of the taxing power. The payment was collected by the Internal Revenue Service, calculated by reference to income, not so large as to be coercive, and carried no other legal consequence. It could therefore fairly be read as a tax on going without insurance. On Medicaid, seven justices agreed that the expansion, as structured, was unconstitutionally coercive. The Spending Clause allows Congress to attach conditions to federal grants, but a condition becomes coercion when the financial pressure leaves states no real choice. Threatening a state with the loss of all existing Medicaid funding, which in many states amounted to more than 10 percent of the state budget, to force it into a new program was, in the Chief Justice's phrase, "a gun to the head." The remedy, supported by five justices, was not to strike the expansion but to limit the penalty: the Secretary of Health and Human Services could withhold new expansion funding from states that declined, but not their existing Medicaid funding. The expansion survived, but as an option. That remedy is one of the most consequential judicial rewrites of a federal statute in modern times. It turned Medicaid expansion from a national floor into a state decision, which is the subject of Chapter 3. And it gave the Spending Clause a new, if uncertain, limit that states have invoked in many later challenges to federal grant conditions. Reading the statute to save it: King v. Burwell The second existential challenge was statutory. The ACA authorized premium tax credits for people who enrolled in coverage "through an Exchange established by the State under section 1311." Most states declined to build their own exchanges, and the federal government operated the marketplace for them through HealthCare.gov. The plaintiffs in King v. Burwell argued that the statute's words meant exactly what they said: tax credits were available only in states that had established their own exchanges, not in the thirty-odd states using the federal platform. The stakes were enormous. Without tax credits, most marketplace enrollees in federal-platform states could not afford coverage. And because the ACA's individual mandate exempted people for whom coverage was unaffordable, while its insurance rules still required guaranteed issue and community rating, the loss of subsidies would have left mostly sicker people in the market, driving up premiums in the pattern economists call a death spiral. In June 2015 the Court ruled 6 to 3 that tax credits were available in all states. Chief Justice Roberts, writing for the majority, concluded that the phrase was ambiguous when read in the context of the whole statute and that the plaintiffs' reading would destroy the very markets the law was designed to create. "Congress passed the Affordable Care Act to improve health insurance markets, not to destroy them," he wrote. Notably, the Court declined to defer to the IRS's interpretation under the Chevron doctrine, reasoning that a question of such economic and political significance was not one Congress would have implicitly delegated to the agency. That move anticipated the Court's later abandonment of Chevron altogether in Loper Bright Enterprises v. Raimondo in 2024, a development that makes agency rules implementing the ACA more vulnerable today than they were a decade ago. Money owed: cost-sharing reductions and risk corridors A third line of litigation concerned money that the ACA promised to insurers and that the government did not pay. The cost-sharing reduction program requires insurers to reduce deductibles and copayments for lower-income marketplace enrollees in silver plans, and the statute directs the government to reimburse insurers for those reductions. But Congress never enacted a specific appropriation for the reimbursements. In House of Representatives v. Burwell, the House sued the Obama administration for paying them anyway, and in 2016 a federal district judge in Washington agreed that the payments were being made without an appropriation, though she stayed her ruling pending appeal. In October 2017 the Trump administration stopped the payments. Insurers responded in two ways. In the market, most loaded the cost of the unfunded reductions onto the premiums of silver plans, a practice known as silver loading. Because premium tax credits are pegged to the benchmark silver premium, silver loading increased tax credits for subsidized enrollees and, paradoxically, made coverage cheaper for many of them while raising federal spending. In court, insurers sued the United States under the Tucker Act for the unpaid reimbursements. The Federal Circuit held in 2020 that the statute created an enforceable obligation to pay despite the lack of an appropriation, but in Community Health Choice v. United States it also held that insurers' damages must be reduced by the additional premium tax credit revenue they received through silver loading. The risk corridors program produced the most consequential of these money cases. The ACA created temporary risk corridors for 2014 through 2016 to cushion insurers against unexpectedly high losses in the new marketplaces: insurers that did better than expected would pay into the program, and those that did worse would be paid out of it. Losses far exceeded gains, and Congress, in appropriations riders beginning in late 2014, barred HHS from using other funds to cover the shortfall. Several insurers, including some of the nonprofit cooperatives the ACA had created, failed. In Maine Community Health Options v. United States, decided in April 2020, the Supreme Court held 8 to 1 that the statute's command that the Secretary "shall pay" created a binding obligation that the appropriations riders did not repeal, and that insurers could recover the unpaid amounts, totaling about $12 billion, in the Court of Federal Claims. The case stands as an important lesson in statutory drafting and in the difference between an obligation to pay and the availability of funds to pay it. The mandate that became zero: California v. Texas The Tax Cuts and Jobs Act of 2017 reduced the individual mandate penalty to zero, effective in 2019, without repealing the mandate itself. A group of states led by Texas then argued that a mandate with no penalty could no longer be sustained as a tax, was therefore unconstitutional under NFIB's Commerce Clause holding, and was so central to the statute that the entire ACA must fall with it. In December 2018 a federal district judge in Fort Worth agreed and declared the whole law invalid, a ruling stayed pending appeal. The Fifth Circuit agreed that the zeroed mandate was unconstitutional but sent the question of severability back for further analysis. In California v. Texas, decided in June 2021, the Supreme Court did not reach the merits. By a vote of 7 to 2, it held that neither the states nor the individual plaintiffs had standing to challenge a provision that imposed no penalty and that no government official could enforce. The practical result is that the mandate remains on the books as an unenforceable command, and the rest of the ACA remains in force. The economic result was more modest than many predicted. The loss of the penalty did not cause the marketplaces to collapse, largely because premium tax credits shield subsidized enrollees from premium increases, which keeps healthier people in the market even without a mandate. A handful of states, including California, Massachusetts, New Jersey, Rhode Island, and the District of Columbia, enacted their own individual mandates with state tax penalties. Preventive care and the appointments question: Braidwood The ACA requires most private plans, including self-funded plans, to cover four categories of preventive services without cost-sharing: items and services with an "A" or "B" rating from the United States Preventive Services Task Force; immunizations recommended by the Advisory Committee on Immunization Practices of the Centers for Disease Control and Prevention; preventive care for children recommended by the Health Resources and Services Administration; and preventive care for women under HRSA guidelines, which include contraception. The contraception requirement generated its own lengthy litigation over religious objections. In Burwell v. Hobby Lobby Stores in 2014, the Supreme Court held 5 to 4 that the Religious Freedom Restoration Act exempted closely held for-profit corporations with religious objections. In Zubik v. Burwell in 2016 it remanded challenges by religious nonprofits for the parties to seek a compromise. In Little Sisters of the Poor v. Pennsylvania in 2020 it upheld the agencies' authority to issue broad religious and moral exemptions, leaving challenges to how those exemptions were adopted to continue in the lower courts. The more fundamental challenge came in Braidwood Management v. Becerra, brought by Christian-owned businesses and individuals in Texas. They objected on religious grounds to covering pre-exposure prophylaxis drugs that prevent HIV infection, which the Task Force had rated "A," and they also argued that the Task Force's members were officers of the United States who had not been appointed in the manner the Constitution's Appointments Clause requires. The district court agreed with the appointments argument and enjoined enforcement of Task Force recommendations nationwide. The Fifth Circuit agreed on the constitutional point but limited relief to the plaintiffs. In Kennedy v. Braidwood Management, decided on June 27, 2025, the Supreme Court reversed by a vote of 6 to 3. Justice Kavanaugh's opinion held that Task Force members are inferior officers, not principal officers, because the Secretary of Health and Human Services may remove them at will and may review and block their recommendations before those recommendations take effect. Inferior officers may be appointed by a department head, and the Court concluded that the Secretary had authority to appoint them. The preventive-services requirement therefore stands. The decision preserved free coverage of dozens of preventive services for well over a hundred million people. But its reasoning had a second edge. The constitutional validity of the Task Force now rests expressly on the Secretary's supervisory control, which means that the content of the ACA's preventive-care guarantee depends more heavily on the judgment of whoever holds that office. That dependence became concrete almost immediately. In June 2025 the Secretary removed all seventeen members of the immunization advisory committee and appointed replacements, and the reconstituted committee began revisiting longstanding vaccine recommendations. Because the ACA ties no-cost vaccine coverage to that committee's recommendations, changes in its advice flow directly into what private plans must cover without cost-sharing. The litigation that once threatened the preventive-care mandate from outside has given way to a contest over who controls it from inside. The administrative turn As the existential challenges receded, ACA litigation shifted toward the rules that implement it. Each administration has used rulemaking to push the law in its preferred direction: expanding or restricting short-term plans that escape ACA rules, loosening or tightening standards for association health plans, adjusting open enrollment periods and eligibility verification, redefining the scope of the ACA's nondiscrimination provision, section 1557. Opponents of each change have sued, and courts have frequently blocked or vacated the rules. A recent example illustrates the pattern. In June 2025 CMS finalized a "marketplace integrity" rule that, among other things, shortened open enrollment, tightened income verification, allowed insurers to require payment of past-due premiums before new coverage began, and excluded certain immigrants from eligibility. A coalition of cities and physician groups challenged it, and in August 2025 a federal district court in Maryland, in City of Columbus v. Kennedy, stayed several of its provisions before they took effect, finding them likely arbitrary or contrary to the statute. Meanwhile the largest change to the marketplaces in years occurred not through litigation or regulation but through legislative inaction. The enhanced premium tax credits first enacted in the American Rescue Plan Act of 2021, and extended through 2025 by the Inflation Reduction Act, removed the 400 percent of poverty cap on eligibility and increased credits at every income level. Enrollment in the marketplaces more than doubled while they were in effect. Congress did not extend them, and they expired at the start of 2026. Marketplace enrollment fell in 2026 for the first time in seven years, declining in every state except New Mexico, which used state funds to replace the lost federal subsidies. Insurers' premium filings for 2027 propose further double-digit increases. What the litigation reveals Three lessons emerge from fifteen years of ACA litigation. First, the ACA's durability comes from its entanglement. By the time the major existential challenges reached the Court, millions of people depended on the marketplaces and Medicaid expansion, insurers had priced products around the law's rules, and states had built programs on its funding. Justices across the spectrum have been reluctant to unravel an arrangement of that scale, and the Court has repeatedly chosen narrow grounds: a saving construction in NFIB and King, standing in California v. Texas, an appointments analysis in Braidwood that preserved the existing structure. Second, the law's vulnerabilities lie in its dependence on appropriations, on executive discretion, and on the states. The cost-sharing and risk-corridor episodes showed how a program can be undermined through the power of the purse even when courts eventually enforce the government's obligations. Braidwood showed how a coverage guarantee can survive in court while becoming more dependent on executive judgment. And NFIB's Medicaid holding showed how a national program can be converted into a set of state choices. Third, the ACA's fate is now determined mostly outside the Supreme Court. Whether the marketplaces remain affordable depends on the level of federal subsidy, which is a legislative choice. Whether Medicaid covers low-income adults depends on state decisions and on the federal conditions attached to expansion funding, which the next chapter examines. And whether the law's consumer protections have practical force depends on the administrative rules that implement them and on the willingness of agencies to enforce them. Chapter 3: Medicaid Expansion: From Coercion to Conditions Medicaid is the largest health insurer in the United States by enrollment, covering more than seventy million people across its core program and the Children's Health Insurance Program. It is also the clearest example in American law of what the Supreme Court calls cooperative federalism: a program in which the federal government sets terms and pays most of the bill, and states choose whether to participate and administer the program within federal limits. Every state has participated since Arizona joined in 1982. But participation is a legal choice, renewed in effect each year, and the terms of the bargain are always contestable. The Affordable Care Act tried to settle one of those terms nationally, by requiring states to cover all non-elderly adults with incomes up to 138 percent of the federal poverty line. The Supreme Court's decision in NFIB v. Sebelius unsettled it again, and for more than a decade since, the expansion has been fought out state by state, in legislatures, at the ballot box, in negotiations over federal waivers, and in federal court. In 2025 Congress changed the terms once more, attaching work requirements and other conditions to expansion coverage. This chapter follows that contest and then turns to a question that has grown steadily more important: when a state fails to honor Medicaid's requirements, who can make it comply? The optional expansion Before the ACA, Medicaid did not cover poor adults simply because they were poor. Eligibility was built around categories: children, pregnant women, parents of dependent children, the elderly, the blind, and people with disabilities. Within those categories, states set income limits, and for parents they were often far below the poverty line. Non-disabled adults without dependent children were generally ineligible no matter how low their income, unless a state obtained a waiver to cover them. The ACA replaced that structure for adults with a single income test. It created a new eligibility group of adults aged nineteen to sixty-four with incomes up to 133 percent of the poverty line, effectively 138 percent after a standard income disregard, and it promised states that the federal government would pay 100 percent of the costs for newly eligible adults from 2014 through 2016, phasing down to a permanent 90 percent from 2020. That is a far richer match than the regular rate, which ranges from 50 percent in the wealthiest states to over 75 percent in the poorest. After NFIB, states that declined the expansion lost only the expansion funding. Twenty-five states and the District of Columbia expanded in the first wave, most effective January 1, 2014. Others followed over the next decade, some through legislation, some through negotiated waivers, and several through ballot initiatives that bypassed reluctant legislatures. North Carolina, after years of debate, expanded in December 2023. By 2025, forty states and the District of Columbia had adopted the expansion. Ten had not: Alabama, Florida, Georgia, Kansas, Mississippi, South Carolina, Tennessee, Texas, Wisconsin, and Wyoming. Wisconsin covers adults up to the poverty line through a waiver without taking the expansion match, and Georgia covers a limited population through its Pathways program, discussed below. In non-expansion states, a coverage gap opened. Adults with incomes below the poverty line generally earn too much for the state's traditional Medicaid categories and too little for marketplace premium tax credits, which the ACA made available only above 100 percent of poverty on the assumption that everyone below that level would have Medicaid. Millions of people fell into that gap. Expansion by ballot and by lawsuit Where legislatures refused, advocates turned to direct democracy. Maine voters approved expansion by initiative in 2017; Idaho, Nebraska, and Utah followed in 2018; Oklahoma and Missouri in 2020; and South Dakota, through a constitutional amendment, in 2022. The ballot route generated its own legal fights, because an initiative can create an entitlement but cannot by itself appropriate money or compel a reluctant executive to implement it. In Maine, the governor refused to implement the voter-approved expansion, and advocates had to go to state court to force the state to submit the required plan amendment to federal regulators; implementation came only after a new governor took office in 2019. In Missouri, voters approved a constitutional amendment in August 2020 requiring expansion, but the legislature passed a budget that did not fund it, and the state withdrew its plan amendment. In Doyle v. Tidball in 2021, the Missouri Supreme Court unanimously held that the amendment was valid and that the newly eligible adults were entitled to enroll under the existing Medicaid appropriation, and coverage began that fall. Missouri's legislature later placed a measure on the ballot designed to make future initiatives harder to pass, a reminder that direct democracy on health coverage invites a response in kind. Waivers, work requirements, and the limits of Secretary discretion The second route for expansion ran through section 1115 of the Social Security Act, which authorizes the Secretary of Health and Human Services to waive many Medicaid requirements for "experimental, pilot, or demonstration" projects that the Secretary judges "likely to assist in promoting the objectives" of the program. Republican-led states used waivers to design expansions that looked more like private insurance. Arkansas used Medicaid funds to buy marketplace coverage for expansion adults, the so-called private option. Indiana's program required enrollees to contribute to health savings-style accounts and imposed lockouts for nonpayment above the poverty line. In January 2018 the first Trump administration invited states to go further by conditioning coverage on work or "community engagement." It approved requirements in Kentucky and Arkansas that year and later in several other states. Arkansas implemented its requirement in mid-2018, requiring many expansion adults to report eighty hours a month of work or qualifying activities through an online portal. Within months, roughly eighteen thousand people had lost coverage, the great majority because of reporting failures rather than documented unemployment. Beneficiaries sued. In Stewart v. Azar in 2018 and Gresham v. Azar in 2019, Judge James Boasberg of the federal district court in Washington vacated the Kentucky and Arkansas approvals. In February 2020 the D.C. Circuit affirmed in Gresham, holding that the principal objective of Medicaid, as the statute defines it, is to furnish medical assistance, and that the Secretary had acted arbitrarily by approving a project without adequately considering its effect on coverage. The Supreme Court agreed to review the decision, but after the Biden administration withdrew the work requirement approvals, the Court vacated the ruling as moot in 2022. The legal question of how far section 1115 permits the Secretary to condition coverage was left formally open. Georgia took a different path. Its Pathways to Coverage program, approved in 2020, extended Medicaid only to adults up to 100 percent of the poverty line, without taking the full expansion, and required eighty hours a month of qualifying activity from the outset. When the Biden administration moved to rescind the work requirement, Georgia sued and won; a federal district court held in 2022 that the rescission was arbitrary. Pathways launched in July 2023. Its enrollment remained in the low thousands during its first year, far below the number of Georgians who would have been eligible under a full expansion, and much of the debate over the program turned on whether the reporting burden, rather than the work requirement itself, explained the gap. Georgia later sued again when CMS declined to extend the program's full term, and in 2024 a federal court again sided with the state. The unwinding Medicaid enrollment grew sharply during the COVID-19 public health emergency, because the Families First Coronavirus Response Act of 2020 offered states a higher federal match on the condition that they keep enrollees continuously covered. When Congress ended that condition, effective March 31, 2023, states began redetermining the eligibility of everyone on their rolls. Over the following year and a half, more than twenty-five million people were disenrolled. A large share lost coverage for procedural reasons, such as failing to return renewal paperwork, rather than because they were found ineligible. Some regained coverage after a gap; others moved to employer or marketplace coverage; others became uninsured. The unwinding matters to the legal story for two reasons. It demonstrated at national scale what the Arkansas experiment had shown locally: that administrative burdens, even without a change in substantive eligibility, determine who keeps coverage. And it showed the limits of federal oversight. CMS paused procedural disenrollments in some states and required others to reinstate people wrongly terminated when their systems failed to renew individuals automatically, but it had to work largely through negotiation with states rather than through litigation. The 2025 reconciliation law The legislation Congress passed through budget reconciliation and the President signed on July 4, 2025, commonly known as the One Big Beautiful Bill Act, made the most significant changes to Medicaid since the ACA. Its central features for expansion states can be summarized as follows. First, it requires states to impose work or community engagement requirements on expansion adults aged nineteen to sixty-four as a condition of eligibility: eighty hours a month of work, community service, participation in a work program, or school enrollment, or income equivalent to eighty hours at the federal minimum wage. The requirement is due to take effect by the start of 2027, with authority for the Secretary to give states making good-faith efforts additional time through the end of 2028. The statute exempts, among others, pregnant and postpartum women, people who are medically frail, parents and caretakers of young children and of people with disabilities, and people in treatment for substance use disorders. Unlike the earlier waiver programs, these requirements do not depend on the Secretary's discretion under section 1115; they are written into the statute, which removes the principal ground on which Gresham was decided. Second, it requires states to redetermine the eligibility of expansion adults every six months rather than annually. Third, it limits retroactive coverage, which previously reached back three months before application. Fourth, it requires states to impose cost-sharing of up to $35 per service on expansion adults with incomes above the poverty line, with exceptions for certain services. Fifth, it restricts the financing tools states have long used to fund their share of Medicaid: it limits new or increased provider taxes, phases down the permitted level of those taxes in expansion states over several years, and caps state-directed payments to providers at or near Medicare rates. Sixth, it ended the temporary financial incentive the American Rescue Plan had offered to states that newly expanded, and narrowed eligibility for some lawfully present immigrants. The Congressional Budget Office projected that the law would increase the number of uninsured Americans by about ten million by 2034, with Medicaid changes accounting for the largest part. The financing provisions may prove as consequential as the eligibility provisions. Provider taxes and directed payments have allowed many states to draw down federal funds at low net state cost, and limits on them raise the effective price of expansion to state budgets. That matters because of a feature of state law that has received less attention than it deserves. A number of expansion states, including Arizona, Arkansas, Illinois, Indiana, Montana, New Hampshire, North Carolina, Utah, and Virginia, adopted "trigger" provisions that end or reconsider the expansion if the federal match falls below 90 percent or if the state's financing mechanisms are disrupted. The 2025 law leaves the 90 percent match intact, but its financing limits may test some of those triggers, and every expansion state will weigh the higher administrative costs of work requirements and more frequent redeterminations against the federal dollars expansion brings. The main Medicaid expansion and enforcement cases discussed in this chapter are collected in Table 2. Table 2. Leading cases on Medicaid expansion and enforcement. Case Court and year Question Outcome NFIB v. Sebelius Supreme Court, 2012 Could Congress require expansion on pain of losing all Medicaid funds? No; expansion became optional Gresham v. Azar D.C. Circuit, 2020 Was approval of Arkansas work requirements lawful under section 1115? No; Secretary ignored coverage effects Doyle v. Tidball Missouri Supreme Court, 2021 Was the voter-approved expansion enforceable without a specific appropriation? Yes; enrollment ordered Armstrong v. Exceptional Child Center Supreme Court, 2015 Can providers sue in equity to enforce Medicaid rate requirements? No Health and Hospital Corp. of Marion County v. Talevski Supreme Court, 2023 Can Spending Clause statutes create rights enforceable under section 1983? Yes, if rights are unambiguously conferred Medina v. Planned Parenthood South Atlantic Supreme Court, 2025 Does the free-choice-of-provider provision create an enforceable right? No Who enforces Medicaid? A federal-state program creates a distinctive enforcement problem. The primary sanction for a state's noncompliance is the Secretary's power to withhold federal funds, a power so drastic it is almost never used. For decades, the practical enforcement of Medicaid's requirements came instead from private lawsuits, chiefly suits by beneficiaries and providers under 42 U.S.C. section 1983, the Reconstruction-era statute that allows individuals to sue state officials who deprive them of federal "rights." That avenue has narrowed steadily. In Wilder v. Virginia Hospital Association in 1990, the Supreme Court allowed hospitals to sue under section 1983 to enforce Medicaid's then-existing rate provisions. Congress repealed that provision in 1997, and in Gonzaga University v. Doe in 2002 the Court tightened the test: a spending statute creates enforceable rights only if its text unambiguously confers them on an identifiable class. In Armstrong v. Exceptional Child Center in 2015, the Court held that Medicaid providers could not bypass section 1983 by suing directly in equity under the Supremacy Clause to enforce the requirement that payment rates be sufficient to ensure access to care; that provision, the Court said, was enforceable only by the Secretary. In Health and Hospital Corporation of Marion County v. Talevski in 2023, the Court rejected a broad argument that Spending Clause statutes can never create section 1983 rights, holding that the nursing-home residents' rights provisions of the Federal Nursing Home Reform Amendments were enforceable. But in Medina v. Planned Parenthood South Atlantic, decided in June 2025, the Court held 6 to 3 that Medicaid's "any qualified provider" provision, which entitles beneficiaries to obtain care from any qualified and willing provider, does not create a right enforceable under section 1983. South Carolina had excluded Planned Parenthood from its Medicaid program, and a beneficiary who wished to receive care there sued. Justice Gorsuch's opinion stressed that section 1983 enforcement of spending statutes is the atypical case and that the provision, though phrased in terms of what state plans must provide, did not speak with the clarity Gonzaga requires. The effect of Medina extends beyond family planning. Lower courts are now reassessing which other Medicaid provisions, from reasonable promptness in providing assistance to the requirements of the children's screening and treatment benefit, remain enforceable by beneficiaries. Where private enforcement is unavailable, compliance depends on CMS oversight, on state administrative hearings, and on whatever state-law remedies exist. As the 2025 law shifts Medicaid toward more frequent eligibility checks and new conditions, the question of who can hold states to the federal rules, including the statutory exemptions from work requirements, is likely to be among the most litigated in the field. The lesson of the expansion fight Medicaid expansion began as a national entitlement and has become a negotiated condition. NFIB gave states the right to refuse; section 1115 gave them room to redesign; the 2025 law now tells them what they must require of expansion adults in return for the enhanced federal match. At each stage, the decisive legal questions have concerned federalism and administrative power rather than the substance of health care. And at each stage, the evidence from Arkansas, Georgia, and the unwinding has pointed to the same practical truth: the rules about proving eligibility can matter as much as the rules about who is eligible. Whether courts will police that distinction, now that the statutory basis for work requirements is explicit and private enforcement is narrower, is the open question the next few years will answer. Chapter 4: ERISA Preemption: The Clause That Shapes Everything If one provision of federal law explains why American health insurance regulation looks the way it does, it is section 514 of ERISA. Its preemption clause has been called one of the broadest in the United States Code, and for two decades the Supreme Court read it almost literally, striking down state laws with only a tenuous connection to employee benefit plans. The Court has since pulled back, developing a more functional test that allows states to regulate the health care market in ways that affect plans indirectly. But the clause still defines the outer limit of state health reform. Whenever a state tries to control costs, protect patients from a billing practice, gather data about prices, or regulate the intermediaries that manage drug benefits, the first legal question is whether ERISA allows it. The text and its three clauses Section 514 has three moving parts. The preemption clause, section 514(a), provides that ERISA shall "supersede any and all State laws insofar as they may now or hereafter relate to any employee benefit plan." The savings clause, section 514(b)(2)(A), provides that nothing in ERISA "shall be construed to exempt or relieve any person from any law of any State which regulates insurance, banking, or securities." And the deemer clause, section 514(b)(2)(B), provides that no employee benefit plan "shall be deemed to be an insurance company or other insurer" for purposes of any state law "purporting to regulate insurance companies [or] insurance contracts." The legislative history suggests that the breadth of the preemption clause was deliberate. Earlier drafts would have preempted state law only on subjects ERISA specifically regulated. The conference committee substituted the broader "relate to" language, reportedly out of concern that multistate employers should not face a patchwork of inconsistent state rules governing their plans. For pension plans, whose central problems were funding and vesting, that choice made obvious sense. For health plans, which ERISA barely regulated substantively, it created a vacuum: state law was displaced, but nothing federal took its place. The expansive era The Supreme Court's early cases took the "relate to" language at face value. In Shaw v. Delta Air Lines in 1983, the Court held that a state law "relates to" a benefit plan "if it has a connection with or reference to such a plan," and it held that New York's laws requiring employers to provide certain disability and pregnancy benefits were preempted as applied to ERISA plans. In Pilot Life Insurance Co. v. Dedeaux in 1987, it held that state-law claims for bad-faith handling of a benefit claim were preempted, because they related to the plan and were not saved as laws regulating insurance, and because ERISA's civil enforcement provisions were meant to be the exclusive remedy for claims about plan benefits. That holding, discussed further in the next chapter, has shaped managed care litigation ever since. The savings clause received a comparatively generous reading. In Metropolitan Life Insurance Co. v. Massachusetts in 1985, the Court upheld a Massachusetts law requiring health insurance policies to cover mental health treatment, reasoning that a mandated-benefit law regulates the content of insurance contracts and is therefore saved as a law that regulates insurance. That holding is the legal foundation for every state benefit mandate applied to insured employer plans. But the deemer clause marked its limit. In FMC Corp. v. Holliday in 1990, the Court held that a self-funded plan could not be subjected to state insurance law at all, even through the savings clause. The combined effect of Metropolitan Life and FMC is the insured-versus-self-funded division described in Chapter 1: mandates reach insured plans; they do not reach self-funded ones. The retreat: Travelers and the functional test By the early 1990s, it had become clear that a literal reading of "relate to" was unworkable. As Justice Scalia later observed, everything relates to everything else in some sense. The turning point came in New York State Conference of Blue Cross and Blue Shield Plans v. Travelers Insurance Co. in 1995. New York imposed surcharges on hospital bills paid by commercial insurers and HMOs, but not on bills paid by Blue Cross plans, as a way to subsidize the Blues' role in insuring high-risk individuals. Commercial insurers argued that the surcharges were preempted because they affected the cost of benefits provided by ERISA plans. A unanimous Court disagreed. It began with the presumption that Congress does not casually displace state regulation in areas of traditional state concern, including health care. It held that the "relate to" language must be read in light of ERISA's objectives, which were to protect plan participants and to allow uniform plan administration, not to free plans from every state law that might affect their costs. A state law that merely creates economic incentives influencing a plan's choice of insurer, without binding plan administrators to any particular choice, has only an "indirect economic influence" on plans and is not preempted. Travelers established the modern framework. A state law is preempted if it has an impermissible "reference to" ERISA plans, meaning it acts immediately and exclusively on them or depends on their existence to operate, or if it has an impermissible "connection with" them, meaning it governs a central matter of plan administration or interferes with nationally uniform plan administration. Subsequent cases applied the functional approach to uphold a range of general state laws: a state tax on hospitals that affected plan-operated facilities in De Buono v. NYSA-ILA Medical and Clinical Services Fund in 1997, and a prevailing-wage law in California Division of Labor Standards Enforcement v. Dillingham Construction the same year. The line has not always been easy to draw. In Egelhoff v. Egelhoff in 2001, the Court held preempted a Washington law that automatically revoked a spouse's designation as beneficiary upon divorce, because it dictated who would receive plan benefits, a core matter of plan administration. Pay-or-play laws and the employer mandate question The functional test was soon tested by state and local efforts to make employers pay for health coverage. In 2006 Maryland enacted the Fair Share Health Care Fund Act, which required employers with more than ten thousand employees in the state to spend at least 8 percent of payroll on health care or pay the difference to the state. Only one employer, Wal-Mart, met the definition. In Retail Industry Leaders Association v. Fielder in 2007, the Fourth Circuit held the law preempted, reasoning that the only rational choice for an employer was to restructure its ERISA plan to meet the spending threshold, which amounted to a mandate on plan design. San Francisco's Health Care Security Ordinance, enacted the same year, reached a different result. It required medium and large employers to make specified health care expenditures for their employees but allowed them to satisfy the obligation through payments to the city's health access program as well as through their own plans. In Golden Gate Restaurant Association v. City and County of San Francisco in 2008, the Ninth Circuit upheld the ordinance, concluding that it did not require employers to establish or alter an ERISA plan and did not bind plan administration. The Supreme Court declined review. The difference between the two cases illustrates the practical meaning of the functional test. A law that effectively forces employers to change their plans is preempted; a law that imposes a financial obligation employers may meet outside their plans is not. The distinction also illuminates why the ACA's employer responsibility provision was enacted at the federal level. Congress is not bound by ERISA preemption, and it could impose a pay-or-play requirement directly. Data, cost control, and Gobeille The limits of the functional test reappeared when states tried to understand what health care actually costs. Beginning in the 2000s, many states created all-payer claims databases, which collect data on claims paid by insurers and plans in order to study prices, utilization, and quality. Vermont's law required all health insurers, including third-party administrators of self-funded plans, to report claims data. Liberty Mutual, which self-funded its employee plan and used Blue Cross Blue Shield of Massachusetts as its administrator, instructed Blue Cross not to comply. In Gobeille v. Liberty Mutual Insurance Co. in 2016, the Supreme Court held 6 to 2 that ERISA preempted Vermont's law as applied to self-funded plans. Reporting, disclosure, and recordkeeping, the Court reasoned, are central to plan administration and are among the core subjects ERISA regulates. Allowing each state to impose its own reporting obligations would expose plans to exactly the kind of inconsistent requirements the preemption clause was meant to prevent. Gobeille left all-payer claims databases dependent on voluntary submissions from self-funded plans, which account for a large share of the market in most states. It prompted calls for federal action, and in the Consolidated Appropriations Act of 2021 Congress authorized grants to states for databases and directed the Department of Labor to develop a standardized reporting format that self-funded plans could voluntarily use. The case is a reminder that information itself is regulated territory, a point that becomes important in the chapters on surprise billing and price transparency. Pharmacy benefit managers: Rutledge and Mulready The most active preemption battleground of the past decade concerns pharmacy benefit managers. PBMs administer prescription drug benefits for insurers and self-funded plans: they build pharmacy networks, set reimbursement rates for pharmacies, negotiate rebates with manufacturers, and manage formularies. Three PBMs handle the large majority of prescriptions in the United States. Independent pharmacies, which often complain that PBMs reimburse them below their costs while steering patients to PBM-affiliated pharmacies, have pressed state legislatures for protection, and every state has now enacted some form of PBM regulation. In 2015 Arkansas enacted a law requiring PBMs to reimburse pharmacies at no less than the pharmacy's acquisition cost, to update their reimbursement lists promptly, and to allow pharmacies to decline to dispense drugs when reimbursement would be below cost. The PBM industry's trade association, the Pharmaceutical Care Management Association, argued that the law was preempted as applied to PBMs serving ERISA plans, and the Eighth Circuit agreed. In Rutledge v. Pharmaceutical Care Management Association, decided in December 2020, the Supreme Court unanimously reversed. Justice Sotomayor's opinion held that the Arkansas law was a form of cost regulation that did not bind plan administrators to any particular choice or dictate plan design. It regulated the rates PBMs pay pharmacies, and while that might increase the costs plans bear, a law that merely affects costs, as in Travelers, does not have an impermissible connection with plans. Nor did it make reference to ERISA plans, because it applied to PBMs whether or not they served ERISA plans. States read Rutledge as an invitation, and a new wave of PBM laws followed. But the Tenth Circuit soon showed that the invitation had limits. Oklahoma's Patient's Right to Pharmacy Choice Act regulated PBM networks: it imposed geographic access standards, barred PBMs from offering lower cost-sharing for use of preferred pharmacies, required PBMs to admit any willing pharmacy to their networks, and barred them from excluding pharmacies based on a pharmacist's probationary status. In Pharmaceutical Care Management Association v. Mulready in 2023, the Tenth Circuit held those network provisions preempted. Unlike Arkansas's rate regulation, they dictated the structure of the pharmacy benefit itself, which is a matter of plan design, and they undermined the ability of plans to administer a uniform benefit nationwide. The Supreme Court declined to hear Oklahoma's appeal in 2024. The line between Rutledge and Mulready, between regulating what PBMs pay and regulating how plans structure benefits, is now the principal frontier of ERISA preemption. It is being tested by laws that ban spread pricing, require pass-through of rebates, prohibit PBM ownership of pharmacies, or restrict a plan's ability to steer patients. Arkansas, for example, enacted a law in 2025 barring PBMs or their affiliates from owning or operating pharmacies in the state, and a federal court enjoined it before it took effect, a dispute that raises constitutional questions beyond ERISA as well. Table 3 collects the leading preemption cases discussed in this chapter and shows how the modern line has settled. Table 3. Leading ERISA preemption cases on health regulation. Case Year State law at issue Result Shaw v. Delta Air Lines 1983 Required employer disability and pregnancy benefits Preempted Metropolitan Life Ins. Co. v. Massachusetts 1985 Mandated mental health coverage in insurance policies Saved as insurance regulation FMC Corp. v. Holliday 1990 Anti-subrogation rule applied to self-funded plan Preempted (deemer clause) N.Y. Blue Cross v. Travelers 1995 Hospital rate surcharges on commercial payers Not preempted Gobeille v. Liberty Mutual 2016 Claims data reporting by self-funded plans Preempted Rutledge v. PCMA 2020 PBM reimbursement floor for pharmacies Not preempted PCMA v. Mulready (10th Cir.) 2023 PBM network access and any-willing-pharmacy rules Preempted What preemption means for reform Three practical rules emerge from this body of law. First, states may regulate the health care market, but not the plan. Laws aimed at hospitals, physicians, pharmacies, and the prices they charge or are paid generally survive, even when they raise plans' costs. Laws that tell plans what to cover, how to structure networks, how to process claims, or what to report generally fail as applied to self-funded plans. Second, the savings clause remains powerful for insured coverage, but the deemer clause confines it. A state can require every insured policy sold within its borders to cover a service, submit to external review, or meet network adequacy standards. It cannot impose the same requirements on self-funded plans. That is why so many state reforms, including the pre-2022 surprise-billing laws discussed in Chapter 6, protected only a minority of the employer-covered population. Third, when a problem requires uniform rules across insured and self-funded plans, only Congress can supply them. That is the pattern of the past fifteen years: the ACA's market reforms, the No Surprises Act, the transparency requirements of the Consolidated Appropriations Act of 2021, and the pharmacy benefit reforms of 2026 all reflect congressional recognition that states cannot reach the whole market. ERISA preemption does not merely constrain state reform; it steadily pushes health regulation toward the federal level. The functional test that emerged from Travelers is sensible, but it is also indeterminate at the margins, and those margins are where the most valuable state policies tend to sit. A state that wants to control drug costs or protect independent pharmacies must frame its law as regulating PBMs rather than plans, and then hope that a court sees it the same way. As states experiment with ever more ambitious regulation of intermediaries, the Supreme Court may eventually need to decide whether Mulready's concept of plan design has room for the kinds of network and pricing rules that states are now enacting. Chapter 5: Managed Care on Trial: Denials, Remedies, and the Limits of ERISA Managed care is the practice of paying for health care while simultaneously trying to control its use. It includes provider networks, negotiated rates, prior authorization, utilization review, step therapy, formularies, and the judgment, applied claim by claim, of whether a service is "medically necessary." Nearly every American with private coverage, and most people in Medicaid and Medicare Advantage, now receive care through some version of it. Its legal history is a history of disputes about denied care, and those disputes have been shaped less by health law than by the remedial structure of ERISA, a statute that did not contemplate them. This chapter looks at managed care from the perspective of the person whose claim has been denied. It asks what that person can do, where, and with what prospect of a meaningful remedy, and it examines how the law has responded to the newest forms of utilization management, including the use of algorithms to make or support coverage decisions. From fee-for-service to managed care For most of the twentieth century, American health insurance paid for care after the fact. A physician ordered treatment, the patient received it, and the insurer paid a large share of the bill with little inquiry. The Health Maintenance Organization Act of 1973 encouraged an alternative: organizations that took a fixed payment per enrollee and provided or arranged all needed care, giving them an incentive to avoid unnecessary services. During the 1980s and 1990s, as employers sought to contain rapidly rising premiums, managed care techniques spread from HMOs to nearly every form of coverage. The backlash was intense. By the late 1990s, stories of patients denied emergency care or specialist referrals had produced a wave of state "patients' bill of rights" laws, requiring direct access to certain specialists, coverage of emergency care without prior approval, and independent external review of denials. Congress debated a federal patients' bill of rights for several years, with the central dispute being whether patients should be able to sue health plans for damages in state court. The legislation never passed. The courts, in a series of cases that followed, largely settled the question by holding that ERISA itself answers it. Treatment decisions and eligibility decisions: Pegram The first of those cases, Pegram v. Herdrich in 2000, arose when a patient's physician at an HMO owned by physicians delayed an ultrasound for eight days so that it could be performed at a facility the HMO staffed, rather than at a closer hospital. The patient's appendix ruptured. She sued, arguing among other things that the HMO had breached its fiduciary duties under ERISA by rewarding its physician-owners for limiting care. The Supreme Court unanimously rejected the fiduciary claim. Justice Souter's opinion distinguished between pure eligibility decisions, which determine whether a plan covers a condition or procedure, and treatment decisions, which concern how to diagnose and treat a patient. HMO physicians routinely make "mixed" eligibility and treatment decisions in the course of caring for patients, and the Court held that those mixed decisions were not fiduciary acts under ERISA. Treating them as fiduciary breaches would, the Court reasoned, effectively make every malpractice claim against an HMO physician a federal fiduciary claim and would call into question the basic structure of managed care, which Congress had itself promoted. Pegram left room for state malpractice claims against physicians. But what about claims against the plan itself for denying coverage? Complete preemption: Davila That question was answered in Aetna Health Inc. v. Davila in 2004. Texas had enacted a Health Care Liability Act that imposed a duty on HMOs and managed care entities to exercise ordinary care when making health care treatment decisions and made them liable for harm caused by a failure to do so. Two plaintiffs sued under the law. One, Juan Davila, was prescribed a painkiller for arthritis; his plan required him to try a cheaper drug first, and he suffered serious internal bleeding. The other, Ruby Calad, was discharged from the hospital after a hysterectomy after the plan's discharge nurse determined she did not qualify for a longer stay, and she suffered complications. A unanimous Supreme Court held that both claims were completely preempted. The doctrine of complete preemption, which the Court had applied to ERISA in Metropolitan Life Insurance Co. v. Taylor in 1987, holds that when a federal statute's civil enforcement provision is intended to be exclusive, any state-law claim that falls within its scope is really a federal claim. It can be removed to federal court and decided under federal law. Because the plaintiffs' injuries arose from the plan's denial of coverage, and because they could have sued under ERISA section 502(a)(1)(B) to obtain the benefits or sought an injunction before harm occurred, their claims duplicated the ERISA remedy and were preempted. It did not matter that ERISA provided no damages for the injuries the plaintiffs suffered. That last point is the heart of the matter. Under section 502(a)(1)(B), a participant may sue to recover benefits due under the plan, to enforce rights under the plan, or to clarify rights to future benefits. Under section 502(a)(3), a participant may obtain "appropriate equitable relief" to redress violations. Neither provision allows recovery of compensatory damages for injuries caused by a wrongful denial, and neither allows punitive damages. A patient whose plan wrongly refuses to authorize surgery can, in principle, obtain an order requiring the plan to pay for it. If the patient suffers injury or dies because the surgery was delayed, ERISA generally provides no remedy beyond the cost of the benefit itself. Justice Ginsburg, joined by Justice Breyer, wrote a concurrence urging Congress and the Court to revisit what she described, quoting lower-court judges, as a "regulatory vacuum" in which "virtually all state law remedies are preempted but very few federal substitutes are provided." Neither has done so. Standards of review and the discretion problem When a denied claim does reach federal court under section 502(a)(1)(B), the standard of review often determines the outcome. In Firestone Tire and Rubber Co. v. Bruch in 1989, the Supreme Court held that a denial of benefits is reviewed de novo, meaning the court decides for itself what the plan requires, unless the plan gives the administrator discretionary authority to determine eligibility or construe the plan's terms. When it does, courts review denials only for abuse of discretion, upholding them if they are reasonable. Plans responded, predictably, by including discretionary clauses in nearly every plan document. In Metropolitan Life Insurance Co. v. Glenn in 2008, the Court held that when the same entity both decides claims and pays them, it has a conflict of interest that the reviewing court must weigh as a factor in determining whether discretion was abused. The weight given to the conflict varies with the circumstances. The states have pushed back through the savings clause. Many states, following a model regulation of the National Association of Insurance Commissioners, prohibit discretionary clauses in insurance policies. Because those prohibitions regulate insurance, they are saved from preemption for insured plans, and denials under insured plans in those states receive de novo review. Self-funded plans are unaffected. The same denial may thus be reviewed deferentially or independently depending on how the plan is funded and where the policy was issued. Courts reviewing denials generally confine themselves to the administrative record assembled during the plan's internal claims process. That makes the internal appeal, often treated by patients as a formality, the stage at which the case is actually won or lost. A participant who does not submit medical records, physician letters, and relevant clinical guidelines during the appeal may be barred from introducing them later. Claims procedures and external review ERISA requires plans to maintain reasonable claims procedures, and the Department of Labor's claims regulation, substantially revised in 2000, sets deadlines and content requirements. Plans must decide urgent pre-service claims within seventy-two hours, other pre-service claims within fifteen days, and post-service claims within thirty days, with limited extensions. Denials must explain the specific reasons, cite the plan provisions relied upon, and describe any additional information needed. Participants must have at least 180 days to appeal a health claim denial, and the appeal must be decided by someone other than the original decision-maker, with consultation of a health care professional for medical judgments. Before the ACA, independent external review existed only under state law and therefore reached only insured plans. The Supreme Court upheld such laws in Rush Prudential HMO, Inc. v. Moran in 2002, holding that an Illinois law requiring independent review of medical-necessity disputes regulated insurance and was not displaced by ERISA's remedial scheme. The next year, in Kentucky Association of Health Plans v. Miller, the Court simplified the savings-clause test, holding that a state law regulates insurance if it is specifically directed toward entities engaged in insurance and substantially affects the risk-pooling arrangement between insurer and insured. That decision upheld Kentucky's "any willing provider" laws. The ACA extended external review nationally. It requires all non-grandfathered plans, including self-funded ones, to provide internal appeals and access to external review by an independent review organization, either through a state process meeting federal standards or through a federal process. The reviewer's decision is binding on the plan. External review is now the most accessible check on managed care denials, and in categories such as medical necessity and experimental treatment, reviewers overturn a meaningful share of denials. Yet few denied claims are ever appealed, internally or externally, which suggests that the formal right is underused. Mental health parity and the guidelines problem Behavioral health has become a proving ground for managed care law. The Mental Health Parity and Addiction Equity Act of 2008 requires group plans and insurers that offer mental health or substance use disorder benefits to impose financial requirements and treatment limitations on those benefits that are no more restrictive than those applied to medical and surgical benefits. Quantitative limits, such as copays and visit caps, are relatively easy to compare. The harder problem is nonquantitative treatment limitations: medical necessity criteria, prior authorization, network standards, and reimbursement rates. The leading litigation involved United Behavioral Health, a large behavioral health administrator. In Wit v. United Behavioral Health, class plaintiffs alleged that UBH's internal guidelines for determining medical necessity were more restrictive than generally accepted standards of care and focused on stabilizing acute symptoms rather than treating underlying conditions. In 2019 the district court in San Francisco agreed after a trial and ordered UBH to reprocess tens of thousands of claims. The Ninth Circuit reversed much of that judgment, ultimately issuing a revised opinion in 2023 that held the plaintiffs had not shown that the plans required coverage consistent with generally accepted standards, and that reprocessing was not an appropriate remedy on the record, while leaving some claims open on remand. The litigation nonetheless influenced state law: California enacted legislation in 2020 requiring insurers to base medical necessity decisions for behavioral health on criteria from nonprofit professional associations. Congress strengthened parity enforcement in the Consolidated Appropriations Act of 2021, requiring plans to perform and document comparative analyses of their nonquantitative treatment limitations and to produce them on request. In September 2024 the three federal agencies finalized a rule that went further, requiring plans to collect and evaluate data on outcomes, such as network composition and authorization denial rates, and to take action when the data revealed material differences. An employer benefits trade group challenged the rule in January 2025, and in May 2025 the agencies announced that they would not enforce the rule's new requirements while they reconsidered them. The underlying statute and the 2021 comparative-analysis requirement remain in force, but the more aggressive enforcement approach is suspended. Prior authorization and algorithms Prior authorization, the requirement that a plan approve a service before it is provided, has become the most contested managed care tool. Physicians describe it as a major administrative burden and a source of dangerous delays; plans describe it as a necessary check on low-value and unsafe care. Regulation has advanced on several fronts. CMS finalized a rule in January 2024 that applies to Medicare Advantage plans, Medicaid and CHIP programs and their managed care plans, and marketplace plans on the federal platform. Beginning in 2026, affected plans must decide expedited requests within seventy-two hours and standard requests within seven calendar days, give specific reasons for denials, and publicly report authorization metrics; by 2027 they must support electronic prior authorization through standard interfaces. The rule does not reach employer plans governed by ERISA. Many states have enacted their own reforms for insured coverage, including "gold card" laws, pioneered by Texas in 2021, that exempt physicians with high approval rates from prior authorization for particular services. In June 2025 a large group of insurers announced voluntary commitments to reduce the scope of prior authorization and speed decisions. The newest front concerns algorithms. Lawsuits filed beginning in 2023 alleged that large insurers used automated tools to deny claims at scale. In one line of cases, plaintiffs alleged that UnitedHealth Group and Humana used a predictive tool to cut off post-acute care for Medicare Advantage patients based on projected lengths of stay rather than individual medical judgment. In another, plaintiffs alleged that Cigna used a system that allowed its medical directors to reject batches of claims without opening the patient files. Because Medicare Advantage claims fall outside ERISA, and because some of the Cigna plaintiffs challenged the process rather than particular benefit determinations, these suits have tested theories that ERISA's remedial scheme would ordinarily foreclose, and some claims have survived motions to dismiss. CMS has told Medicare Advantage plans that an algorithm may assist but cannot replace an individualized determination based on the patient's circumstances, and California enacted a law in 2024 requiring that medical necessity denials by health plans be made by a licensed physician or other qualified professional rather than by an automated tool alone. What a patient can actually do For the reader facing a denial, the law in this area yields concrete guidance. Identify the regime first: the path runs through ERISA for employer plans, through state law and external review for individual coverage, through the fair hearing process for Medicaid, and through Medicare appeals for Medicare Advantage. Use the internal appeal as though it were the trial, because in an ERISA case it effectively is. Request the plan documents and the clinical criteria used to make the decision, which ERISA and the claims regulation entitle a participant to receive. Ask for external review, which is binding on the plan and costs the patient nothing or very little. Where the denial is urgent, invoke the expedited timelines. For the policymaker, the lesson is different. The architecture built by Pilot Life, Davila, and Firestone channels disputes into a process that works reasonably well for recovering the cost of a denied benefit and poorly for anything else. The ACA's external review requirement softened that architecture without changing it. The growth of automated decision-making puts new pressure on it, because an error made at scale and corrected one appeal at a time is not really corrected at all. Whether the answer is a stronger federal remedy, broader state authority, or better administrative oversight is a question Congress has avoided since the patients' bill of rights debates a quarter-century ago. Chapter 6: Surprise Medical Billing and the Arbitration Wars Surprise medical billing is the problem that shows most plainly how the structural features described in the earlier chapters produce real harm. A patient does everything the system asks: carries insurance, chooses an in-network hospital, or has no choice at all because she arrives by ambulance unconscious. She is treated by a physician she never selected and whose network status she could not have known. That physician bills her for the difference between the provider's charge and what her plan paid, a practice called balance billing. Before 2022, bills of this kind arrived after an estimated one in five emergency visits by privately insured patients and a substantial share of elective admissions to in-network hospitals, and they sometimes ran to tens of thousands of dollars. The No Surprises Act, which took effect on January 1, 2022, largely ended the practice for people with private coverage. It is one of the most significant consumer protections in American health law. But it did not end the underlying dispute, which was never really between patients and providers. It was between providers and plans, about the price of out-of-network care, and the statute resolved that dispute by handing it to private arbitrators. That choice produced one of the most litigated regulatory programs in recent memory. How surprise bills happened Surprise billing had a structural cause. Hospitals contract with insurers to be in-network, but many of the physicians who work in hospitals are not hospital employees. Emergency physicians, anesthesiologists, radiologists, pathologists, and neonatologists often practice in independent groups that contract with hospitals to staff departments. Those groups negotiate with insurers separately. Patients do not choose these physicians; they are assigned. The groups therefore had an unusual bargaining position. Unlike most providers, they did not need to be in-network to attract patients, because patients came through the hospital regardless. For some groups, remaining out-of-network was a business strategy. A group could bill its full charges, collect whatever the plan chose to pay, and bill the patient for the rest, or use the threat of balance billing to negotiate high in-network rates. Research published in the late 2010s documented that certain physician staffing companies, several of them owned by private equity firms, had markedly higher out-of-network billing rates after taking over emergency departments. Air ambulance companies, whose charges often ran to tens of thousands of dollars per flight, presented the same problem in a more extreme form. The state response and the ERISA gap States acted first. New York enacted comprehensive protection in 2015, barring balance billing for emergency services and for surprise bills from out-of-network physicians at in-network facilities, and creating an independent dispute resolution process in which an arbitrator chooses either the provider's bill or the plan's payment. New York directed arbitrators to consider, among other things, the eightieth percentile of charges for the service in the region as reported by an independent database. California took a different approach in 2016, setting a payment standard based on the greater of the plan's average contracted rate or 125 percent of Medicare, with a dispute process available. Texas, after years of mediation-based protection, enacted a broader law with arbitration in 2019. By the end of the decade, roughly half the states had some form of protection, though many were partial. Every one of those laws had the same limit. Because they regulated insurance, they applied only to insured plans. ERISA preemption, as described in Chapter 4, barred them from reaching self-funded employer plans, which covered most workers with employer coverage. A patient's protection against a surprise bill depended on whether her employer self-funded, which she almost certainly did not know. Air ambulance bills presented a further problem: the federal Airline Deregulation Act preempts state laws relating to the prices of air carriers, and courts held that it barred states from regulating air ambulance charges. Only Congress could close those gaps. After two years of intense lobbying, in which physician staffing firms and hospital groups funded advertising campaigns against a benchmark rate, Congress enacted the No Surprises Act as part of the Consolidated Appropriations Act of 2021, signed on December 27, 2020. How the No Surprises Act works The statute has two distinct functions: it protects patients, and it establishes a process for resolving what the plan pays the provider. The patient protections apply to three situations. The first is emergency services, whether provided by an out-of-network facility or out-of-network physicians, and including post-stabilization services until the patient can safely travel to an in-network facility and gives consent to further out-of-network care. The second is non-emergency services provided by out-of-network providers at in-network facilities, including hospitals, hospital outpatient departments, critical access hospitals, and ambulatory surgery centers. The third is air ambulance services. In each case, the patient's cost-sharing is calculated as if the care were in-network, based on a "recognized amount," and the provider may not bill the patient for anything more. Ground ambulances were excluded, and Congress created an advisory committee to study them. There is a narrow exception for non-emergency care. A provider can obtain the patient's written consent, given in advance on a prescribed form, to waive the protection and be billed at out-of-network rates. But the exception is unavailable for ancillary services, including emergency medicine, anesthesiology, pathology, radiology, neonatology, assistant surgeons, hospitalists, and intensivists, and for diagnostic services, and when no in-network provider is available. Those are the services where patients have no real choice. The statute also addresses the uninsured and people paying for care themselves. Providers must give them a good-faith estimate of expected charges in advance, and if the final bill exceeds the estimate by at least $400, the patient may use a federal patient-provider dispute resolution process. The protections at the heart of the statute are summarized in Table 4. Table 4. Principal protections under the No Surprises Act. Situation Balance billing barred? Can patient waive by consent? Patient cost-sharing Emergency services, out-of-network facility or physician Yes No (until stabilized and able to travel) In-network level Non-emergency ancillary services at in-network facility Yes No In-network level Other non-emergency services by out-of-network provider at in-network facility Yes Yes, with advance written notice and consent In-network level unless waived Air ambulance Yes No In-network level Ground ambulance No Not applicable Plan terms and state law Uninsured or self-pay patients Not applicable Not applicable Good-faith estimate; dispute process if bill exceeds it by $400 or more Setting the price: the QPA and arbitration Having removed the patient from the dispute, the statute had to determine what the plan pays the provider. Where a state has a surprise-billing law that sets a payment standard for the plan in question, the state law, called a specified state law, governs. Otherwise, the plan makes an initial payment or denial within thirty days of receiving the bill. If the provider is dissatisfied, the parties must negotiate for thirty days. If negotiation fails, either party may initiate the federal independent dispute resolution process, commonly called IDR. IDR is "baseball-style" arbitration. Each side submits a single offer, and a certified independent entity must choose one or the other; it cannot split the difference. The theory is that the need to win pushes both sides toward reasonable offers. The statute tells arbitrators what to consider. They must consider the qualifying payment amount, or QPA, which is roughly the plan's median contracted rate for the same service in the same geographic area as of 2019, adjusted for inflation. They must also consider additional information submitted by the parties, including the provider's training and experience, the market shares of the parties, the acuity of the patient and complexity of the case, the teaching status and case mix of the facility, and good-faith efforts to enter network agreements, including contracted rates during the previous four years. They may not consider the provider's billed charges or public program rates such as Medicare and Medicaid. The losing party pays the arbitrator's fee. The QPA sits at the center of the design. The QPA also determines the patient's cost-sharing in most cases, so it affects patients as well as providers. But the question of how much weight arbitrators should give it, compared with the other factors, was left ambiguous. That ambiguity was where the litigation began. The Texas Medical Association cases The first implementing regulation, an interim final rule issued in the fall of 2021, instructed arbitrators to presume that the QPA was the appropriate payment amount and to deviate from it only when the other information clearly demonstrated that the QPA was materially different from the appropriate value. Providers objected that the statute listed the QPA as one factor among several and did not make it presumptively correct. Because the QPA is derived from plans' own contracted rates, they argued, the rule tilted arbitration toward insurers and would drive down rates across the market. The Texas Medical Association and others sued in the Eastern District of Texas, and in February 2022 Judge Jeremy Kernodle agreed, vacating the rebuttable presumption as contrary to the statute. The departments issued a final rule in August 2022 that dropped the presumption but still told arbitrators to consider the QPA first and to give weight to other information only if it was credible and not already reflected in the QPA. Judge Kernodle vacated that provision as well in February 2023, and in 2024 the Fifth Circuit affirmed, agreeing that the statute does not permit the agencies to elevate the QPA above the other statutory factors. Two further suits followed. In August 2023 the district court vacated several provisions of the methodology the agencies had prescribed for calculating the QPA, agreeing with providers that the methodology improperly depressed the QPA, for example by counting contracted rates for providers who never actually furnished the service in question. The same month, in a separate case, it vacated a sharp increase in the administrative fee for IDR and restrictions on bundling multiple claims into a single dispute. Each ruling forced the agencies to suspend and then restart parts of the IDR process, and appellate proceedings over the QPA methodology continued into 2025. The agencies proposed a comprehensive rule on IDR operations in late 2023 to address administrative problems, and much of the process continues to operate under guidance adjusted in response to the litigation. The process under strain The arbitration process has proven far larger and more favorable to providers than its designers expected. The agencies had estimated that roughly seventeen thousand disputes would be filed each year. Instead, filings ran into the hundreds of thousands annually, with a large share coming from a few provider groups and revenue-cycle companies, most active in a handful of states, including Texas, Florida, and New Jersey. Many disputes were initially found ineligible, often because the service was not covered by the federal process or should have gone to a state process, and determining eligibility became a major bottleneck. Providers have prevailed in roughly four of every five disputes decided on the merits, and the winning amounts have frequently been several times the QPA. Several explanations have been offered. Providers argue that the QPA is artificially low because plans' methodology counts rates that providers never accept, and that arbitrators are correctly recognizing the value of their services. Insurers argue that a small number of high-volume filers are using the process to extract prices well above market rates, that the rulings against the QPA-centered rules have made arbitration outcomes unpredictable, and that the resulting costs will eventually appear in premiums. Economists have noted that high arbitration awards shape negotiations outside IDR, because a provider that expects to win in arbitration has little reason to accept a lower in-network rate. Enforcement of awards has added a further layer of dispute. The statute requires plans to pay awards within thirty days, but it does not expressly create a private right of action to enforce them, and several federal courts have held that providers cannot sue under the No Surprises Act to collect unpaid awards, leaving enforcement to the agencies and to state regulators. Plans, for their part, have sought to vacate awards they consider tainted by ineligibility or error, with limited success, because the statute makes IDR determinations binding absent fraud or misrepresentation. What the No Surprises Act achieved It is easy, amid the litigation, to lose sight of what the statute accomplished. For privately insured patients, surprise bills of the classic type have become rare, and the protection extends to self-funded plans that no state could reach. Complaints and enforcement actions against providers who continued to balance bill have been handled by CMS and state regulators, and awareness among providers of the law's requirements is high. That is a genuine achievement of federal law filling a gap that ERISA preemption created. The unresolved question is who sets the price of out-of-network care. Congress rejected a fixed benchmark, which physicians' groups strongly opposed, and chose arbitration with the QPA as a central but not dominant factor. The courts, applying the statute's text without deference to the agencies, have kept the QPA from becoming a de facto benchmark. The result is a system in which prices are set case by case by arbitrators, subject to periodic litigation over the rules they apply. That system may prove stable. But it is expensive to administer, and its effect on premiums and on in-network rates is uncertain. It also raises a question that goes beyond surprise billing and leads directly to the next chapter: if arbitrators, regulators, and courts struggle to identify a fair price, can the market do better if prices are made visible? Chapter 7: Price Transparency: Disclosure as Regulation For most of the history of American health care, the prices that actually determined what care cost were secret. Hospitals maintained chargemasters listing "gross charges" for every service, but those figures bore little relation to what anyone paid. The real prices were the rates negotiated between each provider and each insurer, and those were confidential, often protected by contract clauses that barred either side from disclosing them. Patients could not learn what a service would cost before receiving it. Employers could not learn what their plans were paying. Researchers had to rely on claims samples obtained under confidentiality agreements. Beginning in 2019, the federal government set out to change that through a series of disclosure mandates. Hospitals must now publish their negotiated rates with every insurer. Health plans must publish their negotiated rates with every provider and the amounts they have allowed for out-of-network care. Contracts that prevent plans from sharing price information are prohibited. These rules represent a distinct regulatory theory. Rather than setting prices, as Medicare does, or choosing among competing offers, as arbitrators under the No Surprises Act do, the government is trying to make prices visible and let patients, employers, and competitors respond. This chapter examines the legal foundations of that theory, the litigation it provoked, the state of compliance, and the evidence on what disclosure can accomplish. The statutory hook and the hospital rule The legal basis for hospital price transparency is a single sentence in the Affordable Care Act. Section 2718(e) of the Public Health Service Act, added by the ACA, requires each hospital to "establish (and update) and make public (in accordance with guidelines developed by the Secretary) a list of the hospital's standard charges for items and services." For almost a decade, CMS interpreted the provision modestly, requiring hospitals to post their chargemasters online. Because chargemaster prices are largely fictional, the disclosure had little practical value. In June 2019 President Trump issued an executive order directing HHS to require hospitals to disclose the prices they actually negotiate with insurers. The resulting rule, finalized in November 2019 and effective January 1, 2021, reinterpreted "standard charges" to include not only gross charges but also payer-specific negotiated charges, the discounted cash price, and the minimum and maximum negotiated charges across all payers. Hospitals must publish all of those figures, for all items and services, in a single machine-readable file. They must also present at least three hundred "shoppable" services, seventy of them specified by CMS, in a consumer-friendly format, or offer an online price estimator tool. The hospital industry sued. In American Hospital Association v. Azar, the hospitals argued that "standard charges" could not reasonably be read to include rates privately negotiated with individual insurers, which are by definition not standard; that the rule compelled speech in violation of the First Amendment; and that the rule was arbitrary because disclosing negotiated rates would not help patients and might lead to higher prices. The district court rejected all three arguments in June 2020, and the D.C. Circuit affirmed in December 2020. The court concluded that a negotiated rate that a hospital charges a particular insurer is the standard charge for that insurer's enrollees; that the disclosure of factual commercial information serving the government's interest in informing consumers does not violate the First Amendment; and that the agency had reasonably explained its predictive judgment that transparency would benefit patients even if the evidence was uncertain. Compliance and enforcement Legal victory did not produce compliance. Early studies after the rule took effect found that a large share of hospitals had not posted complete machine-readable files, and many that did posted files that were difficult to use, incomplete, or inconsistent. The initial penalty, a maximum of $300 per day, amounted to about $110,000 a year, a trivial sum for a large hospital system weighing the competitive consequences of disclosing its rates. CMS responded by raising the stakes. Beginning in 2022, penalties scale with hospital size: hospitals with more than thirty beds face $10 per bed per day, up to $5,500 per day, or more than $2 million per year for the largest facilities. CMS imposed its first civil money penalties in 2022, including a penalty of more than $880,000 against a large hospital in the Atlanta area, and has issued many more since, though the agency's enforcement has relied mostly on warning notices and corrective action plans rather than penalties. The agency has also tightened the substance of the requirement. Rules effective in 2024 and 2025 required hospitals to use a standardized template for their files, to encode specific data elements, and to affirm the accuracy and completeness of their data. The calendar year 2026 outpatient payment rule went further, requiring hospitals to disclose more information about the amounts actually paid when their negotiated rates are expressed as percentages or formulas rather than dollar figures, and to have a senior official attest to the accuracy of the files, with enforcement of the new requirements beginning April 1, 2026. A February 2025 executive order directed the agencies to require the disclosure of actual prices rather than estimates and to strengthen enforcement, and CMS has continued to seek comment on standardizing the data further. States have added their own tools. Several, including Colorado and Texas, have enacted laws that bar hospitals out of compliance with federal price transparency requirements from pursuing collection actions against patients, a sanction that gives patients a direct stake in compliance and that hospitals have taken seriously. Transparency in coverage The hospital rule addresses one side of each negotiated price. The Transparency in Coverage rule, issued jointly by the Departments of Health and Human Services, Labor, and the Treasury in October 2020, addresses the other. It applies to non-grandfathered group health plans, including self-funded ERISA plans, and to health insurers in the individual and group markets. The rule has two main components. The first requires plans and insurers to publish machine-readable files, updated monthly, disclosing their negotiated rates with every in-network provider for every covered item and service, and the historical amounts they allowed for out-of-network services. Those requirements took effect in July 2022. A third file, on prescription drug prices, was deferred indefinitely. The second component requires plans to provide enrollees with an online tool that estimates the enrollee's cost-sharing for a given service from a given provider, initially for five hundred specified services beginning in 2023 and for all covered items and services beginning in 2024. The Consolidated Appropriations Act of 2021 codified overlapping requirements in statute. It requires plans to offer a price-comparison tool and telephone assistance, to provide an "advanced explanation of benefits" before scheduled care, and to report annually on prescription drug spending. It also prohibits "gag clauses": plans may not enter into contracts with providers, networks, or administrators that restrict them from disclosing provider-specific price or quality information or from accessing de-identified claims data. Plans must attest annually that they are in compliance. The advanced explanation of benefits requirement has not yet been enforced, because the agencies have not issued the rules and data standards needed to implement it. The legal basis for the Transparency in Coverage rule is more diffuse than for the hospital rule, resting on the ACA's transparency provisions for plans and on general rulemaking authority. It has not faced a challenge comparable to American Hospital Association v. Azar, in part because the insurers who would bring one also benefit from knowing their competitors' rates. The requirements that now apply to hospitals and to health plans are compared in Table 5. Table 5. Federal price transparency requirements. Requirement Who must comply What is disclosed In effect since Hospital machine-readable file Hospitals Gross charges, payer-specific negotiated rates, cash price, minimum and maximum negotiated rates January 2021 Hospital shoppable services display Hospitals Prices for at least 300 common services, or an estimator tool January 2021 Plan in-network and out-of-network files Insurers and group health plans, including self-funded Negotiated rates with each provider; historical out-of-network allowed amounts July 2022 Enrollee cost-sharing tool Insurers and group health plans Personalized estimate of out-of-pocket costs 2023 (500 services); 2024 (all services) Gag clause prohibition and attestation Insurers and group health plans Access to price, quality and claims data December 2020 (attestations from 2023) What disclosure has produced The files produced by these rules are enormous. A single large insurer's monthly in-network files can run to many terabytes, and the combined public dataset is among the largest bodies of economic information ever released. That scale has made the files difficult for individual patients to use directly, but it has spawned an industry of data firms, researchers, journalists, and consultants who clean, standardize, and analyze them. The data have confirmed what economists long suspected. Prices for the same service vary widely, not only among hospitals in the same market but among insurers at the same hospital, and sometimes the same hospital's negotiated rate for a service is higher than its cash price. They have also revealed data quality problems: rates listed for services a provider never performs, rates expressed in forms that cannot be compared, and files that omit key information. Those problems have driven the successive tightening of CMS's templates and attestation requirements. Can transparency lower prices? The harder question is whether disclosure changes behavior. The theory has two versions. The consumer version holds that patients who know prices will choose cheaper providers, forcing high-price providers to compete. The market version holds that employers, insurers, and providers themselves will use price information in negotiations and plan design, even if individual patients rarely shop. The evidence for the consumer version is modest. A substantial share of health spending is for emergency, urgent, or complex care that patients cannot realistically shop for, and patients with insurance often face cost-sharing that does not vary much across providers. Studies of insurer and employer price-shopping tools have found that relatively few enrollees use them. An influential study published in the Journal of the American Medical Association in 2016 found that employees given access to a price transparency tool did not reduce their outpatient spending; if anything, spending rose slightly. Studies of state programs that publish prices for specific services, such as New Hampshire's HealthCost website, have found savings for shoppable services like imaging, where patients have both choice and a financial stake. The market version is more promising and harder to measure. Employers can use the data to identify high-priced providers, design networks, and hold plan administrators accountable, a subject taken up in Chapter 8. Insurers can use it to benchmark rates. States can use it to set policy; some have used price data to support caps on hospital prices in their state employee plans or public options. There is also a risk. Economists have long warned that transparency among sellers can facilitate tacit coordination. The standard illustration comes from Denmark, where the government's publication of transaction prices for ready-mixed concrete in the 1990s was followed by price increases, as firms apparently used the data to avoid undercutting one another. Health care markets, where many local markets are highly concentrated, are vulnerable to the same effect. The D.C. Circuit in American Hospital Association v. Azar acknowledged the argument but held that the agency had reasonably concluded the benefits would outweigh the risk. Whether that prediction holds is an empirical question the data now make it possible to study. Transparency's place in the legal system Price transparency occupies a particular position in health insurance law. It is the rare reform that applies uniformly to insured and self-funded plans and to hospitals regardless of payer, because it was designed at the federal level from the start. It relies on a regulatory tool, compelled disclosure, that is less intrusive than price-setting and correspondingly less effective at controlling prices directly. And it depends heavily on enforcement, since the entities required to disclose often have strong incentives not to. For the patient, transparency has so far made the most difference in combination with other rules: the good-faith estimates the No Surprises Act requires for uninsured patients, the state laws that deny collection rights to noncompliant hospitals, and the cost-estimate tools that plans must provide. For the system as a whole, its value will depend on whether the parties with bargaining power, particularly employers acting on behalf of their workers, use the data. Chapter 8: The Employer as Fiduciary: Plan Costs, Pharmacy Benefit Managers, and the New Accountability For most of ERISA's history, the fiduciary duties it imposes were understood as a pension matter. The statute requires those who manage plan assets and administer plans to act "solely in the interest of the participants and beneficiaries," for the "exclusive purpose" of providing benefits and "defraying reasonable expenses of administering the plan," and with the care, skill, prudence, and diligence of a prudent person familiar with such matters. Over the past two decades, those words became the basis for a large body of litigation over retirement plans, much of it alleging that employers allowed their 401(k) plans to pay excessive fees or offered imprudent investment options. Health plans were largely left out. Employers treated them as a cost to be controlled, not a trust to be administered, and litigants rarely tested whether the fiduciary standards applied with any force to the prices plans paid for care. That has begun to change. Congress, in the Consolidated Appropriations Act of 2021, gave employers new rights and obligations to obtain information about their plans' costs and the compensation of those who serve them. The transparency rules described in the previous chapter made price data available. Plaintiffs' lawyers who built the retirement-fee litigation have turned their attention to health plans, filing suits alleging that employers breached their fiduciary duties by overpaying for prescription drugs. The Supreme Court in 2025 made certain ERISA claims easier to plead. And Congress in early 2026 enacted reforms directed at pharmacy benefit managers that expressly strengthen the hand of employer plans. Together, these developments are converting the employer from a purchaser of health benefits into something closer to a trustee of health-plan money. What fiduciary duty means for a health plan An employer that sponsors a health plan plays two roles. When it decides whether to offer a plan, what benefits to include, and how much employees must contribute, it acts as a "settlor," exercising business judgment that ERISA does not regulate. When it administers the plan, selects and monitors service providers, and manages plan assets, it acts as a fiduciary, subject to ERISA's duties of loyalty and prudence. The line between the two roles can be difficult to draw, but the selection and monitoring of service providers, including claims administrators and pharmacy benefit managers, falls on the fiduciary side. A self-funded plan is, in economic terms, a pool of money consisting of employer and employee contributions, used to pay claims and administrative expenses. Where employee contributions are held and used for plan purposes, they may be plan assets. A fiduciary who allows the plan to pay unreasonable prices for services or unreasonable fees to administrators, or who fails to monitor whether a vendor is acting in the plan's interest, may breach the duty of prudence. The same logic applies in modified form to fully insured plans, though there the insurer bears the risk and the employer's fiduciary role is narrower. ERISA also prohibits certain transactions outright. Section 406 bars a fiduciary from causing the plan to engage in transactions with a "party in interest," a category that includes service providers, unless an exemption applies. The most important exemption, in section 408(b)(2), permits contracts for necessary services if no more than reasonable compensation is paid. For retirement plans, a 2012 regulation required covered service providers to disclose their direct and indirect compensation so that fiduciaries could evaluate whether it was reasonable. Health plans were not covered. Service providers to health plans have long been compensated in ways that are not obvious to the employer. The early warning came in Hi-Lex Controls, Inc. v. Blue Cross Blue Shield of Michigan, in which the Sixth Circuit in 2014 affirmed a judgment that Blue Cross, acting as third-party administrator for self-funded plans, had breached its fiduciary duties by retaining undisclosed "access fees" added to the hospital claims it paid on the plans' behalf. The case showed that an administrator exercising control over plan funds could itself be a fiduciary, and that hidden compensation could be recovered. But it also showed how much depended on employers discovering what their vendors were doing. The 2021 disclosure and data rights The Consolidated Appropriations Act of 2021 addressed that information problem directly. It extended ERISA's section 408(b)(2) disclosure requirements to group health plans, requiring brokers and consultants who expect to receive $1,000 or more in direct or indirect compensation to disclose that compensation in writing to the plan fiduciary. It prohibited gag clauses that restrict plans from accessing their own claims data or disclosing price and quality information. It required plans to report annually on prescription drug spending, including rebates. And it strengthened the parity comparative analyses discussed in Chapter 5. These provisions gave employers tools and, implicitly, responsibilities. A fiduciary who has a legal right to obtain claims data and compensation disclosures, and does not use it, is in a weaker position to argue that it acted prudently in overpaying. The provisions also produced friction between employers and their administrators. Some employers have sued claims administrators for refusing to provide complete claims data or for alleged mismanagement of claims, and those suits test how far a self-funded employer's rights under ERISA and the 2021 law override the administrator's contractual restrictions. Pharmacy benefit managers and the drug-price suits The largest target of the new scrutiny has been pharmacy benefit management. As Chapter 4 explained, pharmacy benefit managers administer drug benefits for most employer plans, and three firms dominate the market, each affiliated with a large insurer and each owning mail-order and specialty pharmacies. PBMs earn revenue in several ways: administrative fees paid by the plan; the "spread" between what the plan pays for a drug and what the PBM pays the pharmacy; a share of manufacturer rebates; fees paid by manufacturers; and margins earned by their affiliated pharmacies. Many of those revenue streams are invisible to the plan sponsor. Critics argue that this structure gives PBMs incentives to favor high-list-price drugs with large rebates over cheaper alternatives, to steer patients to affiliated pharmacies, and to retain value that should flow to plans. The Federal Trade Commission, in interim staff reports issued in 2024 and 2025, found that the largest PBMs had marked up many specialty generic drugs dispensed through their affiliated pharmacies far above acquisition cost. In September 2024 the Commission filed an administrative complaint against the three largest PBMs over their rebating practices for insulin, and in 2026 two of them reached settlements that required changes in their business practices. In early 2024 a Johnson & Johnson employee enrolled in the company's health plan filed a proposed class action, Lewandowski v. Johnson & Johnson, alleging that the company breached its fiduciary duties by agreeing to pay its PBM prices for certain generic drugs many times what the same drugs cost at retail pharmacies without insurance, and by failing to monitor its PBM. Similar suits followed, including Navarro v. Wells Fargo & Co. The early results were not favorable to the plaintiffs. In 2025 federal district courts dismissed both the Johnson & Johnson and Wells Fargo cases for lack of Article III standing. The courts reasoned that the plaintiffs had not plausibly alleged that the overpayments caused them concrete harm, because any connection between the plan's drug costs and their own premium contributions, cost-sharing, or wages was too speculative. The same reasoning had defeated earlier ERISA suits by participants in fully funded pension plans in Thole v. U.S. Bank in 2020, where the Supreme Court held that participants whose defined benefits were unaffected by alleged mismanagement had no stake in the outcome. The analogy is imperfect for health plans, where employees often pay a share of premiums and coinsurance tied to drug prices, and plaintiffs have refined their pleadings to allege more direct links between overpayments and their own costs. Whether they can clear the standing barrier in future cases, particularly where participants pay a percentage of drug prices as coinsurance, remains open. Cunningham and the prohibited-transaction route The Supreme Court's April 2025 decision in Cunningham v. Cornell University arose from a retirement plan, but its logic reaches health plans. The plaintiffs alleged that Cornell's retirement plans paid excessive recordkeeping fees to service providers, and they framed part of their case as a prohibited-transaction claim under section 406(a)(1)(C), which bars furnishing services between a plan and a party in interest. The lower courts had required plaintiffs to allege not only that such a transaction occurred but also that it was unnecessary or involved unreasonable compensation, the conditions of the section 408(b)(2) exemption. The Supreme Court unanimously reversed. Justice Sotomayor's opinion held that the exemptions in section 408 are affirmative defenses, which the defendant must plead and prove, rather than elements of the plaintiff's claim. A plaintiff states a claim under section 406(a)(1)(C) by alleging that the fiduciary caused the plan to engage in a transaction for services with a party in interest. The Court acknowledged concern that this could allow meritless suits to survive motions to dismiss, and it pointed to tools that courts can use to screen them, including requiring plaintiffs to respond to an asserted exemption and to meet Article III standing requirements. For health plans, Cunningham matters because nearly every service arrangement, with claims administrators, PBMs, brokers, and consultants, involves a party in interest. Plaintiffs who can establish standing may now plead prohibited-transaction claims without first proving that the compensation was unreasonable, shifting the burden to fiduciaries to show that their contracts fall within an exemption. That, in turn, makes documentation of the process by which the fiduciary evaluated the vendor's compensation more important than ever. The 2026 pharmacy benefit reforms After years of hearings and competing bills, Congress enacted significant reforms of pharmacy benefit management in the Consolidated Appropriations Act, 2026, signed in February 2026. For employer plans, the law requires PBMs to pass through all manufacturer rebates to the plan and to provide plan sponsors with detailed reporting on drug spending, rebates, fees, and the prices paid to pharmacies, including those affiliated with the PBM. For Medicare Part D, it moves PBM compensation away from a percentage of drug list prices toward flat service fees, beginning in 2028. The Department of Labor, separately, proposed a rule in January 2026 that would require PBMs serving ERISA plans to disclose, as covered service providers, the compensation they receive from manufacturers and others, and other information bearing on the net cost of drugs. The significance of the reforms lies partly in their form. As Chapter 4 explained, state regulation of PBMs has repeatedly run into ERISA preemption when it touches the structure of employer plan benefits, and it cannot reach self-funded plans at all through insurance regulation. Federal law faces no such obstacle. The 2026 reforms establish a nationwide floor for PBM transparency in employer plans and, by requiring information to flow to plan sponsors, reinforce the fiduciary model. They do not tell employers what drugs to cover or which PBM to choose. They give employers the information to make those choices, and, by implication, make it harder for employers to explain why they did not. Other emerging fiduciary claims The drug-price suits are the most visible part of a broader trend. Class actions filed since 2023 have challenged tobacco surcharges in employer health plans, alleging that plans failed to offer a reasonable alternative standard and retroactive refunds as required by federal wellness program rules and that the surcharges were therefore unlawful discrimination and a breach of fiduciary duty. Other suits have challenged the handling of medical loss ratio rebates paid to insured plans, alleging that employers kept rebates that should have benefited participants. And employers themselves have sued their administrators, alleging that the administrators overpaid claims, withheld data, or retained undisclosed compensation. The Third Circuit's 2024 decision rejecting a participant's challenge to an employer's use of rebates, on standing grounds, suggests that the standing question will remain central to all of these cases. What employers should take from this The practical implications for employers can be stated directly. The selection and monitoring of health plan vendors is a fiduciary function and should be conducted with the same documented, deliberate process that prudent employers apply to retirement plan vendors. The rights to claims data and compensation disclosure that the 2021 law created should be exercised, and the information should be reviewed. Contracts with PBMs and administrators should be read for revenue streams the plan cannot see, and the 2026 reforms and the Department of Labor's proposed rule should be treated as the minimum level of disclosure, not the maximum. Price transparency data can be used to benchmark what the plan pays against the market. For the system as a whole, the development is double-edged. Fiduciary litigation can be a powerful tool for aligning intermediaries' incentives with the interests of employees, as it arguably was for retirement plan fees, where litigation contributed to large declines in average costs. It can also produce defensive behavior, higher insurance and legal costs, and suits that reward pleading skill more than genuine misconduct. The standing doctrine, the pleading rules announced in Cunningham, and the new statutory disclosure regime will together determine which of those outcomes predominates. What is clear is that employers, long the most important and least regulated purchasers of American health care, are being asked to take responsibility for the prices their plans pay. Conclusion: The Seams Will Hold, and Keep Tearing The chapters of this book have examined six battlegrounds: the constitutional and statutory survival of the Affordable Care Act, the long contest over Medicaid expansion, the preemptive force of ERISA, the remedies available when managed care denies treatment, the arbitration of surprise medical bills, and the effort to make prices visible. Taken one at a time, each looks like a distinct problem with its own history and cast of litigants. Taken together, they reveal something about the character of American health insurance law that is not visible from inside any one of them. The pattern In every one of these fights, the decisive question was not what good health coverage requires but who gets to decide. NFIB v. Sebelius did not rule on the wisdom of covering poor adults; it ruled on how hard Washington could push the states. Gobeille and Rutledge did not assess the value of claims databases or pharmacy reimbursement floors; they asked whether the state was regulating the plan or the market. Davila did not weigh whether patients harmed by wrongful denials deserve compensation; it asked whether ERISA's remedial scheme was exclusive. The Texas Medical Association cases did not decide what emergency physicians should be paid; they decided whether the agencies could tell arbitrators how to weigh one factor. Even Braidwood, which preserved free preventive care for well over a hundred million people, turned on the chain of command inside a federal department. This is not an accident of litigation strategy. It reflects the way the system was assembled. Because the United States never chose a single framework for health coverage, every major reform has had to be layered onto structures built for other purposes: a state insurance regime designed around insurers, a federal benefits statute written for pensions, and public programs built on federal-state bargains. The seams between those structures are where the conflicts occur, and the conflicts are almost always framed as jurisdictional because the underlying substantive disagreements cannot be resolved by the institutions asked to resolve them. Courts are ill-equipped to decide what a fair price for anesthesia is. They are well equipped to decide whether a statute lets an agency presume one. The practical consequence is that American health insurance law is unusually procedural. The rights that matter most to individuals often turn on which regime governs them, what process they must follow, and which institution reviews the result. A patient who knows that her employer's plan is self-funded, that her internal appeal will form the record for any later lawsuit, and that she is entitled to binding external review is materially better protected than one who does not, even though the substantive law is the same for both. The direction of travel Three trends visible across the chapters point toward the next several years. The first is a steady migration of regulatory authority from the states to the federal government, driven by ERISA. Each time a problem proved impossible to solve through state insurance law because self-funded plans were beyond reach, Congress eventually stepped in: with the ACA's market reforms, with the No Surprises Act, with the transparency provisions of 2021, and with the pharmacy benefit reforms of 2026. That pattern is likely to continue. States will keep experimenting, particularly in regulating providers and intermediaries, and those experiments will keep testing the boundary drawn in Rutledge and Mulready. But where a uniform national rule for employer plans is needed, only Congress can supply it. The second trend runs in the opposite direction for public coverage. In Medicaid, the federal government is not so much centralizing as conditioning: the 2025 reconciliation law imposes new federal requirements on expansion adults while shrinking the financing tools that make expansion affordable for states, and the Supreme Court's decision in Medina has narrowed the ability of beneficiaries to enforce federal requirements against states in court. The combined effect is to push decisions about coverage back toward state budgets and administrative capacity, with federal requirements that are more restrictive of eligibility than before and less enforceable by the people they govern. The third trend is the weakening of agency power. The end of Chevron deference in Loper Bright, the willingness of courts to vacate rules nationwide under the Administrative Procedure Act, and the growing practice of successive administrations announcing non-enforcement of their predecessors' rules have all made regulations less durable. The No Surprises Act litigation shows the consequence: a statute that depends on agencies to fill in its operational details becomes, in practice, whatever survives review in the most active district court. That instability is likely to affect every regulatory program in this field, from mental health parity to price transparency to the forthcoming implementation of Medicaid work requirements. Coverage in a contracting period These legal trends are unfolding against a changed coverage picture. The enhanced premium tax credits that supported record marketplace enrollment expired at the start of 2026, and enrollment fell for the first time in years. The Medicaid provisions of the 2025 law, most of which take effect from 2027 onward, are projected to increase the uninsured population by millions. Premiums in the individual market are rising by double digits for a second consecutive year, and employer premiums continue to grow faster than wages. After more than a decade in which the share of Americans with coverage rose, the coming years are likely to see it fall. That shift will change which legal questions matter most. In a period of expanding coverage, the central legal issues concerned how to build and defend new programs. In a period of contraction, they will concern how eligibility is determined and terminated, whether procedural protections are honored, who pays for uncompensated care, and how providers and states respond to losing revenue. The experience of the Arkansas work requirement and the post-pandemic unwinding suggests that administrative process, rather than substantive eligibility rules, will determine a large share of who loses coverage. Litigation over those processes, constrained by Medina but not eliminated, will be one of the principal ways the law responds. What remains unsettled Several questions examined in this book remain open and are likely to be resolved, or at least sharply contested, in the near term. Whether courts will treat the new statutory Medicaid work requirements as a license for any implementation the states choose, or will enforce the statutory exemptions and procedural protections, remains to be seen, as does the question of who may bring such challenges after Medina. Whether the independent dispute resolution process under the No Surprises Act can be made administratively stable, and whether its outcomes will push in-network rates upward, will determine whether the statute's greatest achievement, protecting patients from surprise bills, comes at a meaningful cost to premiums. Whether ERISA fiduciary litigation over health plan costs can overcome the standing barrier, and whether the 2026 reforms give employers enough information to act, will determine whether the employer becomes an effective check on intermediaries or remains a passive payer. Whether the executive's control over the bodies that define the ACA's preventive-care guarantee, which Braidwood confirmed, will be used to alter the scope of free coverage, and whether those changes will be challenged successfully, is a question that has moved from the courtroom to the Department of Health and Human Services. And whether price transparency will lower prices, or merely document them, will depend on whether the purchasers with the most leverage use the data it has produced. What the reader should do differently For the individual, the lessons of this book reduce to a few habits. Know which legal regime governs your coverage: ask whether your employer's plan is insured or self-funded, and understand that the answer determines which protections apply. Treat every appeal of a denied claim as the decisive stage, request the plan documents and clinical criteria, and use external review. Know that surprise bills for emergency care and for ancillary services at in-network facilities are prohibited, and that the uninsured are entitled to good-faith estimates. For employers, the lesson is that health plan administration is now a fiduciary responsibility in substance as well as form. For state policymakers, it is that ERISA's boundary can be navigated by regulating markets and intermediaries rather than plans, but not crossed. For federal policymakers, it is that statutes which leave their central operating choices to agencies will be only as durable as those agencies' rules are in court. The seams in American health insurance law are not going to close. The system is too large, too entrenched, and too dependent on the separate institutions that built it. But the seams can be understood, and a reader who understands them can see the next dispute coming, recognize what it is really about, and know where the decision will actually be made. Notes Introduction 1. McCarran-Ferguson Act, 15 U.S.C. §§ 1011–1015 (1945). 1. Employee Retirement Income Security Act of 1974, Pub. L. No. 93-406, 88 Stat. 829. 2. Patient Protection and Affordable Care Act, Pub. L. No. 111-148, 124 Stat. 119 (2010), as amended by the Health Care and Education Reconciliation Act of 2010, Pub. L. No. 111-152. 3. Loper Bright Enterprises v. Raimondo, 603 U.S. 369 (2024). Chapter 1 1. United States v. South-Eastern Underwriters Association, 322 U.S. 533 (1944). 1. ERISA §§ 502(a), 514(a), 514(b)(2)(A)–(B), 29 U.S.C. §§ 1132(a), 1144. 2. FMC Corp. v. Holliday, 498 U.S. 52 (1990). 3. KFF, Employer Health Benefits Survey (annual), for the share of covered workers in self-funded plans. 4. Public Health Service Act §§ 2701–2719A, 42 U.S.C. §§ 300gg et seq. (ACA market reforms); 42 U.S.C. § 18022 (essential health benefits); 42 U.S.C. § 300gg-18 (medical loss ratio). 5. 42 U.S.C. § 1395w-26(b)(3) (Medicare Advantage preemption). Chapter 2 1. National Federation of Independent Business v. Sebelius, 567 U.S. 519 (2012). 1. King v. Burwell, 576 U.S. 473 (2015). 2. U.S. House of Representatives v. Burwell, 185 F. Supp. 3d 165 (D.D.C. 2016). 3. Community Health Choice, Inc. v. United States, 970 F.3d 1364 (Fed. Cir. 2020); Sanford Health Plan v. United States, 969 F.3d 1370 (Fed. Cir. 2020). 4. Maine Community Health Options v. United States, 590 U.S. 296 (2020). 5. Texas v. United States, 340 F. Supp. 3d 579 (N.D. Tex. 2018); Texas v. United States, 945 F.3d 355 (5th Cir. 2019); California v. Texas, 593 U.S. 659 (2021). 6. Burwell v. Hobby Lobby Stores, Inc., 573 U.S. 682 (2014); Zubik v. Burwell, 578 U.S. 403 (2016); Little Sisters of the Poor Saints Peter and Paul Home v. Pennsylvania, 591 U.S. 657 (2020). 7. Kennedy v. Braidwood Management, Inc., 606 U.S. ___ (2025) (decided June 27, 2025). 8. City of Columbus v. Kennedy (D. Md. Aug. 2025) (order staying portions of the 2025 Marketplace Integrity and Affordability rule). 9. KFF analyses of 2026 ACA Marketplace enrollment and 2027 proposed premium changes (2026). Chapter 3 1. 42 U.S.C. § 1396a(a)(10)(A)(i)(VIII) (expansion group); 42 U.S.C. § 1396d(y) (expansion federal match). 1. Doyle v. Tidball, 625 S.W.3d 459 (Mo. 2021). 2. Stewart v. Azar, 313 F. Supp. 3d 237 (D.D.C. 2018); Gresham v. Azar, 363 F. Supp. 3d 165 (D.D.C. 2019), aff'd, 950 F.3d 93 (D.C. Cir. 2020), vacated as moot sub nom. Becerra v. Gresham, 142 S. Ct. 1665 (2022). 3. Georgia v. Brooks-LaSure (S.D. Ga.) (litigation over the Georgia Pathways approvals, 2022 and 2024 rulings). 4. Consolidated Appropriations Act, 2023, Pub. L. No. 117-328, div. FF, § 5131 (ending the Medicaid continuous enrollment condition). 5. An Act to provide for reconciliation pursuant to title II of H. Con. Res. 14, Pub. L. No. 119-21 (July 4, 2025) (commonly called the One Big Beautiful Bill Act), title VII (Medicaid provisions). 6. Congressional Budget Office, estimates of the effects of Pub. L. No. 119-21 on health insurance coverage (2025). 7. Wilder v. Virginia Hospital Association, 496 U.S. 498 (1990); Gonzaga University v. Doe, 536 U.S. 273 (2002); Armstrong v. Exceptional Child Center, Inc., 575 U.S. 320 (2015). 8. Health and Hospital Corp. of Marion County v. Talevski, 599 U.S. 166 (2023). 9. Medina v. Planned Parenthood South Atlantic, 606 U.S. ___ (2025) (decided June 26, 2025). Chapter 4 1. Shaw v. Delta Air Lines, Inc., 463 U.S. 85 (1983). 1. Pilot Life Insurance Co. v. Dedeaux, 481 U.S. 41 (1987). 2. Metropolitan Life Insurance Co. v. Massachusetts, 471 U.S. 724 (1985). 3. New York State Conference of Blue Cross & Blue Shield Plans v. Travelers Insurance Co., 514 U.S. 645 (1995). 4. De Buono v. NYSA-ILA Medical and Clinical Services Fund, 520 U.S. 806 (1997); California Division of Labor Standards Enforcement v. Dillingham Construction, N.A., Inc., 519 U.S. 316 (1997). 5. Egelhoff v. Egelhoff, 532 U.S. 141 (2001). 6. Retail Industry Leaders Association v. Fielder, 475 F.3d 180 (4th Cir. 2007). 7. Golden Gate Restaurant Association v. City and County of San Francisco, 546 F.3d 639 (9th Cir. 2008). 8. Gobeille v. Liberty Mutual Insurance Co., 577 U.S. 312 (2016). 9. Rutledge v. Pharmaceutical Care Management Association, 592 U.S. 80 (2020). 10. Pharmaceutical Care Management Association v. Mulready, 78 F.4th 1183 (10th Cir. 2023), cert. denied (2024). Chapter 5 1. Health Maintenance Organization Act of 1973, Pub. L. No. 93-222. 1. Pegram v. Herdrich, 530 U.S. 211 (2000). 2. Metropolitan Life Insurance Co. v. Taylor, 481 U.S. 58 (1987); Aetna Health Inc. v. Davila, 542 U.S. 200 (2004). 3. Firestone Tire & Rubber Co. v. Bruch, 489 U.S. 101 (1989); Metropolitan Life Insurance Co. v. Glenn, 554 U.S. 105 (2008). 4. 29 C.F.R. § 2560.503-1 (ERISA claims procedure regulation). 5. Rush Prudential HMO, Inc. v. Moran, 536 U.S. 355 (2002); Kentucky Association of Health Plans, Inc. v. Miller, 538 U.S. 329 (2003). 6. Public Health Service Act § 2719, 42 U.S.C. § 300gg-19 (internal appeals and external review). 7. Mental Health Parity and Addiction Equity Act of 2008, Pub. L. No. 110-343, div. C, title V, subtitle B; Requirements Related to the Mental Health Parity and Addiction Equity Act, final rules (September 2024). 8. Wit v. United Behavioral Health, 79 F.4th 1068 (9th Cir. 2023). 9. CMS Interoperability and Prior Authorization Final Rule (CMS-0057-F), 89 Fed. Reg. 8758 (Feb. 8, 2024). 10. Estate of Lokken v. UnitedHealth Group, Inc., No. 0:23-cv-03514 (D. Minn.); Kisting-Leung v. Cigna Corp., No. 2:23-cv-01477 (E.D. Cal.). 11. California Senate Bill 1120 (2024) (physician review of utilization management decisions). Chapter 6 1. No Surprises Act, Consolidated Appropriations Act, 2021, Pub. L. No. 116-260, div. BB, title I; 42 U.S.C. §§ 300gg-111 to 300gg-139. 1. Zack Cooper, Fiona Scott Morton, and Nathan Shekita, "Surprise! Out-of-Network Billing for Emergency Care in the United States," Journal of Political Economy 128, no. 9 (2020). 2. N.Y. Financial Services Law, art. 6 (2015); Cal. Health & Safety Code § 1371.31 (AB 72, 2016); Texas S.B. 1264 (2019). 3. Requirements Related to Surprise Billing; Part II, 86 Fed. Reg. 55980 (Oct. 7, 2021); Requirements Related to Surprise Billing, Final Rules, 87 Fed. Reg. 52618 (Aug. 26, 2022). 4. Texas Medical Association v. U.S. Department of Health and Human Services, 587 F. Supp. 3d 528 (E.D. Tex. 2022); Texas Medical Association v. HHS, 654 F. Supp. 3d 575 (E.D. Tex. 2023), aff'd, 110 F.4th 762 (5th Cir. 2024). 5. Texas Medical Association v. HHS, No. 6:22-cv-450 (E.D. Tex. Aug. 24, 2023) (QPA methodology); Texas Medical Association v. HHS, No. 6:23-cv-59 (E.D. Tex. Aug. 3, 2023) (administrative fee and batching). 6. Centers for Medicare & Medicaid Services, Federal Independent Dispute Resolution Process public use files and reports (2023–2025). Chapter 7 1. Public Health Service Act § 2718(e), 42 U.S.C. § 300gg-18(e). 1. Executive Order 13877, Improving Price and Quality Transparency in American Healthcare to Put Patients First, 84 Fed. Reg. 30849 (June 24, 2019). 2. Price Transparency Requirements for Hospitals To Make Standard Charges Public, 84 Fed. Reg. 65524 (Nov. 27, 2019); 45 C.F.R. part 180. 3. American Hospital Association v. Azar, 468 F. Supp. 3d 372 (D.D.C. 2020), aff'd, 983 F.3d 528 (D.C. Cir. 2020). 4. Transparency in Coverage, 85 Fed. Reg. 72158 (Nov. 12, 2020). 5. Consolidated Appropriations Act, 2021, div. BB, title II, §§ 201–204 (gag clauses, compensation disclosure, parity analyses, prescription drug reporting). 6. Executive Order 14221, Making America Healthy Again by Empowering Patients With Clear, Accurate, and Actionable Healthcare Pricing Information (Feb. 25, 2025). 7. Centers for Medicare & Medicaid Services, CY 2026 Hospital Outpatient Prospective Payment System final rule (hospital price transparency provisions; enforcement from April 1, 2026). 8. Sunita Desai et al., "Association Between Availability of a Price Transparency Tool and Outpatient Spending," JAMA 315, no. 17 (2016). 9. Svend Albæk, Peter Møllgaard, and Per Baltzer Overgaard, "Government-Assisted Oligopoly Coordination? A Concrete Case," Journal of Industrial Economics 45, no. 4 (1997). Chapter 8 1. ERISA §§ 404(a), 406(a), 408(b)(2), 29 U.S.C. §§ 1104(a), 1106(a), 1108(b)(2). 1. Hi-Lex Controls, Inc. v. Blue Cross Blue Shield of Michigan, 751 F.3d 740 (6th Cir. 2014). 2. Federal Trade Commission, Pharmacy Benefit Managers: The Powerful Middlemen Inflating Drug Costs and Squeezing Main Street Pharmacies, Interim Staff Report (July 2024), and Second Interim Staff Report (January 2025). 3. Lewandowski v. Johnson & Johnson (D.N.J.); Navarro v. Wells Fargo & Co. (D. Minn.). 4. Thole v. U.S. Bank N.A., 590 U.S. 538 (2020). 5. Cunningham v. Cornell University, 604 U.S. ___ (2025) (decided April 17, 2025). 6. Consolidated Appropriations Act, 2026 (enacted February 2026) (pharmacy benefit manager provisions). 7. U.S. Department of Labor, Employee Benefits Security Administration, proposed rule on pharmacy benefit manager compensation disclosure (January 2026). Glossary Balance billing. A provider's billing of a patient for the difference between the provider's charge and the amount the patient's plan paid. Largely prohibited for surprise bills since 2022. Deemer clause. ERISA section 514(b)(2)(B), which provides that an employee benefit plan is not to be treated as an insurer for purposes of state insurance law. It places self-funded plans beyond the reach of state insurance regulation. Essential health benefits. The ten categories of services that non-grandfathered individual and small-group insurance policies must cover under the Affordable Care Act. External review. Review of a denied claim by an independent review organization whose decision binds the plan. Required for most plans by the Affordable Care Act. Federal Medical Assistance Percentage (FMAP). The share of a state's Medicaid costs paid by the federal government. For the expansion population it is 90 percent. Fiduciary. Under ERISA, a person who exercises discretionary authority over plan management or administration or over plan assets, and who must act solely in the interest of participants and beneficiaries. Fully insured plan. An employer plan that pays for benefits by buying an insurance policy. The insurer bears the risk and state insurance law applies to the policy. Independent dispute resolution (IDR). The federal baseball-style arbitration process under the No Surprises Act for resolving payment disputes between out-of-network providers and plans. Nonquantitative treatment limitation (NQTL). A non-numerical limit on benefits, such as prior authorization, medical necessity criteria, or network standards, subject to comparison under the mental health parity law. Pharmacy benefit manager (PBM). An intermediary that administers prescription drug benefits for insurers and plans, building pharmacy networks, negotiating rebates, and managing formularies. Qualifying payment amount (QPA). Under the No Surprises Act, generally a plan's median contracted rate for a service in a geographic area as of 2019, adjusted for inflation. It determines patient cost-sharing and is a factor in arbitration. Savings clause. ERISA section 514(b)(2)(A), which preserves state laws that regulate insurance from ERISA preemption. Section 1115 waiver. Authority for the Secretary of Health and Human Services to waive Medicaid requirements for demonstration projects likely to promote the program's objectives. Self-funded plan. An employer plan that pays claims from the employer's own assets, usually through a third-party administrator. It is governed by ERISA and largely exempt from state insurance law. Further Reading Timothy S. Jost, Health Care at Risk: A Critique of the Consumer-Driven Movement (Duke University Press, 2007).** A leading health law scholar's examination of the theory that informed consumers can discipline health care prices, useful background for the debates over price transparency. Mark A. Hall and co-authors, Health Care Law and Ethics (Aspen, multiple editions).** A standard law school casebook with thorough coverage of ERISA preemption, managed care liability, and the Affordable Care Act. Barry R. Furrow and co-authors, Health Law: Cases, Materials and Problems (West Academic, multiple editions).** Another comprehensive casebook, strong on the regulation of insurers and providers. Abbe R. Gluck and Nicole Huberfeld, "What Is Federalism in Healthcare For?," Stanford Law Review 70 (2018).** An influential account of how the Affordable Care Act relies on states and how implementation reshaped federal-state relations in health care. Nicholas Bagley, "Federalism and the End of Obamacare," Yale Law Journal Forum 127 (2017).** A short, clear analysis of the structural vulnerabilities of the ACA's reliance on states and federal appropriations. Zack Cooper, Fiona Scott Morton, and Nathan Shekita, "Surprise! Out-of-Network Billing for Emergency Care in the United States," Journal of Political Economy 128 (2020).** The empirical study that documented how physician staffing firms used out-of-network status, which shaped the congressional debate over the No Surprises Act. KFF (kff.org). The most reliable single source of current data and plain-language analysis on the Affordable Care Act marketplaces, Medicaid expansion and work requirements, employer coverage, and the No Surprises Act, including its annual Employer Health Benefits Survey. Georgetown University Center on Health Insurance Reforms (chir.georgetown.edu). Close tracking of federal and state insurance regulation, surprise billing implementation, and state reforms. Health Affairs Forefront (healthaffairs.org). Timely commentary by legal and policy experts on litigation and rulemaking affecting the ACA, ERISA plans, and transparency rules. U.S. Department of Labor, Employee Benefits Security Administration (dol.gov/ebsa). Primary source for ERISA regulations, fiduciary guidance, the claims procedure rule, and compliance materials on the Consolidated Appropriations Act requirements. Centers for Medicare & Medicaid Services, No Surprises and Hospital Price Transparency resources (cms.gov). Official guidance, independent dispute resolution reports, and enforcement information. Hashtags: #HealthInsuranceLaw #AffordableCareAct #ACA #ERISA #ManagedCare #HealthcareLaw #HealthLaw #InsuranceRegulation #HealthcarePolicy #USHealthcareSystem #Medicaid #Medicare #MedicaidExpansion #EmployerSponsoredHealthInsurance #SelfFundedHealthPlans #ERISAPreemption #HealthInsuranceCoverage #PatientRights #HealthcareLitigation #InsuranceClaimDenials #PriorAuthorization #MedicalNecessity #NoSurprisesAct #SurpriseMedicalBilling #HealthcarePriceTransparency #PharmacyBenefitManagers #PBMRegulation #MentalHealthParity #HealthcareCompliance #HealthPlanFiduciaryDuties

Download the Book (PDF): Introduction Most people who move into a residence hall have never been solely responsible for keeping food safe. At home, someone else bought the refrigerator, set its dial, noticed when the milk turned, and decided how long the leftover lasagna could stay on the shelf. Then, in the space of a weekend, a student inherits a refrigerator the size of a filing cabinet, a microwave that may be older than they are, a shared kitchen down the hall used by forty strangers, and a housing contract that forbids half the appliances they were planning to bring. Nobody hands them an owner's manual for this arrangement. They improvise, and most of the time the improvising works well enough. Sometimes it does not. A pot of rice cooked on Sunday and eaten through Thursday produces a night of violent vomiting. A container of chicken reheated for ninety seconds comes out steaming on the edges and cold in the middle. A roommate with a tree nut allergy reacts to a cutting board nobody thought to wash. A pan of oil left on a floor-kitchen burner while its owner answers a text fills the hallway with smoke and sends three hundred people onto the lawn at eleven at night. None of these events requires carelessness in any dramatic sense. They require only ordinary habits carried into an environment that punishes them. This book is about that environment, and about the small body of knowledge that makes it manageable. The argument running through it is simple: dorm food safety is less a matter of memorizing rules than of understanding a handful of mechanisms, because the dorm is exactly the kind of setting in which rules break down. A rule says to keep the refrigerator at 40°F (4°C) or colder. Understanding says why that number matters, why a small refrigerator packed full on the Sunday after a grocery run may not hold it, and how to find out in thirty seconds with a three-dollar thermometer. A rule says to reheat leftovers to 165°F (74°C). Understanding explains why a microwave leaves cold pockets, why the standing time printed on a frozen meal is part of the cooking rather than a courtesy, and why a thermometer pushed into three places tells you more than a timer ever will. Students who grasp the mechanisms can handle situations no rulebook anticipated. Students who only know the rules tend either to follow them anxiously or to abandon them the first time they seem inconvenient. Why the dorm is a special case Four features set residence-hall cooking apart from cooking in a family kitchen. The first is scale. Compact refrigerators have little thermal mass, small compressors, and thin walls. They warm quickly when the door is opened, struggle when loaded with warm food, and often include a small freezer compartment that is colder than the rest of the box without being cold enough to function as a true freezer. The margins that make a full-size household refrigerator forgiving are simply smaller. The second is sharing. Floor kitchens, communal refrigerators, and shared sinks bring together people with very different ideas about cleanliness, very different diets, and in some cases serious food allergies. A shared space multiplies the number of hands, sponges, and surfaces that food touches, and it removes the sense of ownership that makes people wipe down a counter they know they will use again tomorrow. The third is schedule. Students eat at irregular hours, order food late at night, fall asleep before putting leftovers away, and cook in bursts: a large batch on Sunday meant to last the week. Each of those habits interacts with the basic biology of bacterial growth in ways that are predictable once you know what to look for. The fourth is fire. Cooking is the leading cause of fires in campus housing, and it has been for as long as national data have been collected. The National Fire Protection Association estimates that U.S. fire departments responded to an average of more than 3,300 structure fires a year in dormitories, fraternity and sorority houses, and barracks between 2018 and 2022, and that cooking equipment was involved in nearly eight out of ten of them. Most of those fires are small and confined. They still empty buildings, damage rooms, injure people, and occasionally do far worse. The rules about which appliances are allowed in residence halls are not arbitrary; they reflect decades of experience with what happens when heating elements, bedding, overloaded outlets, and distracted people share a small room. What this book covers The chapters move from the microbes themselves to the equipment, then to the settings in which students cook and eat, and finally to what to do when something goes wrong. Chapter 1 explains what actually causes foodborne illness, which organisms matter most, and why the "danger zone" between 40°F and 140°F (4°C and 60°C) is the central idea in food safety. Chapter 2 is devoted to the compact refrigerator and its freezer compartment: how to measure what they are really doing, how to load them, how to defrost them, and what they can and cannot be trusted with. Chapter 3 turns to time, the other half of the temperature-and-time equation, and deals with takeout, late-night pizza, and leftovers. Chapter 4 covers the microwave, which is the most important cooking appliance in most dorm rooms and one of the least understood. Chapter 5 looks at rice, pasta, and batch cooking, where a spore-forming bacterium called Bacillus cereus creates a specific and underappreciated risk. Chapter 6 moves into the shared kitchen, with its sponges, sinks, and unlabeled containers, and Chapter 7 addresses food allergies and the problem of cross-contact in shared spaces. Chapter 8 deals with fire: why cooking dominates campus fire statistics, how to think about permitted and banned appliances, and what to do about extension cords, power strips, and the moment a pan catches. Chapter 9 explains what to do when prevention fails, from recognizing the signs that a stomach illness needs medical attention to handling a burn or an allergic reaction. The numbers in this book come from the agencies and researchers who set and study food safety standards in the United States: the U.S. Department of Agriculture's Food Safety and Inspection Service, the Food and Drug Administration, the Centers for Disease Control and Prevention, the National Fire Protection Association, and the U.S. Fire Administration, along with peer-reviewed studies where they add something. Temperatures are given in Fahrenheit with Celsius equivalents, because both appear on the equipment students actually buy. Scenarios describing individual students are illustrative composites built to show how common situations unfold; they are not accounts of real people. A note on scope and advice This is a practical guide to preventing illness and injury, not a substitute for medical care or for the specific rules of any particular campus. Housing policies differ from school to school and sometimes from building to building, and the policy that governs your room is the one in your housing agreement, not the typical one described here. If you are pregnant, immunocompromised, managing a chronic illness, or living with a severe food allergy, your clinician's guidance takes precedence over anything general. And if you are ever in doubt during an emergency, whether it is a fire, a severe allergic reaction, or someone who cannot be roused, call 911 first and work out the details later. One more point before beginning. Food safety writing has a habit of sliding into fear, as though every refrigerator were a biohazard and every leftover a gamble. That is neither accurate nor useful. Most meals students eat are perfectly safe, and the habits that keep them that way take little time once they become routine. The purpose of understanding the mechanisms is not to make cooking stressful but to make it relaxed: to know which situations actually carry risk, to handle those well, and to stop worrying about the rest. A student who knows why the rules exist can cook confidently in a space that was never designed for cooking at all. Chapter 1: What Actually Makes You Sick The phrase "food poisoning" suggests something like a toxin slipped into a meal, and occasionally that picture is correct. Much more often, the cause is a living organism: a bacterium, a virus, or less commonly a parasite, which either multiplies in food before it is eaten or survives long enough to multiply inside the person who eats it. Some organisms cause illness by infecting the gut directly. Others make toxins in the food itself, so that the damage is done before the first bite even if the bacteria are later killed. The distinction matters, because it explains why some kinds of mistakes can be corrected with heat and others cannot. The scale of the problem is larger than most people assume. The CDC's long-standing estimate, published in 2011, is that roughly 48 million people in the United States get sick from contaminated food each year, about 128,000 are hospitalized, and about 3,000 die. In 2025, CDC researchers published updated estimates for seven major pathogens using 2019 data. Those seven alone accounted for an estimated 9.9 million domestically acquired foodborne illnesses, 53,300 hospitalizations, and 931 deaths in a single year. Norovirus caused the majority of the illnesses. Campylobacter and nontyphoidal Salmonella followed, and Salmonella caused the most deaths among the pathogens studied. Most of these illnesses were never diagnosed as anything more specific than a bad night; people recovered, blamed the last thing they ate, and moved on. The last thing they ate is often the wrong suspect. Different organisms have very different incubation periods, from under an hour to several weeks, and a stomach illness that begins on Tuesday evening may trace back to a Sunday lunch rather than Tuesday's dinner. Understanding the main organisms, and the conditions each prefers, is the foundation for everything else in this book. The organisms that matter most Table 1 summarizes the pathogens most relevant to how students eat. It is not a complete list, and onset times vary from person to person, but it shows why no single rule covers every risk. Table 1. Common foodborne pathogens and how they typically reach students. Organism Typical sources Usual onset What prevents it Norovirus Food handled by an infected person; shared surfaces 12 to 48 hours Handwashing; sick people not cooking for others Salmonella Undercooked poultry and eggs; raw produce; cross-contamination 6 hours to 6 days Full cooking; separating raw meat Campylobacter Undercooked poultry; raw poultry juices 2 to 5 days Full cooking; clean boards and hands Clostridium perfringens Large batches of meat, stews, chili cooled slowly 6 to 24 hours Rapid cooling in small portions Bacillus cereus Cooked rice and pasta left warm 30 minutes to 15 hours Rapid cooling; short storage Listeria monocytogenes Deli meats, soft cheeses, refrigerated ready-to-eat foods Days to weeks Cold fridge; short storage; reheating deli meat when at high risk Sources: CDC and FDA pathogen fact sheets; University of Florida IFAS Extension. Norovirus deserves special attention in residence halls. It is extraordinarily contagious, spreads easily from person to person as well as through food, survives on surfaces, and is not reliably killed by alcohol-based hand sanitizer. Outbreaks in dormitories, cruise ships, and nursing homes share the same underlying feature: many people in close quarters touching the same doorknobs, faucets, and serving spoons. When norovirus moves through a residence hall, food safety and general hygiene become the same problem. The CDC advises that people sick with norovirus should not prepare food for others while they are ill and for at least two days after their symptoms stop, because they continue to shed the virus. Salmonella and Campylobacter are the classic raw-poultry organisms. Both are commonly present on raw chicken as it is sold, both are killed by thorough cooking, and both cause trouble mainly when chicken is undercooked or when its juices reach something that will not be cooked, such as a salad, a piece of bread, or a hand that then picks up a snack. The practical lesson is not that raw chicken is uniquely dangerous but that it should be treated as contaminated by default: cooked to a safe internal temperature and kept away from ready-to-eat food. Clostridium perfringens and Bacillus cereus belong to a different category. Both form spores, dormant structures that survive normal cooking. When the cooked food then sits warm, as a big pot of chili does while it slowly cools on a stove, or as rice does in a cooker left on "keep warm" with the lid cracked, the spores germinate and the bacteria multiply rapidly. C. perfringens is especially associated with large quantities of meat and gravy; B. cereus with starchy foods, above all rice. Cooking cannot solve these problems after the fact, because the spores were never killed in the first place and some of what they produce resists heat. Chapter 5 is devoted to this category, because batch cooking is so common in student life. Listeria monocytogenes is the odd one out. Most bacteria slow to a crawl at refrigerator temperatures, but Listeria can grow slowly in the cold. It is associated with ready-to-eat foods that are not cooked before eating, such as deli meats, hot dogs eaten cold, smoked fish, and some soft cheeses. For most healthy young adults, Listeria rarely causes serious illness. For pregnant people, it is a genuine hazard, because it can cross the placenta and cause miscarriage, stillbirth, or severe infection in a newborn even when the mother feels only mildly ill. It is also dangerous for people with weakened immune systems. Anyone in those groups should follow their clinician's advice on deli meats and soft cheeses, which often includes heating deli meats until steaming before eating them. The danger zone and why it works Almost every piece of food safety advice rests on one idea: bacteria that cause foodborne illness grow fastest between about 40°F and 140°F (4°C and 60°C). The USDA calls this range the danger zone. Below it, most bacteria grow very slowly or not at all, although they are not killed. Above it, they stop growing, and at sufficiently high temperatures they die. Inside it, a few bacteria can become millions within hours, because under favorable conditions many foodborne bacteria can double their numbers roughly every twenty minutes. Doubling is the key. Consider a portion of cooked chicken that carries a small, harmless number of bacteria after it cools on the counter. At room temperature, if those bacteria double every twenty minutes, one becomes eight in an hour, sixty-four in two hours, and more than a quarter of a million in six. Real growth depends on the organism, the food, and the starting temperature, and it rarely proceeds at the fastest possible rate throughout. But the shape of the curve is what matters. For the first stretch, the numbers stay low and the food is fine. Then, relatively suddenly, the population becomes large enough to cause illness or, in the case of toxin-producing species, to accumulate enough toxin to make someone sick even if the bacteria are later killed. This is why the time limits in food safety are fairly short and fairly strict. The USDA's guidance is that perishable food should not sit in the danger zone for more than two hours in total, or for more than one hour if the surrounding temperature is above 90°F (32°C), such as in a hot car or at an outdoor event in summer. The two-hour rule is not a claim that food becomes dangerous at minute 121 and was perfectly safe at minute 119. It is a conservative line drawn to stay well on the safe side of the curve, given that nobody knows exactly how many bacteria a particular piece of food started with. The danger zone also explains the two target temperatures that dominate the rest of this book. A refrigerator at 40°F (4°C) or colder keeps food below the zone, slowing bacterial growth enough that cooked leftovers stay safe for several days. Cooking food to its safe minimum internal temperature, which ranges from 145°F (63°C) for whole cuts of beef and pork to 165°F (74°C) for poultry and leftovers, pushes it far enough above the zone, for long enough, to kill the vegetative bacteria that are present. Everything between those two numbers is territory to move through quickly: cooling hot food, thawing frozen food, carrying food from one place to another. Why "it smells fine" is not a test Many students rely on smell and appearance to decide whether food is safe. Those senses are good at detecting spoilage and poor at detecting danger. Spoilage bacteria, the organisms that make milk sour, bread moldy, and meat slimy, are mostly different from the pathogens that cause illness. A container of rice contaminated with enough B. cereus toxin to cause hours of vomiting can look, smell, and taste completely normal. A piece of chicken that smells slightly off may be unpleasant without being dangerous. The two processes overlap only loosely. This does not mean that smell is useless. Food that smells rotten, has visible mold where mold does not belong, or has an obviously wrong texture should be thrown out. The point is that the reverse inference does not hold. Passing the smell test tells you only that spoilage organisms have not yet done their work. For the danger of illness, the better guides are the temperature history of the food and how long it has been stored, which is why the rest of this book spends so much time on thermometers and clocks. Tracing an illness backward Because incubation periods vary so widely, the instinct to blame the most recent meal is often wrong, and getting it wrong has consequences. A student who wrongly blames Tuesday's dining hall stir-fry may keep eating from the container of leftovers in their own refrigerator that actually caused the problem. Consider a composite case. A junior wakes at 3 a.m. on a Thursday with cramps and diarrhea, followed by a fever the next day. The obvious suspect is the takeout burrito she ate at 9 p.m. Wednesday. But a fever with diarrhea beginning a few hours after a meal fits poorly with most bacterial infections, which usually take a day or more to develop. Looking back, she remembers the chicken she cooked in her floor kitchen on Monday, which she sliced on the same board she then used for a salad without washing it in between. Campylobacter, with an incubation period of two to five days, fits that timeline better than the burrito does. The reverse pattern is also informative. Vomiting that begins within about thirty minutes to six hours of eating, often abruptly and often in several people who shared the same dish, suggests a toxin that was already present in the food. That is the typical picture of Staphylococcus aureus toxin, which forms when food handled with contaminated hands is left warm, and of the emetic form of B. cereus illness associated with rice. Here the most recent meal really is the likely culprit, and anyone else who ate from the same batch may want to throw out what remains. None of this is meant to turn students into amateur epidemiologists. Most mild cases never receive a specific diagnosis, and most do not need one. But two practical conclusions follow. First, if you suspect a meal made you ill, think back several days, not just several hours, and look in particular at anything involving raw poultry, undercooked eggs, or food that sat warm for a long time. Second, when several people become sick after eating the same food, it is worth reporting to the campus health service or the local health department. They can identify outbreaks that no individual would recognize, and a cluster of cases in one residence hall is exactly the kind of signal public health investigators rely on. Chapter 9 describes how to make such a report. Why students are a particular case Healthy young adults are not the group most likely to die from foodborne illness; that burden falls most heavily on older adults, young children, pregnant people, and people with weakened immune systems. But students carry a combination of risk factors that is unusual. The first is exposure. Students often eat in more places, from more sources, and at more unusual hours than almost any other group: dining halls, food trucks, late-night delivery, club events with catered trays sitting out for hours, potlucks, and their own cooking in unfamiliar kitchens. Every additional source is another chance for something to have gone wrong upstream. The second is equipment. The appliances available in a dorm room are designed for convenience and compactness, not for precise temperature control. A compact refrigerator and a microwave can do a great deal, but both have weaknesses that are easy to miss, and both are often shared, borrowed, or inherited from a previous occupant. The third is density. Residence halls pack hundreds of people into shared bathrooms, lounges, and kitchens. Infections that spread from person to person, including norovirus, move through those buildings efficiently, and a single sick person who prepares food for a floor event can make dozens of others ill. The fourth is cost. An illness that keeps someone in bed for two days is an inconvenience for many adults. For a student, it can mean a missed exam, a lost shift at a job that pays rent, or a week of catching up on coursework. The consequences of an ordinary foodborne illness land differently in a semester with little slack. None of this is cause for alarm. It is a reason to understand the handful of mechanisms that make the difference, which is the subject of the chapters that follow. Chill, cook, clean, separate Public agencies in the United States have organized consumer food safety around four actions: clean, separate, cook, and chill. They are worth naming here because they give the rest of this book its skeleton. Clean covers handwashing and the washing of surfaces and utensils. Separate covers keeping raw meat, poultry, seafood, and eggs away from ready-to-eat food, and in the context of allergies, keeping allergenic foods away from people who react to them. Cook means reaching the safe internal temperature for the food in question and verifying it with a thermometer rather than guessing. Chill means refrigerating promptly and keeping the refrigerator cold. In a family kitchen, each of these steps has a certain amount of built-in support: a large refrigerator, a stove with reliable burners, a sink with hot water, space to keep things apart. In a dorm, several of those supports are weaker or missing, and each of the four actions needs a small adaptation. The compact refrigerator makes chill harder to guarantee. The microwave makes cook uneven. The shared kitchen makes clean and separate depend on other people. The chapters that follow take these adaptations one at a time, starting with the appliance that most students rely on more than they realize: the small refrigerator humming in the corner of the room. Chapter 2: The Micro-Fridge Reality The compact refrigerator is the quiet center of dorm food safety. It holds the milk for cereal, the leftovers from Friday's takeout, the yogurt bought in bulk, the deli meat for sandwiches, and on a good week, the ingredients for real meals. Many students treat it as a smaller version of the refrigerator at home. It is not. A compact unit shares the same basic principle as a full-size refrigerator, but it operates with much less margin for error, and its weaknesses show up in exactly the ways dorm life tends to test them. The standard is straightforward. The FDA and USDA advise keeping a refrigerator at 40°F (4°C) or below and a freezer at 0°F (−18°C). At 40°F, most foodborne bacteria grow very slowly, which is why cooked leftovers can be stored for three to four days rather than a few hours. At 0°F, bacterial growth stops altogether, although freezing does not kill most bacteria; they resume growing once food thaws. The question in a dorm is not whether these targets are correct but whether a particular small refrigerator is actually meeting them. Why small refrigerators struggle A refrigerator does not create cold. It moves heat out of an insulated box, using a compressor and a refrigerant loop, and dumps that heat into the room through coils at the back or sides. How well it holds temperature depends on how much heat is entering the box, how quickly the compressor can remove it, and how well the insulation resists the outside air in the meantime. On each of those counts, a compact unit is at a disadvantage. Its walls are thinner. Its compressor is small. Its interior volume is small relative to the area of its door, so each opening lets in a larger share of warm air than the same opening of a full-size refrigerator. And it is often placed where it cannot shed heat well: wedged under a desk, pushed flush against a wall, surrounded by boxes, or sitting in a room that reaches 80°F (27°C) on a September afternoon in a building without air conditioning. Heat that cannot escape from the coils stays near the refrigerator and makes the compressor work harder for less effect. Many compact units also use a simple mechanical thermostat, a dial numbered from one to seven or marked from "min" to "max." Those numbers are not temperatures. They describe a relative setting, and the actual temperature at any given setting depends on the unit, the room, and how full the box is. A dial set to the middle may produce 36°F (2°C) in one unit and 45°F (7°C) in another. The only way to know is to measure. The three-dollar solution An appliance thermometer is the single most useful piece of food safety equipment a student can own. The FDA recommends buying an inexpensive freestanding thermometer for the refrigerator and another for the freezer, and checking them often. They cost a few dollars at a grocery or hardware store, need no batteries in their simplest form, and remove the guesswork from every other decision in this chapter. Using one well takes a little attention. Place the thermometer in the middle of the main compartment, not in the door and not pressed against the back wall, since those are the warmest and coldest spots respectively. Leave it for several hours, ideally overnight, before trusting the reading, because the thermometer itself needs time to reach the temperature of the air around it. Read it first thing in the morning, when the door has been closed for hours, and again in the evening after a normal day of opening and closing. If the reading is above 40°F (4°C), turn the dial one step colder, wait another day, and read it again. The readings will often be surprising. Students who measure for the first time commonly discover that the door shelf, where they keep milk, runs several degrees warmer than the center; that the back of the top shelf, near the cooling plate, sometimes freezes lettuce; and that the whole box warms by several degrees after a grocery run. These are not defects. They are the ordinary behavior of a small refrigerator, and knowing them allows food to be placed where it belongs. Consider a composite case. A first-year student plugs in a compact refrigerator he bought secondhand from a graduating senior, sets the dial to "4" because that is where it was, and stocks it with milk, yogurt, eggs, and a rotisserie chicken. The milk sours two days before its date, which he blames on the store. A thermometer, bought as an afterthought, reads 46°F (8°C) the next morning. Turning the dial to "6" brings the center down to 38°F (3°C) within a day. Nothing about the refrigerator was broken; it was simply set for a different room and a different load. Without the thermometer, he would have spent the semester storing chicken six degrees too warm. Loading, placement, and the Sunday problem Refrigerators cool food by circulating cold air around it. When a compact refrigerator is packed full, air cannot reach the middle of the load, and items in the center stay warmer than the thermometer suggests. When it is nearly empty, it has little thermal mass, so each door opening causes a large swing in temperature. The best-performing state is in between: reasonably full, with space between items and a clear path for air near the cooling surface. The most demanding moment in a compact refrigerator's week is the return from the grocery store. A student comes back with bags of food that have been warming in a car or on a bus, opens the door for several minutes to find space, and packs everything in at once. The internal temperature can climb well above 40°F (4°C), and a small compressor may need hours to pull it back down. A few habits soften the effect. Shop with a cooler bag or insulated tote when possible, especially for meat and dairy. Go directly home rather than making other stops. Put the most perishable items, meat and poultry first, into the coldest part of the refrigerator right away. And avoid putting a large pot of hot food into the refrigerator at the same time, since that adds a great deal of heat at once. Where food goes inside the box matters too. Raw meat, poultry, and seafood belong on the lowest shelf or in a sealed container, where their juices cannot drip onto anything else. Ready-to-eat foods, such as leftovers, deli items, and cut fruit, belong above them. The door, which is the warmest part of the refrigerator and swings through room air every time it opens, is best used for condiments, drinks, and other items that tolerate some warmth, rather than for milk, eggs, or leftovers. The freezer compartment Many compact refrigerators include a small freezer compartment behind a flap door inside the main box. Students reasonably assume it is a freezer. Often it is not, or not a very good one. In many single-door compact units, the "freezer" is a small metal enclosure wrapped around the cooling element, colder than the main compartment but not isolated from it. It may hold ice cubes perfectly well while running well above the 0°F (−18°C) target for frozen food storage, and its temperature can rise quickly when the main door is opened. The practical consequence is that a compact unit's freezer compartment should be tested, not trusted. Put a second appliance thermometer inside it overnight. If it reads 0°F or colder, it can be used for longer-term frozen storage. If it reads warmer, such as 10°F or 15°F (−12°C or −9°C), it is better treated as a place for ice and for food that will be eaten within a few days. Food held there is still frozen and still safe as long as it stays frozen solid; the concern is mainly quality, since ice cream turns grainy and meat develops freezer burn more quickly at warmer freezing temperatures. What matters for safety is that frozen food stays frozen, and that food which has partially thawed is handled as thawed food. Units with a separate freezer door, sometimes called two-door compacts, generally do a better job, because the freezer section has its own insulated space. Even so, a thermometer is the only way to know. Defrosting, cleaning, and power Frost builds up on the cooling surfaces of manual-defrost refrigerators whenever moist air enters the box. A thin layer does no harm. A thick layer acts as insulation, making the refrigerator work harder to cool its contents, and can eventually prevent the freezer flap from closing. Manufacturers generally advise defrosting well before the frost becomes thick; a common rule of thumb is to defrost when it reaches roughly a quarter of an inch. Defrosting a compact refrigerator is a small, slightly messy job that takes an afternoon. Move the food into a cooler with ice packs, or into a friend's refrigerator. Unplug the unit, prop the door open, and lay towels on the floor and in the bottom of the box to catch water. Let the frost melt on its own, or speed it by placing a bowl of hot water inside with the door closed. Do not chip at the ice with a knife or a screwdriver. The cooling element often carries refrigerant through thin tubing just beneath its surface, and puncturing it can ruin the unit. Once the frost is gone, wipe the interior dry, plug it back in, let it cool for a few hours, and confirm the temperature with the thermometer before returning the food. The defrost is also a good moment to clean. Wipe spills with hot soapy water, rinse, and dry. Pay particular attention to the bottom shelf and the drip area, where juices from raw meat collect, and to any sticky residue in the door shelves. The FDA notes that promptly cleaning up spills in the refrigerator helps limit the growth of Listeria, which, unlike most pathogens, can grow slowly at refrigerator temperatures. A lingering odor can be absorbed with an open box of baking soda. Compact refrigerators in residence halls also experience something home refrigerators rarely do: they are unplugged. Students unplug them to use the outlet for something else, trip a breaker by running a hair dryer and a microwave on the same circuit, or leave for a long weekend and switch everything off. The USDA guidance for power interruptions is that a closed refrigerator keeps food safe for about four hours, while a full freezer holds its temperature for about 48 hours and a half-full one for about 24 hours, provided the doors stay closed. Perishable food that has been above 40°F (4°C) for more than two hours should be discarded. For students, the most common version of this problem is discovering that a refrigerator has been off for an unknown length of time. When the duration is unknown and the thermometer reads above 40°F, the safe choice is to throw out meat, poultry, seafood, dairy, cooked leftovers, and cut produce. Before long breaks, the simplest approach is to eat or give away perishable food, then either leave the refrigerator running with a thermometer inside or empty it completely, unplug it, clean it, and prop the door open so it does not develop mildew. Many housing offices publish specific instructions for winter break, and some require refrigerators to be emptied and unplugged. Micro-fridge combinations and shared circuits Many campuses rent or provide combination units that stack a compact refrigerator, a small freezer compartment, and a microwave in a single appliance. These units are popular with housing offices because they limit the number of separate appliances in each room and can be designed to control total electrical load. Some combination units share a single power cord and include circuitry that prevents the refrigerator's compressor and the microwave from drawing heavy power at the same moment, which reduces the risk of tripping breakers in older buildings. That design has a side effect worth knowing. If the refrigerator pauses while the microwave runs, a unit used heavily for microwave cooking may spend a little more time without active cooling than a standalone refrigerator would. In ordinary use this makes little difference, since the refrigerator's insulation carries it through a few minutes of microwaving. But it is another reason to check the thermometer from time to time, especially in rooms where the microwave runs frequently. How long food keeps Refrigeration slows bacterial growth; it does not stop it. Every perishable food has a practical limit in the refrigerator, and the limits for common dorm foods are shorter than many people assume. Table 2 lists storage times drawn from the USDA and FoodSafety.gov cold storage guidance for foods that students often keep. These times assume a refrigerator at 40°F (4°C) or colder; in a warmer refrigerator, they shrink. Table 2. Refrigerator storage limits for common dorm foods at 40°F (4°C) or colder. Food Storage limit Notes Cooked leftovers, including takeout and pizza 3 to 4 days Count from the day it was cooked Deli meat, opened or sliced at the counter 3 to 5 days Unopened packages last longer Hot dogs, opened package 1 week Unopened: about 2 weeks Raw chicken or ground meat 1 to 2 days Freeze if not cooking soon Raw steaks and chops 3 to 5 days Keep on the lowest shelf Egg, chicken, tuna, or pasta salads 3 to 5 days Store-made or homemade Hard-cooked eggs 1 week Shell on or off Raw eggs in shell 3 to 5 weeks Keep in the carton, not the door Source: USDA Food Safety and Inspection Service; FoodSafety.gov Cold Food Storage Chart. Two tools make these limits easier to follow. The first is labeling: a strip of masking tape and a marker on each container of leftovers, with the date it was cooked. It takes five seconds and eliminates the guessing that otherwise begins around day three. The second is the USDA's free FoodKeeper app, which gives storage times for hundreds of specific foods. Neither substitutes for a cold refrigerator, but together with a thermometer they cover almost every question a student will face. The best way to think about a compact refrigerator is as a tool that works well within its limits and poorly outside them. Measured, sensibly loaded, kept clean, and not asked to store too much for too long, it does exactly what students need. The next chapter turns from the refrigerator to the clock, and to the question of what happens to food during the hours it spends outside one. Chapter 3: The Clock Is Always Running Temperature gets most of the attention in food safety, but time is the other half of every judgment. A refrigerator at 40°F (4°C) does not make food safe forever; it slows the clock. Food on a counter at room temperature does not become dangerous instantly; the clock runs faster. Nearly every practical question students face about leftovers, takeout, and food left out overnight comes down to how long a food has spent at which temperature. This chapter is about keeping track of that clock in a life that rarely runs on a predictable schedule. The two-hour rule in practice The USDA's rule is that perishable food should not be left in the danger zone, between 40°F and 140°F (4°C and 60°C), for more than two hours. When the surrounding air is above 90°F (32°C), the limit drops to one hour. Perishable food means anything that needs refrigeration to stay safe: cooked meat, poultry, and fish; cooked rice, pasta, and grains; dairy products; eggs and egg dishes; cut fruit and cooked vegetables; and most takeout. The two hours are cumulative. A container of pad thai that sat for forty minutes on a delivery driver's seat, another thirty on a desk while its owner finished an assignment, and then an hour on the floor during a movie has used up its two hours even though no single interval felt long. The clock does not reset when the food goes briefly into a colder place and then comes back out, either. A pizza that went into the refrigerator at 2 a.m. after sitting out since 10 p.m. has spent four hours in the danger zone, and the refrigerator does nothing to undo them. It will slow further growth, but the bacteria that multiplied during those four hours are still there, and in the case of toxin-producing species, so are the toxins they made. This is the part of food safety that most often collides with real life. Students eat late, fall asleep in front of screens, host people in small rooms, and share food that arrives all at once but gets eaten over hours. The practical solution is not constant vigilance but a few default habits. When food arrives, divide it immediately into what will be eaten now and what will be saved, and put the saved portion into the refrigerator before starting to eat. When hosting, put out smaller amounts and replenish from the refrigerator rather than leaving the whole tray out. And when falling asleep with food on the desk is a realistic possibility, move the leftovers to the refrigerator before settling in. The overnight pizza question No food raises the time question more often than pizza left out overnight. It is worth addressing directly because the answer surprises people. The USDA's guidance is that pizza, like other perishable food containing cheese and meat, should not be eaten if it has been left at room temperature for more than two hours. By that standard, pizza left on the counter overnight should be thrown away. Many people have eaten overnight pizza without getting sick, and they reasonably ask why the guidance is so cautious. The answer is that food safety rules are drawn to cover the range of cases, not the typical one. Most overnight pizza probably carries low bacterial numbers because it was baked at a high temperature and sat in a cool room. But the rule cannot know whether a particular pizza was topped with cheese handled by someone who had been sick, whether the room was warm, or whether the box was left open. The fact that most overnight pizza does not cause illness does not make any particular slice safe. It means that the risk is modest and unpredictable, and the cost of avoiding it is the price of a few slices. The broader point applies to every food question in this chapter. Food safety rules are not predictions that a given food will make you sick; they are lines drawn where the odds begin to rise in ways nobody can check at the table. Each person decides how to handle those odds. What this book aims to supply is an accurate picture of them, so that the decision is an informed one. Takeout, delivery, and catered trays Delivery food arrives with an unknown history. It may have spent ten minutes or an hour in transit, in a hot car or in an insulated bag. The safest assumption is that the clock started when the food left the restaurant, and that the time already spent counts against the two-hour limit. Food that will be saved for later should be refrigerated promptly after it arrives, not after the meal is over. Takeout containers themselves are rarely designed for storage or reheating. Many are fine for the refrigerator. Fewer are safe for the microwave; Chapter 4 explains why. Transferring leftovers into a container with a proper lid also makes them cool faster, if the container is shallow, and keeps them from drying out or absorbing refrigerator odors. Campus events pose a special version of the time problem. A club meeting with a catered tray of sandwiches, a study session with a spread of wraps, and a floor social with a platter of chicken wings all share a common pattern: the food arrives before the event, sits out while people trickle in, and is still there hours later when the last guests help themselves. Food at events is often handled appropriately by caterers and then left out far longer than intended by organizers who have many other things to think about. A sensible approach for attendees is to eat early rather than late, prefer foods that are either hot from a warming unit or cold from ice, and pass on the cream-based salads and meat platters that have obviously been sitting since the event began. For organizers, the fix is simple: put out part of the food and keep the rest cold until it is needed, and put a clear time on when the spread will be cleared. Cooling leftovers quickly Hot food cools through the danger zone on its way into the refrigerator, and how fast it passes through matters. A small bowl of soup cools quickly. A deep pot of chili can take many hours to cool in its center, because the outside of the mass insulates the inside. That slow middle is exactly the environment in which spore-forming bacteria such as Clostridium perfringens and Bacillus cereus thrive. The USDA's advice is to divide large quantities of hot food into small, shallow containers for refrigeration, so that they cool quickly and evenly. Food service rules are more specific: under the FDA Food Code, cooked food should be cooled from 135°F to 70°F (57°C to 21°C) within two hours and then to 41°F (5°C) or below within a further four hours. Students do not need to track those intervals with a thermometer, but the principle behind them is useful. The faster food moves from hot to cold, the less time bacteria spend at their preferred temperatures. In a dorm, three practical steps accomplish this. First, divide leftovers into portions no more than a couple of inches deep. Second, let them cool briefly on the counter, uncovered or loosely covered, until the steam stops rising, but not for longer than the two-hour window; many people wait too long out of a mistaken belief that putting hot food in the refrigerator is harmful. It is not harmful to the food, though a large amount of hot food can temporarily warm a small refrigerator, which is another reason to use small portions. Third, place the containers where cold air can reach them, not buried behind other items. Leftovers: how long and how to reheat Cooked leftovers keep safely in a refrigerator at 40°F (4°C) or below for three to four days, according to the USDA. After that, the risk of illness rises even if the food looks and smells fine, and the leftovers should be discarded or should have been frozen earlier. Frozen leftovers are safe indefinitely as long as they stay frozen at 0°F (−18°C), although their quality declines; the USDA suggests using them within three to four months for best quality. Counting days is easier with labels. A piece of masking tape with the date is enough. So is a simple routine: the leftovers from Sunday's cooking are eaten or frozen by Wednesday, and anything unlabeled whose history cannot be reconstructed goes in the trash. The cost of throwing out a container of uncertain leftovers is a few dollars. The cost of eating one that turned is potentially days of illness. When reheating leftovers, the USDA's target is an internal temperature of 165°F (74°C), measured with a food thermometer. Sauces, soups, and gravies should be brought to a rolling boil. Reheating does not make up for poor storage: it will kill most vegetative bacteria, but not heat-resistant toxins, and food that has been left out for many hours or stored too long should not be eaten no matter how thoroughly it is reheated. The point of reheating to 165°F is to handle the modest number of bacteria that may have grown during correct storage, not to rescue food that was mishandled. A common question is how many times leftovers can be reheated. The USDA's answer is that safely stored leftovers can be reheated more than once as long as they are handled correctly each time, but quality suffers, and each cycle adds another period in and out of the danger zone. The practical habit is to reheat only the portion you intend to eat, leaving the rest in the refrigerator. Thawing: the other direction Freezing is one of the most useful tools available to students with small refrigerators and irregular schedules. Portions of chili, soup, or cooked grains frozen on Sunday can be eaten over several weeks. But thawing is a trip back through the danger zone, and the way it is done matters. The USDA lists three safe methods. The first is thawing in the refrigerator, which is slow but keeps food at safe temperatures throughout. Small portions usually thaw overnight; larger items can take a day or more. Food thawed this way can be kept in the refrigerator for a period afterward and can even be refrozen, though quality may decline. The second is thawing in cold water, with the food sealed in a leak-proof bag and submerged in cold tap water that is changed every thirty minutes. This is faster, but the food should be cooked right away. The third is thawing in the microwave, which is fastest of all but tends to begin cooking the edges while the center is still frozen, so food thawed this way should also be cooked immediately. What the USDA advises against is thawing on the counter. The outer layers of a frozen chicken breast left out all afternoon can sit in the danger zone for hours while the center is still frozen. In a dorm, where counter space is often a desk next to a sunny window, this is an easy habit to fall into and an easy one to replace. Moving tomorrow's dinner from the freezer compartment to the main shelf of the refrigerator the night before takes seconds and avoids the problem entirely. A week planned around the clock The rules in this chapter can sound like a list of prohibitions. In practice, they are easier to follow as a rhythm, and a worked example shows how the pieces fit together. Consider a composite sophomore who cooks for herself on a limited budget, with a compact refrigerator, a small freezer compartment that her thermometer shows holds about 5°F (−15°C), and access to a floor kitchen. On Sunday afternoon she cooks a pot of black bean chili and a batch of roasted chicken thighs. When the chili finishes, she divides it into four shallow containers rather than leaving it in the pot, lets them stand until they stop steaming, and has them in the refrigerator within an hour. Two containers go into the freezer compartment instead, because she knows she will not eat all four by Wednesday and her freezer, while not reaching 0°F, keeps food frozen solid. The chicken is portioned the same way. Each container gets a strip of tape reading "Sun." Monday and Tuesday she eats chili and chicken, reheating one portion at a time and checking with a food thermometer until the center reads 165°F (74°C). Tuesday night she orders a pizza with friends. Before anyone starts eating, she puts the three slices she wants for lunch into a container and into the refrigerator; the rest of the box sits out for the evening and goes in the trash the next morning without debate. On Wednesday she moves one frozen chili container to the main compartment to thaw overnight. By Thursday, the last of Sunday's chicken is on its fourth day; she eats it at lunch and discards what she does not finish. On Friday she eats the thawed chili. The second frozen container waits for the following week. Nothing about this routine is elaborate. It depends on a few habits that each take seconds: dividing food as soon as it is cooked or delivered, labeling containers, checking the refrigerator thermometer when grabbing breakfast, moving food from freezer to refrigerator a day ahead, and throwing away anything that has outlasted its window. The result is a week of home-cooked food at low cost with very little risk. The alternative, one big pot left in the refrigerator and eaten from until it runs out or starts to smell, is how many students end up sick on a Friday night without any clear idea why. Dining hall food and meal plans Many students carry food out of the dining hall, whether in take-out containers the dining service provides or informally in a napkin. The same rules apply to that food as to any other. A sandwich or a container of pasta taken at lunch and left in a backpack until evening has almost certainly spent more than two hours in the danger zone. Food carried from the dining hall should either be eaten within two hours or put into the refrigerator promptly, and once refrigerated, it follows the three-to-four-day rule for leftovers. Some students on tight budgets rely on carrying dining hall food back to stretch their meal plans. That is a sensible strategy, and it works well with a little planning: an insulated lunch bag with a small ice pack in the backpack, a direct route back to the refrigerator, and a label with the date. Food insecurity is common among college students, and making the most of every meal is reasonable. Doing it safely just means treating the food as the perishable item it is. The clock, in short, is always running, and much of dorm food safety is simply the habit of noticing it. The next chapter turns to the appliance that does most of the cooking and reheating in dorm rooms, and to the reasons it heats food less evenly than most people expect. Chapter 4: Cooking Evenly in a Microwave For many students, the microwave is the kitchen. It reheats leftovers, cooks frozen meals, steams vegetables, makes oatmeal and popcorn and mug cakes, and on a busy night, it is the only appliance used at all. It is also the appliance whose weaknesses are least understood. People trust the microwave because it is fast and because the outside of the food gets hot, but a microwave's greatest limitation is precisely that the outside of the food and the inside can be at very different temperatures when the timer beeps. Cooking safely with one means understanding why that happens and correcting for it. How a microwave heats, and why it leaves cold spots A microwave oven generates electromagnetic waves, typically at a frequency of about 2.45 gigahertz, and bounces them around the inside of a metal box. Water molecules in food absorb that energy and heat up, and the heat then spreads through the food by ordinary conduction. Two features of this process explain almost all of the unevenness that people notice. The first is penetration. Microwaves heat only the outer layer of food directly, typically to a depth of a few centimeters depending on the food. The center of a thick item is heated by conduction from the outside in, which is slow. A thin layer of soup heats through quickly; a thick block of frozen lasagna does not. The second is pattern. The waves inside the oven reflect off the walls and interfere with each other, creating zones where energy is concentrated and zones where it is weak. That is why a plate of leftovers can come out with one edge bubbling and another barely warm, and why most microwaves have a rotating turntable: moving the food through the pattern evens out the heating, though it does not eliminate the problem. The center of the turntable, which moves least, is often a weak spot. Frozen food adds a third complication. Ice absorbs microwave energy much less readily than liquid water. When a frozen meal starts to thaw, the parts that melt first begin absorbing energy faster than the parts still frozen, and they heat faster still. The result can be boiling edges surrounding an icy center. This is why defrost settings cycle power on and off, giving heat time to spread, and why frozen meals often instruct users to stir halfway through. The USDA puts the conclusion bluntly: microwave ovens can cook unevenly and leave cold spots where harmful bacteria can survive. For reheating leftovers that were cooked safely and stored correctly, a cold spot is a moderate concern. For foods that start out raw or partially cooked, it can be the difference between a safe meal and a serious infection. The pot pie lesson In 2007, the CDC and state health departments investigated an outbreak of Salmonella infections that ultimately included 401 cases in 41 states, with 32 percent of ill people hospitalized. The investigation traced the illnesses to a brand of frozen pot pies. The pies were sold as not-ready-to-eat products, meaning they contained ingredients that had not been fully cooked and needed thorough cooking before eating. Many consumers did not understand that. The investigators found that 77 percent of the patients who ate the pies had cooked them in microwave ovens. Only 29 percent of those who used a home microwave knew its wattage, even though the package instructions gave different cooking times depending on wattage. And 68 percent had not let the pies stand for the full recommended time after cooking. The CDC's report concluded that consumer confusion about microwaving instructions might have led people to undercook the product, and it called for not-ready-to-eat frozen foods to be clearly labeled as requiring full cooking, with instructions validated to account for differences in microwave power. The outbreak is a useful case for students because every one of its ingredients is present in a typical dorm room: a frozen meal eaten late at night, a microwave of unknown wattage, instructions skimmed rather than followed, and a hungry person who does not wait for the standing time. The pies themselves were contaminated; the microwaves did not create the problem. But thorough, even cooking would have killed the bacteria, and that is the step that failed. Frozen foods in the United States are increasingly labeled to distinguish products that are fully cooked from those that are not. Phrases such as "cook thoroughly," "not ready to eat," or "for food safety, must be cooked to 165°F" signal that the product contains raw or partially cooked ingredients. Frozen chicken strips, nuggets, and stuffed entrées that look browned and cooked are a common source of confusion, since breading can make raw chicken look finished. If the label says to cook, it means cook, not warm. Know your microwave Microwave ovens differ in power, usually stated in watts on a label inside the door, on the back, or in the manual. Full-size household microwaves commonly range from about 1,000 to 1,200 watts. Compact models sold for dorm rooms and combination micro-fridge units are often lower. A meal whose package says four minutes for a 1,100-watt oven may need considerably longer in a 700-watt unit, and the package instructions usually say so. The first step, then, is to find the wattage and remember it. The second is to read the whole package instruction, including the parts about stirring, rotating, and standing, and to follow the time given for your wattage rather than the first number on the box. If your wattage is lower than any listed, start with the longest time given and check the temperature before eating. Technique: cover, stir, rotate, stand The USDA's guidance for microwave cooking reduces to a few techniques, each aimed at making the heat more even. Arrange food evenly in a covered dish. Thicker parts of food should point toward the outside of the dish, where heating is strongest, and thinner parts toward the center. Pieces of similar size cook more evenly than a mix of large and small. Adding a little liquid helps, since steam spreads heat. Cover the dish with a lid or microwave-safe plastic wrap, leaving a small gap to vent steam. Covering traps moisture and steam, which heats the food more evenly and helps kill bacteria. Plastic wrap should not touch the food. Stir or rotate the food partway through cooking, even when the microwave has a turntable. Stirring moves hot outer portions inward and cold inner portions outward. For items that cannot be stirred, such as a pot pie or a whole burrito, rotating the dish and turning the item over does something similar. Let the food stand for the time the instructions give after the microwave stops. Standing time is part of the cooking. During those minutes, heat continues to move from the hotter outer layers into the cooler center, and the temperature at the center often rises significantly. The pot pie investigators found that most patients skipped it. Using standing time well means leaving the food covered, in the microwave or on the counter, and checking its temperature at the end of the standing period rather than at the beep. The food thermometer The only reliable way to know whether food has reached a safe temperature is to measure it. A digital instant-read food thermometer costs little more than a takeout meal and will last for years. It is as important a piece of equipment as the microwave itself. In microwave cooking, the thermometer's job is to find the coldest part of the food. Insert it into the thickest part, away from any bone, and check several places, especially the center of the dish and the middle of the turntable. If any spot is below the target temperature, cover the food and continue cooking. Clean the thermometer probe with hot soapy water after each use, particularly after testing raw or partially cooked meat. Table 3 lists the USDA's safe minimum internal temperatures for the foods students most often cook. For whole cuts of beef, pork, veal, and lamb, the USDA also specifies a three-minute rest after cooking, during which the temperature holds or continues to rise. Table 3. Safe minimum internal temperatures for foods commonly cooked in dorms. Food Minimum temperature Notes Poultry, whole or ground, including nuggets and strips 165°F (74°C) Check the thickest part Ground beef, pork, or lamb, including burgers 160°F (71°C) Color is not a reliable guide Beef, pork, veal, or lamb steaks, chops, and roasts 145°F (63°C) Plus a three-minute rest Fish and shellfish 145°F (63°C) Flesh opaque and flakes easily Egg dishes 160°F (71°C) Eggs cooked until yolk and white are firm Leftovers and casseroles 165°F (74°C) Sauces and soups to a rolling boil Source: USDA Food Safety and Inspection Service; FoodSafety.gov safe minimum internal temperature chart. The thermometer also resolves arguments that cannot otherwise be settled. The color of ground beef, for instance, is not a reliable indicator of doneness; meat can turn brown before reaching a safe temperature, or stay pink after reaching it. The only way to know is to measure. Cooking raw food in a microwave Some students cook raw food in the microwave by choice or necessity: a chicken breast, a fillet of fish, scrambled eggs, a raw sausage. It can be done safely, but it is the use that most demands the techniques above, and it rewards a little planning. Fish is the most forgiving. A thin fillet placed in a covered dish with a splash of water or lemon juice, thicker end facing outward, cooks through in a few minutes on high power, and its doneness is easy to see: the flesh turns opaque and flakes with a fork. A thermometer reading of 145°F (63°C) at the thickest point confirms it. Eggs are nearly as simple. Beaten eggs cooked in a microwave-safe mug or bowl, stirred every thirty seconds or so, set quickly and evenly. Stirring is what makes the difference; left alone, the outside turns rubbery while the center stays liquid. Cooked until no visible liquid egg remains, they are safe. Chicken is the hardest, because it is thick, dense, and often contaminated. A boneless breast cooked whole in a microwave frequently reaches 165°F (74°C) at its edges long before its center. Several habits help. Cut the chicken into pieces of similar size, or pound a breast flat to an even thickness before cooking. Arrange pieces in a single layer around the edge of the dish, leaving the center empty. Cover and vent. Cook at medium power for longer rather than full power for a short time, which gives heat time to conduct inward. Turn the pieces over and rearrange them halfway through. Then check the thickest piece, not the thinnest, with the thermometer, and let everything stand covered for a few minutes before checking again. Juices running clear and meat that looks white are encouraging signs but not proof. A composite case shows how this goes wrong and how it goes right. A sophomore on a high-protein diet buys a family pack of raw chicken breasts and microwaves one each evening for four minutes on high, because that is what a video suggested. The first two nights, the chicken is dry at the edges and faintly pink near the middle, which she attributes to the brand. A food thermometer, borrowed from a floormate, reads 138°F (59°C) at the center of the third breast. After she switches to slicing the chicken into strips, cooking it covered at seventy percent power, turning it halfway, and checking three pieces, it reaches 170°F (77°C) throughout and is noticeably juicier. The change took a knife, a thermometer, and two extra minutes. Partial cooking is a separate trap. The USDA advises against partially cooking meat or poultry in the microwave and finishing it later, because the partly heated food may sit in exactly the temperature range where bacteria grow fastest. If microwaving is used to start food that will be finished on a grill or in a pan, the food should go directly from one appliance to the other without a break. Containers and burns Not every container belongs in a microwave. The USDA advises using only cookware labeled as microwave safe. Cold-storage containers such as margarine tubs, yogurt cups, and many takeout containers are not designed for heating; they can warp or melt, and chemicals in the plastic may migrate into food when it is heated. Plastic storage bags, grocery bags, brown paper bags, newspaper, and aluminum foil should not be used in the microwave. Metal can cause sparking, known as arcing, which can damage the oven and start a fire. Glass and ceramic dishes labeled microwave safe are the most dependable choices, and a few of them, along with a microwave-safe cover, are a worthwhile investment for a dorm room. Microwaves also cause burns in ways that are easy to overlook. Steam builds up under covers and inside sealed pouches and escapes with force when they are opened; peel covers away from your face and hands. The outer container may stay cool while the food inside is scalding, or the reverse. Popcorn bags release steam hot enough to burn when opened. Whole eggs in their shells, and sometimes even hard-cooked eggs reheated whole, can burst because steam builds inside them; eggs should be removed from the shell and the yolk pierced before microwaving. Water deserves a special note. The FDA has warned that water heated in a microwave can become superheated, rising above its normal boiling point without visibly boiling, particularly in a clean, smooth cup. When the cup is moved or something such as instant coffee is added, the water can erupt violently. Heating water only for the time recommended, not reheating it repeatedly, and placing a microwave-safe non-metal object such as a wooden stir stick in the cup can reduce the risk, as can waiting a moment before removing it. What a microwave does well It would be wrong to leave the impression that microwaves are poor cooking tools. Used with some care, they are excellent at many of the things students most need. They steam vegetables quickly while preserving nutrients. They cook oatmeal, eggs, grains, and potatoes. They reheat soups and stews very well, since liquids can be stirred and heat evenly. And they are far safer from a fire standpoint than a hot plate or an open flame. The adjustments this chapter describes, covering, stirring, standing, and measuring, add perhaps a minute to most meals. They are the difference between using a microwave as a heater and using it as a cooking appliance, and they matter most for the foods where undercooking carries real risk: raw poultry, frozen not-ready-to-eat meals, and anything reheated from a large batch. The next chapter turns to one of those batch foods, which carries a risk that no amount of reheating can fix. Chapter 5: Rice, Pasta, and the Batch-Cooking Trap Rice is one of the best foods a student can cook. It is cheap, filling, stores well dry, pairs with almost anything, and can be made in a microwave, a rice cooker, or a pot on a floor-kitchen stove. Pasta shares most of those virtues. Both are staples of the batch-cooking approach that makes eating well on a student budget possible: cook once on Sunday, portion into containers, and eat through the week. Yet cooked rice and pasta carry a specific food safety risk that is poorly understood, cannot be fixed by reheating, and happens to align almost perfectly with the way many students store them. The organism responsible is Bacillus cereus, and understanding it changes how batch cooking should be done. A bacterium that survives the pot Bacillus cereus lives in soil and is common on rice and other grains as they come from the field. Washing rice does not remove it reliably, and there is no practical way for consumers to buy rice free of it. Most of the time this does not matter, because the numbers of bacteria on raw rice are low and cooking kills the active cells. What cooking does not kill are the spores. B. cereus forms spores, dormant structures with tough coats that survive boiling. When rice is cooked, the spores come through intact. Then, as the rice cools, the conditions become ideal for them: warm, moist, and full of starch. The spores germinate into active bacteria, and those bacteria multiply. The University of Florida's extension service describes B. cereus as growing between about 39°F and 118°F (4°C and 48°C), with its fastest growth between about 82°F and 95°F (28°C and 35°C), which is to say at the temperatures of a warm room or of a pot of rice slowly cooling on a stove. As the population grows, some strains produce a toxin called cereulide. This is the crucial point. Cereulide is extremely heat stable; the same extension publication notes that it withstands temperatures up to 250°F (121°C). No microwave, stovetop, or frying pan will destroy it. Rice that has accumulated toxin during a long, slow cool or a day on the counter will still make people sick after being reheated until it steams, and it will look, smell, and taste normal. The illness this toxin causes is the emetic form of B. cereus food poisoning. It typically begins between about thirty minutes and six hours after eating, with nausea, vomiting, and abdominal cramps, and usually resolves within a day. A second form, the diarrheal type, results from toxins produced after the bacteria are swallowed and usually appears six to fifteen hours after eating, more often associated with meat and vegetable dishes. Both are usually unpleasant rather than dangerous for healthy adults. The emetic form in particular has earned the nickname "fried rice syndrome," because many early outbreaks traced back to restaurants that cooked large batches of rice, left them at room temperature, and fried them to order. When it goes further For the vast majority of people, B. cereus illness means a miserable few hours and nothing more. But the toxin can, rarely, cause serious harm, and two published case reports from Belgium show what that looks like. In 2005, researchers described a family outbreak in which several children became ill after eating a pasta salad, and one child died of liver failure. The authors concluded that the case demonstrated the possible severity of the emetic syndrome. In 2011, a second report described the death of a 20-year-old man in Brussels after he ate a spaghetti meal contaminated with B. cereus. Investigators isolated emetic strains of the bacterium and found high levels of cereulide in the food. Press coverage of that case, which circulated widely on social media years later, emphasized that the pasta had been left out for days before being reheated. These cases are rare and should not be read as evidence that leftover rice is deadly. Millions of people eat leftover rice every day without harm. They are useful because they show that the mechanism is real: toxin produced in food that was stored warm can survive reheating and, at high enough levels, do serious damage. The practical response is not to stop eating leftovers but to store starchy foods in a way that never gives the toxin a chance to accumulate. Cooling rice and pasta the right way Because the problem arises while cooked rice sits warm, the solution is to minimize that time. The general two-hour rule applies, but with rice it is worth doing better than the minimum. Cooked rice that will be saved should be cooled quickly and refrigerated promptly, ideally within an hour of cooking. Spreading rice in a thin layer is the single most effective step. A pot of rice holds heat for a long time, and its center can stay warm for hours. The same rice spread in a shallow container, a layer an inch or two deep, loses heat quickly. Dividing it into several meal-sized portions does the same thing and makes later reheating easier. Rice can go into the refrigerator while still warm; there is no need to wait until it reaches room temperature, and waiting is counterproductive. Once refrigerated at 40°F (4°C) or below, cooked rice follows the general rule for leftovers: eat it within three to four days, or freeze it. Some public health agencies outside the United States recommend a shorter window for rice specifically. The United Kingdom's National Health Service, for example, advises eating leftover rice within one day. That stricter advice reflects how rarely consumers cool rice as quickly as they should. Students who cool rice properly and keep a cold refrigerator can reasonably use the three-to-four-day window; those who are less sure about either condition have good reason to use leftover rice sooner. When reheating rice, heat it until it is steaming hot throughout, stir partway through, and heat only the portion you will eat. Reheating rice more than once adds another cycle of warming and cooling without any benefit. The same principles apply to cooked pasta, couscous, quinoa, and other grains, as well as to cooked potatoes and dishes built from them. Fried rice made from day-old rice is a special case worth addressing because it is so common, and because good fried rice genuinely depends on cold, slightly dried rice. It is perfectly safe to make fried rice from rice that was cooled quickly and refrigerated overnight. The risk comes from rice left out on the counter overnight, which is exactly how some home cooks and restaurants used to prepare it. Rice cookers and the "keep warm" setting Many students use rice cookers, and many campuses permit them as appliances with automatic shut-off. A rice cooker's "keep warm" function holds cooked rice at an elevated temperature, and as long as it keeps the rice hot, above the range in which B. cereus grows, the rice is protected. Food service regulations define hot holding as keeping food at 135°F (57°C) or above. The trouble is that the keep-warm setting works only if it is used as intended. If the lid is left open, if the cooker is unplugged to use the outlet for something else, or if a small amount of rice sits in a large cooker for many hours, parts of the rice can drop into the growth range without anyone noticing. Rice that has spent many hours on keep-warm also dries out and develops off flavors. The simplest approach is to treat keep-warm as a way to hold rice through a meal or an evening, not as storage. Rice that will not be eaten within a few hours should come out of the cooker, be spread thin, and go into the refrigerator. Big pots and Clostridium perfringens Rice is not the only batch food with a spore problem. Clostridium perfringens, another spore-forming bacterium, is among the most common causes of foodborne illness in the United States; the CDC's 2025 estimates ranked it fourth among the seven major pathogens studied. It is associated above all with large quantities of cooked meat, poultry, gravy, beans, stews, and chili, the very dishes students make in quantity to eat through the week. The mechanism is similar to that of B. cereus. Spores survive cooking. As a large pot cools slowly, its center spends hours in the temperature range where the bacteria multiply quickly. When the food is eaten, the bacteria produce a toxin in the intestine that causes cramps and diarrhea, usually six to twenty-four hours later. Outbreaks are often linked to catered events and institutional kitchens, where large volumes of food are cooked ahead of time and cooled or held improperly. For students, the lesson is the same as for rice. Large quantities of hot food cool slowly and should be divided into shallow containers before refrigeration. A pot of chili left on the stove overnight to cool, then refrigerated in the morning, has given C. perfringens the conditions it needs. The same chili divided into four shallow containers and refrigerated within an hour or two has not. Meal prep without the risk Meal prep has become a popular approach to eating on a budget and on a schedule, and much of the advice circulating online is sound: cook in quantity, portion into containers, and eat a planned meal each day. One common pattern, however, runs into trouble. Recipes and videos often suggest preparing five or even seven days of meals at once and storing them all in the refrigerator. That exceeds the USDA's three-to-four-day guidance for cooked leftovers, and meals built on rice or pasta are exactly the ones where the later days carry the most risk. The fix is to split the week in two. Meals intended for the first three or four days go into the refrigerator. Meals for the remainder of the week go into the freezer immediately, while they are fresh, and move to the refrigerator the night before they will be eaten. Many meal-prep containers are designed for both refrigerator and freezer, and cooked rice, grains, and most stews freeze and reheat well. Students whose compact refrigerator has a weak freezer compartment face a harder version of this problem. If the thermometer shows the freezer compartment running well above 0°F (−18°C) but still holding food frozen solid, it can serve for a few days of storage. If it does not keep food frozen, the practical solution is either to cook twice a week rather than once or to arrange space in a shared freezer if the residence hall has one. A composite case illustrates the difference. Two roommates cook the same meal-prep recipe, chicken, broccoli, and rice in five containers, every Sunday. One refrigerates all five and eats them through Friday, reheating each for ninety seconds. The other spreads the rice on a tray to cool before portioning, puts three containers in the refrigerator and two in the freezer, and moves the frozen ones to the refrigerator on Wednesday night. For most weeks, both roommates stay well. But the first roommate is exposed, every week, to rice that has spent five days in a refrigerator that runs a few degrees warm after grocery trips, reheated unevenly in a low-wattage microwave. Over a semester, the difference in risk adds up. The second roommate's routine costs perhaps ten extra minutes a week. Why the advice sounds contradictory Students who look into rice safety quickly run into advice that seems to conflict. One agency says three to four days; another says one. A viral video warns that leftover rice can kill; a parent points out that the family left rice in the cooker overnight for decades without a problem. A food scientist explains that good fried rice requires day-old rice; a food safety poster says to throw out rice after two hours. It is worth sorting these out, because confusion tends to push people toward ignoring the advice altogether. The first thing to notice is that most of the apparent conflict comes from different assumptions about how the rice was handled before it reached the refrigerator. Guidance that allows three to four days assumes rapid cooling and a refrigerator at 40°F (4°C) or below. Guidance that allows only one day is hedging against the reality that many people cool rice slowly and keep their refrigerators too warm. Both are internally consistent. The stricter advice simply builds in more margin for common mistakes. The second thing to notice is that the family experience of eating rice left out overnight is real evidence, but of a limited kind. It shows that the risk from any single occurrence is modest. It does not show that the risk is zero, and it cannot show what happened on the occasions when someone felt ill without connecting it to the rice. B. cereus illness is common, usually brief, and often blamed on something else. A family could experience several episodes over the years without ever suspecting the rice cooker. The third thing to notice is that the viral warnings, while grounded in real cases, lose the context of how rare those cases are and how extreme the handling usually was. The published fatal cases involved food that had been stored badly for a long time. They are reasons to take storage seriously, not reasons to fear a container of rice cooked yesterday and refrigerated promptly. Put together, the evidence supports a clear position. Cooked rice is safe when it is cooled quickly, refrigerated at 40°F (4°C) or below, eaten within a few days, and reheated until steaming. Cooked rice left warm for hours is a gamble whose odds are usually good and occasionally very bad, and no amount of reheating improves them. Students can decide how much margin they want within that frame, but the frame itself is not in dispute. Other starches and cooked foods The same logic extends beyond rice and pasta. Cooked potatoes, including baked potatoes wrapped in foil and left at room temperature, can support the growth of Clostridium botulinum, a rare but serious hazard, because foil creates a low-oxygen environment. Potatoes baked in foil should be kept hot or refrigerated promptly after the foil is removed. Cooked beans, oatmeal, and other grains all fall under the general leftover rules. Garlic or herbs stored in oil at room temperature can also support botulism and should be kept refrigerated and used within a few days, or bought as commercially prepared products formulated for safety. None of these foods is dangerous in itself. Each becomes a problem only when it sits warm for long periods, the same condition that runs through every chapter so far. Starches, beans, and big batches of meat simply give spore-forming bacteria a more hospitable place to grow than most foods do, and they reward faster cooling more than most. The habits that manage them, cooling quickly in shallow containers, refrigerating promptly, and freezing what will not be eaten within a few days, are the same habits that make batch cooking a sound strategy rather than a gamble. The next chapter leaves the individual dorm room for the shared kitchen, where the challenge is less about time and temperature than about other people's hands, other people's sponges, and other people's food. Chapter 6: Shared Kitchens and Other People's Hands The floor kitchen is one of the stranger rooms in university life. It usually contains a stove, sometimes an oven, a sink, a microwave, a counter of uncertain cleanliness, and a refrigerator shared by dozens of people, few of whom know each other well. It is used at every hour of the day by people with very different cooking skills and very different standards. No one owns it, which in practice often means no one cleans it. Housekeeping staff may wipe surfaces on a schedule, but they are not there when someone cuts raw chicken on the counter at 9 p.m. and leaves the juices behind. The problems in a shared kitchen are mostly problems of the clean and separate steps: getting harmful organisms off hands and surfaces, and keeping raw foods away from foods that will not be cooked. Neither step is difficult. What makes them harder in a shared space is that your own care is only part of the picture. Every surface you touch has also been touched by strangers, and every item you leave behind becomes part of someone else's environment. Good practice in a shared kitchen therefore has two aims: protecting yourself from what others leave, and not leaving anything for others. What the research shows about kitchens The most instructive evidence on how contamination spreads in kitchens comes from observational studies, in which researchers watch people cook. In 2018, the USDA released results from one such study, conducted with RTI International and North Carolina State University. Participants cooked in test kitchens while researchers observed them, and the meat they handled had been inoculated with a harmless tracer organism, so that its spread could be measured. The results were sobering. The USDA reported that participants failed to wash their hands correctly 97 percent of the time they should have, most often by not scrubbing for the recommended twenty seconds or by not drying their hands with a clean towel. The tracer organism from raw poultry turned up on spice containers in 48 percent of cases, on refrigerator handles in 11 percent, and in 5 percent of salads, which had never touched raw meat directly but were contaminated through hands and surfaces. Only 34 percent of participants used a food thermometer to check whether their burgers were done, and nearly half of those who did still did not cook them to a safe temperature. The study was not conducted with students, and its test kitchens were cleaner than most floor kitchens. But it shows how contamination actually moves. It rarely travels in dramatic ways. It travels on a hand that touched a raw chicken breast and then opened the salt shaker, on a towel used to wipe a board and then to dry a clean plate, on the handle of a refrigerator that everyone uses. In a shared kitchen, each of those paths also runs through other people. Hands first Handwashing is the cheapest and most effective food safety step, and the most often done badly. The CDC's guidance is to wet hands with clean running water, apply soap, lather all surfaces including the backs of the hands, between the fingers, and under the nails, scrub for at least twenty seconds, rinse, and dry with a clean towel or air dryer. Twenty seconds feels long the first few times; humming the "Happy Birthday" song twice is the standard way to measure it. When to wash matters as much as how. In the kitchen, the key moments are before starting to prepare food, after touching raw meat, poultry, seafood, or eggs, after touching the garbage, after using the bathroom, after blowing your nose or coughing into your hands, and before eating. In a shared kitchen, add one more: after touching common surfaces such as the refrigerator handle and the faucet, if you are about to handle food that will not be cooked. Hand sanitizer is useful when soap and water are not available, but it is not a substitute in the kitchen. It does not remove grease or food residue, and alcohol-based sanitizers are not reliably effective against norovirus, the leading cause of foodborne illness in the United States. Chapter 7 adds another reason: sanitizer does not remove food allergens from hands, while soap and water do. Drying deserves more attention than it gets. A shared cloth towel hanging on the oven door of a floor kitchen has probably dried dozens of hands and wiped several counters since it was last laundered. Paper towels, or your own towel brought and taken away, avoid the problem. Sponges, boards, and counters The kitchen sponge is a remarkably effective device for collecting and spreading bacteria. It is porous, stays damp, and is regularly loaded with food residue, which makes it an excellent habitat. A 2017 study that analyzed used kitchen sponges with genetic sequencing and microscopy found bacterial densities in parts of the sponges reaching tens of billions of cells per cubic centimeter, with dense colonization by several groups of bacteria, some related to species that can cause infections. The researchers also found that sponges their owners reported regularly sanitizing did not simply become clean; their bacterial communities shifted in ways that the authors said called such sanitation methods into question over the long term. For a student, the practical conclusions are straightforward. Do not use the communal sponge in a floor kitchen. Bring your own sponge or dish brush, let it dry between uses, keep it out of the shared space, and replace it frequently. Sponges that are microwaved damp or run through a dishwasher's drying cycle carry fewer bacteria in the short term, but frequent replacement is the more reliable approach. Use paper towels, not a sponge, to wipe up juices from raw meat, and throw them away immediately. Cutting boards come next. Raw meat, poultry, and seafood should be cut on a different board from produce, bread, and other foods that will not be cooked. In a dorm, the easiest approach is to own two inexpensive boards in different colors and never mix them up. Plastic boards can go through a dishwasher, if one is available; otherwise, wash them in hot soapy water after each use. Boards with deep grooves are hard to clean and should be replaced. Counters in a shared kitchen should be treated as dirty until you have cleaned them yourself. Wipe the area you will use with hot soapy water before starting, or with a disinfecting wipe if one is available, and do the same when you finish. Place a clean cutting board on the counter and prepare food on the board rather than directly on the surface. It takes a minute and removes the question of who used the counter last and for what. Separating raw and ready-to-eat The separate step is about the paths that contamination takes from raw foods to foods that will not be cooked. In a shared kitchen, those paths multiply. The key habits are simple to list and easy to forget in the middle of cooking. Keep raw meat, poultry, and seafood in sealed containers or bags, on the bottom shelf of the refrigerator, where they cannot drip onto anything else. Use one set of utensils and one board for raw foods and a different set for everything else. Never put cooked food back on a plate that held raw meat without washing it first. Do not rinse raw chicken in the sink; the USDA advises against washing poultry because splashing spreads bacteria around the sink and nearby surfaces, and thorough cooking kills the bacteria anyway. In a shared kitchen, the next person to use that sink may be rinsing lettuce. Seasoning is a less obvious route. The USDA study found the tracer organism on spice containers in almost half of cases, because people handle them in the middle of preparing raw meat. Measuring out spices before touching raw meat, or washing hands before reaching for the salt, closes that route. The shared refrigerator Communal refrigerators have a particular set of problems. They are opened constantly, which makes them harder to keep cold. They are overfilled at the beginning of the week and clogged with forgotten containers by the end. Items are moved around by other users, so raw meat may end up above someone else's salad. And few people check the temperature or know whom to tell if it is warm. A few habits make a shared refrigerator work better. Label everything with your name and the date. Store raw meat in a sealed container on the lowest available shelf, even if that means asking someone to move their drinks. Take home anything you will not use soon; a shared refrigerator is a poor place for long-term storage. If you have your own appliance thermometer, leave one in the shared refrigerator or check it occasionally, and report a warm refrigerator to residence life or facilities rather than assuming someone else will. Some floors organize a regular clean-out, often weekly, in which unlabeled or expired items are thrown away. If yours does not, suggesting one to a resident assistant is a small act of public health. A refrigerator that is cleared regularly stays colder, smells better, and does not accumulate the forgotten containers of leftovers whose mold spreads spores across everything nearby. Norovirus and the sick roommate Norovirus deserves particular attention in residence halls because it spreads so efficiently in shared spaces. It can be passed through contaminated food, through contaminated surfaces, and directly from person to person. Very small amounts of virus can cause infection, and people who are sick shed large quantities of it in vomit and stool. The virus can remain infectious on surfaces for a long time, and it is resistant to many common cleaners and to alcohol-based hand sanitizer. The CDC's guidance for preventing norovirus spread applies directly to shared living. People who are sick with norovirus should not prepare food for others while they are sick and for at least two days after symptoms stop. They should wash their hands carefully with soap and water. Surfaces contaminated with vomit or diarrhea should be cleaned and then disinfected with a chlorine bleach solution or another disinfectant registered by the Environmental Protection Agency as effective against norovirus; the CDC recommends a bleach concentration of 1,000 to 5,000 parts per million, which corresponds to about 5 to 25 tablespoons of household bleach per gallon of water. Laundry soiled with vomit or diarrhea should be handled carefully, washed on the longest cycle, and machine dried. In practice, this means a student with a sudden stomach illness should stay out of the floor kitchen, avoid sharing food, and use a separate bathroom if one is available. Many campuses provide guidance for residents with contagious illnesses, and some offer temporary isolation spaces during outbreaks. Roommates and floormates of someone who is sick should be especially careful about handwashing and about the surfaces they share, particularly bathroom fixtures, doorknobs, and the microwave handle. A composite case shows how quickly norovirus can move through a shared space. A student comes back from a weekend trip feeling fine, then begins vomiting late on Sunday night in a shared bathroom. By Tuesday, she feels better and bakes cookies for a floor study session. By Thursday, a dozen people on the floor are ill. No one did anything unusual. The virus traveled on bathroom fixtures and on hands that shaped cookie dough, and the timing, symptoms subsiding just before the cookies were made, fits exactly the window the CDC warns about. Cooking for a crowd Sooner or later, most students cook for more people than themselves: a floor potluck, a club fundraiser, a birthday dinner for friends, a holiday meal for students who cannot go home. Cooking for a group raises the stakes of every habit in this chapter, because one mistake now reaches many people, and because large quantities behave differently from small ones. The first consideration is who is cooking. A person who has been sick with vomiting or diarrhea in the previous two days should not prepare food for others, however well they feel. That is not a judgment about their hygiene; it reflects how long norovirus continues to be shed after symptoms end. Anyone with a cut or sore on their hands should cover it and wear a glove. Hands should be washed at the start, after handling raw ingredients, and whenever they have touched a phone, a face, or a bin. The second is quantity. Large pots cool slowly, as Chapter 5 described, so dishes made ahead for an event should be divided into shallow containers and refrigerated, then reheated to 165°F (74°C) shortly before serving. Hot dishes can be held in slow cookers or warming trays above 140°F (60°C); cold dishes can sit on ice. Anything that has neither heat nor ice should follow the two-hour rule, which in practice means putting out modest amounts and replenishing from the refrigerator. The third is labeling. At a potluck, nobody knows what is in anyone else's dish. A folded card next to each dish naming it and listing any of the major allergens it contains, especially peanuts, tree nuts, sesame, shellfish, milk, egg, and wheat, lets guests with allergies make informed decisions and signals that the host has thought about them. Separate serving utensils for each dish prevent a spoon from carrying one dish into another. The fourth is the end of the event. Leftovers from a gathering have often spent the better part of two hours at room temperature. Food that has been out longer should be thrown away rather than divided among guests, however wasteful that feels. Food that went out late and came back quickly can be refrigerated and eaten within the usual three to four days. None of this makes group cooking complicated. It makes it deliberate, and most experienced hosts find the deliberate version less stressful because nothing depends on luck. Shared cooking as a social skill It is easy to present shared kitchen hygiene as a matter of individual discipline, but in practice it is also a social skill. Asking a floormate not to cut chicken on the counter without a board, pointing out that the shared refrigerator smells, or suggesting that a sick friend skip the potluck all require some tact. So does admitting that you left a pan soaking for two days. Students who handle these conversations well tend to focus on the shared goal rather than on blame, and to offer a practical fix. A note on the refrigerator suggesting a weekly clean-out on Sunday evening, signed by several residents, works better than an anonymous complaint. Leaving a roll of paper towels and a spray bottle of cleaner in the kitchen makes good habits easier for everyone. Resident assistants are usually glad to help organize this kind of arrangement, and many campuses provide cleaning supplies for floor kitchens on request. The shared kitchen will never be as clean as a well-kept home kitchen, and it does not need to be. What it needs is for each user to leave it no worse than they found it, and to protect their own food from what others might have left. The next chapter takes up a form of contamination that ordinary cleaning handles differently, and for which the stakes can be much higher: food allergens. Chapter 7: Allergens and Cross-Contact Everything discussed so far concerns organisms: bacteria and viruses that can be killed by heat, slowed by cold, and washed away by soap. Food allergens are a different kind of hazard. They are proteins, part of the food itself, and they are not destroyed by cooking in any reliable way. A peanut butter cookie baked at 350°F (177°C) is just as dangerous to someone with a peanut allergy as a raw peanut. A pan that cooked shrimp and was wiped with a paper towel may carry enough shellfish protein to cause a reaction. The tools of food safety that work against microbes, including heat, time limits, and sanitizer, do little against allergens. What works is separation and thorough cleaning, and in a shared living environment, both depend on communication. How common food allergy is among adults Food allergy is often thought of as a childhood condition, and many children do outgrow allergies to milk and egg. But food allergy in adults is far more common than was once assumed. A national survey of more than 40,000 U.S. adults, published in JAMA Network Open in 2019 by Ruchi Gupta and colleagues, estimated that about 10.8 percent of adults had a convincing food allergy, meaning symptoms consistent with a true allergic reaction, which corresponds to more than 26 million people. Nearly half of the adults with food allergies reported that at least one of their allergies had begun in adulthood. Shellfish was the most common adult food allergy, followed by milk, peanut, tree nuts, and fin fish. The same study found gaps in how adults manage their allergies. Only about half of those with convincing food allergy symptoms had a physician-confirmed diagnosis, and fewer than a quarter reported a current prescription for epinephrine, the first-line treatment for severe reactions. Meanwhile, nearly 20 percent of adults believed they had a food allergy, which is almost twice the proportion whose symptoms were consistent with one. Some of that difference reflects food intolerances, such as lactose intolerance, which cause real discomfort but do not involve the immune system and do not cause anaphylaxis. For residence halls, these numbers mean that on a typical floor of forty students, several are likely to have a real food allergy, some may not yet know it, and some who do know it may not carry epinephrine. Allergy specialists have also long regarded adolescents and young adults as a group at elevated risk of fatal food-induced reactions, partly because they eat away from home more often, are less likely to carry epinephrine, and may be reluctant to ask about ingredients in social settings. The transition to college is precisely when many of those risk factors converge. The nine major allergens and how labels work U.S. law recognizes nine major food allergens. The Food Allergen Labeling and Consumer Protection Act of 2004 established eight: milk, eggs, fish, crustacean shellfish, tree nuts, peanuts, wheat, and soybeans. The Food Allergy Safety, Treatment, Education, and Research (FASTER) Act of 2021 added sesame as the ninth, effective January 1, 2023. Packaged foods regulated by the FDA must declare these allergens in plain language, either in the ingredient list or in a "Contains" statement. Two limitations of labeling matter for students. First, precautionary statements such as "may contain peanuts" or "processed in a facility that also processes tree nuts" are voluntary and not defined by regulation. Their presence or absence does not reliably indicate the level of risk, and products without such statements may still carry traces from shared equipment. People with severe allergies often follow their allergist's guidance on which of these products to avoid. Second, labeling rules apply to packaged foods, not to food served in dining halls, from food trucks, or at campus events, and not to dishes that other students make in shared kitchens. In those settings, the only way to know what is in food is to ask. After sesame was added to the list, some manufacturers responded by adding sesame deliberately to products and labeling it, rather than undertaking the cleaning needed to guarantee sesame-free production. The practical lesson is that labels change, and people with allergies need to read them every time, even on products they have bought before. Cross-contact is not cross-contamination Food safety professionals use two different terms for two different problems. Cross-contamination refers to the transfer of harmful microorganisms from one food or surface to another. Cross-contact refers to the transfer of an allergen from one food to another, so that a food that should not contain the allergen now does. The distinction matters because the remedies differ. Cooking a cross-contaminated food to a safe temperature kills the bacteria; cooking a food that has undergone cross-contact does not remove the allergen. In a shared kitchen, cross-contact happens in ordinary ways. A knife used to spread peanut butter is wiped and then used to spread jam for someone with a peanut allergy. A toaster used for wheat bread is used for gluten-free bread. A frying pan that cooked shrimp is rinsed and used for eggs. A wooden spoon absorbs traces of whatever it has stirred. Oil used to fry breaded food carries wheat and possibly other allergens into whatever is fried next. A cutting board used to chop walnuts is brushed off and used to slice fruit for a roommate with a tree nut allergy. The amounts involved in cross-contact can be very small, and the threshold at which people react varies widely. Some people with food allergies react only to substantial amounts; others react to traces. People with the most sensitive allergies usually know it and plan accordingly, but cross-contact can still catch them off guard in a shared space where others do not understand the risk. Cleaning that actually removes allergens A study published in 2004 in the Journal of Allergy and Clinical Immunology tested how well common cleaning methods removed a major peanut allergen from hands and tabletops. After handwashing with liquid soap, bar soap, or commercial wipes, the allergen was undetectable. Plain water left detectable allergen on three of twelve hands, and antibacterial hand sanitizer left it on six of twelve. Most household cleaning agents removed the allergen from tabletops, although dishwashing liquid alone left detectable allergen on four of twelve tables. The study also offered reassurance. Peanut allergen was rarely found on surfaces in schools, and airborne peanut allergen was not detected when people ate peanut butter, shelled peanuts, or unshelled peanuts in simulated real-life settings. The researchers concluded that the major peanut allergen is relatively easy to clean from hands and surfaces with common agents. For students, the implications are practical. Wash hands with soap and water, not sanitizer, after handling allergenic foods and before handling food for someone with an allergy. Clean counters and tables with a household cleaner or a wipe, not a quick rinse. Wash utensils, boards, and pans thoroughly in hot soapy water, with scrubbing, before using them for allergen-free cooking, or better, keep separate equipment. A dedicated cutting board, knife, and pan, clearly marked, removes most of the uncertainty. Toasters are nearly impossible to clean fully; people with wheat allergy or celiac disease often keep their own or use toaster bags. It is worth noting a distinction here. Celiac disease is an autoimmune condition triggered by gluten, not a food allergy in the immunological sense, and it does not cause anaphylaxis. But people with celiac disease also need to avoid even small amounts of gluten, and the same principles of separate equipment and careful cleaning apply to them. Living with a roommate who has an allergy When a roommate or floormate has a serious food allergy, the conversation about it is best held early and plainly. The person with the allergy is the expert on their own condition: which foods they react to, how sensitive they are, what their reactions look like, whether they carry epinephrine and where they keep it, and what they would like others to do. Some students with allergies ask roommates not to keep the allergen in the room at all. Others are comfortable with it as long as it is sealed and handled carefully. Neither request is unreasonable, and the right arrangement depends on the individual. Roommates without allergies can help in concrete ways. Wash hands after eating the allergen, especially before touching shared surfaces. Keep allergenic foods sealed and in a designated area of the refrigerator or shelf. Do not share utensils or dishes without washing them thoroughly. Learn the signs of a severe reaction and where the epinephrine is kept. And perhaps most importantly, take the allergy seriously without making it a burden: a roommate who treats precautions as a normal part of living together makes it easier for the person with the allergy to speak up when something is wrong. Students with allergies also have options through their institutions. Many campuses have processes through which students with food allergies or celiac disease can request housing or dining accommodations, typically through a disability services or accessibility office. In 2012, the U.S. Department of Justice reached a settlement with Lesley University in Massachusetts under the Americans with Disabilities Act, which required the university to make reasonable modifications to its meal plan for students with celiac disease and food allergies. Since then, many colleges have formalized their approaches. Students who need accommodations are best served by registering with the relevant office before arrival, with documentation from their clinician. Dining halls, parties, and dates Much of a student's eating happens away from their own room, and each setting presents its own version of the allergen problem. Dining halls are, in some ways, the easiest. Many campus dining services post ingredient and allergen information at each station or online, train staff to answer questions, and offer stations designed to exclude the most common allergens. Those systems are only as good as their execution, though, and self-serve stations create their own cross-contact: a serving spoon moved from the stir-fry with peanuts to the plain rice, crumbs from the bread station on the salad bar, a shared fryer used for breaded chicken and French fries. Students with serious allergies tend to do best when they introduce themselves to the dining hall's manager or dietitian early in the year, ask how food for them can be prepared separately, and treat posted labels as a starting point rather than a guarantee. Parties and social events are harder. Food at parties is often unlabeled, homemade, or dropped off by someone who has left. Lighting is dim, attention is divided, and asking about ingredients can feel awkward. Alcohol adds two problems. It can impair a person's ability to recognize the early signs of a reaction and the judgment of those around them, and allergy specialists recognize alcohol, along with exercise and certain pain relievers, as a cofactor that can make some allergic reactions more severe. A student with a serious allergy is safer bringing their own food to parties, eating beforehand, and letting at least one friend know about the allergy and where the epinephrine is. Dating adds a form of exposure that many people never consider. Allergens can be transferred through saliva after the other person has eaten the food. A 2006 study in the Journal of Allergy and Clinical Immunology measured peanut allergen in saliva after people ate peanut butter and found that simple steps such as brushing teeth or rinsing the mouth did not reliably remove it immediately, while waiting several hours and eating a peanut-free meal were more effective. The practical implication is awkward but important: people with severe allergies need partners who know about the allergy and understand that eating the food shortly before kissing is a real risk. It is a conversation worth having early. None of these settings requires a person with an allergy to withdraw from social life. What they require is a little planning and a willingness to explain the allergy to the people who matter. Those around them can help by making the conversation easy: asking before offering food, labeling dishes, and responding to a disclosure with interest rather than a joke. Recognizing and responding to a severe reaction Anaphylaxis is a severe, potentially life-threatening allergic reaction that can affect several body systems at once. Signs can include hives or widespread flushing; swelling of the lips, tongue, or throat; difficulty breathing, wheezing, or a persistent cough; a hoarse voice or trouble swallowing; vomiting, cramps, or diarrhea; dizziness, fainting, or a sense of impending doom; and pale or bluish skin. Symptoms usually begin within minutes to two hours after exposure. Reactions can progress rapidly, and a reaction that begins mildly can become severe. Epinephrine is the first-line treatment for anaphylaxis, and it works best when given early. Antihistamines may help with itching or hives but do not treat the life-threatening features of anaphylaxis and should never be used as a substitute for epinephrine. Epinephrine is available as auto-injectors, and in 2024 the FDA also approved an epinephrine nasal spray. People with prescribed epinephrine should carry it at all times, including to dining halls, parties, and friends' rooms, and should make sure people around them know where it is. If someone appears to be having a severe allergic reaction, the general steps are: help them use their epinephrine, or use it for them if they cannot and they or their prescribed device's instructions allow; call 911 immediately, even if symptoms improve after epinephrine, because a second phase of symptoms can occur; have the person lie down with legs raised, unless they are having trouble breathing, in which case they may sit up; and be ready to give a second dose if symptoms persist or return and a second device is available. Students who share a room or floor with someone at risk of anaphylaxis should learn how to use the specific device that person carries. Most devices come with training versions and clear instructions. A shared responsibility The shared kitchen and the shared room make food allergy a collective issue in a way that it rarely is in a family home. A person with a severe allergy depends on the habits of dozens of others: whether they wash their hands, whether they clean the counter, whether they label potluck dishes, whether they notice when someone is in trouble. Most of those habits cost almost nothing, and many of them are the same habits that prevent foodborne illness. The student with an allergy also has a role. Telling roommates and resident assistants, carrying epinephrine consistently, asking about ingredients without embarrassment, and registering for accommodations where needed all make the environment safer. Neither side can do it alone. The next chapter turns to a hazard that is shared even more completely, because a fire that starts in one room affects everyone in the building. Chapter 8: Heat, Power, and Fire The food safety risks in earlier chapters mostly affect the person who makes a mistake, and occasionally a few friends. Fire is different. A fire that starts in one room can fill a hallway with smoke, force hundreds of people out of a building, destroy other students' belongings, and in the worst cases take lives. It is also the hazard most directly tied to cooking. For as long as national data on campus housing fires have been collected, cooking has been the leading cause. This chapter explains why, and what it means for the choices students make about appliances, outlets, and their own behavior in the kitchen. It also covers the moment most people never rehearse: what to do when something is actually on fire. What the data show The National Fire Protection Association publishes periodic analyses of fires in dormitories, fraternity and sorority houses, and barracks. Its most recent analysis, covering 2018 through 2022, estimated that U.S. fire departments responded to an average of 3,331 structure fires a year in these properties. Cooking equipment was involved in nearly eight of every ten fires, and roughly three of every four began in a kitchen or cooking area. Cooking fires accounted for 58 percent of civilian injuries in these properties, even though, because most were small, they accounted for only about 11 percent of direct property damage. The pattern is remarkably stable. An earlier NFPA analysis of 2003 to 2006 found that cooking equipment was involved in 75 percent of such fires. A U.S. Fire Administration report on university housing fires from 2007 to 2009 estimated about 3,800 fires a year and found that 88 percent were cooking fires. Across these reports, the same timing recurs: fires are most common in the evening, with the USFA report finding the peak between 5 and 10 p.m. and especially between 7 and 9 p.m., and the NFPA's earlier analysis identifying 5 to 11 p.m. and weekends. The start of the academic year stands out as well; the USFA found that September, October, and November together accounted for a third of the year's fires. The evening and early-semester peaks tell a story. Fires cluster when students are cooking dinner, when they are tired or distracted after a day of classes, and when they are new to their living arrangements and still learning how the equipment works. Most of these fires are small and confined to the cooking vessel or the appliance, and residence halls with automatic sprinklers and monitored alarms usually contain them quickly. But even a small fire can cause injuries, particularly burns to the person trying to put it out, and the evacuations they trigger disrupt hundreds of people. It is worth noting where campus fire deaths tend to occur, because it is not primarily in residence halls. Campus Firewatch, which has tracked campus-related fatal fires in the United States since 2000, has consistently found that the large majority occurred in off-campus housing, such as rented houses and apartments, rather than in residence halls. Off-campus buildings often lack sprinklers and monitored alarms, may have missing or disabled smoke alarms, and are frequently the setting for parties; the USFA notes that roughly a quarter of campus fire fatalities follow a party. The same NFPA analysis of 2003 to 2006 found that fires starting in bedrooms, though only 5 percent of fires in dormitory-type properties, accounted for 62 percent of civilian deaths. The lessons of this chapter therefore apply with even more force to students who move off campus, where the protections built into residence halls are often absent. Why cooking causes so many fires Cooking fires are overwhelmingly caused by people rather than equipment. The most common single factor in home cooking fires, according to NFPA analyses, is unattended cooking: a pan left on a burner while the cook steps away. Oil and fat are the usual fuel. Cooking oil heated beyond its smoke point can reach its ignition temperature and catch fire, and a pan left on high heat can get there within minutes. Other common contributors are flammable materials too close to a heat source, such as paper towels, packaging, oven mitts, or loose sleeves; food left in a toaster or toaster oven too long; and, in microwaves, metal, foil, or food heated far too long. The dorm setting adds its own conditions. Floor kitchens are often down a hallway from students' rooms, which makes it tempting to start something on the stove and go back to the room while it cooks. Students cook late at night, sometimes after drinking, sometimes while exhausted. Many are cooking for the first time without supervision. And in rooms, where cooking with heating appliances is usually banned, the appliances that do get smuggled in sit near bedding, papers, and curtains. The most effective fire prevention habits follow directly from those causes. Stay in the kitchen while anything is cooking on a stovetop, especially when frying or using high heat. If you need to leave, turn off the burner. Keep anything that can burn away from the stovetop. Wear short or close-fitting sleeves. Keep a lid nearby when cooking with oil. Do not cook when you are very tired or have been drinking. And when using the microwave, stay close enough to notice if something goes wrong; popcorn left for too long is a leading cause of false alarms in residence halls, and occasionally of real fires. Permitted and banned appliances Every residence hall has a list of appliances that are permitted and appliances that are not. Students often experience these lists as arbitrary, especially when a friend at another college is allowed an appliance that theirs bans. The lists do vary, but the logic behind them is fairly consistent. Appliances with exposed heating elements, open coils, or high surface temperatures that can ignite nearby materials are most often banned. Appliances that draw a lot of power, and can overload circuits in older buildings, are often banned or restricted. Appliances with automatic shut-off features and enclosed heating elements are more often permitted. Table 4 summarizes how residence halls typically sort common appliances. It is a general picture only. Your housing agreement is the authority for your room, and the same appliance may be permitted in one building's kitchen and banned from its bedrooms. Table 4. How residence halls typically classify common cooking appliances. Appliance Typical status in rooms Main concern Microwave, often under a set wattage Permitted Unattended heating; power draw Compact refrigerator, often under a set size Permitted Power draw; placement Coffee maker or kettle with auto shut-off Often permitted Boiling dry without shut-off Rice cooker with auto shut-off Varies Heat near bedding Toaster, toaster oven, air fryer Often banned in rooms High heat; grease; food igniting Hot plate, open-coil appliance, grill Usually banned Exposed heating elements Sources: typical residence-hall appliance policies; U.S. Fire Administration campus fire safety guidance. Several points about these policies are worth understanding. First, the restriction is usually about location, not the appliance itself. A toaster that is banned in a room may be provided in the floor kitchen, where it sits on a noncombustible counter under a smoke detector designed for cooking areas and away from bedding. Second, wattage limits on microwaves and size limits on refrigerators exist largely to protect circuits in buildings that were not designed for the electrical loads of modern student life. Third, "automatic shut-off" is not a marketing detail. An electric kettle or coffee maker that turns itself off when it boils dry removes the most common way such appliances start fires. Students sometimes argue that banned appliances are safe if used carefully. For most of them, that may be true in the hands of a careful user most of the time. But a residence hall is not one careful user; it is hundreds of users with varying habits, and policies are written for the building. The consequences of a violation can include confiscation, fines, disciplinary action, or loss of housing, and more importantly, a fire in a room can harm people who had no say in the decision to bring a hot plate. Extension cords, power strips, and outlets Electrical problems are a separate category of fire risk in campus housing, and they interact with cooking because cooking appliances draw a lot of power. A microwave can draw more than 1,000 watts while running; a compact refrigerator draws less, but its compressor pulls a surge of current each time it starts. Add a hair dryer, a space heater, a gaming computer, and several chargers to the same circuit, and the load can exceed what the wiring was designed for. The USFA's guidance for college students includes a few specific rules. Do not overload outlets. Do not use extension cords with large appliances such as refrigerators; plug them directly into a wall outlet. Use only surge protectors or power strips that have internal overload protection. To those, general electrical safety guidance adds several more. Use only power strips and cords certified by a recognized testing laboratory, such as UL or ETL. Do not plug one power strip into another, a practice called daisy-chaining, which can overload the first strip. Do not run cords under rugs, through doorways, or under furniture, where they can be damaged and overheat unnoticed. Replace any cord that is frayed, cracked, or warm to the touch. And treat extension cords as temporary; they are not designed for permanent use. Most residence halls also restrict which power strips and extension cords are permitted, often requiring surge protectors with circuit breakers and banning ordinary extension cords altogether. Those rules are worth following for the same reason as appliance rules: they are written from experience with what fails. A composite case illustrates how these risks combine. A student sets up a desk area with a power strip plugged into a second power strip, which runs to an outlet behind the bed. On it are a laptop charger, a monitor, a lamp, a phone charger, a small refrigerator, and a microwave. For weeks, nothing happens. Then one evening, with the microwave running and the refrigerator's compressor starting at the same time, the strip behind the bed grows hot enough to scorch the mattress. The building's smoke detector catches it before it spreads. Nothing about the setup looked unusual; it looked like most dorm rooms. The fix, plugging the refrigerator and microwave directly into separate wall outlets and removing the second strip, would have cost nothing. When something catches fire Most people have never practiced responding to a cooking fire, and in the moment, instinct often suggests the wrong thing. The most dangerous instinct is to throw water on a grease fire. Water sinks beneath burning oil, flashes instantly to steam, and throws burning oil out of the pan, spreading the fire and causing severe burns. The second most dangerous is to carry a burning pan to the sink or out of the room. The NFPA's guidance for small cooking fires is simple. For a fire in a pan on the stovetop, slide a lid or a baking sheet over the pan to smother the flames, turn off the burner, and leave the pan covered until it has cooled completely. For a fire in an oven, turn off the heat and keep the door closed. For a fire in a microwave, keep the door closed and unplug the appliance if it is safe to reach the plug; the fire will usually go out once it runs out of oxygen. In a toaster or toaster oven, unplug it and keep it closed. Fire extinguishers can put out small fires, but they are useful only in the hands of someone who knows how to use them and only when the fire is small and contained. The standard technique is summarized by the acronym PASS: pull the pin, aim at the base of the fire, squeeze the handle, and sweep from side to side. Before using one, make sure everyone else is leaving, that the fire department is being called, and that you have a clear exit behind you. Many campuses offer hands-on extinguisher training during fire safety events, and it is worth attending once. Above all, when in doubt, get out. If the fire is spreading, if smoke is filling the room, if the fire is not in a container, or if you are unsure, leave immediately. Close the door behind you to slow the spread of fire and smoke, pull the nearest fire alarm, and call 911 from a safe location. Do not stop to collect belongings. Once outside, stay out until the fire department says it is safe to return. Alarms, sprinklers, and evacuations Residence halls are protected by systems that have dramatically reduced fire deaths in campus housing, and those systems depend on students not defeating them. Smoke detectors in rooms and kitchens should never be covered, disabled, or removed, even when they are triggered by cooking smoke or steam from a shower. Tampering with them is usually a serious violation of housing rules and may violate state fire codes. Sprinkler heads should never have anything hung from them, and nothing should be stacked close enough to block their spray. Fire alarms in residence halls are frequent, and many turn out to be caused by burned food. That frequency breeds complacency, and students sometimes stay in their rooms rather than evacuating. Every alarm should be treated as real. The student who decides to wait it out is betting that this particular alarm is a false one, and when the bet is wrong, the consequences can be severe. It also helps to know the evacuation routes from your room and from the floor kitchen, including a second route in case the first is blocked by smoke. Students who move off campus should bring these habits with them and check what is missing. The USFA recommends that smoke alarms be installed in every sleeping room and on every level, and that they be tested regularly. An off-campus house without working alarms, with extension cords snaking across the floor, a deep fryer on the counter, and a party on Saturday night, combines nearly every factor that appears in fatal campus-related fires. A tenant who checks the alarms, clears the exits, and knows where the extinguisher is changes that picture considerably. Fire safety in the kitchen, then, rests on the same principle as food safety. Most of the risk is concentrated in a few predictable situations: unattended cooking, heat near flammable materials, overloaded circuits, and responses made in panic. Handle those well, and the kitchen, even a shared one at the end of a long hall, becomes a safe place to cook. The final chapter turns to what to do when prevention has not worked, whether the problem is a stomach illness, a burn, or something more serious. Chapter 9: When Things Go Wrong No set of habits eliminates risk entirely. Even students who keep a cold refrigerator, use a food thermometer, cool their rice quickly, and wash their hands correctly will sometimes get sick, whether from a dining hall meal, a restaurant, a friend's cooking, or a bug caught from a roommate. And even careful cooks occasionally burn a hand or cut a finger. This chapter is about the moment after prevention fails: how to recognize what is happening, how to take care of yourself or someone else, when to get help, and how to make sure a problem does not spread. The usual course of a foodborne illness Most foodborne illnesses in healthy young adults follow a familiar course. Symptoms may include nausea, vomiting, stomach cramps, diarrhea, and sometimes fever, chills, headache, or body aches. Depending on the cause, they may begin within a few hours or a few days of eating the contaminated food. Toxin-mediated illnesses, such as those caused by Staphylococcus aureus or the emetic form of Bacillus cereus, tend to begin abruptly within hours and resolve within a day. Norovirus usually lasts one to three days. Bacterial infections such as Salmonella and Campylobacter may last several days to a week. For most people, the illness runs its course without specific treatment. The main job during that time is to stay hydrated, rest, and avoid passing the illness to others. That sounds simple, but when someone is vomiting repeatedly in a shared bathroom, alone in a dorm room at 2 a.m., it can feel far from simple. It helps to know in advance what to do and what signs mean the situation has moved beyond home care. Staying hydrated The main danger of a typical foodborne illness is dehydration. Vomiting and diarrhea remove water and electrolytes from the body quickly, and it can be hard to replace them when nothing stays down. The CDC advises people with diarrhea or vomiting to drink plenty of fluids to prevent dehydration. The most effective approach is small amounts often. After vomiting, wait a little while and then take a few sips of fluid every few minutes, gradually increasing the amount as it stays down. Oral rehydration solutions, sold in pharmacies as powders, liquids, or ice pops, are designed to replace both water and electrolytes in the right proportions, and they are worth keeping in a dorm room first aid kit. Clear broths and diluted sports drinks are reasonable alternatives. Sugary sodas and undiluted fruit juices can sometimes worsen diarrhea, and alcohol and large amounts of caffeine increase fluid loss. Once vomiting settles, most people can return gradually to bland food as they feel ready. There is no need to force food, and no need to follow a strict diet; appetite is a reasonable guide. Anti-diarrheal medicines can relieve symptoms for some people but are not recommended for everyone, particularly when there is fever or bloody diarrhea, and it is worth asking a pharmacist or clinician before using them. When to get medical help The CDC lists specific signs that a foodborne illness needs medical attention: bloody diarrhea; diarrhea lasting more than three days; a fever higher than 102°F (38.9°C); vomiting so often that you cannot keep liquids down; and signs of dehydration, which include not urinating much, a dry mouth and throat, and feeling dizzy when standing up. Table 5 organizes these signs alongside home care and emergency situations. Table 5. Deciding what level of care a stomach illness needs. Situation Examples What to do Manageable at home Vomiting or diarrhea without warning signs; able to keep sips of fluid down Rest, fluids, oral rehydration; avoid preparing food for others See a clinician soon Diarrhea over 3 days; fever over 102°F (38.9°C); bloody diarrhea; can't keep liquids down; signs of dehydration Contact student health or an urgent care clinic Seek emergency care Confusion or fainting; severe abdominal pain; not urinating for many hours; vomiting blood; difficulty breathing or swallowing Call 911 or go to an emergency department Source: CDC, Symptoms of Food Poisoning; general clinical guidance. Some groups should seek care sooner. People who are pregnant should contact a clinician promptly if they develop a fever with flu-like symptoms, since Listeria infection can cause serious complications in pregnancy even when the mother's symptoms are mild. People with weakened immune systems, diabetes, kidney disease, or other chronic conditions should not wait for symptoms to become severe before calling their clinician. And certain unusual symptoms need immediate attention regardless of other signs: blurred or double vision, drooping eyelids, slurred speech, or difficulty swallowing or breathing may indicate botulism, which is rare but a medical emergency. For students, the first point of contact is usually the campus health service. Many offer same-day appointments, and many also provide a nurse advice line after hours that can help decide whether a situation needs urgent care or can wait until morning. Knowing the number before you need it, and saving it in your phone, is one of the more useful things to do in the first week of the semester. Looking after someone else Students often find themselves caring for a sick roommate or friend, sometimes in the middle of the night. A few practical points help. Check in regularly, especially if the person is alone. Help them keep sipping fluids. Watch for the warning signs in Table 5, particularly confusion, fainting, or an inability to keep anything down, and do not hesitate to call campus health, a resident assistant, or 911 if they appear. If the person has been drinking alcohol as well as vomiting, be especially cautious: someone who is very drunk and vomiting can choke, and should be placed on their side and not left alone. Protect yourself at the same time. Norovirus in particular spreads easily from a sick person to their caregivers. Wash your hands with soap and water after helping them, avoid sharing utensils or towels, and clean contaminated surfaces with a bleach solution as described in Chapter 6. If you share a bathroom, cleaning its high-touch surfaces, including the faucet handles, the toilet flush handle, and the door handle, is worth doing more than once a day during the illness. Reporting a suspected outbreak Most foodborne illnesses are never reported, which is one reason public health agencies rely on estimates rather than counts. Individual reports matter most when several people become ill after eating the same food, whether at a restaurant, a campus event, or a dining hall. Health departments can identify outbreaks that no single person would recognize, and their investigations can lead to recalls, closures, and changes in food handling that protect others. If you suspect that food from a restaurant, event, or dining hall made you ill, especially if others who ate it are also sick, report it to your local or state health department. Most have an online form or phone line for suspected foodborne illness. Campus health services can also help, and they often coordinate with local health departments when clusters appear on campus. For problems with packaged meat, poultry, or egg products, the USDA runs a Meat and Poultry Hotline at 1-888-674-6854 (1-888-MPHotline). Problems with other FDA-regulated foods can be reported through the FDA's consumer complaint coordinators or its online Safety Reporting Portal. A useful report includes what you ate, where, and when; when your symptoms began and what they were; and whether others who ate the same food also became ill. If any of the suspect food remains, keep it in the refrigerator, clearly labeled "do not eat," in case the health department wants to test it. Keep packaging and receipts as well. If you see a clinician, mention that you suspect a foodborne illness; a stool test can identify the cause and contributes to the public health data used to track outbreaks. Burns and cuts Kitchen injuries are far more common than serious fires. Most are minor burns and cuts that can be treated at home, but a few need medical attention, and knowing the difference is useful. For a minor burn, such as a brief touch of a hot pan or a splash of hot water, cool the burn under cool running water for several minutes as soon as possible. First aid organizations commonly advise continuing for at least ten minutes, and often up to twenty. Remove rings or tight items near the burn before swelling begins. Do not apply ice, which can damage the skin further, and do not apply butter, oil, or toothpaste. After cooling, cover the burn loosely with a clean, non-stick dressing. Over-the-counter pain relievers can help. Leave small blisters intact. Seek medical care for burns that are larger than the palm of the hand, that are deep (white, leathery, or charred skin, or skin that feels numb), that are on the face, hands, feet, genitals, or over major joints, or that are caused by chemicals or electricity. Burns from hot oil and steam can be deeper than they first appear. Call 911 for extensive burns, burns involving the airway, or any burn in someone who is having trouble breathing. For cuts, apply firm, direct pressure with a clean cloth. Most kitchen cuts stop bleeding within a few minutes. Once bleeding stops, clean the wound with water, and cover it with a clean bandage. Seek medical care for cuts that will not stop bleeding after about ten minutes of steady pressure, that are deep or gaping, that have debris embedded, or that show signs of infection over the following days, such as increasing redness, warmth, swelling, or pus. If it has been many years since your last tetanus booster, a clinician may recommend one for a deep or dirty wound. Any cut or burn on the hands should be covered before preparing food, both to protect the wound and to avoid contaminating food. Staphylococcus aureus, which lives on skin and in infected wounds, can produce a toxin in food that causes rapid-onset vomiting, and a gloved or bandaged hand prevents that route. After an allergic reaction Chapter 7 covered how to recognize and respond to anaphylaxis. The aftermath matters too. Anyone who has used epinephrine for an allergic reaction should be evaluated in an emergency department, even if they feel better, because symptoms can return hours later. The used device needs to be replaced promptly, since people who have had one reaction may have another. And afterward, it is worth working out, with the person's clinician if possible, how the exposure happened. Was an ingredient hidden in a sauce? Was a utensil shared? Did a label change? Each answer points to a change in practice that can prevent the next reaction. For roommates and friends who witnessed a severe reaction, it is also normal to feel shaken afterward. Talking about it, and taking a training session on using epinephrine, can help turn that feeling into preparedness. Classes, deadlines, and the day after A foodborne illness rarely lasts long, but it often lands at a bad time, and students frequently make it worse by trying to push through. Attending an exam while vomiting, or dragging oneself to a lab while shedding norovirus, helps no one: the student performs poorly, and classmates are exposed. Most institutions have procedures for exactly this situation, and using them is not a sign of weakness. The first step is usually to notify instructors as soon as possible, ideally before the class or deadline in question, with a short message explaining that you are ill and asking how to make up the work. Many syllabi describe the process for missed exams or late assignments. If an illness causes you to miss several days or an important assessment, the dean of students office or an equivalent office can often notify all of your instructors at once and help coordinate extensions. Some courses require documentation from a clinician for missed exams; a visit to the campus health service, even a brief one, provides a record if one is needed. Students who are also employees on campus should tell their supervisor promptly, particularly if the job involves handling food, since food service workers with vomiting or diarrhea are generally required to stay away from work until they have recovered. The days after an illness also carry a quiet risk of their own. A student who has been sick for two days returns to a refrigerator whose contents have aged without being eaten, and to a room that may still carry virus on its surfaces. It is worth spending a few minutes checking the refrigerator's thermometer, throwing out leftovers that are now past their three-to-four-day window, and cleaning high-touch surfaces in the room and bathroom. If the illness was norovirus, wait at least two days after symptoms end before cooking for anyone else, and wash hands carefully before handling food even for yourself. Finally, think briefly about what happened. Not every illness has a traceable cause, and the purpose is not self-blame. But if the timing and symptoms point toward a particular meal, and especially if that meal came from your own refrigerator or kitchen, it is useful to identify what might have gone wrong. Was the rice left out? Was the chicken checked with a thermometer? Did a container outlive its label? Each honest answer makes the next semester a little safer, and that kind of learning is what turns a set of rules into judgment. A small first aid kit for the room A few inexpensive items make every situation in this chapter easier to handle. A basic dorm kit might include an appliance thermometer and a food thermometer, since most of the problems in this book start with an unmeasured temperature; oral rehydration packets; a pain and fever reliever; adhesive bandages in several sizes and a few non-stick burn dressings; disposable gloves; a small bottle of household bleach, or disinfecting wipes labeled as effective against norovirus, for cleaning after illness; and a card or phone note listing the campus health number, the after-hours nurse line, campus security, and the address of the nearest urgent care and emergency department. Students with food allergies should add their epinephrine, a second device if prescribed, and a written action plan from their clinician. None of this is elaborate. The point is simply to have what you need at hand at the moment you need it, which is usually late at night, when shops are closed and judgment is not at its best. The final pages of this book step back from specific problems to consider what they have in common, and what that suggests about cooking well in a space that was never built for it. Conclusion Looked at one at a time, the hazards in this book seem unrelated. A compact refrigerator that runs six degrees too warm. A frozen meal eaten before its standing time is up. A pot of rice cooling slowly on a stove. A sponge that has wiped every counter on the floor. A knife that spread peanut butter and then jam. A pan of oil left on a burner while its owner answers the door. They involve different organisms, different appliances, and different kinds of harm. But set side by side, they share a structure, and that structure suggests something more useful than any individual rule. Almost every problem in these chapters happens at a moment of transition that nobody is watching. Food moves from the oven to the refrigerator, and the hours in between go unmeasured. It moves from the freezer to the plate, and the cold center goes unchecked. It moves from one person's hands to another's, from one board to the next, from a raw state to a cooked one, from the grocery bag to the shelf. A pan moves from medium heat to its smoke point while the cook is in another room. Danger rarely lives in the food or the appliance itself. It lives in the gaps: the unobserved interval, the unmeasured temperature, the unwashed surface between one use and the next. This observation has a practical consequence. The most effective safety habits in a dorm are not the ones that require the most effort; they are the ones that close those gaps with the least. A three-dollar appliance thermometer turns an invisible temperature into a number that can be read while reaching for the milk. A food thermometer turns a guess about doneness into a fact. A strip of masking tape with a date turns an unknowable storage history into a known one. A second cutting board in a different color turns a decision that has to be remembered every time into one that is made once. Dividing leftovers into shallow containers turns a slow, unwatched cool into a fast one. Staying in the kitchen while oil heats turns an unobserved interval into an observed one. Each of these habits takes seconds, and each works by making something visible that would otherwise be hidden. That is why this book has emphasized mechanisms over rules. A rule tells you what to do in a situation someone anticipated. A mechanism tells you where to look in a situation nobody anticipated. Student life produces unanticipated situations constantly: a power outage during finals, a roommate's new diagnosis, a potluck with forty guests, a secondhand refrigerator whose dial is marked in letters instead of numbers. A student who knows that bacteria multiply in warm food over time, that microwaves leave cold spots, that spores survive cooking, that allergens survive heat, and that fires start where heat meets fuel without a watcher can reason through each of those situations without a manual. Why the first year matters There is a further reason to take these habits seriously now rather than later. The years in which students first live independently are the years in which lasting kitchen habits form. A person who learns to cook with a food thermometer at nineteen will probably still be using one at forty. A person who learns to leave pans unattended, to thaw chicken on the counter, and to judge leftovers by smell will probably keep those habits too, and will carry them into shared apartments, family kitchens, and eventually into feeding children and older relatives, the people for whom foodborne illness is most dangerous. Seen that way, the residence hall is not a temporary inconvenience to be survived until real kitchens become available. It is a training ground, and an unusually good one, because its constraints make the underlying mechanisms hard to ignore. A full-size refrigerator hides its temperature swings; a compact one exposes them. A household stove heats a pan evenly enough that the cook never has to think about it; a microwave forces the question of how heat actually moves through food. A family kitchen shared by four people who trust each other hides the problem of cross-contact; a floor kitchen shared by forty strangers puts it in plain view. Students who learn to cook safely under these conditions learn things that people who have always cooked in comfortable kitchens often never learn at all. What institutions can do The habits described here are individual, but the environment is not. Housing offices, dining services, and residence life staff shape that environment in ways that make safe habits easier or harder, and a few low-cost steps would make a considerable difference. Appliance thermometers could be included with every rented micro-fridge and placed in every communal refrigerator. Floor kitchens could be stocked with paper towels, soap, a spray cleaner, and a small sign explaining how to handle a pan fire. Residence hall orientation could spend five minutes on cooling rice and five on the two-hour rule, alongside its sessions on fire drills and alcohol. Appliance policies could explain their reasoning in a sentence or two, since students follow rules more consistently when they understand them. And programs that let students with food allergies register for accommodations could be publicized as routinely as meal plans. None of these steps is expensive or complicated. Each addresses one of the unwatched gaps that run through this book, and together they would shift some of the burden of safety from individual memory to the design of the space itself, which is where the most durable safety improvements usually come from. Cooking with confidence The introduction to this book suggested that the purpose of understanding food safety is not to make cooking stressful but to make it relaxed. That remains the right note to end on. A student who knows where the real risks lie can stop worrying about the rest. The leftover pasta cooked last night and refrigerated promptly is fine. The chicken that read 170°F (77°C) in three places is fine. The dining hall salad, the frozen vegetables steamed in a covered bowl, the rice spread thin and chilled within the hour, the microwave oatmeal: all fine. The risks worth attention are few and specific, and the habits that manage them are small. The dorm will never be an ideal kitchen. Its refrigerator is too small, its microwave too weak, its shared stove too far down the hall, its outlets too few. But it can be a safe one, and for many students, it is where they first learn to feed themselves well. That is worth doing properly. The knowledge in these pages is modest in size and durable in value: a few temperatures, a few time limits, a few ways that heat and microbes and allergens behave, and the habit of checking rather than assuming. Carried out of the residence hall and into the rest of life, it is enough to keep a great many meals, and a great many people, safe. Notes Introduction 1. National Fire Protection Association, Structure Fires in Dormitories, Fraternities, Sororities and Barracks (data for 2018–2022), summarized in NFPA and Center for Campus Fire Safety materials for Campus Fire Safety Month, August 2024. https://www.nfpa.org/education-and-research/research/nfpa-research/fire-statistical-reports/fires-in-dormitory-type-properties Chapter 1: What Actually Makes You Sick 1. Scallan E, Hoekstra RM, Angulo FJ, et al. "Foodborne Illness Acquired in the United States—Major Pathogens." Emerging Infectious Diseases 17, no. 1 (2011): 7–15; and Scallan E, Griffin PM, Angulo FJ, et al. "Foodborne Illness Acquired in the United States—Unspecified Agents." Emerging Infectious Diseases 17, no. 1 (2011): 16–22. Source of the estimate of 48 million illnesses, 128,000 hospitalizations, and 3,000 deaths per year. 1. Scallan Walter EJ, Cui Z, Tierney R, et al. "Foodborne Illness Acquired in the United States—Major Pathogens, 2019." Emerging Infectious Diseases 31, no. 4 (2025): 669–677. https://wwwnc.cdc.gov/eid/article/31/4/24-0913_article 2. Centers for Disease Control and Prevention, "Preventing Norovirus" and "Norovirus Facts and Stats." https://www.cdc.gov/norovirus/ 3. U.S. Department of Agriculture, Food Safety and Inspection Service, "Danger Zone 40°F–140°F." https://www.fsis.usda.gov/food-safety/safe-food-handling-and-preparation/food-safety-basics/danger-zone-40f-140f 4. Pathogen incubation periods and sources in Table 1 follow CDC pathogen pages and FDA, Bad Bug Book: Handbook of Foodborne Pathogenic Microorganisms and Natural Toxins, 2nd ed. (2012). https://www.fda.gov/food/foodborne-pathogens/bad-bug-book-second-edition 5. FoodSafety.gov, "4 Steps to Food Safety: Clean, Separate, Cook, Chill." https://www.foodsafety.gov/keep-food-safe/4-steps-to-food-safety Chapter 2: The Micro-Fridge Reality 1. U.S. Food and Drug Administration, "Refrigerator Thermometers: Cold Facts about Food Safety." https://www.fda.gov/food/buy-store-serve-safe-food/refrigerator-thermometers-cold-facts-about-food-safety 2. USDA FSIS, "Refrigeration and Food Safety." https://www.fsis.usda.gov/food-safety/safe-food-handling-and-preparation/food-safety-basics/refrigeration 3. USDA FSIS, "Appliance Thermometers." https://www.fsis.usda.gov/food-safety/safe-food-handling-and-preparation/food-safety-basics/appliance-thermometers 4. USDA FSIS, "Freezing and Food Safety." https://www.fsis.usda.gov/food-safety/safe-food-handling-and-preparation/food-safety-basics/freezing-and-food-safety 5. FoodSafety.gov, "Food Safety During Power Outage" and "Cold Food Storage Chart." https://www.foodsafety.gov/food-safety-charts/cold-food-storage-charts 6. USDA FSIS, FoodKeeper App. https://www.foodsafety.gov/keep-food-safe/foodkeeper-app Chapter 3: The Clock Is Always Running 1. USDA FSIS, "Leftovers and Food Safety." https://www.fsis.usda.gov/food-safety/safe-food-handling-and-preparation/food-safety-basics/leftovers-and-food-safety 2. USDA FSIS, "The Big Thaw—Safe Defrosting Methods." https://www.fsis.usda.gov/food-safety/safe-food-handling-and-preparation/food-safety-basics/big-thaw-safe-defrosting-methods 3. U.S. Food and Drug Administration, Food Code 2022, section 3-501.14 (cooling). https://www.fda.gov/food/fda-food-code/food-code-2022 Chapter 4: Cooking Evenly in a Microwave 1. USDA FSIS, "Cooking Safely in the Microwave Oven." https://www.fsis.usda.gov/food-safety/safe-food-handling-and-preparation/food-safety-basics/cooking-safely-microwave-oven 2. Centers for Disease Control and Prevention. "Multistate Outbreak of Salmonella Infections Associated with Frozen Pot Pies—United States, 2007." MMWR 57, no. 47 (2008): 1277–1280. https://www.cdc.gov/mmwr/preview/mmwrhtml/mm5747a3.htm 3. FoodSafety.gov, "Safe Minimum Internal Temperature Chart." https://www.foodsafety.gov/food-safety-charts/safe-minimum-internal-temperatures 4. U.S. Food and Drug Administration, "Risk of Burns from Eruptions of Hot Water Overheated in Microwave Ovens." https://www.fda.gov/radiation-emitting-products/microwave-ovens/risk-burns-eruptions-hot-water-overheated-microwave-ovens Chapter 5: Rice, Pasta, and the Batch-Cooking Trap 1. University of Florida IFAS Extension, "Preventing Foodborne Illness: Bacillus cereus" (FSHN15-06/FS269). https://ask.ifas.ufl.edu/publication/FS269 2. Dierick K, Van Coillie E, Swiecicka I, et al. "Fatal Family Outbreak of Bacillus cereus-Associated Food Poisoning." Journal of Clinical Microbiology 43, no. 8 (2005): 4277–4279. https://doi.org/10.1128/JCM.43.8.4277-4279.2005 3. Naranjo M, Denayer S, Botteldoorn N, et al. "Sudden Death of a Young Adult Associated with Bacillus cereus Food Poisoning." Journal of Clinical Microbiology 49, no. 12 (2011): 4379–4381. https://doi.org/10.1128/JCM.05129-11 4. National Health Service (UK), "Can reheating rice cause food poisoning?" https://www.nhs.uk/common-health-questions/food-and-diet/can-reheating-rice-cause-food-poisoning/ 5. Centers for Disease Control and Prevention, "About Clostridium perfringens Food Poisoning." https://www.cdc.gov/clostridium-perfringens/ Chapter 6: Shared Kitchens and Other People's Hands 1. U.S. Department of Agriculture, "Study Shows Most People Are Spreading Dangerous Bacteria Around the Kitchen and Don't Even Realize It," press release, June 28, 2018. https://www.usda.gov/about-usda/news/press-releases/2018/06/28/study-shows-most-people-are-spreading-dangerous-bacteria-around-kitchen-and-dont-even-realize-it 2. Centers for Disease Control and Prevention, "Clean Hands" (handwashing guidance). https://www.cdc.gov/clean-hands/ 3. Cardinale M, Kaiser D, Lueders T, Schnell S, Egert M. "Microbiome Analysis and Confocal Microscopy of Used Kitchen Sponges Reveal Massive Colonization by Acinetobacter, Moraxella and Chryseobacterium Species." Scientific Reports 7 (2017): 5791. https://doi.org/10.1038/s41598-017-06055-9 4. USDA FSIS, "Washing Food: Does It Promote Food Safety?" https://www.fsis.usda.gov/food-safety/safe-food-handling-and-preparation/food-safety-basics/washing-food-does-it-promote-food 5. Centers for Disease Control and Prevention, "Preventing Norovirus" (including bleach concentrations for disinfection). https://www.cdc.gov/norovirus/prevention/ Chapter 7: Allergens and Cross-Contact 1. Gupta RS, Warren CM, Smith BM, et al. "Prevalence and Severity of Food Allergies Among US Adults." JAMA Network Open 2, no. 1 (2019): e185630. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2720064 2. U.S. Food and Drug Administration, "Food Allergies" and "FASTER Act: Sesame Is the Ninth Major Food Allergen." https://www.fda.gov/food/food-allergies/faster-act-sesame-ninth-major-food-allergen 3. Perry TT, Conover-Walker MK, Pomés A, Chapman MD, Wood RA. "Distribution of Peanut Allergen in the Environment." Journal of Allergy and Clinical Immunology 113, no. 5 (2004): 973–976. https://doi.org/10.1016/j.jaci.2004.02.035 4. Maloney JM, Chapman MD, Sicherer SH. "Peanut Allergen Exposure through Saliva: Assessment and Interventions to Reduce Exposure." Journal of Allergy and Clinical Immunology 118, no. 3 (2006): 719–724. 5. U.S. Department of Justice, Settlement Agreement between the United States and Lesley University under the Americans with Disabilities Act, December 2012. https://www.ada.gov/lesleyuniversitysa.htm 6. U.S. Food and Drug Administration, "FDA Approves First Nasal Spray for Treatment of Anaphylaxis," August 9, 2024. https://www.fda.gov/news-events/press-announcements/fda-approves-first-nasal-spray-treatment-anaphylaxis Chapter 8: Heat, Power, and Fire 1. National Fire Protection Association, Structure Fires in Dormitories, Fraternities, Sororities and Barracks (2018–2022 data), as reported in "Campus Housing Fires: Cooking, Lithium-Ion Batteries Are Top Concerns," Facility Executive, August 2024. https://facilityexecutive.com/campus-housing-cooking-lithium-ion-batteries-top-fire-worries 2. Flynn JD. Structure Fires in Dormitories, Fraternities, Sororities and Barracks. Quincy, MA: National Fire Protection Association, August 2009 (2003–2006 data). 3. U.S. Fire Administration. "University Housing Fires (2007–2009)." Topical Fire Report Series 12, no. 9 (August 2011). https://www.usfa.fema.gov/downloads/pdf/statistics/v12i9.pdf 4. U.S. Fire Administration, "Fire Safety for College Students." https://www.usfa.fema.gov/prevention/home-fires/at-risk-audiences/college-students/ 5. Campus Firewatch, map and information sheets on campus-related fatal fires since 2000. https://www.campus-firewatch.com/ 6. National Fire Protection Association, "Cooking Safety Tips" and "Campus and Dorm Fires." https://www.nfpa.org/education-and-research/building-and-life-safety/campus-and-dorm-fires Chapter 9: When Things Go Wrong 1. Centers for Disease Control and Prevention, "Symptoms of Food Poisoning." https://www.cdc.gov/food-safety/signs-symptoms/index.html 2. USDA FSIS, Meat and Poultry Hotline, 1-888-674-6854. https://www.fsis.usda.gov/contact-us/meat-poultry-hotline 3. U.S. Food and Drug Administration, "How to Report a Problem with Food." https://www.fda.gov/food/compliance-enforcement-food/how-report-problem-food 4. Centers for Disease Control and Prevention, "Listeria Infection (Listeriosis): People at Risk—Pregnant People and Newborns." https://www.cdc.gov/listeria/ Further Reading FoodSafety.gov. The federal government's consumer food safety site, maintained jointly by the USDA, FDA, and CDC. Its charts of safe minimum internal temperatures and cold storage times are the most practical single reference for a dorm kitchen. https://www.foodsafety.gov USDA FoodKeeper App. A free app with storage guidance for hundreds of specific foods in the pantry, refrigerator, and freezer. Useful for the question "how long does this keep?" when the answer is not obvious. USDA Meat and Poultry Hotline and "Ask USDA." Food safety specialists answer questions by phone, email, and chat on weekdays. A good resource for questions about specific situations, such as power outages or thawing mistakes. 1-888-674-6854. CDC, Food Safety pages. Clear explanations of the major foodborne pathogens, symptoms, and when to seek care, along with current outbreak notices and recall information. https://www.cdc.gov/food-safety/ FDA, Bad Bug Book, 2nd edition (2012). The FDA's handbook of foodborne pathogens and toxins. Technical but readable, it describes each organism's sources, symptoms, onset times, and characteristics. Free online. Food Allergy Research & Education (FARE). The largest U.S. nonprofit devoted to food allergy, with guidance for college students on dining, roommates, and accommodations, and downloadable emergency care plans. https://www.foodallergy.org National Fire Protection Association, campus fire safety resources. Tip sheets, videos, and statistics on campus and off-campus housing fires, including cooking safety and what to do if a fire starts. https://www.nfpa.org Center for Campus Fire Safety. A nonprofit focused on reducing fire deaths among college students, with resources for students, parents, and housing staff, including guidance on evaluating off-campus housing. https://www.campusfiresafety.org U.S. Fire Administration, "Fire Safety for College Students." A concise federal guide covering cooking, candles, electrical safety, smoke alarms, and escape planning. https://www.usfa.fema.gov Harold McGee, On Food and Cooking: The Science and Lore of the Kitchen (Scribner, revised edition 2004).** Not a food safety manual, but the standard reference on what heat, cold, and time actually do to food. Readers who enjoyed the mechanisms in this book will find many more here. Hashtags: #DormCooking #DormRoomCooking #CookingSafely #FoodSafety #FoodborneIllnessPrevention #StudentFoodSafety #CampusLiving #CollegeStudentWellness #DormKitchenSafety #MicroFridge #MiniFridgeSafety #RefrigeratorTemperature #SafeFoodStorage #FoodTemperatureDangerZone #FoodThermometers #MicrowaveCooking #MicrowaveSafety #SafeFoodHandling #CrossContaminationPrevention #FoodAllergySafety #FoodHygiene #LeftoverSafety #MealPrepSafety #BacillusCereus #RiceSafety #PreventingFoodPoisoning #SharedKitchenHygiene #CampusFireSafety #KitchenFirePrevention #StudentHealthAndSafety

Download the Book (PDF): Introduction In the last days of October 2025, Hurricane Melissa sat over the warm Caribbean south of Jamaica and did what forecasters fear most: it intensified with almost no pause. Days before landfall, the National Hurricane Center in Miami was already warning of a storm that would reach the top of the Saffir-Simpson scale, and it said so with a confidence that forecasters rarely allow themselves when rapid intensification is involved. One of the reasons for that confidence was not a supercomputer running the equations of fluid dynamics. It was a neural network, an experimental ensemble model built by Google DeepMind, whose fifty possible futures for the storm were arriving on forecasters' screens alongside the traditional guidance. When the season was over, an independent review by a former branch chief of the hurricane center found that this model had been the best single source of track and intensity guidance in the Atlantic that year. A few months earlier, on the other side of the Atlantic, the European Centre for Medium-Range Weather Forecasts had switched on an operational ensemble forecast produced entirely by machine learning. By ECMWF's own account, it beat its physics-based counterpart on many headline measures, including surface temperature by up to 20 per cent, while using roughly a thousandth of the energy. In December 2025 the United States National Oceanic and Atmospheric Administration followed with its own suite of AI global models, built on a foundation first published by Google DeepMind and retrained on NOAA's own analyses. A single sixteen-day forecast from the American system used around 0.3 per cent of the computing that its physics-based equivalent needed. These are not laboratory curiosities. They are the forecasts that now inform evacuation orders, energy trading, flood preparation and aviation. Something real has changed in how humanity predicts the atmosphere, and it has changed in about four years. What this booklet argues It would be easy to tell this as a story of replacement: neural networks arrive, equations retire. That story is wrong, and believing it would be dangerous. The argument of this booklet is narrower and, I think, more useful. Machine learning has given Earth system science one overwhelming gift, and it is not accuracy. It is cheapness. A trained network can produce a global forecast in seconds or minutes where a physics-based model needs an hour on thousands of processors. Accuracy matters, and the learned models are genuinely accurate within the range of weather they were trained on. But the transformative consequence is what the cheapness can buy: ensembles of hundreds or thousands of members instead of fifty, climate simulations of centuries instead of decades, kilometre-scale local detail instead of hundred-kilometre averages. Each of those purchases is, at heart, a purchase of uncertainty information, and uncertainty information is what decision-makers have always lacked most. Cheapness, however, comes from learning patterns from the past, and the past is a limited teacher. The atmosphere of the coming decades will produce events that have no precedent in the training record, both because records are broken every year and because the climate itself is shifting. A learned model asked to forecast an event outside what it has seen, or a climate it has never experienced, has no physical law to fall back on unless one has been deliberately built in. So the craft of using AI in this field lies in combination: learned speed to buy many samples, physical structure to anchor what cannot be learned from history, and honest verification to know which is which. That idea organises everything that follows. Where the evidence shows learned models succeeding, it is usually because they were asked to reproduce, quickly and faithfully, behaviour that was well represented in their training data. Where they fail, it is usually because they were asked to extrapolate. The most promising work of the present moment is the work that knows the difference. The three tasks The description of this subject names three applications, and they turn out to be three versions of the same problem. The first is emulation: training a network to imitate a complex physics simulation, so that a forecast or a climate projection can be produced at a fraction of the cost. This includes the global weather models that made headlines from 2022 onwards, the learned replacements for small-scale processes such as clouds and convection inside conventional climate models, and the newer emulators that run the whole atmosphere for decades. The second is prediction of extreme weather: heatwaves, tropical cyclones, extreme rainfall, windstorms. These are the events that cause the damage, and they are, by definition, the rarest examples in any training set. This is where the tension between learning from the past and forecasting the unprecedented becomes sharpest. The third is downscaling: taking the coarse output of a global climate model, which might describe a grid cell a hundred kilometres across, and producing the local detail that a water engineer, a farmer or a city planner actually needs. Here machine learning competes with, and increasingly collaborates with, regional physics-based models that are accurate but enormously expensive to run. In each case the same pattern recurs. A network learns a mapping from data; it does so astonishingly cheaply; the question is whether that mapping holds where we most need it. How the booklet is built The first chapter explains why the physics-based approach is expensive, and why that expense is not a sign of inefficiency but a consequence of what the atmosphere is. Without that foundation it is impossible to judge what the learned models have actually achieved. The second and third chapters cover the breakthrough in medium-range weather forecasting: how deterministic models such as GraphCast and Pangu-Weather came to rival the best physics-based systems, and why the field moved quickly from single forecasts to generative ensembles. The fourth chapter looks inside the conventional climate model, at efforts to replace its weakest components with learned ones, and at hybrid models that keep a physical core. The fifth turns to climate itself, and to the much harder question of whether a learned model can be trusted over decades of change. The sixth chapter confronts extremes directly, drawing on the best evaluation studies available as of late 2026, including evidence that physics-based models still outperform AI on record-breaking heat. The seventh chapter treats downscaling, from classical statistical methods to the diffusion models that now generate kilometre-scale weather from coarse inputs. The final chapter addresses trust: how these systems should be verified, how they are entering operational use, and what a sensible user should ask of any learned forecast. The book assumes no background in either meteorology or machine learning beyond general scientific literacy. Technical terms are explained where they first appear. The field moves quickly, and specific model names will date; the underlying distinctions between interpolation and extrapolation, between a single forecast and a calibrated probability, between a pretty picture and a verified one, will not. Chapter 1: The Price of Physics To understand what machine learning has done to weather and climate prediction, it is necessary first to understand what it was competing against. The physics-based forecast is one of the great achievements of twentieth-century science, and it is expensive for reasons that have nothing to do with sloppy engineering. Its cost follows directly from the nature of the atmosphere and from the honest attempt to represent it from first principles. Every claim about AI's advantage is, in the end, a claim about this cost. Equations on a grid The atmosphere is a thin shell of compressible fluid on a rotating sphere, heated unevenly by the sun, coupled to oceans, ice and land, and carrying water that changes phase and releases or absorbs enormous amounts of heat as it does so. Its motion obeys conservation laws that were understood by the end of the nineteenth century: conservation of momentum, of mass, of energy, and of water substance, together with an equation of state relating pressure, density and temperature. In 1904 the Norwegian scientist Vilhelm Bjerknes argued that weather prediction should therefore be treated as an initial-value problem: know the state of the atmosphere now, apply the laws, and compute the state later. The difficulty is that these equations have no general solution on paper. They must be solved numerically, which means chopping the atmosphere into a three-dimensional grid of boxes and stepping the state of each box forward in small increments of time. Lewis Fry Richardson attempted this by hand during the First World War, producing a six-hour forecast for a patch of central Europe that took him weeks and was wildly wrong, mainly because of problems with his initial data. The first successful computer forecast came in 1950, on the ENIAC machine, run by Jule Charney, Ragnar Fjørtoft and John von Neumann. From there the field progressed steadily. A widely cited review by Peter Bauer, Alan Thorpe and Gilbert Brunet in Nature in 2015 called it a "quiet revolution": forecast skill had improved by roughly a day per decade, so that a six-day forecast in the 2010s was about as good as a three-day forecast in the 1980s. That steady progress was bought with computing power, and the reason lies in the grid. Halving the horizontal spacing of a global model quadruples the number of columns. Numerical stability then demands a shorter time step, roughly halving it. If vertical resolution is also increased, the cost climbs further. As a rule of thumb, doubling resolution in all directions multiplies the cost by something between eight and sixteen. The European Centre for Medium-Range Weather Forecasts (ECMWF), generally regarded as the world leader in global prediction, runs its Integrated Forecasting System at a grid spacing of about 9 kilometres for both its single high-resolution forecast and its ensemble. Getting there took decades of supercomputer upgrades, each one costing tens of millions of euros and consuming megawatts of electricity. The problem of what the grid cannot see Even at 9 kilometres, an enormous amount of important physics is too small for the grid to resolve. An individual thunderstorm updraft may be a kilometre or two across. The turbulent eddies that carry heat and moisture up from the surface are smaller still. Cloud droplets form on particles less than a micrometre wide. Radiation is absorbed and emitted by molecules according to their spectroscopic properties, which vary across thousands of wavelengths. None of these processes can be ignored. Convection carries heat vertically and produces most of the heavy rain on Earth. Clouds reflect sunlight and trap outgoing heat, and their net effect on the climate is one of the largest single uncertainties in projections of future warming. The boundary layer, the turbulent lowest kilometre or so of the atmosphere, controls surface temperature, fog, and the mixing of pollution. Models deal with this through parameterization: a set of simplified sub-models, each representing the aggregate effect of unresolved processes on the resolved grid-scale variables. A convection scheme, for example, looks at the temperature and humidity profile in a grid column, decides whether conditions favour thunderstorms, and if so estimates how much heat and moisture the storms would move and how much rain would fall. These schemes are built from a mixture of physical reasoning, observation and tuning. They are, frankly, the weakest part of any model. Many of the persistent biases in weather and climate models, such as the "drizzle problem" in which models rain too often and too lightly, or errors in the timing of the daily cycle of tropical convection, trace back to parameterization. Parameterization also consumes a large fraction of a model's computing budget. Radiation calculations alone are so expensive that many models compute them less frequently than the dynamics, or on a coarser grid, accepting some error in exchange for speed. This matters for the rest of the story, because parameterization is the obvious place where a learned model might do better: here, the physics is already approximate, the governing laws are already being replaced by fitted relationships, and the only question is whether a neural network can fit them better than a human can. Starting from the right place A forecast is only as good as its starting point, and the starting point is not something anyone observes directly. Observations of the atmosphere come from a bewildering variety of sources: weather stations, balloons launched twice a day from hundreds of sites, commercial aircraft, ships and buoys, radar, and above all satellites, which measure radiances in many wavelength bands that relate only indirectly to temperature and humidity. These observations are irregular in space and time, have different errors, and leave large gaps, particularly over oceans and in the upper atmosphere. Data assimilation is the discipline that turns this mess into a complete, consistent three-dimensional state from which a forecast can start. The leading systems use variational methods, of which ECMWF's four-dimensional variational assimilation (4D-Var) is the best known. In essence, 4D-Var searches for the initial state that, when run forward through the model over a window of several hours, best matches all the observations taken in that window, weighted by how much each observation and the previous forecast can be trusted. This requires running the model and its mathematical adjoint repeatedly, and it is one of the most computationally demanding tasks in operational meteorology. Ensemble-based methods, such as the ensemble Kalman filter, achieve similar ends by different routes and at comparable expense. Each cycle at a major centre ingests tens of millions of observations. One by-product of data assimilation turned out to be the single most important ingredient in the AI revolution: reanalysis. A reanalysis takes a modern, fixed version of a forecast model and assimilation system and runs it back over decades of historical observations, producing a consistent record of the atmosphere's state at regular intervals. ECMWF's ERA5, described by Hans Hersbach and colleagues in 2020, covers 1940 to the present at hourly intervals on a grid of roughly 31 kilometres, with 137 levels in the vertical. It is, in effect, a movie of the global atmosphere for more than eighty years, with every frame physically consistent with its neighbours. No other branch of science has a training set quite like it, and nearly every major machine-learning weather model has been trained on it. It is worth pausing on the irony. The learned models that now compete with physics-based forecasting were taught by physics-based forecasting. ERA5 is the output of the very system that the AI models emulate. When a neural network learns to forecast from ERA5, it is learning, in part, the habits of ECMWF's model, filtered through decades of observations. Its best performance is to reproduce that system's analysis faster and with fewer errors at forecast time. Whether it can ever surpass the system that generated its training data, in ways that matter, is a question the later chapters return to. Chaos and the need for many forecasts In 1963 Edward Lorenz showed that a simple model of convection was exquisitely sensitive to its initial conditions: tiny differences in the starting state grew until two forecasts bore no resemblance to each other. In later work he argued that the real atmosphere has a finite limit of predictability for individual weather systems, on the order of two weeks, no matter how good the model or the observations. Small-scale errors grow fastest, and they cascade upward to larger scales. The practical response, developed through the 1980s and made operational at ECMWF and at the US National Meteorological Center in 1992, was the ensemble forecast. Instead of running one forecast, a centre runs many, each starting from a slightly perturbed initial state and often using slightly different model physics. The spread of the ensemble indicates uncertainty: when the members agree, confidence is high; when they diverge, the forecast is uncertain, and the distribution of outcomes tells the user how likely each scenario is. A modern ensemble also reveals risks that a single forecast would hide. If eight of fifty members show a hurricane turning toward a coastline, that is a 16 per cent risk worth preparing for, even if the most likely outcome is a miss. Ensembles are where the cost of physics bites hardest. ECMWF's ensemble runs 50 perturbed members plus a control, each at full resolution. NOAA's Global Ensemble Forecast System runs 31. These numbers are not chosen because fifty is enough; they are chosen because fifty is what the computer can afford. For common outcomes, fifty samples give a reasonable estimate of probability. For rare outcomes, the ones that matter most, they do not. An event with a 1 per cent chance will, on average, appear in half of one member. The tail of the distribution, where the catastrophes live, is precisely the part that a fifty-member ensemble samples worst. Statistics was already in the room It is a common misconception that machine learning entered meteorology from outside in the 2020s. Statistical learning has been part of operational forecasting for half a century, in a role that is worth understanding because it anticipates many of the debates in later chapters. Raw output from a physics-based model is never used quite as it comes. A model grid point represents an average over a box many kilometres wide, with smoothed terrain, so its temperature will be systematically wrong for a weather station in a valley or on a hilltop within that box. The model will also have its own consistent habits: perhaps too warm on clear nights, perhaps too little wind in strong gradients. In 1972 Harry Glahn and Dale Lowry at the US National Weather Service introduced Model Output Statistics, a method that fits regression equations relating model output to observed weather at each station over a training period. Those equations then correct each new forecast. The technique, and its many descendants, became standard in forecast offices worldwide, and it remains part of the chain that converts a model's grid into the temperature on a phone screen. Post-processing of this kind exposed, decades ago, the central tension that now governs learned models. A statistical correction trained on past forecasts works well as long as the model and the climate stay the same. When the model is upgraded, its habits change and the correction must be retrained, which requires a new archive of forecasts. When the weather goes outside the range seen in training, the regression equations extrapolate, sometimes badly. Forecasters learned to be wary of statistically corrected forecasts precisely in unusual conditions, such as an exceptional cold outbreak, when the correction might be pulling the forecast toward what was typical rather than what was coming. Every lesson about generalisation in this book has a precedent here. Ensemble post-processing extended the same ideas to probabilities. Methods developed in the 2000s, such as ensemble model output statistics and Bayesian model averaging, adjust the spread and centre of raw ensembles so that their stated probabilities match observed frequencies. These calibrated outputs are what most weather services actually issue as probability forecasts. Neural networks began to replace the regression equations in post-processing during the late 2010s, with demonstrable gains, before anyone seriously proposed that a network could replace the model itself. The significance is that the physics-based forecast has always been a hybrid: a dynamical model wrapped in layers of statistical correction learned from data. What changed in the 2020s was not the arrival of learning but its promotion from the periphery of the forecast to its core. Climate: the same model, run for a century Climate models are built from the same ingredients as weather models: a dynamical core solving the fluid equations, parameterizations for unresolved processes, and, increasingly, coupled components for ocean, sea ice, land surface, vegetation, atmospheric chemistry and the carbon cycle. A model that includes these interacting pieces is usually called an Earth system model. But the problem they address is different in kind. A weather forecast is an initial-value problem; its skill decays as the initial information is forgotten. A climate projection is mostly a boundary-value problem. It asks how the statistics of weather, averaged over decades, respond to changed conditions such as higher concentrations of carbon dioxide, different aerosol loads, or altered land cover. The initial state matters little; what matters is that the model's internal physics responds correctly to the forcing. That changes the computational arithmetic in two directions at once. Climate runs must cover centuries rather than days, so models are typically run at coarser resolution, commonly around 100 kilometres in the models used for the Intergovernmental Panel on Climate Change assessments. At that resolution, much more of the important physics, including all convection and most of the mountain-induced weather, must be parameterized, so the structural uncertainties grow. And because climate statistics are noisy, especially for extremes, the same scenario must be run many times with different initial conditions to separate the forced signal from natural variability. Large ensembles of this kind, with forty to a hundred members of a single model each spanning more than a century, have become a central tool of climate science, but each one costs a great deal of supercomputer time. The coordinated experiments of the Coupled Model Intercomparison Project, which underpin the IPCC reports, now involve dozens of modelling centres running hundreds of experiments. The resulting archives occupy many petabytes. Yet decision-makers still routinely complain that the outputs are too coarse to use, too few to characterise risk, and too slow to update when new scenarios are needed. Three bottlenecks, one opportunity Put together, the physics-based approach faces three bottlenecks, which map neatly onto the three applications of machine learning in this field. The first is the raw cost of stepping the model forward. A ten-day global forecast at high resolution consumes an hour or more on a large fraction of a national supercomputer. This is where learned emulators of the whole forecast, the subject of the next two chapters, offer the most dramatic savings. The second is the approximate and expensive nature of parameterization. Here machine learning offers the possibility of better representations of clouds, convection and turbulence, learned from high-resolution simulations or observations, and potentially at lower cost. That is the subject of the fourth chapter. The third is resolution and sample size. Because the full model is so expensive, its output is coarse, and its ensembles are small. Machine learning offers ways to add local detail after the fact, known as downscaling, and ways to generate far larger ensembles cheaply. Those are the subjects of the third, fifth, sixth and seventh chapters. None of these opportunities removes the need for physical understanding. Each of them, as we shall see, depends on it in less obvious ways: for training data, for constraints that keep learned models honest, and for the judgment of when a learned answer is likely to be wrong. The price of physics was high because the problem is hard. Machine learning makes certain parts of that problem cheaper. It does not make the problem easier. Chapter 2: Learning to Forecast For most of the history of numerical weather prediction, the idea that a statistical model could forecast global weather better than the equations of motion was treated as faintly absurd. Statistical methods had their place: correcting the systematic biases of physical models, translating grid-box output into station forecasts, predicting seasonal anomalies from ocean temperatures. But the core of the forecast, the evolution of the three-dimensional atmosphere from one day to the next, belonged to physics. Between 2022 and 2024 that assumption collapsed. Understanding why it collapsed so quickly, and what exactly was achieved, is the necessary groundwork for everything else in this book. The shape of the problem At its simplest, a machine-learning weather model is a function. It takes as input the state of the global atmosphere at one moment, or at two consecutive moments, and produces as output the state a fixed interval later, commonly six hours. To forecast ten days ahead, the model is applied to its own output forty times in succession, a procedure called autoregressive rollout. The "state" is a stack of gridded fields: temperature, wind components, humidity and geopotential at a dozen or more pressure levels, plus surface variables such as two-metre temperature, ten-metre wind and sea-level pressure. At the 0.25-degree resolution of ERA5, roughly 28 kilometres at the equator, there are about a million grid points, and with several dozen variables at each, a single atmospheric state contains tens of millions of numbers. The training procedure is conceptually straightforward. Take pairs of consecutive states from four decades of reanalysis, feed the earlier one into the network, compare its prediction to the later one, and adjust the network's millions of weights to reduce the error. Repeat across hundreds of thousands of examples. Then, typically, fine-tune the network on longer multi-step rollouts so that it learns not to accumulate errors over many applications. What makes this feasible is the combination of three things that did not coexist before about 2020: a long, high-quality, consistent training set in ERA5; neural network architectures capable of handling data on a sphere with long-range interactions; and accelerator hardware, graphics processing units and Google's tensor processing units, capable of training such networks in days or weeks. Early attempts, such as work by Peter Dueben and Peter Bauer at ECMWF in 2018 and the WeatherBench benchmark created by Stephan Rasp and colleagues in 2020, operated at coarse resolution and fell well short of operational skill. They established the problem and its metrics. The jump came from scale and from architecture. The architectures that worked Four lines of development converged in 2022 and 2023, each built around a different way of letting information flow across the globe. FourCastNet, published by Jaideep Pathak and colleagues at NVIDIA and partner institutions in 2022, used Fourier neural operators: layers that transform fields into frequency space, mix them there, and transform them back. This is a natural fit for atmospheric data, which has a strong spectral structure, and a later version reformulated the idea using spherical harmonics, the natural frequencies of a sphere, to avoid distortions near the poles. FourCastNet was the first learned model to operate at the full 0.25-degree resolution of ERA5 and produce a credible week-long forecast in seconds. Pangu-Weather, developed at Huawei Cloud and published in Nature in 2023 by Kaifeng Bi and colleagues, used a three-dimensional transformer adapted to Earth's geometry. Transformers, the architecture behind large language models, use a mechanism called attention to let each part of the input weigh its relationship to every other part. Pangu-Weather divided the atmosphere into three-dimensional patches and applied attention within and across them, and it trained separate models for different time steps (one, three, six and twenty-four hours) so that long forecasts could be built with fewer applications and less error accumulation. Its authors reported that it beat ECMWF's operational high-resolution forecast on standard deterministic scores, the first learned model to make that claim credibly. GraphCast, from Google DeepMind, was published in Science in late 2023 by Remi Lam and colleagues. It took a different route, building on earlier work by Ryan Keisler. The model maps the latitude-longitude grid onto an internal graph, a mesh of nodes spread almost uniformly across the sphere, created by repeatedly subdividing an icosahedron. Messages are passed along the edges of this mesh, with edges at several scales, so that information can travel across a continent in a single step or refine detail locally. The output is mapped back to the grid. GraphCast predicted five surface variables and six atmospheric variables at 37 pressure levels, with a six-hour step, and could produce a ten-day forecast in under a minute on a single tensor processing unit. On the verification scorecard its authors used, it outperformed ECMWF's high-resolution forecast on about 90 per cent of 1,380 combinations of variable, level and lead time. FuXi, from Fudan University and published in 2023 by Lei Chen and colleagues, used a cascade of transformer-based models specialised for successive forecast windows, and pushed useful skill out to fifteen days. ECMWF itself joined the effort. Its Artificial Intelligence Forecasting System, AIFS, described by Simon Lang and colleagues in 2024, uses a graph-based encoder and decoder with a transformer processor in between. It became the first fully operational learned global model at a major centre in February 2025. The comparative picture is set out in Table 1. The important point is less the architectural differences than the convergence: four quite different designs, trained on the same data, reached similar levels of skill within about two years. That suggests the decisive factor was not a particular clever idea but the data, the scale, and the formulation of the problem. Table 1. Leading learned medium-range global weather models. Model Developer Core architecture Published FourCastNet NVIDIA and partners Fourier neural operator 2022 (preprint) Pangu-Weather Huawei Cloud 3D Earth-specific transformer Nature, 2023 GraphCast Google DeepMind Multi-scale graph neural network Science, 2023 FuXi Fudan University Cascaded transformers npj Clim. Atmos. Sci., 2023 AIFS ECMWF Graph encoder-decoder, transformer processor 2024; operational Feb. 2025 How good, exactly? Claims of "beating" a physics-based model need careful reading, because the standard scores reward a particular kind of forecast. The most common measures are root-mean-square error, which penalises large departures from the verifying analysis, and anomaly correlation, which measures whether the forecast captures the pattern of departures from climatology. Both are computed against an analysis, usually ERA5 or a centre's own operational analysis, rather than against raw observations. On these measures the learned models are genuinely excellent, particularly for large-scale fields such as the 500-hectopascal geopotential height that describes the positions of jet streams, troughs and ridges. But root-mean-square error has a property that matters enormously for what follows. When a forecast is uncertain, the way to minimise expected squared error is to predict the average of the possible outcomes. A model trained to minimise this error therefore learns, as lead time grows and uncertainty increases, to blur its forecasts. At day one a learned forecast of a storm system may look crisp; by day seven the same system may look smeared out, its peak winds reduced and its rainfall spread thinly. The forecast is "accurate" in the sense of having low average error, but it does not look like any weather that could actually happen. The atmospheric scientists who first scrutinised these models noticed this at once. Plots of energy spectra, which show how much variability a field contains at each spatial scale, revealed that the learned models lost small-scale detail rapidly with lead time, whereas physics-based models preserved it. A physics-based forecast at day seven is a realistic atmosphere that may be wrong about where things are; a learned deterministic forecast at day seven is closer to an ensemble mean, plausible in its large features and implausibly smooth in its small ones. This blurring helps on the scorecards and hurts in practice. Extreme values are underestimated. Tropical cyclones become weaker and broader than reality with lead time. Heavy rainfall is diluted. Comparing a smoothed learned forecast with a sharp physics-based one on root-mean-square error is not a fair contest, which is why ECMWF and others began to publish comparisons against the physics-based ensemble mean rather than the single high-resolution run. On that fairer comparison the learned deterministic models' advantage shrinks, though they often remain competitive. There is a second subtlety. The learned models were evaluated against ERA5, the same dataset they were trained on, though for later years held out from training. Because ERA5 is produced by an older version of ECMWF's system, it has its own biases, and a model that reproduces those biases scores well against ERA5 without necessarily being closer to the true atmosphere. When verified against independent observations, such as weather balloons and surface stations, learned models still perform well, but the margins are less dramatic. None of this diminishes the achievement. Matching the best physics-based global model on large-scale accuracy, from a standing start, at a thousandth of the cost per forecast, is extraordinary. It does mean that the headline claims of 2023 told only part of the story. Where the cheapness comes from A learned forecast is cheap at the point of use because almost all its computational cost has been moved into training. Training GraphCast reportedly took about four weeks on 32 tensor processing units. Once trained, a forecast is a sequence of matrix multiplications. A physics-based model, by contrast, must solve its equations afresh with every time step of every forecast, and its time step must be short, a few minutes at high resolution, to remain numerically stable. The learned model steps six hours at a time. There is a further, subtler saving. The physics-based model must simulate every fast process explicitly, including sound waves and gravity waves that carry little weather information but constrain the time step. The learned model is trained only to predict the slower, weather-relevant evolution over six hours, so it never pays for those fast processes at all. The cost comparison is not quite like for like, because the learned model needs an initial state, and today that state still comes from a conventional data assimilation system running the physics-based model. NOAA's AI global models, announced in December 2025, used Google DeepMind's GraphCast as a starting point and fine-tuned it on NOAA's own Global Data Assimilation System analyses, precisely so that it would start well from the analyses NOAA actually produces. ECMWF's AIFS v2, implemented in 2026 alongside a new cycle of the physics-based system, was fine-tuned on that cycle's analyses after the first version was found to degrade when initialised from it. These details matter: the learned model is only as good as its fit to the initial conditions it will actually be given. A few groups have gone further and tried to cut the physics out of the initialisation as well. Aardvark Weather, developed at the University of Cambridge and the Alan Turing Institute and published in Nature in 2025 by Anna Allen and colleagues, maps raw observations directly to a forecast without a conventional assimilation step. Its skill was lower than the leading operational systems, but it demonstrated that the whole chain could in principle be learned, with enormous savings. The trouble with rain Of all the quantities a forecast must deliver, precipitation is the one the public cares about most and the one learned global models handle least convincingly. The reasons illuminate how these models work. Temperature and pressure fields are smooth and strongly constrained by the large-scale flow, so their evolution over six hours is highly learnable. Precipitation is intermittent, patchy, and controlled by processes at scales far smaller than a 25-kilometre grid. It is also, in ERA5, not an observation at all. ERA5's rainfall is produced by the forecast model inside the reanalysis, using its parameterized convection, and it inherits that model's biases: rain that is too frequent and too light, and convective peaks that occur at the wrong time of day in parts of the tropics. A learned model trained to reproduce ERA5 rainfall learns those biases faithfully. Some developers left precipitation out of their models entirely in early versions for this reason; others included it but reported its skill with caution. Then there is the blurring problem. Because rainfall is so uncertain at the scale of individual storms, a model trained on squared error learns to spread small amounts of rain broadly rather than placing intense rain anywhere specific. Its forecast of the total amount of rain over a large region may be good; its forecast of the heaviest rainfall at any point is systematically too low. That is the opposite of what a flood forecaster needs, and it is why, as later chapters describe, generative methods that produce sharp realistic rainfall fields have become so important. Improvements are coming from several directions. Training against observation-based rainfall products, such as merged satellite and rain-gauge estimates, rather than reanalysis rainfall, pulls models toward reality rather than toward another model. Probabilistic and generative training addresses the blurring. And at the very shortest ranges, from minutes to a few hours, a separate family of learned models working directly from radar has shown real value. In 2021 Suman Ravuri and colleagues at DeepMind, working with the UK Met Office, published a generative model of radar rainfall in Nature that produced nowcasts up to ninety minutes ahead. In a structured evaluation by more than fifty Met Office forecasters, it was preferred to the competing methods in 89 per cent of cases, because it kept rainfall sharp and realistic rather than letting it blur. A worked example of error growth It helps to see concretely how a learned forecast is assembled and where it goes wrong. Suppose a model with a six-hour step is started from the 00 UTC analysis on a Monday. Its first application produces a state for 06 UTC. That state is not an observation; it contains the model's own small errors, perhaps a slight misplacement of a frontal zone and a slight weakening of the jet. The second application takes this imperfect state as input and produces 12 UTC, adding new errors on top of the inherited ones. By the fortieth application, valid ten days later, the forecast is the product of forty compounded predictions, each made from an input that is further from anything in the training data than the last, because no real atmospheric state ever looked quite like a learned model's day-nine forecast. This is why multi-step fine-tuning matters. If a model is trained only on single steps from real analyses, it never learns how to behave when given its own slightly unrealistic output, and its errors can grow rapidly. By training on sequences of several steps, with the loss computed on each, the model learns to produce outputs that remain good inputs. The price is that it also learns to hedge: when it knows it will be judged on states several days ahead, the safest strategy under squared error is to drift toward the average. The blurring described above is, in part, a direct consequence of the rollout training that keeps learned models stable. Physics-based models face a version of the same issue, since their own forecasts also feed back into themselves, but their equations do not care whether the input resembles past weather. A physically odd input produces physically governed consequences. A learned model given a state unlike its training data produces whatever its weights happen to imply. Over ten days, learned models have proved remarkably well behaved. Over a year, as Chapter 5 explains, most of the early ones were not. What a learned forecast is, and is not The deterministic learned models of 2022 to 2024 established three facts that remain true. First, the large-scale evolution of the atmosphere over days is a learnable mapping. Given enough examples of how the atmosphere has evolved, a sufficiently expressive network can reproduce that evolution to a high standard, without ever being told the equations. This is in some ways the most surprising result of the whole enterprise, and it was by no means guaranteed. Second, the learned models learned real physics in an operational sense. They propagate Rossby waves at the right speed, they produce realistic geostrophic balance between pressure and wind, and in several studies they reproduced known atmospheric responses to perturbations. They are not merely matching patterns from a catalogue. Third, and crucially, a deterministic model trained to minimise average error is not a good tool for predicting the specific, sharp, rare events that cause damage. It is a superb estimator of the expected state. It is not a generator of realistic possible states. The field's response to that limitation, the move from single forecasts to learned ensembles, is the subject of the next chapter, and it is where the real power of cheap computation began to show itself. Chapter 3: From One Forecast to a Thousand The deterministic learned models made forecasts cheap. Their limitation, a tendency to blur toward the average as uncertainty grows, made it clear what that cheapness should be spent on. If a learned model can produce a forecast in a minute, the natural question is not how to produce one better forecast but how to produce many plausible ones, each sharp and realistic, whose spread honestly reflects what is and is not known. This chapter traces how the field made that transition, and why it matters more than any improvement in average skill. Why the average is not enough Consider a forecaster responsible for a coastal city five days before a tropical storm may arrive. A single deterministic forecast tells her one possible track. A blurred learned forecast gives her something like the average track and a weakened, smeared storm. Neither tells her what she most needs to know: the probability that the storm makes landfall within fifty kilometres of the city as a major hurricane. That probability can only come from an ensemble, a set of forecasts that represents the range of outcomes consistent with current knowledge. For an ensemble to be useful, three conditions must hold. Each member should be a realistic weather state, not a smoothed compromise. The members should differ from one another by about as much as reality differs from them, a property known as calibration or reliability. And there should be enough members to estimate the probabilities of the outcomes that matter, including rare ones. Meteorologists check these properties with specific tools. The continuous ranked probability score (CRPS) generalises mean absolute error to probabilistic forecasts: it rewards ensembles that are both close to the truth and appropriately confident. The spread-skill ratio compares the average spread of the ensemble to the average error of its mean; a ratio near one indicates good calibration, below one indicates overconfidence. Rank histograms show whether the verifying observation falls uniformly among the ensemble members, as it should if the ensemble is reliable. A model trained to minimise squared error will fail all these tests if one simply perturbs its inputs, because its outputs are biased toward the mean and lack realistic variability. Three ways to make a learned ensemble Three broad approaches emerged. The first and simplest is to keep a deterministic learned model and generate an ensemble by perturbing its initial conditions, its weights, or both, mimicking what physics-based centres do. The "Huge Ensembles" project led by Ankur Mahesh at Lawrence Berkeley National Laboratory, with NVIDIA collaborators, took this approach with a spherical Fourier neural operator model. It combined bred-vector perturbations of the initial state, a classic technique for finding the fastest-growing errors, with 29 independently trained copies of the network, so that differences between the copies would represent model uncertainty. The resulting ensemble was reasonably calibrated, with overall skill lagging ECMWF's physics-based ensemble by twelve to eighteen hours of lead time. A two-week forecast took about a minute on a single graphics processor. That cheapness allowed the team to generate a summer hindcast of 7,424 members, roughly 150 times larger than an operational ensemble, specifically to study low-probability, high-impact extremes. The second approach is to train the network directly on a probabilistic objective. Instead of minimising squared error, the model is trained to minimise CRPS over a small set of outputs generated with random noise injected into the network. Because CRPS penalises both error and overconfidence, the network learns to produce members that are individually sharp and collectively well spread. ECMWF's AIFS ENS takes this route. It became operational on 1 July 2025, running at about 31 kilometres resolution, coarser than the 9 kilometres of the physics-based ensemble. ECMWF reported that it outperformed the physics-based ensemble on many measures, with gains of up to 20 per cent for surface temperature, while running more than ten times faster and using about a thousandth of the energy. Google DeepMind's WeatherNext 2, announced in November 2025, uses a related idea it calls a Functional Generative Network, which injects noise into the model architecture itself. Google reported that it improved on its predecessor on 99.9 per cent of variables and lead times out to 15 days, and could generate hundreds of scenarios in under a minute on one tensor processing unit. The third approach uses diffusion models, the same family of generative methods behind modern image generators. A diffusion model learns to reverse a process of gradually adding noise: starting from pure noise, it repeatedly removes a little, guided by the conditioning information, until it produces a realistic sample. For weather, the conditioning information is the current atmospheric state, and the sample is the state twelve hours later. Each run from different noise yields a different plausible future. GenCast, published in Nature by Ilan Price and colleagues at Google DeepMind (preprint 2023, journal 2025), was the landmark here. Run at 0.25 degrees with 12-hour steps, it was evaluated against ECMWF's physics-based ensemble on 1,320 combinations of variable, level and lead time and outperformed it on 97.2 per cent of them. A fifteen-day ensemble member took about eight minutes on a single tensor processing unit, and members could be run in parallel. Diffusion models produce exceptionally sharp, realistic samples, because they are trained to reproduce the full distribution of the data rather than its average. Their drawback is cost: generating each sample requires many passes through the network. The probabilistic-loss approaches are cheaper per member, which explains why they have tended to reach operations first. What the larger ensembles bought Cheap ensembles change what can be asked of a forecast. The obvious gain is better estimates of probability. In a fifty-member ensemble, the smallest nonzero probability is 2 per cent; anything rarer is either unrepresented or represented by a single lucky member. Statistics built on a handful of members are noisy. With several hundred or several thousand members, the probability of a one-in-a-hundred outcome can be estimated directly, and its uncertainty quantified. For decisions that hinge on rare events, such as whether to pre-position emergency supplies, pay out a parametric insurance contract, or reserve generating capacity for a cold snap, this is the difference between guessing and knowing. A second gain is the ability to combine. NOAA's suite, deployed in December 2025, includes not only a 31-member AI ensemble but also a 62-member hybrid "grand ensemble" that pools those members with the 31 members of its physics-based Global Ensemble Forecast System. Combining ensembles built on different principles is an old idea in meteorology. Multi-model ensembles are often better calibrated than any single model, because different models make different mistakes. A learned ensemble and a physics-based ensemble are about as different as two forecast systems can be, which is exactly why pooling them is attractive. NOAA reported that its AI ensemble alone used about 9 per cent of the computing of the physics-based ensemble while extending skill by eighteen to twenty-four hours in early tests. A third gain is in tropical cyclones, where ensembles have always been both vital and underpowered. In June 2025 Google DeepMind and Google Research launched Weather Lab, a public site showing live predictions from an experimental cyclone model that generates fifty scenarios out to fifteen days, predicting formation, track, intensity, size and shape. It was trained on reanalysis together with a database of nearly 5,000 observed cyclones from the previous 45 years. In evaluations on 2023 and 2024 Atlantic and East Pacific storms, its five-day track predictions were on average 140 kilometres closer to the observed position than those of ECMWF's physics-based ensemble, and its intensity errors were lower than those of NOAA's high-resolution hurricane model, HAFS. The National Hurricane Center began viewing its predictions in real time during the 2025 season. The season-long verification by James Franklin, a retired NHC branch chief, found that this model provided the best guidance for both track and intensity among all the models available that year. That is a notable result, because intensity forecasting is where physics-based models have historically struggled most. Intensity depends on processes in the storm's inner core, a few tens of kilometres across, that global models cannot resolve. The fact that a model trained partly on a historical cyclone database could beat dedicated high-resolution physics models on intensity in a full season suggests that the database contains statistical regularities, relationships between environment and intensity change, that the physics-based models were not fully exploiting. Turning probabilities into decisions Ensembles are valuable only if their probabilities lead to better decisions, and meteorologists have a simple framework for thinking about that, known as the cost-loss model. Suppose a user can take a protective action at a cost C, and if the adverse weather occurs without protection the user suffers a loss L. If the probability of the event is p, the expected loss without protection is pL and the cost of protecting is C. Protecting is worthwhile whenever p exceeds the ratio C/L. Consider a road authority deciding whether to send out gritting lorries ahead of a possible freeze. Gritting costs a known amount; an ungritted freeze brings accidents and closures costing far more. If gritting costs one tenth of the loss it prevents, the authority should grit whenever the probability of ice exceeds 10 per cent. A different user facing the same forecast, say an event organiser for whom cancelling costs half of what a frozen event would lose, should act only above 50 per cent. A single deterministic forecast, which effectively says either yes or no, serves neither user well: it will be too cautious for one and not cautious enough for the other. A calibrated probability serves both. David Richardson at ECMWF used this framework in 2000 to show that the economic value of an ensemble forecast exceeds that of a single forecast of the same or even higher accuracy, for most users, and that the advantage is greatest for users whose decisions turn on rare events with large losses. This is the formal version of the argument of this book. It also clarifies what larger ensembles add. For a user with a cost-loss ratio of one in twenty, the critical probability is 5 per cent. A fifty-member ensemble can represent that only as two or three members out of fifty, a very noisy estimate. A thousand-member ensemble estimates it with fifty members, much more reliably. For users protecting against rare, costly outcomes, such as electricity grid operators planning for wind droughts or emergency managers deciding on evacuations, cheap large ensembles convert directly into better decisions. The cost-loss model is simplified, of course. Real decisions involve sequences of actions, lead-time trade-offs, and losses that depend on the severity of the event, not just its occurrence. But its central insight holds: the value of a forecast to a decision-maker depends on the quality of its probabilities at the threshold that matters to that decision-maker, and for high-stakes decisions that threshold is usually in the tail. The limits of sampling None of this means that a large learned ensemble automatically captures everything uncertain about the future. Several limits deserve attention. An ensemble can only sample the uncertainty its generating model represents. If a learned model has a systematic bias, for example in how quickly heatwaves break down, every member will share it, and adding more members will only produce a more confident wrong answer. The Huge Ensembles authors found their system reliable for warm temperature extremes but overconfident at long lead times for high-probability wind and cold extremes. Size does not cure miscalibration. The independence of members is also a modelling choice. Physics-based ensembles combine perturbations to the initial state with stochastic perturbations to the physics. Learned ensembles use a variety of substitutes, such as noise injected into network layers or multiple trained copies of a model. How well these represent the true sources of forecast uncertainty is still an open research question. Calibration on average, which is what CRPS rewards, does not guarantee that the ensemble spreads in the right directions on any particular day. Resolution remains a constraint. AIFS ENS runs at 31 kilometres, coarser than the physics-based ensemble it now often beats. For large-scale patterns this matters little; for local extremes such as convective rainfall or mountain winds, it matters a great deal. Learned ensembles at kilometre scale are an active research area, and much of the current effort in downscaling, the subject of Chapter 7, is in effect an attempt to bridge this gap. Finally, an ensemble's tails can be no better than its training data's tails. A diffusion model that has learned the distribution of weather from 1979 to 2018 will generate samples from something like that distribution. It can recombine familiar ingredients in unfamiliar ways, but its ability to generate states far outside anything in the training data is limited. This is the core problem of the sixth chapter, and it becomes the core problem of climate emulation in the fifth. Beyond two weeks Lorenz's limit of roughly two weeks applies to individual weather systems, not to all useful prediction. Beyond it lies the subseasonal-to-seasonal range, from about two weeks to a few months, where skill comes not from the initial state of the atmosphere but from slower components that the atmosphere responds to: sea-surface temperatures, soil moisture, snow cover, sea ice, the stratosphere, and organised tropical patterns such as the Madden-Julian Oscillation, a slow eastward-moving region of enhanced tropical rainfall that modulates weather across the globe on timescales of thirty to sixty days. At these ranges, forecasting is inherently probabilistic. Nobody can say whether it will rain in Paris on a particular day five weeks ahead, but it can sometimes be said that the coming month is likely to be warmer than normal across western Europe, or that the odds of a cold outbreak in eastern North America are elevated after a sudden warming of the stratosphere. The signals are weak relative to the noise, which means that very large ensembles are needed to detect them reliably. This is fertile ground for cheap learned ensembles. Progress is real but less mature than in the medium range. Several learned models, including ECMWF's AIFS and Google's models, have been extended to subseasonal lead times, and learned ensembles have shown competitive skill in some measures. The challenge is that subseasonal skill depends heavily on the coupling between atmosphere, ocean and land, and most learned weather models either omit the ocean and land or treat them crudely. They also depend on the faithful representation of slow modes such as the Madden-Julian Oscillation and stratosphere-troposphere coupling, which, as Chapter 5 shows, learned models capture unevenly. Hybrid approaches, and learned models with explicit ocean components, are the likely route to further progress. Uncertainty as the product The deeper lesson of this chapter is a change in what a forecast is understood to be. For most of the twentieth century, a forecast was a single prediction, and its quality was judged by how close it came. The ensemble era redefined the forecast as a probability distribution, but cost constrained that distribution to a few dozen samples. Learned models remove most of that constraint. That shift places new demands on users. A forecast expressed as a thousand possible futures is useless unless someone can turn it into decisions, and that requires knowing what level of risk justifies what action. It also places demands on producers, who must demonstrate not only that their average forecast is accurate but that their stated probabilities are honest, including for the events that occur once a decade. The field has learned to measure the first well. It is still learning to measure the second, and the evaluation of extremes, taken up later in this book, remains the decisive test. Chapter 4: Learning What the Grid Cannot See The learned global forecast models of the previous two chapters replace the entire physics-based model with a network. There is another way to use machine learning, older in conception and arguably more important for climate: keep the physics-based model, with its equations of motion and its conservation laws, and replace only the parts that were never really physics in the first place. Those parts are the parameterizations, the simplified schemes that stand in for clouds, convection, turbulence and radiation. This chapter examines that strategy, its surprising difficulties, and the hybrid models that have grown out of it. Why parameterization is the obvious target Recall from Chapter 1 that a climate model's grid, at around 100 kilometres, cannot resolve thunderstorms, low clouds, or most of the turbulence that moves heat and moisture vertically. Parameterizations estimate what those unresolved processes would do in aggregate. They are the main source of structural uncertainty in climate projections. The spread of estimates for equilibrium climate sensitivity, the long-term warming from a doubling of carbon dioxide, which the IPCC's Sixth Assessment Report placed in a likely range of 2.5 to 4 degrees Celsius, owes much to differences in how models represent low clouds over the subtropical oceans. There is a standard way to know what the right answer should be. Models that run at very high resolution, a few kilometres or finer, resolve deep convection explicitly and need far fewer assumptions. Large-eddy simulations, at tens of metres, resolve the turbulent eddies inside clouds. These simulations are far too expensive to run globally for centuries, but they can be run for limited periods or domains. In principle, they tell us exactly what the coarse model's parameterization ought to output for any given coarse-grained state. That is a supervised learning problem: inputs are the coarse-grained profiles of temperature, humidity and wind in a column, outputs are the heating, moistening and momentum tendencies that the resolved small-scale processes produced. If a neural network can learn that mapping, it can be dropped into the coarse model as a new parameterization, carrying high-resolution knowledge into a cheap model. The first experiments, and the instability problem The idea was tested seriously from 2018. Stephan Rasp, Michael Pritchard and Pierre Gentine, in a paper in the Proceedings of the National Academy of Sciences, trained a neural network on output from a "superparameterized" climate model, in which every coarse grid column contains its own small embedded cloud-resolving model. The network learned to reproduce the embedded models' effects and, when coupled into the host model, produced a multi-year simulation with a realistic climate at a fraction of the cost. In the same year Noah Brenowitz and Christopher Bretherton, working with coarse-grained output from a near-global cloud-resolving simulation, reported similar successes. Both groups also met the problem that has shaped the field ever since: instability. A network that performs beautifully offline, predicting tendencies from held-out data with low error, can drive the coupled model to blow up within days once it is plugged in. The reason is feedback. In the coupled model, the network's outputs change the state, which becomes its next input. Small errors in states the network was never trained on push the model further into unfamiliar territory, where its outputs can be wildly wrong, and the errors compound. A network that has never seen a column so moist, or so unstable, may produce an absurd tendency that makes the column even more extreme. Physics-based schemes, for all their faults, are built from relationships that remain sensible across a wide range of states; a neural network offers no such guarantee. Rasp and colleagues found that different training runs of the same network could differ dramatically in their online stability despite near-identical offline errors. Brenowitz and Bretherton traced instabilities to spurious sensitivities of the network to particular inputs, such as humidity high in the atmosphere, and showed that removing those inputs or penalising unstable behaviour helped. Janni Yuval and Paul O'Gorman, in Nature Communications in 2020, used random forests rather than neural networks. A random forest predicts by averaging stored training outputs, so it cannot produce values outside the range it has seen, and their scheme ran stably and conserved energy. The trade-off was flexibility: random forests generalise less smoothly. The broader lesson was that offline accuracy, the metric that machine learning naturally optimises, is not the metric that matters. What matters is the behaviour of the coupled system over many years. That gap between offline and online performance has driven much of the work since. Building the physics back in Several strategies have emerged to close the gap. One is to impose physical constraints directly. Tom Beucler and colleagues showed in 2021, in Physical Review Letters, that neural networks could be built to satisfy conservation of energy and mass exactly, either by adding penalty terms to the loss or by constructing output layers that enforce the constraints algebraically. A parameterization that cannot create or destroy energy is less likely to drive a climate model into a runaway state. Beucler and colleagues later showed that rescaling inputs and outputs into physically motivated, climate-invariant forms, for example using relative rather than specific humidity, greatly improved a network's ability to generalise to warmer climates than those it was trained on. This matters enormously for climate projection, where the whole point is to simulate conditions the training data never contained. A second strategy is to train online. Rather than fitting a network to tendencies and hoping it behaves when coupled, one can train the network while it runs inside the model, adjusting its weights to keep the coupled trajectories close to the reference. This requires the whole model, or at least the parts that interact with the network, to be differentiable, so that gradients can flow back through many time steps. Writing climate models in differentiable frameworks such as JAX is a substantial engineering effort, but it is precisely what made the hybrid model described below possible. A third strategy is to learn interpretable equations rather than black-box networks. Laure Zanna and Thomas Bolton, in Geophysical Research Letters in 2020, used an equation-discovery method to find compact mathematical expressions for the effects of ocean mesoscale eddies on the resolved flow. The resulting expressions can be read, checked against physical understanding, and implemented in existing ocean models without the opacity of a neural network. This line of work has been pursued by the M2LInES project, an international collaboration on machine learning for climate model parameterizations. A fourth strategy is better data. The ClimSim dataset, released in 2023 by Sungduk Yu and a large team, provided a standardised, very large training set from a multiscale climate model, with defined benchmark tasks for learned parameterizations of clouds and convection. Like WeatherBench for global forecasting, it gave the community a common target and made results comparable. Its paper received an outstanding paper award in the datasets and benchmarks track at the NeurIPS conference that year. NeuralGCM: the hybrid that worked The most influential demonstration that a hybrid can rival both physics-based and fully learned models came from Google Research. NeuralGCM, described by Dmitrii Kochkov and colleagues in Nature in 2024, combines a differentiable dynamical core, solving the equations of large-scale atmospheric motion in the conventional way, with a neural network that replaces all the parameterized physics in each column. The network is trained end to end: the full model is run forward over several days, its trajectory compared with ERA5, and the gradient of the error propagated back through the solver and the network together. This is online training at full scale. The result was a model that could be run as a weather forecast or a climate model. As a deterministic forecast at 0.7 degrees, its skill over the first few days was competitive with the leading learned models and with ECMWF's physics-based system. As an ensemble at 1.4 degrees, it was competitive with ECMWF's physics-based ensemble on many measures out to fifteen days. And, unlike the fully learned weather models, it could be run for decades. In atmosphere-only simulations driven by observed sea-surface temperatures, it reproduced the observed rise in global mean temperature from 1980 to 2020, the climatology of tropical cyclones, and other features of the real climate, with accuracy comparable to or better than conventional climate models at similar resolution. Its authors estimated large computational savings relative to physics-based models of comparable skill, because the learned physics could be evaluated cheaply and the dynamical core could run at coarse resolution. NeuralGCM's success had a clear explanation in terms of the argument of this book. The dynamical core keeps the large-scale motion bound to conservation laws and the geometry of a rotating sphere, things that do not change as the climate changes. The learned component handles exactly the processes that are poorly represented by traditional schemes and that can be learned from data. The division of labour puts the physics where it is reliable and the learning where it is needed. Later work extended the approach. A version trained to reproduce satellite-based precipitation estimates, rather than reanalysis precipitation, was reported in 2025 to improve the intensity distribution and daily cycle of rainfall, long-standing weaknesses of conventional models. That is a reminder that the training target need not be another model's output; with the right architecture, a hybrid can be pulled toward observations directly. Learning from what is actually observed Most of the work described so far learns from models: superparameterized simulations, cloud-resolving runs, or reanalyses produced by physics-based systems. An alternative, harder but potentially more powerful, is to learn directly from observations of the real climate. The motivation is straightforward. A learned parameterization trained on a cloud-resolving model can be no better than that model, and cloud-resolving models still parameterize the microphysics of droplets and ice crystals, still have turbulence schemes, and still disagree with each other about important quantities such as the amount of low cloud over the subtropical oceans. Observations have no such ceiling. Satellites measure the radiation leaving the top of the atmosphere, the distribution of clouds, and increasingly the vertical structure of clouds and precipitation. The obstacle is that observations rarely measure the quantities a parameterization must produce. Nobody observes the convective heating rate in a 100-kilometre column directly. What can be observed are its consequences: the resulting rainfall, cloud cover and radiation. Learning a parameterization from consequences requires running the whole model, comparing its simulated observables with real ones, and adjusting the parameterization to reduce the mismatch. That is an inverse problem, and it is computationally formidable. Two developments are making it tractable. One is differentiable models, such as NeuralGCM, in which gradients can be computed through the whole simulation, allowing the learned components to be adjusted directly to match observed quantities, as in the version trained on satellite-based precipitation. The other is a family of methods developed by Tapio Schneider and colleagues in the Climate Modeling Alliance (CliMA), based at the California Institute of Technology. Their approach uses ensemble Kalman methods, borrowed from data assimilation, to calibrate the parameters of physically based but flexible parameterizations against climate statistics from observations and high-resolution simulations, without requiring gradients. The philosophy is deliberately conservative: keep parameterizations grounded in physical theory, with a modest number of learnable parameters, so that they are more likely to remain valid in a changed climate, and use data to constrain those parameters as tightly as possible. CliMA's argument is that learning from climate statistics, averaged over long periods, rather than from instantaneous tendencies, targets the properties that actually matter for projections. Between the black-box neural network trained on a cloud-resolving model and the physically structured scheme calibrated against observations lies a wide spectrum. Where on that spectrum the best climate models will eventually sit is not yet clear. What is clear is that the choice of training target, model output or observation, is as consequential as the choice of architecture. Problems that remain Hybrid models have not yet replaced the parameterizations in any of the major climate models used for IPCC assessments, and several obstacles explain why. Generalisation remains the central concern. A learned parameterization trained on the present climate must give correct answers in a warmer one. Climate-invariant rescaling helps, but there is no proof that a network will extrapolate correctly, and the most consequential processes, such as the response of low clouds to warming, are exactly those where observational constraints are weakest. A hybrid can match the past well and still have an unknown, perhaps unrealistic, sensitivity to carbon dioxide. Coupling to the rest of the Earth system is unfinished. NeuralGCM's published climate results are atmosphere-only, driven by prescribed ocean temperatures. A full climate model must also simulate the ocean, sea ice, land, and their interactions with the atmosphere, and each coupling introduces new opportunities for drift. Tuning and interpretation are harder. Climate modellers spend years calibrating parameterizations against a range of observational targets, and they understand, at least roughly, why a model behaves as it does. A learned parameterization is harder to adjust by hand and harder to diagnose when something goes wrong. The community's growing interest in explainable machine learning, and in equation discovery, is partly a response to this. Finally, institutional inertia is real and partly justified. The major climate models are the product of decades of development by large teams, and their behaviour under forcing has been scrutinised through multiple rounds of intercomparison. Replacing a central component with a network, however accurate, requires a level of evidence that the field has only begun to accumulate. The quiet importance of the hybrid It would be easy to see the hybrid approach as a cautious halfway house between physics and learning, overtaken by the dramatic success of fully learned models. That would be a mistake. For weather prediction over days, the fully learned models are compelling, because the atmosphere's near future resembles its recent past and the training data covers the relevant range. For climate projection, where the future is by design different from the past, the hybrid's commitment to physical structure is not caution but necessity. The next chapter takes up that difference directly, by asking what happens when fully learned models are asked to simulate climate. Chapter 5: Emulating Climate A weather forecast is a question about the next two weeks. A climate projection is a question about the next century. The learned models that transformed weather prediction were built to answer the first kind of question, and it was natural to ask whether they could be stretched to answer the second. If a network can step the atmosphere forward by six hours in a fraction of a second, why not step it forward by six hours a few hundred thousand times and produce a century of climate? This chapter examines that ambition, the real progress made toward it, and the reason it is fundamentally harder than weather. Why climate is not long weather Two differences separate climate emulation from weather forecasting. The first is stability. A weather model need only stay realistic for fifteen days. A climate model must run for decades or centuries without drifting into an unphysical state. The deterministic learned weather models of 2022 and 2023 generally could not do this. Run for months, many of them became unstable: small-scale noise grew without limit, or the fields smoothed into a featureless state, or the global energy and water budgets slowly drifted. GraphCast and Pangu-Weather were never designed for long rollouts, and their creators did not claim otherwise. The errors that are harmless at day ten accumulate over a year. The second difference is more fundamental. Weather forecasting asks a model to interpolate: the next fortnight of weather almost always resembles weather that has occurred before, under a climate the training data represents. Climate projection asks a model to extrapolate: to simulate the statistics of an atmosphere under carbon dioxide concentrations, ocean temperatures and ice extents that have never existed in the observational record. That is exactly where a learned model is weakest, because it has no direct experience of the target conditions and no guarantee that the patterns it has learned still apply. Climate science has a long-standing way to frame this. The most important property of a climate model is not how well it reproduces the present but how correctly it responds to forcing, the change in the planet's energy balance from greenhouse gases, aerosols, solar variability or volcanoes. A climate model must represent the feedbacks that amplify or dampen that response, from water vapour and clouds to sea ice and the carbon cycle. A learned model trained only on the recent past sees one realisation of a climate that has warmed by roughly 1.2 to 1.3 degrees since pre-industrial times. The question is whether it can infer from that the system's response to two or three more degrees. The first stable emulators The past three years have nonetheless produced learned atmosphere models that run stably for centuries and reproduce the present climate with impressive fidelity. The main examples differ in ambition and method, as Table 2 summarises. Table 2. Representative learned and hybrid climate emulators. Model Developer Approach Coupled components Reported stability ACE2 Ai2 with NOAA GFDL Fully learned (spherical neural operator) Atmosphere; ocean prescribed 1,000-year runs NeuralGCM Google Research Physics core plus learned physics Atmosphere; ocean prescribed Multi-decade runs DLESyM University of Washington Coupled learned atmosphere and ocean Atmosphere and ocean 1,000-year runs ACE2, the Ai2 Climate Emulator, developed by Oliver Watt-Meyer and colleagues at the Allen Institute for AI with the Geophysical Fluid Dynamics Laboratory, and published in npj Climate and Atmospheric Science in 2025, is the clearest case of a fully learned atmosphere built for climate. It operates on a 1-degree grid with only eight vertical layers and a six-hour step, much coarser than the weather models, and it takes as inputs the boundary conditions a climate model needs: sea-surface temperature, sea-ice fraction, global-mean carbon dioxide and incoming solar radiation. Two versions exist, one trained on ERA5 and one on a simulation from GFDL's physics-based SHiELD model. ACE2 hard-codes two conservation laws, exactly conserving global dry air mass and atmospheric moisture, though not total energy. It runs about 1,500 simulated years per day of computer time, and per simulated year it is about 100 times faster and roughly 700 times less energy-intensive than SHiELD at the same horizontal resolution. Run for a millennium under repeating climatological forcing, it showed no instability and no long-term moisture drift. It spontaneously produces tropical cyclones, the Madden-Julian Oscillation, sudden stratospheric warmings, and a realistic atmospheric response to El Niño, none of which it was explicitly told to produce. NeuralGCM, described in the previous chapter, reached the same destination by a different road, keeping a physical dynamical core and learning only the column physics. Its atmosphere-only simulations reproduced the observed warming trend from 1980 to 2020 when driven by observed sea-surface temperatures. DLESyM, the Deep Learning Earth System Model developed by Nathaniel Cresswell-Clay, Dale Durran and colleagues at the University of Washington and published in AGU Advances in 2025, went a step further by coupling a learned atmosphere to a learned ocean. The ocean component updates every four days and the atmosphere every twelve hours. The team reported that it could simulate a thousand years of the present climate in about twelve hours on a single processor, a task they estimated would take about ninety days on a state-of-the-art supercomputer with a conventional model. Compared with four of the physics-based models used in the most recent Coupled Model Intercomparison Project, it simulated tropical cyclones and the seasonal cycle of the Indian summer monsoon better, and it captured mid-latitude variability and atmospheric blocking at least as well. These are genuine scientific achievements. They show that the statistics of the present climate, including its modes of variability, can be learned and reproduced with long-term stability and at negligible cost. The forcing problem But reproducing the present climate is the easy part. The decisive test is the response to forcing, and here the evidence is more mixed. The ACE2 authors were candid about their model's limitations. Its sensitivity to carbon dioxide was not fully realistic: with carbon dioxide held fixed but ocean temperatures following history, the model lost much of the observed near-surface warming over land. In other words, much of what ACE2 knows about warming comes from the prescribed ocean, not from the greenhouse gas itself. Its authors stated that, being trained on roughly the last eighty years, it should not be expected to handle strong climate change, such as a doubling of carbon dioxide. They also noted that it does not reproduce the quasi-biennial oscillation of stratospheric winds with the correct period, and that it lacks ocean and sea-ice components, which they described as a basic requirement for a useful climate emulator. Independent benchmarking by Ian Baxter, Hamid Pahlavan, Pedram Hassanzadeh, Katharine Rucker and Tiffany Shaw at the University of Chicago, posted in late 2025, compared ACE2 and NeuralGCM against observations and physics-based models. Both learned models captured large-scale tropical waves and the interactions between mid-latitude eddies and the mean flow. Both struggled with slower phenomena: the roughly 28-month quasi-biennial oscillation and the roughly 150-day propagation of the Southern Annular Mode. A companion study by the same group found that both models captured regional temperature trends, including the amplified warming of the Arctic, and that ACE2 matched or beat physics-based models on several regional thermodynamic trends. Yet neither learned model captured the observed trend in heat extremes over the south-western United States, and both missed drying trends in arid regions, although on the latter they generally did better than the physics-based comparison models. Those are precisely the kind of regional changes that matter for adaptation. The most direct test of how learned weather models behave in other climates came from Thomas Rackow and colleagues at ECMWF in 2024. They initialised several leading learned weather models from states taken from a physics-based climate model's simulations of pre-industrial, present-day, and much warmer future climates, roughly 2.9 degrees warmer than today. The good news was that the learned models still produced skilful weather forecasts in all three climates, suggesting that the short-timescale dynamics of weather do not change fundamentally as the climate warms. The bad news was that two of the models showed a global-mean cold bias in the warmer climate, drifting back toward the present-day conditions they had been trained on, and two of three warmed the pre-industrial state. A learned model, in effect, has a gravitational pull toward its training climate. Ways forward The research community has pursued several routes around the forcing problem. One is to train on simulations rather than only on observations. A physics-based climate model can be run under any forcing scenario, including quadrupled carbon dioxide or a world without aerosols. Training a learned emulator on such runs exposes it to a wider range of climates than history contains. ACE2-SHiELD is a step in this direction. The trade-off is that the emulator inherits the physics-based model's biases, including its climate sensitivity; it becomes a fast copy of that model, not an independent estimate. For many purposes that is exactly what is wanted. A fast, faithful copy of a trusted model can generate the large ensembles and scenario variations that the original cannot afford. A second route is to build in physical structure that carries over to new climates. The hybrid approach of NeuralGCM, with its physical dynamical core, is one form of this. Climate-invariant rescaling of the kind Tom Beucler and colleagues developed for parameterizations is another. Hard conservation constraints, as in ACE2, are a third. The underlying principle is to reduce the burden on learning by assigning to physics everything that physics already knows and that does not change with climate. A third route abandons step-by-step simulation altogether and emulates the climate response directly. Earlier statistical emulators, sometimes called pattern scaling, assumed that local changes scale linearly with global mean temperature. Machine-learning versions extend this to non-linear responses. ClimateBench, introduced by Duncan Watson-Parris and colleagues in 2022, set a benchmark task: given emissions of carbon dioxide, methane, sulphur dioxide and black carbon, predict maps of annual temperature, precipitation and extremes from a physics-based climate model's response. Interestingly, the ClimateBench authors and later analyses found that relatively simple methods, such as Gaussian processes and even linear pattern scaling, performed competitively with deep networks for temperature, because the forced temperature response really is close to linear in many regions. Deep learning helped more for precipitation and extremes. That is a useful corrective to the assumption that the most complex model is always best. A fourth route is coupling. Most current learned climate emulators prescribe ocean temperatures, which means much of the climate's response to forcing is supplied rather than simulated. Coupled learned models, like DLESyM, and learned ocean emulators developed in parallel are necessary steps toward a learned model that could genuinely project future climate rather than only reproduce the atmosphere's response to a given ocean. A thought experiment on extrapolation The forcing problem can be made concrete with a simple thought experiment. Imagine training a learned atmosphere on the period 1979 to 2020, during which global mean surface temperature rose by roughly 0.8 degrees and atmospheric carbon dioxide rose from about 337 to about 414 parts per million. The training data contains one trajectory of a slowly warming climate, overlaid with a great deal of natural variability: El Niño and La Niña events, volcanic eruptions such as Pinatubo in 1991, decadal swings in ocean circulation. From that record the model must somehow infer how the atmosphere responds to carbon dioxide. But in the record, carbon dioxide rises almost monotonically with time, alongside many other things that also changed with time: aerosols, ozone, land use, the observing system itself. A learned model cannot easily separate the effect of carbon dioxide from everything else that trended in the same direction, any more than a regression can separate two variables that are almost perfectly correlated. It will learn some combination, and there is no guarantee that combination is right. ACE2's lack of near-surface warming with fixed carbon dioxide is exactly the symptom one would expect: much of what the model learned about warming was carried by the ocean temperatures it was given, which rose at the same time. Now ask the model to simulate a world with 560 parts per million. That value lies far outside the training range. A physics-based model computes the extra absorption of infrared radiation from the known spectroscopy of carbon dioxide and propagates the consequences. The learned model has nothing comparable. Whatever it does is an extrapolation of a relationship it was never able to identify cleanly in the first place. The thought experiment explains why the most credible paths forward involve training on physics-based simulations with independently varied forcings, or building the radiative effect of carbon dioxide into the model structure. It also explains why learned emulators can be excellent at reproducing internal variability, which the training record samples thoroughly, while remaining uncertain on forced change, which it samples only once. What climate emulators are good for now Even with these limitations, learned climate emulators have immediate uses that do not depend on extrapolating far beyond the training climate. They can generate large ensembles of the present and near-future climate cheaply, sampling internal variability far more thoroughly than physics-based large ensembles can. That helps answer questions such as how often a particular combination of drought and heat might occur by chance in today's climate, which is the basis of extreme event attribution. They can explore scenarios quickly. A policymaker considering different emissions pathways, or a scientist studying the effect of a volcanic eruption or a proposed intervention, can run hundreds of variations with an emulator trained on a trusted physics-based model, then confirm the most interesting with the original model. They can serve as fast components inside larger systems, such as integrated assessment models that combine climate with economics and need climate responses computed thousands of times. And they can be benchmarks against which physics-based models are judged. If a learned model trained only on observations reproduces a phenomenon better than a physics-based model, the comparison points to deficiencies in the physics worth investigating. The honest position The most accurate summary as of late 2026 is that learned models can reproduce the present climate stably, cheaply and in many respects as well as physics-based models, but their projections of a substantially different climate are not yet trustworthy on their own. The obstacle is not engineering but epistemology. A model learned from the past contains, in principle, no information about how the system responds to conditions outside the past, unless that information has been built into its structure or supplied by training on physics-based simulations that already contain it. This does not make learned climate emulators unimportant. It defines their role. They are powerful amplifiers of existing knowledge: they take what a trusted physics-based model or the observational record already contains and make it available at a scale and speed that was previously impossible. They are not yet independent sources of knowledge about future climate. The same distinction, between amplifying the known and predicting the unknown, turns out to govern the forecasting of extreme weather, which is the subject of the next chapter. Chapter 6: The Problem of the Unprecedented Extreme weather is the reason forecasting matters. Average conditions rarely kill anyone or bankrupt anyone; heatwaves, floods, windstorms and tropical cyclones do. It is therefore in the forecasting of extremes that any new method must ultimately prove itself, and it is here that the strengths and weaknesses of learned models are most sharply exposed. This chapter sets out what the evidence now shows. The short version is that learned models are often excellent at forecasting extremes of a kind they have seen many times before, sometimes better than physics-based models, and systematically poor at forecasting extremes more severe than anything in their training data. Unfortunately, a warming climate keeps producing exactly those. Rare by definition From a statistical point of view, an extreme is an observation in the tail of a distribution. A machine-learning model trained on decades of data encounters the bulk of the distribution millions of times and the tails comparatively rarely. A model trained to minimise average error will, unless deliberately designed otherwise, devote most of its capacity to getting common conditions right. That alone would bias it against extremes. There is a more fundamental issue. Many extremes are defined not just as rare but as beyond previous experience: record-breaking events. In a stationary climate, records become progressively rarer over time, because each new record must exceed all previous values. In a warming climate, records keep falling, and some are broken by startling margins. The Pacific Northwest heatwave of late June 2021 is the canonical example. Lytton, British Columbia, reached 49.6 degrees Celsius on 29 June, beating the previous Canadian national record by nearly five degrees. The town burned in a wildfire the next day. Attribution studies by the World Weather Attribution group concluded that the event would have been virtually impossible without human-caused climate change. No model trained on the climate record before 2021 had ever seen a temperature like that in that place. For a physics-based model, this matters less, because its equations do not know about records; they compute the consequences of a strong, persistent ridge of high pressure, subsidence, dry soils and local heating, and the temperature follows. For a learned model, a value far outside the training range is precisely what it is least equipped to produce. What the case studies show The most useful evidence comes from careful evaluations that compare learned and physics-based models on specific extremes, using the same initial conditions where possible. Table 3 summarises four of the most informative studies. Table 3. Selected evaluations of learned models on extreme events. Study Events examined Main finding Pasche et al., AIES, 2025 2021 Pacific Northwest heat, 2023 South Asian humid heat, 2021 North American winter storm Mixed: learned models worse on aggregated heatwave error, better on the winter storm Sun et al., PNAS, 2025 Category 5 tropical cyclones, 2018 to 2023 Model trained without Category 3 to 5 storms could not forecast them Zhang, Engelke and Fischer, Science Advances, 2026 Record heat, cold and wind, 2018 and 2020 Physics-based HRES consistently better on records Franklin, NHC 2025 season review 2025 Atlantic tropical cyclones Google DeepMind ensemble best guidance for track and intensity Olivier Pasche, Jonathan Wider, Zhongwei Zhang, Jakob Zscheischler and Sebastian Engelke, writing in the American Meteorological Society's journal Artificial Intelligence for the Earth Systems in 2025, examined three high-impact events using GraphCast, Pangu-Weather and FourCastNet alongside ECMWF's high-resolution forecast. For the 2021 Pacific Northwest heatwave, the learned models matched the physics-based forecast locally, but did worse when errors were aggregated across space and time. For the 2021 North American winter storm, which brought catastrophic power failures to Texas, the learned models actually did substantially better than the physics-based model, and the errors accumulated differently in the two systems. For the humid 2023 heatwave in South Asia, the learned models could not directly produce the variable that mattered, surface humidity, because it was not among their outputs; using a substitute, they underestimated the highest levels of heat stress over Bangladesh. That last finding is a reminder that a model can only forecast the variables it was built to forecast, and that the variables chosen for scorecards are not always those that matter for health or infrastructure. The most systematic test came from Zhongwei Zhang, Sebastian Engelke and Erich Fischer, published in Science Advances in 2026 after appearing as a preprint in 2025. They identified every grid point in 2018 and 2020 where ERA5 recorded a value exceeding the previous record from 1979 to 2017, yielding about 160,000 heat records, 33,000 cold records and 53,000 wind records in 2020 alone. They then compared forecasts of those events by ECMWF's physics-based high-resolution model with GraphCast, Pangu-Weather and FuXi. Across all weather in 2020, two of the learned models beat the physics-based model on standard scores. But on record-breaking events, the physics-based model was consistently better, at nearly all lead times, with the gap largest at short range. The learned models underestimated the intensity of hot records and overestimated the temperature during cold records, and the further a record was broken, the larger their error became. The authors described this as something like an implicit soft cap near the most extreme values in the training data. The learned models also produced too few records, and their errors were correlated with each other, suggesting that they failed on the same events, plausibly because they learned from the same data. The controlled experiment by Y. Qiang Sun, Pedram Hassanzadeh and colleagues at the University of Chicago, published in the Proceedings of the National Academy of Sciences in 2025, isolated the mechanism. They retrained FourCastNet on ERA5 data from 1979 to 2015 with all Category 3 to 5 tropical cyclones removed, then tested it on the Category 5 storms of 2018 to 2023. The retrained model's overall skill on global weather was essentially unchanged, which is itself telling: removing the most intense storms barely affects the scorecard. But it could not forecast Category 5 storms. It always underestimated them, producing at most a Category 2 storm. When the strong storms were removed from only one ocean basin, however, the model retained some ability to forecast them there, having learned about intense storms from another basin. The authors called the events that fall outside a model's experience "gray swans": physically possible, perhaps foreseeable, but absent from the training data. The other side of the ledger These findings need to be read alongside the evidence, presented in Chapter 3, that learned models can be outstanding at forecasting tropical cyclones. The Google DeepMind ensemble's performance in the 2025 Atlantic season, including its early confidence in the rapid intensification of Hurricane Melissa to Category 5, appears at first glance to contradict the gray-swan result. It does not. Category 5 hurricanes are rare, but they are present in the historical record; the DeepMind model was trained not only on reanalysis but on a dedicated database of nearly 5,000 observed cyclones. It had seen many intense storms. What it learned was the relationship between a storm's environment and its intensification, and Melissa, extreme as it was, fell within the envelope of that experience. The gray-swan experiment, by contrast, deliberately removed that experience. Together the two results say something consistent: learned models can forecast rare events well if they have seen enough examples of them, from somewhere, and cannot forecast events whose kind they have never encountered. Those who worked with the model in 2025 were measured in their conclusions. Franklin, the independent reviewer, noted that its performance was not uniform across every storm and that its lack of an explicit physical basis made it harder to evaluate. Forecasters at the National Hurricane Center stressed that performance over a full season matters more than any single storm, and that learned models still make mistakes and cannot be relied upon for exact predictions of track, intensity or impact. Google DeepMind itself cautioned against broad conclusions from Melissa alone. That caution is appropriate, and it is notable that it came from all sides. There is a deeper subtlety in the heatwave results. The deterministic learned models evaluated by Zhang and colleagues and by Pasche and colleagues were trained to minimise average error, which, as Chapter 2 explained, pushes them toward the mean and away from extremes at longer lead times. Some of their weakness on extremes is a consequence of that training objective rather than of learning from data as such. Generative and probabilistically trained ensembles, which are designed to produce sharp realistic samples, should in principle represent extremes better, and early evaluations of GenCast and similar systems suggest improvement in their calibration for some extremes. But a generative model still learns its distribution from historical data. It can recombine familiar ingredients to produce a novel combination, which is valuable, but there is no evidence yet that it can reliably produce an intensity far beyond its training range. The soft cap is softened, not removed. Why physics still wins at the edge It is worth spelling out why a physics-based model does not suffer the same problem, because the reason determines how the two approaches can be combined. A physics-based model does not know what temperatures are normal. It computes how air heats when it sinks, how dry soil reduces evaporative cooling, how a blocking anticyclone persists. If the initial state contains a ridge stronger than any previously observed, the equations simply carry its consequences forward. The model can still be wrong, through poor resolution, imperfect parameterization or a bad initial state, and physics-based models have their own systematic biases in extremes, but their errors do not grow systematically with how unusual an event is. In Zhang and colleagues' analysis, the physics-based model's temperature errors stayed roughly constant regardless of how far a record was broken. That is exactly the property one wants in a forecasting system for a warming world. There is also an asymmetry of consequence. A learned model that underestimates an unprecedented heatwave by several degrees produces a warning that is too weak at the moment when a strong warning matters most. The Pacific Northwest heatwave killed hundreds of people, many of them elderly people in homes without air conditioning, in a region unaccustomed to such heat. A forecasting system whose largest errors fall on the most dangerous events would be dangerous, however good its average scores. Rain, floods and the compound problem Heat and tropical cyclones dominate the published evaluations partly because they are well represented in the variables learned models predict. Extreme rainfall, which drives most flood disasters, is harder to evaluate and, on current evidence, harder to forecast. Zhang and colleagues explicitly did not address it. The reasons go back to Chapter 2: learned global models are trained on reanalysis rainfall, which is itself a model product with known biases, at a resolution too coarse to capture the convective cells that deliver the most intense downpours; and deterministic training blurs rainfall toward the mean. Floods such as those in western Germany and Belgium in July 2021, where more than 200 people died after a slow-moving low-pressure system delivered extreme rain over steep valleys, or in Valencia in October 2024, where a cut-off low produced torrential rain in a few hours and more than 200 people died, illustrate the challenge. In both cases the large-scale meteorological setup was signalled days ahead by physics-based models, and warnings were issued. The tragedies lay in the extreme local intensity, the speed of the hydrological response, and failures in translating warnings into action. A learned global model at 25 or 31 kilometres resolution contributes to the first part, identifying the dangerous setup, but cannot by itself resolve the local intensity. That requires either a kilometre-scale physics-based model, a learned downscaling of the kind described in Chapter 7, or both. Many of the most damaging events are also compound: heat combined with drought and wildfire, storm surge combined with river flooding, humid heat that overwhelms the body's cooling. Pasche and colleagues' finding that learned models lacked the surface humidity needed to assess humid heatwaves is a symptom of a broader issue. Learned models predict the variables in their training targets, which were chosen largely for convenience in verification, not for relevance to impacts. As the field matures, there is a growing push to include impact-relevant variables, such as humidity, soil moisture and river discharge, in learned systems, and to evaluate them on the combinations of conditions that cause harm. Practical approaches to the tail Several strategies are under way to make learned forecasts more reliable for extremes. The most immediate is not to rely on them alone. Operational centres run learned and physics-based systems side by side, and forecasters compare them. NOAA's hybrid grand ensemble, combining physics-based and learned members, formalises this. Where the two systems agree, confidence rises; where a physics-based model shows a record-breaking extreme that the learned model softens, forecasters have reason to trust the physics. A second approach is to train with loss functions that weight extremes more heavily or to use objectives drawn from extreme value statistics, so that the model is penalised more for missing the tail than for small errors near the mean. This helps within the training range but does not solve extrapolation. A third is to augment training data with physically simulated extremes. Physics-based climate models, especially large ensembles and simulations of warmer climates, contain many more extreme events than the observational record, including events more intense than any yet observed. Training on them exposes a learned model to the tails it would otherwise miss. The cost is that the learned model inherits whatever biases the physics-based model has in those tails, but those biases are at least physically grounded. A fourth uses the cheapness of learned ensembles to explore rare events in the present climate. The Huge Ensembles project described in Chapter 3 generated 7,424 members for a single summer precisely to sample low-probability, high-impact heat extremes. Approaches from rare-event statistics, which steer simulations toward trajectories that lead to extremes, can be combined with fast learned models to estimate the probability of events that would almost never appear in a conventional ensemble. Studies of this kind have asked, for instance, how close past summers came to producing a heatwave as severe as the Pacific Northwest event in other places. A fifth is to build more physics into the learned model, as in the hybrid approaches of Chapter 4, so that unprecedented states are handled by equations rather than by pattern matching. The lesson for users For anyone using AI weather forecasts, the evidence suggests a simple rule. Learned models are now among the best guidance available for common weather and for extremes of a type that is well represented in history, including many tropical cyclones. They should be treated with suspicion precisely when the situation is unprecedented: when a physics-based model or a forecaster's judgment suggests that a record may be broken by a wide margin. In those moments, the learned model's soft cap is most likely to bite, and its agreement with other learned models is weak evidence, because they share training data and therefore share blind spots. This is not a temporary limitation that more data or bigger networks will simply remove. As long as a model's knowledge comes from the past, it will be least reliable for events that break with the past, and a warming climate guarantees a steady supply of such events. The resolution lies in combining the cheap breadth of learned models with the extrapolative honesty of physics, which is also the theme of the next chapter, on producing the local detail that climate adaptation requires. Chapter 7: Bringing Global Projections Down to Earth A global climate model can tell a city that its region will warm by a certain amount by mid-century and that heavy rainfall will intensify. It cannot tell the city's drainage engineers how much rain will fall in the worst hour of the worst storm on a particular catchment, because the model's grid cell may be a hundred kilometres across and contain the city, its surrounding hills, a stretch of coastline and a river valley all averaged together. Yet that local, hourly, extreme quantity is exactly what an engineer needs to size a culvert or a flood wall. The process of translating coarse global output into the local detail that decisions require is called downscaling, and it is one of the places where machine learning is most clearly changing what climate science can deliver. Why local detail is hard Local climate is shaped by features far smaller than a global model grid. Mountains force air upward, wringing out rain on windward slopes and leaving dry shadows behind. Coastlines produce sea breezes and sharp temperature gradients. Lakes generate snow bands. Cities form heat islands. Valleys trap cold air on still nights. Convective storms, which deliver most extreme short-duration rainfall in many parts of the world, are a few kilometres across. A 100-kilometre grid sees none of this directly. The problem is not only one of averaging. Coarse models often get the statistics of local weather systematically wrong: too much light rain, too little heavy rain, temperatures that are too mild over mountains and too extreme over flat terrain. A useful downscaling method must therefore do two things at once. It must add spatial detail that the coarse model lacks, and it must correct the coarse model's systematic biases. And it must do both in a way that preserves the climate change signal, so that a projection of the future is not silently pulled back toward the present. The established toolkit Before machine learning, two families of methods dominated. Dynamical downscaling runs a regional climate model, a physics-based model at fine resolution over a limited domain, using the global model's output as boundary conditions. The regional model resolves mountains, coastlines and, at kilometre scales, convection itself, so it can produce realistic local weather that responds physically to the large-scale state. International programmes such as CORDEX, the Coordinated Regional Climate Downscaling Experiment, organise this work across continents. Convection-permitting regional models, running at a few kilometres, have been shown to represent extreme short-duration rainfall far better than coarser models. The drawback is cost. Running a regional model at a few kilometres for a century, for many global models and scenarios, is so expensive that only a handful of such projections exist for most regions, which means the full uncertainty across models and scenarios is rarely sampled at fine scale. Statistical downscaling instead learns relationships between coarse model output, or coarse observations, and fine-scale observations over a historical period, then applies them to future projections. The simplest methods, such as bias-correction and spatial disaggregation, combine quantile mapping, which adjusts the coarse model's distribution of values to match observed distributions, with interpolation guided by fine-scale climatology. Datasets produced this way, covering whole continents at a few kilometres for multiple global models and scenarios, are the backbone of practical climate risk assessment in many countries, precisely because they are cheap enough to apply to every model and scenario. Their weakness is that they rely on the assumption that the statistical relationship between scales stays the same in the future, and they often produce local fields that are statistically right but physically inconsistent: rain that falls in implausible spatial patterns, or temperature and humidity that do not fit together. The trade-offs among these approaches, and the machine-learning methods that have joined them, are set out in Table 4. Table 4. Approaches to downscaling global climate output. Approach Physical consistency Cost per projection Represents extremes Main risk Statistical (e.g. quantile mapping) Low Very low Partly, via bias correction Stationarity assumption Dynamical (regional model) High Very high Well at kilometre scale Too few projections Deep learning regression Moderate Low after training Poorly (smoothed) Smooth fields, extrapolation Generative (diffusion) Moderate to high Low after training Better, with sharp samples Extrapolation Dynamical-generative hybrid High Moderate Well Inherits regional model's biases The deep learning turn The first wave of deep-learning downscaling borrowed directly from image super-resolution, the computer vision task of producing a sharp high-resolution image from a blurred low-resolution one. Thomas Vandal and colleagues' DeepSD, presented in 2017, stacked convolutional super-resolution networks to downscale precipitation over the United States. Many variants followed, using convolutional networks, U-Nets and later transformers, trained to map coarse fields to fine ones using observations or regional model output as targets. These methods learned richer spatial relationships than classical statistical downscaling, and they could exploit many predictors at once: not just coarse rainfall but coarse winds, humidity and pressure, which together carry information about how the rain will be distributed over local terrain. But they shared the weakness of the deterministic weather models described in Chapter 2. Trained to minimise squared error, they produced the average of the possible fine-scale fields consistent with the coarse input. For temperature, which is smooth, that was often acceptable. For precipitation, which is patchy and intermittent, it was not: the networks produced broad, gentle rain where reality would have produced intense, localised cells. The mean was right and the extremes were wrong, which is the opposite of what a flood engineer needs. Generative models address this directly. Rather than predicting the average fine-scale field, a generative downscaling model learns the full distribution of fine-scale fields consistent with a given coarse state, and draws samples from it. Each sample is a sharp, realistic field; multiple samples represent the genuine uncertainty about local detail that the coarse state cannot resolve. Generative adversarial networks were tried first, with some success but notoriously unstable training. Diffusion models, more stable and better at representing full distributions, have largely taken over. Two landmark examples CorrDiff, developed by Morteza Mardani and colleagues at NVIDIA with Taiwan's Central Weather Administration and published in Communications Earth & Environment in 2025, downscaled from 25-kilometre global reanalysis to 2-kilometre fields over Taiwan, training on output from a high-resolution regional weather model. Its design illustrates a powerful general principle. It works in two stages. A conventional regression network first predicts the mean fine-scale field, capturing the predictable part of the detail, such as how temperature varies with elevation. A diffusion model then learns only the residual, the difference between that mean and the actual fine-scale field, which carries the unpredictable, small-scale variability such as the exact placement of convective cells. Splitting the problem this way makes the diffusion model's job much easier, because it need only learn the variability, not the whole field. CorrDiff produced realistic kilometre-scale structure, including the intense rainfall and sharp wind gradients of typhoons approaching the island, at a small fraction of the cost of running the regional model. The second example extends the idea from weather to climate projection. Ignacio Lopez-Gomez and colleagues at Google Research, publishing in the Proceedings of the National Academy of Sciences in 2025, proposed what they called dynamical-generative downscaling. The physics-based regional model, WRF, is used to downscale global climate model output only part of the way, to about 50 kilometres, which is relatively cheap. A diffusion model called R2D2 then completes the step to about 9 kilometres, learning, like CorrDiff, the residual between the intermediate and fine-scale fields. The method was tested on the western United States using a dataset of regional projections from multiple global models. Trained on the projection from one global model, it was evaluated on seven others. According to the authors, it reduced fine-scale errors by over 40 per cent compared with established statistical methods, reduced errors in projected extreme summer heat by over 20 per cent and in winter precipitation by over 10 per cent, and cut the computational cost by about 85 per cent for the eight-model ensemble tested, with larger savings expected for larger ensembles. The key feature of the dynamical-generative design is where it places physics and where it places learning. The physics-based regional model handles the step in which the large-scale climate change signal is translated into regional circulation, the step where extrapolation to new climates matters most. The generative model handles the final step, adding fine-scale detail whose statistical relationship to the intermediate field is likely to remain stable as the climate changes. The division follows the same logic as NeuralGCM's division of labour in Chapter 4: physics where the system is changing in ways that data cannot teach, learning where the relationships are stable and learnable. Keeping the climate signal honest The central risk in any downscaling method, learned or statistical, is that it distorts the climate change signal. A method trained on the present may implicitly learn present-day relationships between scales and impose them on the future. If, for example, extreme rainfall intensifies faster than average rainfall as the climate warms, which theory and observations suggest is often the case for short-duration extremes because a warmer atmosphere holds more water vapour and convective storms can intensify further, then a downscaling method that ties extremes to the average in a fixed way will underestimate future extremes. Several practices help. Training on regional model output from future climate simulations, as well as the present, exposes the learned model to changed relationships. Evaluating a method by training it on one global model's projection and testing it on others, as the R2D2 study did, checks whether it generalises across different representations of future climate. A related test, sometimes called the "perfect model" or "pseudo-reality" experiment, treats one physics-based simulation as the truth, trains a downscaling model on its present-day period, and checks whether it reproduces that simulation's own future. Downscaling methods that fail this test cannot be trusted for real projections. Physical constraints matter here as well. A downscaled rainfall field should, when averaged back to the coarse grid, match the coarse rainfall it was conditioned on; a downscaled temperature field should be consistent with the terrain. Methods that enforce these consistency conditions are less likely to drift. What downscaling makes possible The ultimate value of cheap, high-quality downscaling is that it allows the full range of climate uncertainty to reach the local scale. Today, a city or a utility often bases its plans on one or two regional projections, because that is all that exists at fine resolution. Those projections may come from global models that happen to be warmer or wetter than the average, and the user rarely knows how representative they are. With learned downscaling, every member of a multi-model ensemble, under every scenario, can be brought to kilometre scale. The user can then see not just one local future but the distribution of local futures consistent with current knowledge. That has practical consequences. Design standards for drainage, bridges and dams are based on return periods: the rainfall depth expected once in fifty or a hundred years. Estimating such quantities requires many years of data, and estimating how they will change requires many years of projected data. Generative downscaling of large ensembles can supply those samples, and in combination with the extreme-value methods engineers already use, can produce updated design values with honest uncertainty ranges. Agricultural planning, wildfire risk assessment, water resource management and public health planning for heat all face analogous needs. The same tools are also entering operational weather forecasting. Downscaling an ensemble forecast to kilometre scale lets forecasters see probabilities for local extremes, such as the chance of more than a given amount of rain in an hour over a particular catchment, that coarse ensembles cannot provide. Several national meteorological services are experimenting with generative downscaling of learned global ensembles for exactly this purpose. A worked case: designing for a hundred-year storm To see what changes in practice, consider an engineer responsible for the storm drainage of a mid-sized city on a coastal plain, asked to design a system that will cope with the rainfall expected once in a hundred years, on average, through to 2070. Under the traditional approach, she would take a design value from historical rain-gauge records: perhaps a few decades of hourly data, fitted with an extreme value distribution to estimate the hundred-year one-hour rainfall. She would then be asked to account for climate change. The available guidance might be an uplift factor, a percentage increase drawn from a national assessment, based on a small number of regional model projections or on a theoretical scaling of rainfall intensity with temperature. She would have little idea how uncertain that uplift was, or whether it applied to her catchment. With a well-validated generative downscaling system, the problem looks different. Every global model in a multi-model ensemble, under two or three emissions scenarios, can be downscaled to kilometre scale over her region, with multiple generative samples per model to represent the local detail the coarse models cannot resolve. That yields thousands of simulated years of hourly rainfall for the 2050s and 2060s, enough to estimate the future hundred-year one-hour rainfall directly, and to see how much the estimate varies between global models and scenarios. She might find that most projections cluster around a 15 to 25 per cent increase, with a few outliers much higher. She can then choose a design that meets the central estimate, assess the cost of protecting against the upper range, and identify cheaper adaptive measures, such as reserving land for future storage, that keep options open. None of this is guaranteed to be right. The downscaled projections are only as good as the global models' large-scale circulation and the downscaling model's preservation of the climate signal. But the engineer now has a quantified distribution of plausible futures rather than a single uplift factor, and her decisions can be made, justified and revisited on that basis. That is the practical meaning of bringing uncertainty to the local scale. A note of caution Downscaling adds detail, but it cannot add information that is not, in some form, present in the inputs and the training. A beautiful kilometre-scale rainfall map produced from a coarse global model inherits all of that global model's errors in the large-scale circulation. If the global model puts the storm track in the wrong place, the downscaled output will show realistic storms in the wrong place. Learned downscaling can make projections look far more authoritative than their underlying uncertainty warrants, because sharp, detailed fields are persuasive. Responsible use requires communicating that the fine detail represents plausible realisations, not a prediction of exactly what will happen where. Downscaling is, in this sense, the clearest example of the book's central argument. Its value lies not in making any single local projection more accurate but in making it cheap enough to generate many projections, and so to represent uncertainty honestly at the scale where decisions are made. Its risks lie where it is asked to extrapolate, and the most robust designs are those that keep physics in charge of the parts of the problem that change with the climate. Chapter 8: Earning Trust A forecast is useful only if people act on it, and people act on forecasts they trust. For physics-based models, trust was built over decades: through verification statistics accumulated over thousands of forecasts, through understanding of why the models behave as they do, and through the fact that their failures could usually be traced to identifiable causes. Learned models have arrived in operations in a few years, with spectacular headline results and a fundamentally different relationship between their behaviour and its causes. This final chapter asks how trust in them should be built, what operational centres are actually doing, and what users should ask of any learned forecast. What verification can and cannot tell us Verification, the systematic comparison of forecasts with what actually happened, is the foundation of trust in meteorology. It is also more subtle than it appears. The first subtlety, raised in Chapter 2, is the choice of truth. Learned models are often verified against reanalysis or analysis fields, which are themselves model products. A learned model trained on ERA5 and verified against ERA5 is partly being rewarded for reproducing ERA5's habits. Verification against independent observations, such as surface stations, weather balloons and satellite retrievals, is essential, and the margins of superiority claimed for learned models generally shrink when it is done. ECMWF and NOAA both verify their learned systems against observations as well as analyses, which is good practice that should be expected of any producer. The second subtlety is the choice of score. Root-mean-square error rewards smooth forecasts; CRPS rewards calibrated ensembles; neither directly measures performance on extremes, physical realism, or usefulness for a particular decision. The benchmarking effort WeatherBench 2, described by Stephan Rasp and colleagues in 2024, broadened the standard set of evaluations to include probabilistic scores, energy spectra that reveal blurring, and comparisons against observations as well as reanalysis. That made it much harder for a model to look good on one metric while failing on others, and it has become a common reference point. But no fixed benchmark can anticipate every use. The third subtlety is sample size. A scorecard aggregated over a year of forecasts across the globe contains millions of comparisons and is dominated by ordinary weather. As the gray-swan experiment in Chapter 6 showed, removing the most intense tropical cyclones from training barely changed global skill while destroying the model's ability to forecast those storms. Aggregate scores are almost blind to the events that matter most. Event-based and impact-based evaluation, of the kind pioneered by Pasche and colleagues and extended by Zhang, Engelke and Fischer to every record-breaking event in a year, is the necessary complement. It is also harder to do, requires careful statistical treatment, and produces less flattering numbers. The fourth subtlety is independence. Many leading learned models are trained on the same dataset, ERA5, with broadly similar objectives. When they agree, it is tempting to treat that as corroboration. But Zhang and colleagues found that the errors of different learned models on record-breaking events were positively correlated: they fail on the same events. Agreement among models that share training data is weaker evidence than agreement between a learned model and a physics-based one. How the operational centres are proceeding The operational meteorological services have moved quickly but deliberately, and their choices reveal a consensus about how learned models should be used. ECMWF has gone furthest. Its learned deterministic model, AIFS Single, became operational in February 2025, and its learned ensemble, AIFS ENS, on 1 July 2025. A second version of both, AIFS v2, was implemented in 2026 alongside a new cycle of the physics-based system. According to ECMWF, the update added forecasts of ocean wave height, direction and period, snow cover and additional land-surface and stratospheric fields; reduced medium-range errors in significant wave height by roughly 10 per cent relative to the physics-based system; and made the deterministic model's output somewhat less smooth and its vertical motions more physically consistent. ECMWF was also open about what had not improved, including a small persistent degradation in Arctic two-metre temperature that it attributed to limited winter sea-ice training data, and the fact that more physically consistent circulation did not translate into better verification scores. That candour is itself a model of good practice. Crucially, ECMWF continues to run its physics-based system, which also generates the analyses from which AIFS starts and the reanalyses on which it trains. ECMWF's data are openly licensed, which has allowed many other organisations to use and evaluate AIFS. NOAA's approach, launched in December 2025, explicitly combines rather than replaces. Its suite comprises a learned deterministic model, a learned 31-member ensemble, and a hybrid 62-member grand ensemble that pools learned and physics-based members. The learned models were adapted from Google DeepMind's GraphCast and fine-tuned on NOAA's own analyses. The emphasis on a hybrid ensemble reflects the evidence that learned and physics-based systems make different errors and that their combination is more reliable than either alone. Other services, including the UK Met Office with the Alan Turing Institute, Germany's Deutscher Wetterdienst, Météo-France, the China Meteorological Administration and the Australian Bureau of Meteorology, have developed or are developing learned models, often building on open frameworks such as ECMWF's Anemoi, a shared software toolkit for training and running learned weather models that several European services contribute to. The private sector has become a major producer of operational-grade forecasts in its own right. Google's WeatherNext models supply weather information across its consumer products and are offered to businesses through its cloud services; Microsoft's Aurora, described in Nature in 2025 by Cristian Bodnar and colleagues, was trained across a wide range of atmospheric data and fine-tuned for tasks including air quality, ocean waves and tropical cyclone tracks; NVIDIA provides open tools and models through its Earth-2 platform. This plurality is healthy in that it accelerates progress, but it raises questions that the physics-based era rarely faced: about openness of model weights and training data, about the sustainability of public forecasting services that depend on models developed by private firms, and about who is accountable when a forecast fails. The forecaster's role One of the most consistent findings from operational experience is that learned models change the forecaster's job rather than eliminating it. The National Hurricane Center's use of the DeepMind cyclone model in 2025 is instructive. Forecasters saw it alongside all their other guidance. When it showed early, confident rapid intensification of Hurricane Melissa, they weighed it against physics-based models, satellite observations and their own experience, and their published forecasts reflected that combined judgment. The value of the learned model lay in shifting the evidence, not in replacing the decision. That arrangement has a vulnerability. A model that is usually right becomes, over time, a model that forecasters defer to. If its failures are concentrated in unprecedented events, as the evidence on records suggests, deference will be greatest exactly when it is least warranted. Training forecasters to recognise the situations in which learned guidance is likely to fail, and building tools that flag when a forecast state lies outside the training distribution, are therefore as important as improving the models themselves. Research on detecting out-of-distribution inputs, and on quantifying the uncertainty that arises from limited training data rather than from chaotic atmospheric dynamics, is accordingly an active and important area. When the model changes Physics-based forecasting systems change on a regular cycle, and each change is preceded by months of parallel testing in which the new version runs alongside the old and is verified across seasons. Users who build products on the forecasts, such as energy traders, hydrological services and road authorities, are warned in advance and given test data, because a change in a model's habits can break the statistical corrections they have built on top of it. Learned models complicate this discipline in two ways. They can be retrained quickly and cheaply, which tempts developers to update often. And a retrained network can change its behaviour in ways that are not easy to anticipate from its design: a new version may improve on average while behaving differently in particular situations that users depended on. ECMWF has applied its usual practice of announced implementation dates, test data and published evaluation to AIFS, as the documentation of AIFS v2 shows. Commercial providers vary. For any organisation building decisions on learned forecasts, version control, advance notice of changes, and access to reforecasts, long archives of past dates run with the current model version, are not technical niceties but preconditions for reliable use. Without reforecasts, a user cannot recalibrate their own post-processing, and cannot know whether a forecast probability of 30 per cent from the new model means the same as it did from the old one. Explanation and its limits Learned models are often described as black boxes, and that is partly fair. One cannot inspect GraphCast's weights and read off a theory of baroclinic instability. But the situation is less stark than the label suggests, for two reasons. First, learned models can be probed experimentally. Researchers have perturbed their initial conditions to see whether disturbances propagate as atmospheric dynamics predicts, tested whether they reproduce known teleconnections such as the response to tropical heating, and examined whether their forecasts respect physical balances. Several studies have found that leading learned models do reproduce such behaviour within their training range, which offers some reassurance that they have learned something like physics rather than a lookup table. Other tests, such as the slow phenomena on which the Chicago group found ACE2 and NeuralGCM struggling, reveal where the learning stops. Second, physics-based models are not as transparent as they are sometimes presented. Their parameterizations contain tuned constants and empirical relationships, and their behaviour emerges from interactions among many components in ways that no one fully understands. The real difference is not between transparency and opacity but between models whose structure encodes physical principles that hold everywhere and models whose structure encodes only what the data showed. That difference is what makes physics-based models more trustworthy for the unprecedented, and it is why hybrid models that retain physical structure are attractive for climate. Who benefits One of the most consequential effects of learned models may be geographic. Running a competitive physics-based global model requires a large supercomputer, a substantial staff of specialists, and decades of accumulated development. Only a handful of centres in the world have that capacity. Most national meteorological services, particularly in lower-income countries, rely on global guidance from those centres and run, at most, a regional model over their own territory. Many of the people most exposed to extreme weather, in South Asia, sub-Saharan Africa, Central America and small island states, live in countries with the least forecasting capacity. A trained learned model changes the arithmetic. Running it requires a single graphics processor or a modest cloud allocation rather than a supercomputer. If its weights are openly released, as many have been, a national service can run global forecasts itself, adapt them to its region, fine-tune them on local observations, and add downscaling for its own territory. The World Meteorological Organization's Early Warnings for All initiative, launched in 2022 with the goal of protecting everyone on Earth with early warning systems by 2027, has drawn attention to exactly this opportunity. The opportunity has limits. A learned model still needs good initial conditions, which come from the data assimilation systems of the major centres. Regions with sparse observations are poorly represented in the training data as well as in the initial state, so learned models may perform worse there than global average scores suggest. Fine-tuning and evaluation require expertise and local data that may be scarce. And dependence on models released by foreign companies or agencies raises questions about continuity: what happens to a national warning system if a model's licence changes or its developer withdraws support? Open models, open data, and investment in local observation and expertise are what turn the potential into lasting capacity. The question of cost, revisited The case for learned models rests heavily on their cheapness, and that case needs one qualification. The cost of a learned forecast is low at the point of use, but the total cost of the system includes data assimilation, which still mostly relies on physics-based models, and training, which can be substantial for the largest models and must be repeated whenever the model is updated. It also includes the continued existence of physics-based reanalysis, without which there would be no training data. ECMWF's figure of roughly a thousandfold energy saving for AIFS ENS compared with its physics-based ensemble refers to the forecast itself, which is the right comparison for running an ensemble, but not for the whole enterprise. A world in which physics-based models were abandoned would soon find its learned models training on an ageing reanalysis and starting from analyses of degrading quality. The savings are real, and they are what makes the large ensembles and downscaled projections of earlier chapters possible. But they are savings within a system that still depends on physics at its foundation. What users should ask For decision-makers who receive AI-based forecasts or projections, whether from a national service, a commercial provider or a research group, the arguments of this book reduce to a short set of questions. What was the model trained on, and does the situation in question resemble that training data? A forecast for ordinary weather in a well-observed region is very different from a projection of a climate two degrees warmer or a heatwave that breaks records by five degrees. How was the model verified, against what truth, and on which events? Aggregate skill scores against reanalysis are a starting point, not an answer. Evidence on comparable extremes, against independent observations, is what matters. Is the forecast a single prediction or a calibrated distribution, and how many samples does it contain? A learned forecast that offers only one future is discarding its main advantage. What does a physics-based model say? Disagreement between learned and physics-based guidance is informative, especially when the physics-based model shows the more extreme outcome. For climate projections and downscaled products specifically: does the method preserve the climate change signal, and has it been tested on climates other than the one it was trained on? A provider who cannot answer these questions clearly is offering a product whose reliability is unknown, however impressive it looks. Conclusion When the first learned global weather models appeared, the question most often asked about them was whether they would replace physics. Four years later the question looks badly posed. The learned models did not replace physics; they changed its economics. The interesting questions now are about how to spend the savings, and about how to know when the learned answer can be trusted. What has actually changed The single most important consequence of machine learning in this field is that sampling has become cheap. A forecast centre that could afford fifty ensemble members can now afford thousands. A climate group that could afford a few centuries of simulation can now produce millennia in a day. A regional planner who had access to one or two kilometre-scale projections can, in principle, have one for every global model and every scenario. Each of these is an increase in the amount of uncertainty information available to decision-makers, and uncertainty information is what matters most for decisions about risk. This is a quieter claim than the headlines about AI beating the world's best forecast model, but it is more durable. Average skill scores will continue to rise for both learned and physics-based systems, and which is ahead on a given scorecard in a given year will shift. The ability to produce many realistic samples cheaply is a structural change that will not reverse. The second consequence is that learned models have exposed, in a new way, where the knowledge in a forecast comes from. A physics-based model's knowledge comes from laws that apply everywhere and an initial state that is never perfectly known. A learned model's knowledge comes from the record of what has happened. For most weather on most days, the record is an excellent teacher, which is why learned models now match or beat physics on standard measures. For events that break the record, the teacher has nothing to say. The evidence assembled in Chapter 6, from controlled experiments on tropical cyclones to the systematic study of record-breaking heat, points the same way: learned models are least reliable precisely where the stakes are highest. What follows Three practical conclusions follow from the argument of this book. The first is that the future of prediction is hybrid, in several senses. Operational centres are running learned and physics-based systems side by side and combining their ensembles. The most promising climate models keep a physical core and learn only what physics represents poorly. The most robust downscaling methods let a physics-based regional model handle the part of the problem that changes with the climate and a generative model handle the fine detail that does not. In each case, the division of labour follows a single principle: physics where extrapolation is required, learning where the relationships are stable and well sampled. That principle is more useful than any specific architecture, and it will outlast them. The second is that verification must change. The evaluation culture inherited from machine learning rewards average performance on held-out data, and the evaluation culture of meteorology was built around scorecards dominated by ordinary weather. Neither is adequate for systems whose most consequential failures are concentrated in rare, unprecedented events. Event-based evaluation of extremes, testing against independent observations, tests of generalisation to other climates, and explicit checks of whether downscaled projections preserve the climate change signal should be the minimum standard for any learned system used for warnings or adaptation. The organisations that publish these less flattering evaluations, as ECMWF has done in documenting the remaining weaknesses of its learned systems, deserve more trust, not less. The third is that physics-based modelling and observation remain indispensable, and their funding should not be cut on the strength of AI's success. Learned forecasts depend on physics-based data assimilation for their starting point and on physics-based reanalysis for their training data. Learned climate emulators depend on physics-based simulations to learn about climates that have not yet happened. A world that let its physics-based capability decay would soon find its learned models training on an ageing record and starting from deteriorating analyses. The observing system, from weather balloons to satellites, is the ultimate source of everything in this book, and it is under financial pressure in several countries. What remains unsettled Several important questions are open. It is not known how far learned models can generalise beyond their training data if they are given the right structure and the right training. Hybrid models and climate-invariant formulations have improved generalisation, and training on physics-based simulations of other climates extends the range of experience. Whether a learned or hybrid model will ever provide an independent, trustworthy estimate of climate sensitivity, rather than a fast copy of a physics-based model's estimate, is a deep question that remains unanswered. It is not known whether learned data assimilation, mapping observations directly to forecasts as in Aardvark Weather, will mature to the point where the whole forecast chain can be learned without a physics-based model at its core, and what would be lost if it did. It is not settled how the balance between public and private forecasting will evolve. Some of the best-performing learned models are developed by technology companies, and public services increasingly build on them. The arrangements that will keep warnings reliable, open and accountable as this happens are still being worked out. And it is not known how the tails will behave as the climate continues to warm. Every year produces new records, and some will be broken by large margins. Each of those events is a test of whether the forecasting system, learned and physical together, can see what has never been seen. The evidence so far says that learned models alone cannot be relied on for that, and that the combination can. For the reader For a reader who uses forecasts or climate projections in work, the practical lesson is to value learned systems for what they do best and to be alert to where they fail. Treat a large learned ensemble as the best available description of the range of ordinary outcomes. Treat agreement between learned and physics-based guidance as strong evidence and disagreement as a reason for closer attention, especially when the physics-based model shows the more extreme outcome. Ask any provider how their model was trained and tested, and be wary of answers that offer only average skill scores. For a reader in research or policy, the lesson is to invest in the combination: in hybrid models, in rigorous evaluation of extremes, in the observing system and reanalyses that all learned models depend on, and in the open data and shared benchmarks that have allowed this field to progress so quickly. The atmosphere has not become more predictable because of machine learning. Its limits of predictability were set by chaos long before any neural network was trained. What has changed is how cheaply we can explore the futures it might produce. Used with honesty about where the knowledge comes from, that cheapness is one of the most valuable tools climate adaptation has gained in a generation. Further Reading Background on weather and climate prediction Paul N. Edwards, A Vast Machine: Computer Models, Climate Data, and the Politics of Global Warming (MIT Press, 2010). A history of how global observation and computer modelling together made climate knowledge possible. It is the best account of why models and data cannot be separated. Peter Lynch, The Emergence of Numerical Weather Prediction: Richardson's Dream (Cambridge University Press, 2006). A careful reconstruction of Richardson's hand-computed forecast and the scientific path from it to modern numerical prediction. Andrew Blum, The Weather Machine: A Journey Inside the Forecast (Ecco, 2019). An accessible tour of the observing systems, supercomputers and institutions behind the modern forecast, including ECMWF. Peter Bauer, Alan Thorpe and Gilbert Brunet, "The quiet revolution of numerical weather prediction," Nature 525 (2015). The standard short review of how physics-based forecasting improved, and the baseline against which learned models are judged. Machine learning for Earth system science Markus Reichstein and colleagues, "Deep learning and process understanding for data-driven Earth system science," Nature 566 (2019). An influential early argument for combining learned and physical models rather than choosing between them. Gustau Camps-Valls, Devis Tuia, Xiao Xiang Zhu and Markus Reichstein (eds.), Deep Learning for the Earth Sciences: A Comprehensive Approach to Remote Sensing, Climate Science and Geosciences (Wiley, 2021). A technical edited volume covering methods and applications across the geosciences. Veronika Eyring and colleagues, "Pushing the frontiers in climate modelling and analysis with machine learning," Nature Climate Change 14 (2024). A review of learned parameterizations, emulators and downscaling from the climate modelling community's perspective. Learned weather models Remi Lam and colleagues, "Learning skillful medium-range global weather forecasting," Science 382 (2023). The GraphCast paper. Kaifeng Bi and colleagues, "Accurate medium-range global weather forecasting with 3D neural networks," Nature 619 (2023). The Pangu-Weather paper. Ilan Price and colleagues, "Probabilistic weather forecasting with machine learning," Nature 637 (2025). The GenCast paper, and the clearest demonstration of diffusion-based ensemble forecasting. Stephan Rasp and colleagues, "WeatherBench 2: A benchmark for the next generation of data-driven global weather models," Journal of Advances in Modeling Earth Systems (2024). The reference framework for evaluating learned forecasts, with a useful discussion of metrics. Cristian Bodnar and colleagues, "A foundation model for the Earth system," Nature (2025). The Aurora paper, on a single model fine-tuned for several Earth system tasks. Hybrid models and climate emulation Dmitrii Kochkov and colleagues, "Neural general circulation models for weather and climate," Nature 632 (2024). The NeuralGCM paper, the leading demonstration of a hybrid physics and learning model. Oliver Watt-Meyer and colleagues, "ACE2: accurately learning subseasonal to decadal atmospheric variability and forced responses," npj Climate and Atmospheric Science (2025). Notable for its frank discussion of the emulator's limitations under forcing. Thomas Rackow and colleagues, "Robustness of AI-based weather forecasts in a changing climate" (arXiv preprint 2409.18529, 2024). The test of learned weather models in pre-industrial and much warmer climates. Extremes and downscaling Zhongwei Zhang, Sebastian Engelke and Erich Fischer, "Physics-based models outperform AI weather forecasts of record-breaking extremes," Science Advances (2026). The systematic study of record-breaking events discussed in Chapter 6. Y. Qiang Sun and colleagues, "Can AI weather models predict out-of-distribution gray swan tropical cyclones?" Proceedings of the National Academy of Sciences 122 (2025). The controlled experiment on intense tropical cyclones. Olivier C. Pasche and colleagues, "Validating deep learning weather forecast models on recent high-impact extreme events," Artificial Intelligence for the Earth Systems 4 (2025). Case studies of three recent extremes. Morteza Mardani and colleagues, "Residual corrective diffusion modeling for km-scale atmospheric downscaling," Communications Earth & Environment (2025). The CorrDiff paper. Ignacio Lopez-Gomez and colleagues, "Dynamical-generative downscaling of climate model ensembles," Proceedings of the National Academy of Sciences 122 (2025). The hybrid regional-model and diffusion approach discussed in Chapter 7. Hashtags: #AIAndClimateModeling #ArtificialIntelligence #ClimateModeling #ClimateChange #EarthSystemModeling #EarthSystemScience #ClimatePrediction #MachineLearning #DeepLearning #AIWeatherForecasting #ClimateForecasting #NumericalWeatherPrediction #ClimateSimulation #ClimateEmulation #GraphCast #PanguWeather #FourCastNet #NeuralGCM #GenCast #WeatherNext #ExtremeWeatherPrediction #TropicalCycloneForecasting #ClimateDownscaling #ClimateProjections #ClimateRiskAssessment #EnsembleForecasting #ProbabilisticForecasting #PhysicsInformedMachineLearning #ClimateDataScience #ClimateResilience