When Code Is Cheap, Judgment Compounds

AI is changing how much software a large company can economically build for itself. That creates an opportunity to encode more of the operating knowledge that makes the company distinctive: how it allocates capacity, interprets exceptions, serves different markets, and resolves competing priorities.

The strategic investment extends beyond the programs themselves. It includes the requirements, domain knowledge, counterexamples, and evaluation evidence that make each subsequent program easier to produce and justify.

The Economics of In-House Software#

This shift is already influencing purchasing decisions. McKinsey’s August 2026 survey of 1,719 participants across 97 countries found that 32% of respondents said their organizations had decided against purchasing at least one software product or feature because agentic coding tools enabled them to build it internally. Across AI use more broadly, 37% reported a positive contribution to enterprise EBIT.¹ Building capability and capturing enterprise value are becoming separate management questions.

For leaders, this opens a larger architectural question: what should the company own so that its ability to build useful software improves over time?

In The Abstract Machine, I described an approach developed through merchandising software. A system takes a business brief, generates a specialized solver, evaluates its outputs, and preserves an accepted implementation for deterministic execution. The model participates in creating the program; the resulting software can then operate without a model in its routine execution path.

The same pattern is worth examining wherever operational decisions repeat, legitimate differences exist between business units, and outcomes can be evaluated with sufficient precision. Production scheduling, inventory allocation, distribution planning, and product configuration offer examples.

The next question concerns what this machine remembers.

The economics make that question increasingly consequential. Stanford’s 2025 AI Index documented a greater than 280-fold decline in inference cost at approximately GPT-3.5-level performance between November 2022 and October 2024.² That measures the price of obtaining a given capability. Enterprise implementation costs also include specification, integration, evaluation, and maintenance.

Research on coding productivity shows why those surrounding activities matter. A February 2025 analysis of three field experiments involving 4,867 developers at Microsoft, Accenture, and a Fortune 100 company estimated that using a coding assistant increased completed tasks, measured through weekly pull requests, by approximately 26%.³

In a different setting, METR’s early-2025 randomized study of 16 experienced developers working on 246 tasks in familiar repositories found that AI use increased completion time by 19%.⁴ METR subsequently reported that newer tools likely produced greater benefits, while selection effects prevented its follow-up experiment from reliably establishing the size of the improvement.⁵

These studies examine different tools, populations, and tasks. Their management implication is that productivity needs to be measured against the actual work and its acceptance requirements. An enterprise cannot derive its delivery economics from a model benchmark or a general productivity multiplier.

From Code Generation to System Evaluation#

There is also evidence of what becomes possible when generation is tightly connected to measurable outcomes.

Google DeepMind’s AlphaEvolve combines language models that propose programs with automated evaluators that assess candidate solutions. Google reported that an AlphaEvolve-discovered scheduling heuristic recovered, on average, 0.7% of its worldwide compute resources. It also reported a 23% improvement in a matrix-multiplication kernel, translating into a 1% reduction in Gemini’s training time.⁶ Those are company-reported results in specific computing applications.

A local technical gain acquires business meaning through its effect on the larger system. Evaluation connects the generated artifact to that effect.

This suggests an enterprise architecture built around a durable capacity to generate and judge specialized implementations.

Code reuse remains an important engineering choice within that architecture. Established libraries and shared infrastructure can improve reliability and reduce cost. But every reusable component also carries assumptions, interfaces, and compatibility obligations. As those obligations accumulate, a new requirement may become expensive to express within the existing design.

Where generation and validation are economical, a complete specialized program becomes another option. Its implementation can reflect the business requirement directly.

The fiftieth program may still contain five hundred freshly generated lines. Progress means that the organization can establish its fitness with less effort and stronger evidence.

Institutional Memory for the Machine#

That requires preserving knowledge at a different level.

Consider the instruction to keep a brand together on a retail shelf. It describes a business outcome. A solver might achieve it by allocating space to brands first, repairing discontinuities after placement, or searching across complete arrangements. The most suitable approach can depend on the fixture, assortment, and competing priorities.

The domain vocabulary should establish what the requirement means. Evaluation should establish whether the result satisfies it. The implementation should retain room to adapt.

For a multinational company, this distinction is particularly valuable. Shared definitions can coexist with policies whose scope is a market, customer, facility, or product line. A local commercial decision need not become a permanent exception inside a global algorithm.

The machine should therefore retain several kinds of institutional knowledge:

These assets make experience usable beyond the project that produced it.

A counterexample from one distribution center might reveal a capacity assumption that other facilities should test. A product-data ruling might resolve a recurring ambiguity across a category. A failed strategy might help the next generator avoid an approach that performs poorly under similar conditions.

Each lesson needs provenance and scope. A local policy should remain local until there is a reason to generalize it. An explanation of a failure should remain a hypothesis until evidence supports it. Knowledge also needs effective dates and owners, because the business continues to change.

Research offers antecedents for this form of learning. The ExpeL framework gathers agent experiences, extracts insights in natural language, and retrieves those insights and examples for subsequent tasks without updating the underlying model’s parameters.⁷ Its experiments concern agent benchmarks; the enterprise opportunity is to apply that mechanism to governed operational knowledge and measure whether lessons transfer.

Our merchandising work provides a practical setting for developing this direction. A September 2026 repository review recorded 35 categories and 64 solver versions, together with rule documents, audits, run histories, comparison reports, and visual shelf outputs.

The version history also contains lessons about evaluation. Tea v11 documents repairs to earlier gaps in audit coverage. That finding matters because a system can improve its apparent performance without improving the business outcome if its checks leave important requirements unexamined.

The Independence of Acceptance#

Imagine a shelf arrangement that fits perfectly but omits a required product. The geometry can be valid. The rendering can look convincing. An audit that never compares the output with the required assortment can pass the result.

The enterprise has received a plausible answer to an incomplete question.

There is a related problem when the generator and evaluator share the same misunderstanding. An agent can interpret a requirement incorrectly, implement that interpretation, and create a checker that confirms it. Agreement between the programs then gives a misleading impression of assurance.

Acceptance criteria need an identity independent of the candidate attempting to satisfy them. Agents can help create and improve evaluators, but a candidate cannot quietly change the conditions of its own acceptance. Changes to requirements or evaluation logic should be recorded and followed by reassessment.

The evaluator also needs challenges. Remove a required product. Break a brand block. Exceed the fixture’s capacity. Check that the corresponding failure is detected.

This is productive use of failure: each discovered blind spot can strengthen the judgment applied to future candidates.

Anthropic’s engineering guidance similarly recommends combining automated evaluations, production monitoring, and human review.⁸ It distinguishes the outcome actually achieved in the operating environment from the agent’s account of what it accomplished.

For enterprise deployment, these evaluations should preserve distinct claims. Technical validity establishes that the program operates within its constraints. Policy compliance establishes that it respects the stated business rules. Completeness accounts for required inputs, outputs, and exceptions. Robustness examines behavior across varied conditions.

Commercial performance requires operating evidence. A compliant allocation can still produce weak service levels. A better simulation score can justify a live experiment without establishing margin improvement.

An Operating Model for Compounding Judgment#

This leads to a concrete operating model.

The enterprise platform team should provide common execution infrastructure, interfaces, evaluation records, and release processes. Business owners and domain experts should own the meaning of requirements and the decisions about precedence. Acceptance should have accountable ownership with the authority to reject a candidate or require stronger evidence.

Local teams can then generate specialized implementations within a shared operating framework. The organization gains flexibility while retaining a common way to inspect, compare, deploy, and replace what it produces.

That shared framework becomes more important as specialization expands. The economic benefit of inexpensive authorship can disappear if each generated program introduces a separate integration approach, an unidentified owner, or an irreproducible deployment.

An accepted release therefore needs more than source code. It needs an identifiable environment and dependencies, the relevant requirement and evaluator versions, and the inputs used to establish acceptance.

This enables two useful operations. The organization can replay the release under its original conditions. It can also reassess it against changed policies or stronger checks.

When new evidence exposes a defect, the previous decision remains understandable. A counterexample enters the institutional record, and a replacement can be generated or a direct repair made.

For the executive team, investment should follow the economics of this complete process:

These measures should be tracked across tasks of comparable difficulty and under recorded model and evaluation versions. Stronger evaluation can initially lower acceptance rates by revealing failures that previous checks missed. Leadership needs that context to interpret the numbers.

The compounding claim can also be tested directly. Give comparable tasks to the system with and without selected prior evidence. Record whether accumulated knowledge reduces intervention, improves acceptance quality, or lowers total cost. This separates the contribution of institutional learning from improvements supplied by a newer model.

The most useful starting point is a bounded operational domain with repeated decisions, meaningful variation, accessible examples, and measurable outcomes. In our work, tea offers that setting because its history contains both functioning solvers and documented failures of judgment. The next milestone is a complete generation-and-evaluation experiment with recorded attempts, withheld assessment cases, and a reproducible accepted release.

For a large enterprise, the broader objective is an expanding portfolio of domains in which this process becomes dependable.

That changes the asset created by an AI program. A project delivers working software. It also leaves behind clarified decisions, difficult examples, better evaluators, and evidence that future teams can use.

When a stronger model arrives, those assets give the enterprise a concrete basis for testing it. When a business policy changes, they establish what must be reconsidered. When an implementation is replaced, they preserve what the organization learned.

The company’s knowledge can therefore improve across generations of software and models. That is the compounding opportunity: an enterprise that becomes progressively better at turning its operating intent into systems whose performance it can demonstrate.

Footnotes#

1. QuantumBlack, McKinsey & Company, The State of AI in 2026 (August 2026). ↩

2. Stanford Institute for Human-Centered AI (HAI), The 2025 AI Index Report (2025). ↩

3. Z. Cui et al., The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers (February 2025). ↩

4. METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (July 2025). ↩

5. METR, Update on Developer Productivity Measurement (February 2026). ↩

6. Google DeepMind, AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms (2025). ↩

7. A. Zhao et al., ExpeL: LLM Agents Are Experiential Learners (arXiv:2308.10144, 2023). ↩

8. Anthropic Engineering, Demystifying Evals for AI Agents (2025). ↩