Agentic AI Masterclass Curated by Arunabh Bhattacharjee
Part IX - Appendices · Appendix A
A

Appendix A — The Reference Agent Capstone

A ninety-day plan to build the whole system — what to do in what order, what to skip, and the six checkpoints that stop you fooling yourself.

28 min read · Part IX - Appendices · Updated 9 November 2026

This illustrative ninety-day plan targets a governed, measured pilot. Each stage establishes controls needed for the next stage’s authority.

The plan assumes two to four engineers, an existing model provider relationship, and one real task with measurable outcomes. Treat the durations as estimates. Preserve the dependency order when you adjust the schedule.

A.1What you are building#

By day ninety: an agent that takes real work from real users, in one bounded domain, with durable state, a permission boundary, budgets, guardrails, a trace, an eval suite with confidence intervals, an approval flow with a measured refusal rate, and a governance pack that answers a security review. Running at autonomy level 2 on its consequential capabilities and level 3 on its safe ones.

Out of scope: multi-agent coordination, knowledge graphs, fine-tuning, self-authored skills and voice.

DAY 1 10 25 45 60 75 90 1 · Spine loop · state Ch 3-5, 13 — the twelve lines, the event log, three tools 2 · Bounds budgets · trace Ch 20, 23, 15 — ceilings, spans, kill switch, policy 3 · Knowledge context · retrieval Ch 8-11, 14 — assembler, hybrid search, rerank 4 · Human seam approvals Ch 19, 6 — plan artefact, approval flow 5 · Measurement evals · analysis Ch 24, 25 6 · Production rollout · gov CHECKPOINTS — each one is a thing you can fail ① day 10 kill a worker mid-run; resume ② day 25 breach all five ceilings by name ③ day 45 tenant A cannot read tenant B ④ day 60 a reviewer can answer a request ⑤ day 75 paired evidence for the change proposed ⑥ day 90 the pack, cold, to a reviewer
Figure A.1 - Ninety days. The phases overlap deliberately; the checkpoints do not — each one is a specific test you can fail, and failing it means the next phase is premature.

A.2Turn the calendar into a capacity budget#

Ninety calendar days sounds like a capacity number. It is not. The team does not receive ninety uninterrupted engineering days from each engineer, and the six phase durations are not six allocations you can add. Reviews, on-call work, leave, security conversations and the waiting time between a test and a decision all consume the calendar without advancing a feature. Convert the plan into engineer-days before you assign work, or the first checkpoint failure will expose a schedule that never had slack.

Take the smallest team assumed by this appendix: two engineers, thirteen working weeks.

Step Calculation Engineer-days
Ideal capacity 2 engineers × 5 days × 13 weeks 130
Capacity after normal availability 130 × 75% 97.5, budget as 98
Explicit recovery reserve 98 × 20% 19.6, budget as 20
Work you may schedule 98 − 20 78

The 75% factor and 20% reserve are illustrative planning choices, not universal benchmarks. Replace them with your team’s measured availability and interruption rate. Their purpose is to force two honest statements: you do not own every hour on the calendar, and a checkpoint designed to find defects needs capacity to fix the defects it finds.

A defensible first allocation of those 78 scheduled engineer-days looks like this:

Phase Scheduled work Why the calendar window is longer
Spine 10 days Ten engineer-days are about five paired working days inside the days 1–10 calendar window; little work is safely parallel
Bounds and observability 17 days Policy, trace and guardrail tests proceed in parallel, then converge at checkpoint ②
Knowledge and tools 19 days Retrieval evaluation can run while the tool catalogue and tenant-scoping tests are built
Human seam 9 days Engineering is short; reviewer recruitment and response time occupy the window
Measurement 11 days Case authoring, labelling and harness work overlap; eval runs consume elapsed time
Production and governance 12 days Most artefacts are assembled from prior evidence, while rollout observation continues
Total 78 days The remaining 20 days are unassigned recovery capacity

This is not a work breakdown structure. It is a capacity test. If the named work requires 95 days before you have reserved any recovery time, changing a Gantt chart will not make ninety days feasible. Reduce the pilot’s authority or tool surface. Do not silently set availability to 100% and call the problem solved.

The phase windows in Figure A.1 overlap because waiting and work are different resources. While one engineer waits for ten cold approval reviews, the other can add eval cases from read-only traffic. While a shadow run accumulates enough production examples to inspect, the team can assemble the data map. The checkpoint itself never overlaps: the next increase in authority waits for a pass. Figure A.3 shows the capacity conversion and reserve after the dependency graph.

Four engineers do not justify twice the scope. With the same assumptions, four engineers have 260 ideal days, 195 available days and 156 scheduled days after a 39-day reserve. Use the extra capacity to shorten feedback paths: a second person independently reviews the permission engine, another labels eval cases, and governance evidence is checked while it is produced. Adding multi-agent coordination, voice or a knowledge graph consumes the extra capacity while also creating new dependencies. The result is a broader system with no stronger checkpoint.

Assign one directly responsible engineer to each checkpoint, but never make that person the sole reviewer. The owner prepares the test and evidence; someone who did not build the mechanism runs or reads it cold. For checkpoint ③, for example, the storage owner writes the tenant-isolation test and a second engineer proves that the assertion fails when tenant scoping is removed. Passing a test that cannot fail is not evidence.

Treat reserve consumption as a product signal. Record planned days, actual days and reserve burned per phase. If the spine consumes 14 rather than 10 days, the four-day difference comes out of the reserve immediately; it does not disappear into a revised estimate. After two phases, suppose the team has spent 12 days on the spine and 22 on bounds. Against a plan of 10 + 17, it has burned 7 of 20 reserve days and has 13 left. That arithmetic should trigger a scope conversation while there is still scope to remove.

The scope conversation has a fixed order. First remove an unrequired tool. Then reduce the pilot’s input variety. Then defer a convenience surface. Reduce evaluation coverage or a control only if you are also reducing the authority whose risk required it. A schedule recovered by weakening its checkpoint is merely late in a way the dashboard cannot see.

Engineer-days also hide the critical path. Seventeen days of bounds work do not necessarily fit into nine calendar days merely because two engineers are available. Suppose budget fixtures require four engineer-days, trace parentage five, permission rules five, and the integrated breach rehearsal three: seventeen in total. The rehearsal depends on all three preceding packages and is a single-owner, non-divisible task — two engineers cannot turn its three days into one and a half. If one engineer takes budgets then permission while the other takes tracing, the two lanes take nine and five days; integration starts on working day ten and completes on day twelve. The ideal arithmetic says 8.5 days, but the dependency graph says twelve. Put the dependency into the plan before you promise the checkpoint date.

Limit work in progress for the same reason. A two-person team can own one primary build lane and one independent evidence lane; three simultaneous subsystem changes create waiting, review debt and partial integrations. During the bounds phase, one engineer might implement a ceiling while the other writes the fixture that breaches it, then swap roles for the next ceiling. The apparent loss of parallel coding buys an observation neither author can unconsciously tailor to their own implementation.

Account for non-engineering wait explicitly but do not convert it into fictional capacity. A security reviewer taking three days to answer does not consume three engineer-days; it does consume three calendar days on the checkpoint’s critical path. Record both numbers. The capacity ledger answers whether the team can do the work. The milestone network answers when the required evidence can exist. You need both to make the ninety-day claim honest.

Review the ledger twice a week. Each line should show original estimate, engineer-days spent, remaining estimate, reserve burned and the authority that depends on completion. “Eighty per cent done” is not a useful field. “Two of five breach fixtures pass; wall-clock and cost remain; three reserve days burned” tells you what remains and whether the next gate is at risk.

A.3Days 1-10 · The spine#

Build: the loop, the event log, and three tools.

Start with Chapter 4’s twelve lines and Chapter 5’s append-only log. Do not start with a framework: write the loop yourself, because everything in Parts VI and VII has to attach to it and a loop you cannot see is a loop you cannot instrument. You can adopt a framework later for durability or deployment (Chapter 7’s caution applies) — but own the loop.

Three tools, no more. One read, one search, one write scoped to an output directory. Chapter 13’s discipline from the first commit: generated schemas, per-argument descriptions with examples, and rung-4 error messages. This is the cheapest quality investment in the whole plan and it is the one most often deferred.

Name terminal outcomes on day one so dashboards, evaluation cases and alerts share the same vocabulary (Chapter 4).

Checkpoint ① — day 10. Kill the process mid-run, restart it, and resume. If the side effect repeats or the run restarts from zero, stop and fix it. Everything after this assumes durability, and retrofitting it means rewriting the loop.

Deliberately skip: planning, memory, retrieval, compaction, sub-agents.

A.4Days 11-25 · Bounds and observability#

Build: budgets, guardrails, the trace, the permission engine, and the kill switch.

This phase is what makes the agent safe to run unattended, and its order within itself matters. Budgets and the kill switch first (Chapter 20) — all five ceilings plus the consecutive-null-result progress signal, which is the cheapest guardrail in the book and the one that would have caught the eleven-minute runaway. Then the trace (Chapter 23): one run span with a named outcome, a chat span with tokens and cost and context utilisation, a tool span with the policy decision. Render the tree and look at it; assert it has one root.

Then the permission engine (Chapter 15): path and egress policies, ordered rules, the three-valued verdict, and the test that a rule cannot widen a resource denial. Egress default-deny, with the allowlist derived by running your suite with it empty and collecting the failures.

Run it read-only against real traffic for the last week of this phase. This observation period informs the later phases (Chapter 29’s stage 1). It produces the failure taxonomy, the first eval cases, the cost model and the trajectory corpus that phases 3 through 6 all consume. Do not skip it to move faster; it is the fast path.

Checkpoint ② — day 25. Deliberately breach each of the five ceilings and confirm the outcome names the specific one. Then flip the kill switch, confirm in-flight runs stop at a turn boundary with a usable handoff record, and time the whole thing. That duration is your incident-response floor.

A.5Days 26-45 · Knowledge and tools#

Build: the context assembler, retrieval, compaction, and the real tool set.

Chapter 8’s assembler first, with the ledger instrumentation: sections with priority and stability, stable-first ordering for cache alignment, an explicit drop log. Look at the ledger on a long run before tuning anything; the composition is reliably surprising.

Retrieval next (Chapter 11), as a tool rather than a preprocessing step. Hybrid from the start — BM25 alongside vectors, fused by rank — because enterprise questions contain identifiers and embeddings miss them. Add the reranker and lower your top_k; four excellent chunks beat ten mediocre ones on both quality and cost.

Compaction (Chapter 9) with the six-field structured handoff record and a test asserting the goal survives byte-identically.

Then widen the tool set, and apply Chapter 14’s discipline immediately rather than at tool forty: keep the visible set under twenty, build the selection benchmark, and look at the confusion pairs. If you are integrating third-party tools, pin their definitions including descriptions.

Multi-tenancy is this phase’s non-negotiable (Chapter 29): tenant scope as a required argument on every data access, and the CI test that asserts an empty result rather than a cross-tenant one.

Checkpoint ③ — day 45. In CI, run as tenant A and assert that tenant B’s content cannot be retrieved. Verify that the test detects a deliberately broken tenant filter.

Defer the knowledge graph until observed questions meet Chapter 12’s threshold test.

A.6Days 46-60 · The human seam#

Build: the plan artefact and the approval flow.

Chapter 6’s plan as first-class state, with BLOCKED distinct from PENDING, a required verification criterion per step, and dependencies making it a graph. Then run the read-only steps before committing to the plan, and re-prompt for a revision — the cheapest quality improvement available that does not involve a better model.

Chapter 19’s approval flow, designed around the reviewer’s attention rather than around the actions. Approve the plan and its bounds, once, not ten actions individually. Requests carry all six fields — especially what is uncertain and what happens if declined. on_timeout is never proceed.

Instrument the gate itself from the start: refusal rate, decision latency, approval-to-action ratio. These are the gate’s health metrics and without them you will discover the four-second-median problem from a compliance review rather than a dashboard.

Checkpoint ④ — day 60. Show ten real approval requests to an actual reviewer, cold, and ask whether they could evaluate each. The ones they cannot are missing fields. Then check that your approval-to-action ratio is well below one.

A.7Days 61-75 · Measurement#

Build: the eval suite, the error-analysis cadence, and online sampling.

Use an initial target of two hundred cases, then size the suite from coverage, risk and a power analysis. Draw cases from the traffic you have been collecting since day twenty-five. Stratified by risk rather than frequency. Including cases that should escalate, cases whose answer is not in the corpus, and cases containing injection attempts. Trajectory contracts — must_call and must_not_call — on the high-risk cases. Confidence intervals printed in the default output, and repeats so you have a pass^k number rather than only pass@1.

Then the cadence: one hour a week, a named owner, twenty failures read and clustered by mechanism (Chapter 25). Let the traces determine which layers own the top clusters. Every failure read becomes a case before the meeting ends.

Add sampled judging on production traffic (for example, 2–5%) to detect gaps in the offline suite.

Checkpoint ⑤ — day 75. Run your suite three times without changing anything. Use the repeats to find flaky cases, then compare candidate and incumbent on paired cases. Predeclare a minimum useful change, safety gates and case-clustered uncertainty; a single-version interval width is not a test of a change.

A.8Days 76-90 · Production and governance#

Build: the rollout, the operational surface, and the governance pack.

Shadow mode first (Chapter 29): the agent runs on real traffic with output diverted to a comparison log. Two weeks of this produces the agreement-rate document — “on two thousand real cases the agent agreed with the human 91% of the time, and here are the 9%” — which unblocks more security reviews than any architecture diagram.

Then the deployment shape: async over a queue, stateless workers over the event log, one model gateway owning caching, routing, fallback and the shared rate-limit budget. Per-tenant queues, quotas and cost attribution.

Then governance (Chapter 30), and most of it is assembly rather than new work: the capability manifest with owners, the blast-radius table, the data map including your trace and trajectory stores, the identity model, the trifecta audit, the red-team results, the eval report, the approval metrics, the kill-switch rehearsal record, the regulatory classification with a named signatory, the incident runbook, and the shadow-mode report.

Begin regulatory classification in week one and revisit it before rollout (Chapter 30). Its depth and duration depend on the use and jurisdictions; it determines required controls and evidence.

Checkpoint ⑥ — day 90. Take the twelve-item governance pack to your hardest internal reviewer, cold, and let them ask questions. What it does not answer is the pack’s next revision. If they approve, you are at Chapter 29’s stage 5.

A.9Make every checkpoint leave executable evidence#

A checkpoint is not a meeting, a demonstration or a sentence in a project update. It is an executable test, the immutable inputs used by that test, the result, and a decision tied to the result. Store those four things together. Otherwise day ninety produces a scavenger hunt through screenshots, chat threads and dashboards whose queries have since changed.

Use one evidence record per checkpoint:

Field What it must contain Example for checkpoint ②
Claim The invariant this checkpoint is meant to establish Every run stops at its configured ceiling and names the breached resource
Procedure A command or reproducible sequence, including configuration Five fixtures, each with one ceiling below the consumption required by its scripted run
Observation Raw result plus a stable link to trace or event log budget_exhausted, detail tool-call ceiling reached (3/3), run id and log digest
Decision Pass, fail or exception; owner; time; next authority allowed Pass, signed by runtime owner; read-only unattended traffic may begin

Keep the claim narrow enough to test. “The system is safe” cannot pass because no finite observation establishes it. “A worker killed after a recorded tool completion resumes without executing that logical call again” can pass. It also exposes the boundary: it says nothing about a worker killed after the external effect but before the completion record. Chapter 5’s uncertain-outcome protocol owns that second claim.

The reference implementation gives you executable starting points, not a production certification. BudgetTracker.exceeded() checks turns, tool calls, tokens, wall-clock and cost in that order, and returns the first breached ceiling it finds. If one turn crosses the token and cost ceilings together, the result therefore names tokens. Your checkpoint must breach one ceiling at a time when it is testing classification, then breach several together and assert that documented precedence. Otherwise a refactor can change operational attribution while every individual ceiling test still passes.

Be exact about the reference’s progress signal. BudgetTracker.record_call() passively increments a counter for each call signature, and repeats() can report that count. The loop never consults repeats() or Budget.max_repeat_signature, so neither one stops a run. The active halt comes from the separate LoopDetector action guardrail, which maintains its own recent-signature window and returns a HALT violation for exact repetition or the four-step A, B, A, B oscillation. The reference still does not implement the consecutive-null-result counter specified for the production capstone in Chapter 20. Add that counter as a production extension and give it its own test; do not describe passive telemetry as enforcement.

Make the same distinction at all six checkpoints:

Checkpoint What the reference can demonstrate Production evidence you still need
① resumption A JSONL event log, state replay, and a completed-call map keyed by logical action Receiver-enforced idempotency or reconciliation for the external-effect crash window; durable shared storage rather than one local file
② bounds Five named budget ceilings, repeat and oscillation detection, run outcomes and an in-memory trace Consecutive-null progress, distributed kill-switch propagation, exported traces and measured stop latency
③ tenancy Nothing tenant-specific in the reference storage or retrieval path Tenant required at every data boundary, adversarial cross-tenant fixtures, and a test that is known to fail without the scope
④ human seam An ASK path, deny-by-default approver and approval events The six-field request, bounded plan approval, authenticated reviewer identity, timeout behaviour and decision-latency reporting
⑤ evaluation Outcome, trajectory and efficiency scorers, repeats, cost totals and a Wilson interval Risk-stratified cases from real traffic, stable datasets, judge calibration, run-to-run comparison and release policy
⑥ governance Event types and trace attributes that can feed evidence Retention, access, export, signatures, data lineage and the assembled twelve-item pack

That right-hand column is not a criticism of the reference. The package is a dependency-free teaching implementation. The production system needs shared infrastructure, identity and operating ownership that would make the teaching code harder to read. Your capstone fails when it mistakes a clear interface for a completed service.

Checkpoint ⑤ deserves arithmetic because evaluation can consume both budget and time without producing decisive evidence. Suppose Reference Agent’s suite contains 200 cases and 176 pass: an observed rate of 88%. Using the Wilson calculation implemented in EvalReport, the illustrative 95% interval is about 82.8% to 91.8% — roughly nine percentage points wide. These intervals estimate one version’s rate. They cannot establish whether an 88%-to-91% change is real; retain the paired outcomes and analyse the difference with base cases as clusters.

Run each case three times and the harness produces 600 scored reports. At an illustrative average of $0.08 per run, that evaluation costs 600 × $0.08 = $48. At ten minutes per run, it needs 6,000 worker-minutes. Twenty workers with perfect parallelism would finish the model work in 6,000 ÷ 20 = 300 minutes, or five hours, before rate limits, queueing and judge time. Put those numbers into the phase budget before you promise a daily release gate.

Do not then treat 600 repeated reports as 600 independent questions. Repeats measure stochastic consistency on the same case; new cases measure coverage of the task distribution. If the 528 of 600 repeated runs pass, a naive Wilson interval tightens to about 85.2%–90.4%, but the suite still contains only 200 distinct situations. Use repeats to compute reliability and expose flaky cases. Add independently sampled, risk-stratified cases when the question is whether the suite covers the work.

The release decision should state both. For example: “176 of 200 distinct cases passed; 162 cases passed all three repeats; no must_not_call violation occurred in the 47 high-risk cases; evaluation cost was $48.” The first number describes single-trial task success, the second consistency, the third a hard trajectory invariant and the fourth the recurring cost of the gate. One blended score would hide all four decisions.

Finally, rehearse evidence retrieval. Ask an engineer who did not build Reference Agent to select checkpoint ②, rerun its procedure from a clean checkout, and find the original run record. Ask a security reviewer to follow checkpoint ③ from claim to failing adversarial fixture to passing result. If either must ask the author what a field means, the checkpoint is not yet an organisational asset. It is personal memory wearing a test name.

Make checkpoint ① adversarial across the whole action boundary. Choose four kill points: before the intent event, after intent but before dispatch, after the receiver acknowledges but before the completion event, and after completion but before the next model turn. Run each point five times for one read-only and one mutating tool: 4 × 5 × 2 = 40 crash scenarios. A read may execute again safely, although the trace must say that it did. A mutation must return the recorded result, reconcile with the receiver or escalate as uncertain; it must never guess that absence of a local completion means absence of an external effect.

The shipped reference can exercise the local cases around its JSONL event log and completed-call map. It cannot prove receiver-side idempotency because the receiver is outside the package, and its tool-start event is not a transactional outbox. Your production checkpoint must therefore capture the receiver’s idempotency record or reconciliation response beside the local log. Otherwise the hardest of the forty scenarios has an assertion against the wrong system.

Measure checkpoint ②’s kill switch as a distribution, not a single live demonstration. Trigger it at random turn positions in thirty scripted runs, measure from control-plane acknowledgement to the run_finished event, and sort the thirty durations. Using the nearest-rank convention, the illustrative p95 is the 29th ordered value because ceil(0.95 × 30) = 29. Declare the threshold before running the test. If your operational requirement is p95 below ten seconds, a 9.8-second result passes and a 10.1-second result fails even if the median looks excellent.

Pin enough identity to explain a changed result: source revision, model and configuration, tool-schema digest, eval-dataset digest, policy version, tenant fixture and environment. A checkpoint result without those fields is not reproducible evidence; it is a historical observation. Do not require bit-for-bit identical model output. Require the same fixture, boundaries and scorer to be reconstructable, then compare the outcome and trajectory contracts the gate actually owns.

Define the advance rule beside the test, before seeing its output. Some invariants are absolute: zero cross-tenant disclosures, zero forbidden calls in high-risk trajectories, exactly one root run span. Others are statistical: a latency percentile, a refusal-rate band, or a confidence bound on the paired change that clears the declared release margin. Mixing them produces bargaining after the run. A 99.9% aggregate pass rate cannot compensate for one tenant-boundary violation, while one slow synthetic case need not block a rollout whose declared p95 still passes. Mark each assertion hard invariant, threshold or diagnostic. Hard invariants block the dependent authority on one failure; thresholds use the declared sample and calculation; diagnostics trigger investigation without pretending to be gates.

Give exceptions an expiry test as well as an expiry date. “Allowed until day 105” invites renewal by inertia. “Allowed until the cache-key migration passes the forty crash-and-isolation scenarios, no later than day 105” states both the evidence that closes the exception and the final date beyond which authority falls back. The fallback must be executable configuration — usually read-only or disabled — rather than a promise that somebody will remember.

A.10What to skip, and what never to skip#

Skip until you have a reason Why Where the reason comes from
Knowledge graph Expensive, and only pays for multi-hop relationship questions Ch 12’s threshold test — ten questions, count the hops
Multi-agent Coordination overhead needs a measured benefit Ch 28’s five decision questions
Fine-tuning Rungs 1-6 are cheaper, faster and reversible Ch 27’s eight-point checklist
Self-authored skills Needs a verification gate and an eval suite first Ch 26 — after day 75, not before
Voice Changes latency architecture, not reasoning Ch 18 — a separate project, not a phase
Code execution Needs real isolation, and reopens the permission boundary Ch 16 — worth it, after phase 2

Six controls must arrive early to avoid later migrations:

Named terminal outcomes. Day one. Free, and everything depends on it.

The event log as the authoritative store. Day one. Retrofitting durability means rewriting the loop.

Budgets including a progress signal. Phase 2, before any unattended run.

Tenant scope as a required argument. Phase 3, before the second tenant exists.

The trace with correct parentage. Phase 2. A trace that lies about causality is worse than none.

Identity, the audit record, and data boundaries. Chapter 30’s three architectural governance requirements, decided in week one even if implemented later.

The one-page plan

If you take one artefact from this appendix, take this. Fill it in before you start.

text
TASK          the one bounded task, chosen for measurability and blast radius
DONE MEANS    the observation that proves a run correct
OUTCOMES      the terminal outcomes your loop can return
CEILINGS      turns / tokens / wall-clock / cost / tool-calls / null-results
AUTHORITY     what it may read, write, and reach — per capability, with owners
IRREVERSIBLE  which actions have no undo, and therefore need a human
STOP          how to stop it, granularity, who can, how long it takes
EVIDENCE      what you will show a reviewer on day 90
OWNER         quality, cost, and on-call — three names, possibly the same one
NEVER OPTIONAL — each is a migration if it arrives late named outcomes event log = truth budgets + progress tenant scope required trace with correct parentage THESE DEPEND ON THEM fleet view needs outcomes resumption needs the log unattended running needs bounds second tenant needs scope debugging non-determinism needs the trace + the log idempotency keys · approval gate · audit chain need durable, attributable action records eval suite with intervals start with fixtures; add observed traffic autonomy promotion needs the eval suite’s numbers DEFER UNTIL EVIDENCE ASKS — and use each chapter’s own threshold test as the trigger knowledge graph multi-agent fine-tuning self-authored skills voice code execution Sequence authority by evidence. The ninety-day duration is illustrative; repair failures or reduce scope.
Figure A.2 - The dependency graph the plan follows. Nothing is built speculatively, nothing twice, and every deferred item has a written trigger rather than an opinion.
CALENDAR TO CAPACITY — ILLUSTRATIVE TWO-ENGINEER PLAN 130 ideal days 2 × 5 days × 13 weeks × 75% 98 available days after normal interruptions split 78 planned assigned work 20 reserve unassigned repair THE 78 PLANNED ENGINEER-DAYS Spine10 Bounds17 Knowledge19 Human9 Measurement11 Production + gov12 RESERVE IS A CHECKPOINT REPAIR BUDGET — NOT A FEATURE BACKLOG Example burn after phase 2 spine +2 · bounds +5 = 7 used 20 − 7 = 13 reserve days remain Checkpoint fails? repair from reserve, rerun gate record plan, actual and delta Reserve exhausted? cut scope or move date never weaken the checkpoint
Figure A.3 - A two-engineer capacity budget for the illustrative ninety-day plan. Only 78 of 98 available engineer-days are assigned; the 20-day reserve exists to repair what checkpoints reveal, not to finance new scope.

A.11The five things that go wrong#

The pilot task was chosen to impress. Choose for measurability and blast radius. The unglamorous high-volume task with a clear correctness criterion is the one that reaches production.

Autonomy was requested all at once. Chapter 15’s ladder exists so the answer can be “read-only now, level 2 in a quarter” rather than yes or no. A binary ask gets a binary refusal.

Cost was discovered at scale. Do Chapter 7’s arithmetic in week one on a spreadsheet. It is frequently decisive and almost never done.

Nobody owned quality. An unowned eval suite decays and nobody notices until a customer does. One name, on a wall.

The prompt became the whole project. Four months of justified additions moving nothing (Chapter 25). Read your failures before you edit your prompt; their mechanisms may live outside the prompt.

A.12When a checkpoint fails, protect the gate#

The most dangerous moment in the ninety days is not a model failure. It is a checkpoint failure after stakeholders have planned around the next date. The system has produced evidence that authority must not increase, while the calendar creates pressure to reinterpret the evidence. Decide the response before that moment arrives.

Consider an illustrative failure on day forty-three. Reference Agent’s tenant-isolation fixture passes for ordinary document retrieval, but fails when two tenants issue the same query within the cache lifetime. Tenant A’s authorised result is cached under a key derived only from the normalised query. Tenant B then receives that entry without reaching the tenant-scoped store. The dashboard shows fewer retrieval calls and lower latency; the adversarial test shows a data-boundary violation.

The first assumption is that the storage filter is wrong. It is not. The filter never runs on a cache hit. The mechanism is an identity error one layer earlier: the cache key describes what was asked but not who was allowed to receive the answer. Adding another predicate to the database query changes nothing. Clearing the cache makes the test pass once and guarantees the incident returns.

The fix has three parts. Include tenant identity and the relevant authorisation version in the cache key. Purge entries written under the incomplete scheme. Add two paired fixtures: same query across different tenants must miss, and the same authorised query within one tenant should hit. The first proves isolation; the second stops the security fix from quietly disabling caching altogether. Chapter 21’s least-privilege argument and Chapter 29’s tenant architecture meet at this key.

Now do the schedule arithmetic rather than negotiating with the test. Suppose phases 1 and 2 already burned 7 of the 20 reserve engineer-days, and unrelated retrieval defects consumed another 9. Four reserve days remain. Investigation, migration and fixtures for this failure are estimated at 6 engineer-days. The plan is short by 2 engineer-days. With two engineers fully paired, that is at least one ideal calendar day; with one cache specialist and review latency, budget several. Checkpoint ③ moves. Phase 4 may continue only on work that does not assume tenant isolation, such as drafting the approval-request schema against synthetic data.

Do not take the two days from the purge, the negative fixture or the reviewer. Take them from scope. Remove the lowest-value write tool from the day-ninety manifest, which removes its schema, policy rules, approval copy and eval cases. Or reduce the pilot to one input channel. The gate remains intact and the system reaches day ninety with less authority than planned. That is a successful correction, not a failed project.

Write the checkpoint outcome as a small incident record even though no customer saw it:

Field Record
Symptom Cross-tenant cache hit in the adversarial fixture
Assumed cause Missing tenant predicate in the retrieval query
Actual mechanism Cache identity omitted tenant and authorisation version, bypassing retrieval
Repair Key migration, purge, negative isolation fixture, positive same-tenant cache fixture
Schedule effect 6 engineer-days needed, 4 in reserve, checkpoint moved and one tool removed
General rule Every cache containing authorised data inherits the identity dimensions of the source boundary

This is useful eval material. Preserve the failing fixture and tag it tenant, cache, authorisation and high-risk. Add a trajectory constraint that the run must not return a cached entry whose scope differs from the request. If the system cannot expose cache scope in the trajectory, that missing observability becomes part of the repair. A fixed incident that leaves no regression case is a loan taken against the next release.

There are only three valid outcomes when a checkpoint fails: repair and rerun; reduce the authority under test; or stop the capstone. An exception can be valid only when it is narrower than the failed claim, time-bounded, owned and reflected in the capability manifest. “Proceed because the demo is booked” is not an exception. It is the decision to make checkpoint theatre the operating model.

The general lesson reaches beyond tenancy. A failed resumption test blocks mutating tools, not read-only analysis. Insufficient paired evidence blocks the gain claim, not necessarily a change justified by a separate safety invariant. A reviewer who cannot evaluate an approval request blocks that class of consequential action, not all use of the agent. State precisely which authority depended on the failed evidence, and you can preserve the gate without turning one failure into a programme-wide halt.

Rerun the complete checkpoint after the repair, not only the new regression fixture. The cache-key migration could preserve isolation while destroying same-tenant hit rate; the purge could leave workers on the old key scheme; the new trace field could expose tenant identity where it should be redacted. A checkpoint is a bundle of claims, and repairing one failed assertion can invalidate another. Record the old failure and new pass under separate evidence ids so the governance pack shows the decision changing for a reason.

Then update the capability manifest before work resumes. Remove the deferred tool, revise its owner and autonomy level, and state which evidence is still missing. This keeps product scope, runtime authority and governance evidence aligned. A plan that says one tool was cut while an old deployment manifest still exposes it has not actually recovered schedule; it has created configuration drift at the most sensitive boundary.

A.13After day ninety#

The system now runs and can be improved. The next ninety days are chosen by evidence rather than by a plan, and the evidence comes from the weekly error-analysis session.

If your top clusters are retrieval, invest in Chapter 11 and consider Chapter 12’s threshold test. If they are context loss on long tasks, invest in Chapter 9’s externalisation. If they are cost, work Chapter 29’s six levers in order. If they are repeated procedures the agent re-derives, you are ready for Chapter 26’s skills — with the gate. If one narrow high-volume step dominates your bill, Chapter 27’s distillation is the highest-return training project.

And if your top cluster is “the task genuinely needs breadth one context cannot hold,” you have earned the right to Chapter 28 — with the single-agent baseline measured first.

Key Takeaways

  • Ninety days is an illustrative target for a bounded pilot, dependent on integration and review capacity: each phase’s controls are the precondition for the next phase’s authority.
  • Own the loop from day one. Adopt a framework later for durability or deployment if you like, but everything in Parts VI and VII has to attach to a loop you can see.
  • The read-only fortnight in phase 2 is the fast path, not a delay. It produces the failure taxonomy, first eval cases, cost model and trajectory corpus that every later phase consumes.
  • Six checkpoints, each a test you can fail: resume after a kill, breach every ceiling by name, tenant isolation in CI, a reviewer who can actually evaluate a request, paired, cluster-aware release evidence, and the governance pack read cold.
  • Skip the knowledge graph, multi-agent, fine-tuning, self-authored skills, voice and code execution until evidence asks for them — and use each chapter’s own threshold test as the trigger rather than enthusiasm.
  • Six things are never optional because they are migrations later: named outcomes, the authoritative event log, budgets with a progress signal, required tenant scope, a correctly-parented trace, and the three architectural governance requirements.
  • The five failure patterns are a pilot chosen to impress, an all-or-nothing autonomy ask, cost discovered late, unowned quality, and the prompt becoming the whole project.
  • Budget the ninety days in engineer-days, reserve recovery capacity explicitly, and spend that reserve only on evidence-driven repair — extra engineers should shorten feedback paths before they widen scope.
  • A checkpoint leaves a reproducible claim, procedure, observation and decision. When it fails, repair and rerun, reduce the authority under test, or stop; never weaken the gate to recover a date.

Hands-on

Start the ninety days 1. Fill in the one-page plan. The line you cannot complete is your first conversation, and it is cheapest now. 2. Choose your pilot task on two axes only: can you state what done means, and what is the worst it can do. Anything failing either is the wrong pilot regardless of how impressive it is. 3. Do Chapter 7’s cost arithmetic on a spreadsheet before writing code — cost per task at expected volume, for the agent design and for one design to its left. 4. Write checkpoint ① as a test before you write the loop. Making resumption a test you must pass changes the design you produce. 5. Put three names on a wall: quality, cost, on-call. If they are all the same name, that is fine — if any of them is blank, that is the project’s real risk. 6. Convert ninety calendar days into available engineer-days using your team’s measured availability. Hold back 20% as an initial recovery reserve, then cut planned scope until every remaining day has an owner and every checkpoint has repair capacity. 7. Create the four-field evidence record for checkpoint ②, breach one ceiling at a time, then breach two together. Record which outcome wins and make that precedence an explicit contract.