Knowledge Graphs & Structured Memory
When the answer is a relationship — LLM-driven construction, entity resolution, bi-temporal facts, and an honest account of what graphs cost.
“If we deprecate the settlement connector, what breaks, and who has to sign off?”
Reference Agent’s retrieval stack could not answer this, and no amount of reranking would have helped. The question is not about finding a document; it is about traversing relationships that no single document contains. The connector is referenced in three services, owned by two teams, all of which have change-approval requirements recorded in a policy document that never mentions the connector. The answer exists — distributed across a dozen sources, as a path through a graph — and one similarity query does not retrieve the complete path. Iterative retrieval, relational joins and graph traversal are alternative ways to assemble it.
Knowledge graphs make these paths explicit.
12.1Three shapes of question#
Sort your agent’s questions into three buckets, because the right storage follows directly.
Lookup questions have their answer inside one document. What is the reconciliation cutoff? Vector or lexical retrieval may suffice; Chapter 11 handles this pattern.
Aggregate questions need many documents summarised. What are the recurring themes in last quarter’s postmortems? No single chunk contains a theme, and top-k retrieval returns five postmortems out of ninety. This needs broad coverage: long context, query-time map/reduce summaries or a precomputed structure.
Relationship questions need paths. What depends on this, who owns those things, what approvals do they require, and did anything change since the last release? The answer is a subgraph. A single similarity query can fragment the required relationships. A graph makes these joins explicit; it is not the only way to compute them.
Design Rule
Build a graph when relationships are the answer, not when relationships exist. Almost every domain has relationships; only some domains have questions whose answers are paths. Write real questions and compare the simplest viable retrieval, relational and graph implementations before committing to extraction and maintenance costs.
12.2What a graph is, operationally#
An agent-facing knowledge graph contains three elements:
Entities with a stable identity and type: Service:reports, Person:priya, Policy:change-approval-v3, Tenant:acme.
Relationships, typed and directed, between entities: reports --depends_on--> settlement_connector, settlement_connector --owned_by--> platform. The type vocabulary should be small — a dozen or two — because a large uncontrolled vocabulary makes querying, extraction and evaluation harder.
Provenance and time on every edge. Where this edge came from, and when it was true. This is what separates a knowledge graph you can operate from a diagram: without provenance you cannot verify or revoke; without time you cannot answer anything historical, and you cannot distinguish “no longer true” from “never true.”
The schema question — rigid ontology or emergent types — is a real fork. A rigid schema gives consistent queries and clean joins, at the cost of an extraction step that fails whenever reality does not fit. An emergent schema, where the extractor proposes types, captures more and produces a graph where owns, owned_by, and is_owner_of are three different edge types describing one relationship. The workable middle: a fixed core of the dozen relationship types your questions actually need, plus an other bucket that captures the rest as text for later promotion. Review the other bucket monthly; it tells you what your schema is missing.
12.3Construction, and what it costs#
Building a graph from text is an extraction problem, and it is expensive in a way that is easy to underestimate.
EXTRACT = """Extract entities and relationships from the document below.
Use ONLY these entity types: Service, Person, Team, Policy, Tenant, Incident, Document
Use ONLY these relationship types: depends_on, owned_by, requires_approval_from,
caused_by, references, member_of, applies_to
Rules:
- Use the exact name as written; do not normalise or expand abbreviations.
- Emit a relationship only if the document states it. Do not infer.
- For each relationship, quote the span that supports it.
- If the document states when something became true, record it.
Return JSON: {"entities": [...], "relationships": [
{"from": ..., "type": ..., "to": ..., "evidence": "<quoted span>",
"valid_from": "<ISO date or null>"}]}
"""Three instructions in that prompt are doing most of the work. “Do not normalise” keeps resolution as a separate, auditable stage instead of hiding it inside extraction where you cannot inspect it. “Do not infer” is the most important line: an extractor that infers produces a graph full of plausible edges nobody stated, and a plausible false edge in a graph is worse than a missing one because traversal will find and trust it. “Quote the span” makes every edge verifiable and gives you the provenance the previous section demanded.
The cost is where teams get surprised. Extraction is one or more model calls per document, so a corpus of 200,000 documents is 200,000+ calls — and it is not once, because the schema will change, the extractor will improve, and re-extraction is a full re-run. Budget for three or four full passes in the first year.
Two mitigations can reduce cost. Extract incrementally: process changed documents so steady-state work follows the change rate rather than corpus size. Extract selectively: architecture documents, ownership registries, policies and incident reports may contain more useful relationships than general prose. Measure useful relationship yield by document class before choosing the extraction scope.
11The uncomfortable arithmetic that kills most graph projects: extraction cost scales with corpus size, but graph value scales with the number of multi-hop questions you actually ask. Those two numbers are unrelated, and the second is usually far smaller than anyone estimated.12.4Entity resolution is the hard part#
reports, reports-service, ReportingService, the reporting pipeline, and svc-reports-prod are one thing. If your graph has five nodes for it, every traversal is wrong — silently, because a traversal from the wrong node returns a valid-looking empty result rather than an error.
Use four identity-resolution techniques together, in this order:
Anchor on identifiers, not names. If the entity exists in a system of record — a service catalogue, an HR directory, a CMDB — use its identifier as the canonical node key and treat names as aliases. This converts an unbounded matching problem into a bounded lookup, and it is by far the highest-value move available. Most enterprises have these registries and most graph projects ignore them.
Blocking then matching. Compare only candidates that share a cheap key — a normalised prefix, a token in common — then apply expensive matching within blocks. Comparing every pair is quadratic. Blocking reduces work but can exclude a true match; measure its recall and allow alternate blocks.
Model-assisted matching with abstention. For ambiguous pairs, ask a model with the surrounding context — and let it answer “unclear.” An abstention that queues for human review is enormously more valuable than a coin flip, because a wrong merge is very hard to undo once traversals depend on it.
An alias table you can edit. Whatever the automation, someone will need to say “these two are the same” or “these two are not.” Make that a first-class, audited operation rather than a database surgery.
Failure Mode
A wrong merge is worse than a missed merge, and much harder to detect. Two distinct services merged into one node produce a graph where every dependency query returns a superset — plausible, connected, and wrong. A missed merge produces a sparse node whose emptiness is at least visible. Bias your matcher toward precision, queue the ambiguous cases, and keep merges reversible with an audit trail.
12.5Time: facts that were true#
Most graph tooling models a single present state, and agents need more than that. Two temporal dimensions matter, and conflating them causes real errors.
Valid time is when the fact was true in the world. The connector was owned by analytics from March to June.
Transaction time is when the system learned it. We recorded that ownership change in July, backdated to March.
Both are needed because agents get asked both kinds of question. Who owned this in April? is valid time. What did we believe in May, when that decision was made? is transaction time — and it is the question every incident review and every audit asks. A graph with only present state cannot answer either.
The mechanism is edges that are never deleted, only closed:
@dataclass(frozen=True)
class Edge:
src: str
type: str
dst: str
recorded_at: datetime # trusted ingestion clock; required
valid_from: date | None = None # None = effective start unknown
valid_to: date | None = None # None = no known end
invalidated_at: datetime | None = None
evidence: str = "" # the span that supports it
source: str = "" # document id + version
confidence: float = 0.8An ownership correction closes the old system-time version and appends corrected valid-time versions, as the next example shows. Mutating valid_to in place would destroy the earlier belief. Historical retention is deliberate, with deletion and access policy still applied. This is the model Zep’s temporal knowledge graph architecture formalises for agent memory — a bi-temporal graph where relationships carry both when they held and when they were learned, precisely so an agent can reason about change rather than only about the present.1
The cost is that every query needs a time predicate, and a query without one has silently asked “at any point in history,” which returns a union of contradictory states. Default the predicate to now() in your query layer so the common case is right, and make the historical form explicit.
12.6The late correction that creates three answers#
Consider an illustrative ownership incident. On 1 April, the settlement connector moves from the Analytics team to the Platform team. The transfer is agreed in a meeting and takes effect immediately, but the service catalogue is not updated until 3 May. On 22 April, Reference Agent is asked who must approve a connector change. It reads the graph, finds owned_by Analytics, and routes the request to the Analytics approver. The request waits for seven hours before Analytics rejects it and points to Platform.
The first diagnosis is “the graph was stale.” That is true but incomplete. Staleness explains why the operational answer on 22 April was wrong. It does not tell you what the graph should say after the correction arrives. If the 3 May ingestion job overwrites owned_by Analytics with owned_by Platform, three different questions now collapse into one current-state answer:
- Who owns the connector now? Platform.
- Who actually owned it on 22 April? Platform, because the transfer was already valid.
- Who did Reference Agent believe owned it when it routed the request on 22 April? Analytics, because the correction had not yet been recorded.
The third answer matters most during review. A present-state overwrite destroys it. The trace still says Reference Agent chose Analytics, while the graph now says Platform owned the connector. Without transaction time, an investigator can only conclude that Reference Agent ignored its knowledge. The system has rewritten the evidence beneath the decision.
Treat correction as a new system version, not a destructive edit. Before 3 May, the graph has A0: Analytics · valid=[start,∞) · system=[t0,2026-05-03). Closing A0’s system interval preserves it for an as-known query on 22 April.
On 3 May, append A1: Analytics · valid=[start,2026-04-01) · system=[2026-05-03,∞) and P1: Platform · valid=[2026-04-01,∞) · system=[2026-05-03,∞). A current valid-time query for 22 April returns P1; the same query at transaction time 22 April returns A0. Separate system intervals preserve the old belief and the corrected history without asserting simultaneous owners. Both predicates are mandatory: omitting either silently converts a historical question into a different query that may still return plausible data.
Keep the clocks separate. The source supplies valid time; the ingestion system supplies transaction time and never accepts a source-authored value for it. Otherwise a backdated document can claim Reference Agent knew something earlier. With no effective date, record valid_from as unknown rather than copying recorded_at: learning a fact on 3 May does not make it true from 3 May.
Now make the correction workflow explicit:
- Resolve the connector and both teams to canonical identifiers.
- Find the open
owned_byedges for that connector. - End the original assertion’s system interval at the correction time; retain the row as history.
- Append corrected Analytics and Platform versions with their source spans, source versions, valid intervals and new system intervals.
- Re-run affected materialised views and invalidate cached traversal answers whose evidence includes those edges.
- Emit a correction event listing the old and new edge identifiers, so Chapter 23’s trace can connect changed knowledge to changed answers.
These are target production contracts, not features of the reference Reference Agent harness. Bi-temporal history lookup, evidence-to-edge joins, traversal-cache invalidation and edge-identifier-to-trace correlation are all production extensions; the reference harness implements none of them.
Step 4 still requires semantic judgement. An open owned_by Finance Operations edge from an escalation policy may describe financial approval rather than operational ownership. A “one owner” constraint would close it incorrectly. If the edge type cannot distinguish the roles, abstain and queue the case for review. Temporal machinery must not preserve semantic ambiguity more precisely.
Late facts require selective replay, not blind re-answering. If the edge appeared in 47 retrieval bundles, join its identifier to traces and prioritise runs where ownership selected an approver, changed permission or triggered an external action (Chapter 19). In this illustrative incident, nine historical answers contain two approval routings and one completed change. Review those three action-bearing runs first.
The general lesson is sharper than “store history.” Your graph must reproduce the information boundary at decision time. Valid time reconstructs the world. Transaction time reconstructs the agent’s evidence. If you cannot reproduce both, you can explain neither a wrong historical answer nor a correct decision made from information that was later corrected.
12.7Querying: three interfaces, and which to give an agent#
Generated query language. The model writes Cypher, SPARQL or SQL. Maximally expressive, and it fails in the way generated code fails: syntax errors are cheap and semantic errors are not. A query that runs can return a plausible but wrong subgraph; syntax success alone will not reveal the error. Mitigations: a strict schema in the prompt, validation before execution, a read-only role, a row and time limit, and — most usefully — returning the query alongside the result so the agent and a human can see what was actually asked.
Parameterised traversals. A small set of typed tools: dependents_of(entity, depth), owners_of(entity), approvals_required_for(change), changed_between(entity, t1, t2). Far less expressive and far more reliable, and — the important part — each one is testable. This is the right default for an agent.
Natural-language-to-subgraph with verification. The model describes what it wants, a resolver translates it into one of the parameterised traversals, and the result comes back with the traversal that produced it. A middle path that keeps expressiveness while preserving auditability.
The recommendation is unambiguous: give the agent parameterised traversals, and reserve generated queries for an escape hatch behind a read-only role with hard limits. The reasoning is Chapter 13’s, arriving early: a tool whose failure mode is a silently wrong answer is a worse tool than one whose failure mode is “cannot express that.”
| Interface | Expressiveness | Failure mode | Testable | Agent-appropriate |
|---|---|---|---|---|
| Generated Cypher/SPARQL | High | Silently wrong subgraph | Yes; wider test space | Escape hatch only, read-only, limited |
| Parameterised traversals | Medium | “Cannot express that” | ✓ | ✓ Default |
| NL → traversal + echo | Medium-high | Wrong traversal chosen, but visible | ✓ | ✓ When the tool set grows large |
12.8Work a traversal before you trust it#
A parameterised traversal is reliable only when you can account for every expansion, deduction and limit. Take the opening question: If we deprecate the settlement connector, what breaks, and who has to sign off? Reference Agent should not issue one unconstrained query and summarise whatever comes back. It should run a typed traversal with an explicit plan.
Start with this illustrative graph:
| Hop | Relationship read | Edge records read | Distinct results after deduplication |
|---|---|---|---|
| 1 | settlement_connector <- depends_on - Service |
3 edges | 3 services |
| 2 | Service - owned_by -> Team |
3 edges | 2 teams |
| 3 | Team - governed_by -> Policy |
2 edges | 2 policies |
| 4 | Policy - requires -> ApprovalRole |
5 edges | 3 roles |
The three services are Reports, Nightly ETL and Dashboard API. Reports and Dashboard API share Analytics as owner; Nightly ETL belongs to Platform. Analytics is governed by Policy A and Platform by Policy B. Policy A requires a service owner and a risk approver; Policy B requires a service owner, a change manager and the risk approver. The shared risk role appears in both policies.
Raw edge counts are not answer counts. The traversal reads 13 unique edge records: 3 + 3 + 2 + 5. Deduplication leaves 1 connector + 3 services + 2 teams + 2 policies + 3 approval roles = 11 nodes. Yet the user-facing answer has three affected services and three distinct approval role types. Reporting “five approvers” would count the same risk role twice and the service-owner role once per policy. Reporting “two policies” would omit that those policies jointly name three role types; the number of people depends on role assignments and separation-of-duties rules. The graph gives structure; the query contract still has to define the unit being counted.
The final answer should preserve enough path information to show how each conclusion was reached:
| Affected service | Owner | Policy | Required roles |
|---|---|---|---|
| Reports | Analytics | Policy A | service owner, risk approver |
| Dashboard API | Analytics | Policy A | service owner, risk approver |
| Nightly ETL | Platform | Policy B | service owner, change manager, risk approver |
Then Reference Agent can say: “Three services are affected. Analytics and Platform own them. Across the applicable policies, approval is required from the service owner, change manager and risk approver.” It should attach the seven service-to-role paths and their supporting source spans, not a bag of eleven nodes. A node list proves that things exist. A path proves why they are in the answer.
Now cost the same traversal. Suppose each returned edge contributes an illustrative 34 tokens of identifiers, type, validity and provenance, and each supporting evidence span contributes 52 tokens. Returning all 13 edges with evidence costs 13 × (34 + 52) = 1,118 tokens before the answer prompt. If the tool instead emits seven compact answer-bearing paths while retaining one evidence span per unique edge, the payload is 7 path records × 46 tokens + 13 unique evidence records × 52 tokens = 998 tokens. The saving is only 120 tokens here. The reason to normalise the result is not the saving; it is that Reference Agent sees one identity for each fact and cannot accidentally vote for a duplicated edge.
The dangerous arithmetic is frontier growth. With an average branching factor of four, a depth-four traversal can inspect 4 + 16 + 64 + 256 = 340 edges. Increasing the depth to six raises that to 4 + 16 + 64 + 256 + 1,024 + 4,096 = 5,460 edges — sixteen times as many for two extra hops. A single high-degree node such as Team:all-engineering makes the average misleading and can consume the entire query budget on one branch.
Put four ceilings and one relationship allow-list in the tool contract:
max_depth: maximum edges, or hops, from the anchor; depth 4 reaches nodes four edges away even when a type repeats;allowed_edge_types: the relationship types and directions this traversal may cross — here,depends_onin reverse, thenowned_by,governed_byandrequires;max_expansions: total edges read, regardless of how many survive filtering;max_paths: complete paths returned to the harness;max_evidence_tokens: provenance text admitted to context.
For this traversal, a defensible illustrative budget is depth 4, 200 edge expansions, 25 complete paths and 2,000 evidence tokens. If any ceiling is reached, return truncated=true, the frontier that was cut, and counts before and after deduplication. Do not return a partial subgraph as though it were complete. “I found three dependants before the 200-edge limit” is operationally different from “there are three dependants.”
Filter at the edge, not after expansion. The question asks what breaks now, so every hop needs valid_at=now, the caller’s tenant, allowed entity types and provenance visibility. Expanding historical edges and filtering them after six hops wastes the budget and can leak the existence of another tenant’s entities through counts or timing (Chapter 15). The query planner must reject a traversal that omits the tenant predicate; the model must not be responsible for remembering it.
Finally, test the traversal with metamorphic properties rather than only fixed examples. Adding an unrelated service must not change the result. Adding an expired dependency must not change a present-time query. Replacing an alias with its canonical entity must not change path count. With no other budget truncation and exhaustive traversal, tightening max_depth must not introduce a result absent at the larger depth. Under other ceilings, exploration order can change the returned subset; compare only complete traversals for this property. These tests catch the semantic errors a syntactically valid generated query hides.
The worked example exposes the real advantage of a typed traversal. You can state what the answer counts, where deduplication happens, how time and tenancy constrain each hop, and what the tool says when its budget is exhausted. An unconstrained query language gives you more possible paths. A production tool needs fewer, better-defined ones.
12.9GraphRAG and its variants#
Graphs can also answer aggregate questions across a corpus.
The GraphRAG approach builds an entity graph from the corpus, detects communities of related entities, pre-generates summaries for each community, and answers global questions by combining relevant community summaries. This targets exactly the query class top-k retrieval cannot serve — “what are the main themes across this whole corpus” — where the answer requires the corpus in aggregate rather than any of its parts.2
Several successors trade construction cost against quality in different ways. LightRAG uses a dual-level retrieval scheme over a graph-augmented text index, aiming for comparable quality at substantially lower construction and update cost, with incremental updates rather than full rebuilds.3 HippoRAG takes a memory-inspired approach, using a knowledge graph with a personalised PageRank-style retrieval to support the multi-hop association that single-step retrieval misses.4
The practical guidance for choosing:
- Global/thematic questions over a large corpus → compare a GraphRAG-family approach with hierarchical summarisation and long-context baselines.
- Multi-hop factual questions → a graph with parameterised traversals, or a HippoRAG-style graph retrieval.
- Frequently changing corpus → prioritise incremental-update designs; full-rebuild architectures become operationally untenable fast.
- Lookup questions → start with lexical, vector or structured retrieval and measure whether graph construction adds enough value.
GraphRAG documentation checked on 9 November 2026 still distinguishes local, global, DRIFT and basic search. Choose the search mode for the question. HippoRAG 2 adds passage-level integration and stronger association to the original graph-memory approach. Both broaden the comparison set, but their reported benchmark improvements must be re-tested on your corpus, update rate and access-control requirements.56
12.10The honest cost-benefit#
Graph construction and maintenance have several costs:
What you pay. Extraction, at one or more model calls per document, repeated on every schema change. Entity resolution, which is an ongoing operational cost with a human in it, not a one-time job. A graph store to run, back up, and secure. Query tooling to build and test. Freshness machinery, because a stale graph is worse than a stale index — it produces confidently wrong structure. And a real learning cost for the team.
What you get. Inspectable answers to multi-hop relationship questions. Aggregate views over a corpus. Explicit structure lets a reviewer correct an edge. An embedding error usually requires correcting its source, representation or retrieval configuration. And a natural home for the provenance and temporality that Chapter 10’s semantic memory needed anyway.
The threshold test. Write down the ten questions you most want answered that you currently cannot. Count how many require two or more hops through relationships that no single document states. Use that count to choose a prototype, not as a numerical investment rule. Compare task value, change rate, extraction quality and operating cost against iterative retrieval and relational joins.
A useful intermediate option is to build a small, hand-curated graph covering only the entities and relationships those questions need, populated from systems of record rather than extracted from prose. A service catalogue joined to an ownership registry joined to an approval policy table is a knowledge graph. It can have a few thousand nodes and avoid model extraction costs. Its accuracy still depends on source freshness, identity resolution and complete ownership records. Many organisations need this and build a corpus-wide extraction pipeline instead.
12.11Measure answers at path level#
A graph dashboard full of node counts and ingestion throughput can stay green while every useful answer degrades. Those are capacity measures. Operate the graph against the contract the agent consumes: resolve an entity, traverse valid edges, produce complete paths, attach evidence, or abstain honestly.
Build an evaluation set from real relationship questions and freeze the expected answer as paths, not prose. Each case needs the starting entity identifier, query time, allowed relationship sequence, expected terminal entities, acceptable evidence sources and whether “no answer” is correct. Keep historical cases because a present-only suite will never detect the overwrite failure from the previous section. Keep adversarial aliases because canonical names make entity resolution look solved.
Suppose Reference Agent’s illustrative suite contains 200 cases: 60 answerable one-hop questions, 60 answerable two-hop questions, 40 answerable three-hop questions and 40 questions whose correct result is abstention. The graph returns an answer in 158 cases and abstains in 42. Of the returned answers, 146 are fully correct, 6 omit at least one required path, 4 contain a wrong path on an answerable case, and 2 answer a question that has no supported path. The categories are mutually exclusive: six returned answers are wrong, and six more are incomplete.
Do the arithmetic rather than reporting “79% answered”:
- Strict answer precision:
146 ÷ 158 = 92.4%. An incomplete answer is not fully correct. - Answerable-case success:
146 ÷ 160 = 91.25%. This exposes four false abstentions as well as wrong and incomplete answers. - Unsupported-answer rate:
2 ÷ 40 = 5%on cases where abstention was required. - Correct-abstention precision:
38 ÷ 42 = 90.5%. Four of the 42 abstentions hid an answer the graph should have found.
Now segment the 146 correct cases by hop count. If 58 of 60 one-hop cases, 55 of 60 two-hop cases and 33 of 40 three-hop cases are correct, accuracy falls from 96.7% to 91.7% to 82.5%. The overall score concealed the slope. Longer paths combine more opportunities for a missed resolution, false edge, stale interval or over-tight query budget. That is the operational signal: inspect which hop and edge type first loses the expected path.
| Measure | Exact calculation | Bad value means | First fix |
|---|---|---|---|
| Anchor resolution precision | correctly resolved starting mentions ÷ resolved starting mentions | Wrong graph neighbourhood selected before traversal begins | raise match threshold; expand canonical identifiers; abstain more |
| Expected-path recall | expected complete paths returned ÷ expected complete paths | Missing edge, wrong time predicate, unresolved alias or early budget cut | inspect first missing hop and truncated flag |
| Unsupported-path rate | returned paths with no valid supporting span ÷ returned paths | Extraction inferred, evidence was revoked, or provenance join failed | quarantine the edge and re-run extraction |
| Temporal validity rate | returned edges valid at the requested world time and visible at the requested transaction time ÷ returned edges | Query omitted one clock or correction handling is broken | enforce both predicates in the traversal layer |
| Freshness lag | graph transaction time minus source-change detection time, at p50/p95/p99 | Change capture or resolution queue is falling behind | split ingest, resolve and publish latency |
| Truncation rate | traversals ending at any ceiling ÷ traversals | The tool contract does not cover observed graph shape | add a narrower traversal; do not merely raise every ceiling |
| Coverage-qualified absence | empty results with resolved anchor, required sources current and no truncation ÷ empty results | The system is presenting unknown coverage as “nothing found” | return unknown with the missing coverage condition |
Trace these measures per edge type, source, tenant, extractor version and hop number (Chapter 23). A global 98% anchor-resolution precision can hide that Service anchors run at 99.7% while Team anchors run at 84%. Likewise, a p50 freshness lag of four minutes can coexist with a p99 of eleven days if ambiguous entities wait in a review queue. The tail is where an agent routes work to an owner who changed last week.
Edge precision does not equal answer precision. As an illustrative heuristic, if every edge were independently correct 97% of the time, a three-edge answer path would be fully correct only 0.97³ = 91.3% of the time. Real errors are not independent: one wrong entity merge can corrupt hundreds of paths, while one stale source can invalidate a whole subgraph. Use the multiplication only to reject the comforting idea that “97% accurate edges” means “97% accurate answers.” Measure complete paths directly.
Sampling has to follow the same logic. Randomly reviewing 100 edges mostly selects common, short, recently ingested facts. Review complete answer paths, oversampling long paths, high-degree nodes, late corrections, low-confidence resolutions and results that caused an action. For every sampled path, ask four binary questions: is the start entity right, is every edge stated by its evidence, was every edge valid and known at the requested time, and does the path support the final claim? One “no” makes the path wrong.
Coverage is part of the measurement contract: a graph cannot prove absence unless it can prove the relevant coverage. An empty traversal may mean no service depends on the connector. It may also mean the relevant repository was never ingested, an alias failed to resolve, the ownership import is eleven days late, the caller cannot see one source, or the 200-edge ceiling cut the only useful branch. Graph structure makes positive answers inspectable; it does not turn missing data into negative knowledge.
Return absence with its boundary. Reference Agent should say: “No dependants were found in the service catalogue snapshot recorded at 08:10 UTC; two repositories are outside this index and the traversal was not truncated.” If source coverage is unknown or a required source is stale beyond its service-level objective, the result is unknown, not an empty list. This distinction is especially important before destructive changes: the safe interpretation of “no dependants found” is not “nothing will break.”
Do not set a universal target for answer precision or freshness. A recommendation graph can tolerate an occasional missing relationship; an approval-routing graph cannot tolerate an unsupported owner. Define targets by decision consequence, then gate actions accordingly. Reference Agent may use a lower-confidence path to suggest who might know, but it must require current, source-backed ownership before it selects an approver. The graph tool should return the confidence inputs and coverage statement; policy decides what those facts permit.
Measurement changes the architecture in one productive way. It forces every answer to carry a query identity, graph snapshot or transaction-time boundary, canonical anchor, traversed edge identifiers, evidence identifiers, limits reached and final disposition. That record is larger than a fluent sentence and far more useful. It lets you reproduce the answer, correct the edge, find affected runs and know whether the next version improved the path rather than merely phrasing it better.
Graph, vector, and relational as three tools
Do not choose. Give the agent all three and let it route, with the trade-offs described in each tool’s own description.
@tool(keywords=("search", "documents", "explain", "why", "how"))
def search_docs(query: str, top_k: int = 4) -> str:
"""Search document text for explanations, procedures and prose.
Best for: why something works, how to do something, narrative context.
Not for: what depends on what, who owns something, exact aggregates."""
@tool(keywords=("depends", "owns", "impact", "relationship", "approval"))
def graph_query(entity: str, relation: str, depth: int = 2,
as_of: str = "") -> str:
"""Traverse relationships between known entities, optionally as of a date.
Best for: dependencies, ownership, approval chains, blast radius.
Not for: prose, explanation, anything not modelled as an edge."""
@tool(keywords=("count", "sum", "metric", "how many", "trend"))
def warehouse_query(question: str) -> str:
"""Aggregate over structured operational data.
Best for: counts, sums, trends, exact numbers.
Not for: anything not in the warehouse schema."""The tool descriptions are the routing logic — the “not for” clauses do more work than the “best for” ones, because they prevent the failure where the agent uses the graph for a prose question and reports that it found nothing.
12.12Graphs as structured memory#
There is a use for graphs that has nothing to do with corpora: as the store for Chapter 10’s semantic memory.
The fit is good. Semantic memory needed provenance on every fact, temporality so corrections supersede rather than coexist, contradiction detection, and scoping. A bi-temporal graph can represent all four; the write gate, contradiction checks and scope enforcement still require implementation. “Tenant acme’s reconciliation cutoff is 21:00 UTC” is an edge with a source, a validity interval, and a confidence — and the April misstatement from Chapter 10’s incident would have been a separate low-confidence edge from a distinct source, visibly contradicting the config-derived one rather than silently replacing it.
This is a smaller, better-scoped, far more tractable graph than a corpus-wide extraction: hundreds or a few thousand facts, written through a gate, each with provenance. If you want to build one graph, build this one. It solves a problem you demonstrably have, can be cheaper to construct, and — unlike a corpus graph — its value does not depend on how many multi-hop questions your users happen to ask.
Definition
Entity resolution — determining that several surface forms refer to the same real entity, and merging them under a canonical identity.
Bi-temporal graph — one where every edge records both when the fact was true (valid time) and when the system learned it (transaction time).
GraphRAG — building an entity graph plus pre-computed community summaries to answer corpus-global questions that top-k retrieval structurally cannot.
Part III is complete. The agent now knows what goes in its context, what it remembers, what it can look up, and what it can traverse. Part IV turns to the other half of agency: not what the agent knows, but what it can do.
Key Takeaways
- Sort questions into lookup, aggregate and relationship. The third most directly benefits from explicit relationships, and building one because relationships exist rather than because relationship questions exist is the standard way these projects fail.
- An operational graph is entities with stable identity, a small vocabulary of typed edges, and provenance plus time on every edge. Without provenance you cannot verify or revoke; without time you cannot distinguish “no longer true” from “never true.”
- Extraction must not normalise and must not infer, and every edge must quote its supporting span. A plausible false edge is worse than a missing one, because traversal will find it and trust it.
- Entity resolution is where graph projects die, and quietly. Anchor on identifiers from systems of record, block before matching, let the matcher abstain, and keep merges reversible — a wrong merge is far worse than a missed one.
- Give the agent parameterised traversals, not a generated query language. A tool whose failure mode is a silently wrong subgraph is worse than one whose failure mode is “cannot express that.”
- Cost each traversal in edge expansions, paths and evidence tokens, and return truncation explicitly. A partial subgraph is not evidence that no other path exists.
- Run the threshold test: count the questions you cannot answer that need two or more hops. Use the questions to compare a curated graph, relational joins and iterative retrieval before funding broad extraction.
- A bounded first graph can be a bi-temporal store for semantic memory, where provenance, supersession and contradiction are already requirements.
- Evaluate complete, time-valid, evidenced paths and justified abstentions. Edge accuracy and an empty result do not by themselves establish that an answer is correct.
Hands-on
Run the threshold test before you build anything
1. Write the ten questions you most want answered and currently cannot. For each, count the hops and note whether any single document states the answer. Use this list to define the pilot and its evaluation cases.
2. Pick your three most important entity types and count distinct surface forms for ten real instances of each. That number is your resolution problem, and it is the cost nobody budgets.
3. Find the systems of record you already have — service catalogue, HR directory, CMDB, policy table. Build the small curated graph from those first. Estimate the integration work and measure which target questions it answers.
4. Implement three parameterised traversals rather than a Cypher generator. Write a test for each. Note how much easier that is than testing generated queries — that difference is the argument.
5. Add valid_from/valid_to and a source to every edge from the start. Retrofitting temporality into a graph that has present-state semantics is a migration, and it is one you will need.
6. For one traversal, write down allowed_edge_types, max_depth, max_expansions, max_paths and max_evidence_tokens. Force each ceiling in a test and verify that the tool returns an explicit partial-result status.
7. Build twenty path-level evaluation cases: five one-hop, five two-hop, five three-hop and five correct abstentions. Report accuracy by depth and inspect the first failing hop in every wrong path.
-
Rasmussen, P. et al. Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv:2501.13956. https://arxiv.org/abs/2501.13956 ↩
-
Edge, D. et al. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130. https://arxiv.org/abs/2404.16130 ↩
-
Guo, Z. et al. LightRAG: Simple and Fast Retrieval-Augmented Generation. arXiv:2410.05779. https://arxiv.org/abs/2410.05779 ↩
-
Gutiérrez, B. J. et al. HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. arXiv:2405.14831. https://arxiv.org/abs/2405.14831 ↩
-
Microsoft. GraphRAG query overview. Accessed 9 November 2026. https://microsoft.github.io/graphrag/query/overview/ ↩
-
Gutiérrez, B. J. et al. From RAG to Memory: Non-Parametric Continual Learning for Large Language Models (HippoRAG 2). arXiv:2502.14802. Accessed 8 September 2026. https://arxiv.org/abs/2502.14802 ↩