Knowledge Graphs
Build knowledge graphs for AI: graph databases, semantic relationships, and enhanced information retrieval.
What is a knowledge graph?
A knowledge graph represents facts as entities and named relationships. Instead of storing only a paragraph that says "Atlas depends on Token Service," the graph stores Atlas, Token Service, and a directed DEPENDS_ON edge between them.
That structure matters when an application must answer more than a keyword question. It can follow paths, enforce types, preserve where each fact came from, and explain why two entities are connected. A graph does not make extracted text true by itself: identity resolution, relation validation, provenance, and access control determine whether its answers deserve trust.
The core invariant is that every accepted node and edge remains traceable to evidence, policy, and time. The language model may propose or explain facts; it must not silently promote an unsupported claim into graph truth.
Read a graph as claims, not as a picture
Four parts turn connected data into a usable knowledge system. Each part answers a different question.
Who or what?
Entity
A stable identity such as employee EMP-104, service svc-token, or paper doi:10.1234/example. Names are attributes, not durable identifiers.
How connected?
Relationship
A typed, directed claim such as WORKS_ON, DEPENDS_ON, or CITES. Direction and meaning must be explicit.
What is allowed?
Schema
Entity types, relation types, required properties, and constraints keep accidental strings from becoming arbitrary graph structure.
Why believe it?
Provenance
Source records, extraction versions, confidence, timestamps, and permissions let the system verify or withdraw a claim.
For example, the claim EMP-104 -[WORKS_ON]-> PROJECT-ATLAS is useful only if the system can establish that several records refer to EMP-104, that WORKS_ON means current assignment, and that the supporting source is visible to the person asking.
Build graph truth in explicit stages
Extraction creates candidates. Resolution and validation decide which candidates become authoritative.
1 Observe
Capture source records
Keep the original text, source identity, event time, tenant, and access policy before deriving graph structure.
2 Normalize
Resolve entity identity
Match aliases to canonical IDs using stable keys and several independent signals. Preserve unresolved candidates instead of forcing a merge.
3 Constrain
Validate relationships
Check endpoint types, direction, confidence, corroboration, temporal validity, and conflicts with existing facts.
4 Own
Publish versioned claims
Write accepted nodes and edges with provenance. Quarantine weak candidates for review and make retractions possible.
Entity resolution changes the graph itself
Two records with similar names may describe one entity, while two identical names may describe different people. A false merge transfers every accepted relationship onto the wrong entity. A missed merge creates duplicate islands that hide paths and split history.
Use stable identifiers first, then combine normalized names with attributes such as verified email domains, organization IDs, timestamps, and source authority. Thresholds should reflect the cost of a false merge versus the cost of manual review.
Shape queries around graph operations
A graph query should name the kind of answer it needs. Increasing traversal depth without a question contract usually adds unrelated nodes, latency, and permission risk.
Zero or one hop
Lookup
Read an entity and its direct properties, such as the owner of a service or the current status of a project.
Bounded expansion
Neighborhood
Inspect nearby entities with allowed edge types, hop limits, direction, and fan-out caps.
Connected explanation
Path
Find a typed route between entities and return the relationship chain that supports the conclusion.
Population insight
Aggregate
Count, rank, or cluster over a declared subgraph while preserving the filters and snapshot used for the result.
Bound traversal before optimizing it
- Declare the starting entities and allowed relationship types.
- Set a maximum hop depth and fan-out for every expansion.
- Filter by tenant, authorization, and validity time during traversal, not after it.
- Return path evidence with the answer so users can inspect the connection.
- Measure empty results, explosive neighborhoods, stale facts, and permission-pruned paths separately.
Combine graph paths with document evidence
Graph-enhanced retrieval uses structure to find relevant entities and paths, then gathers source passages that support those claims. The language model receives a compact evidence package rather than an unrestricted graph dump.
Evidence-preserving Graph RAG path
The answer is generated only after graph claims and source passages survive policy, freshness, and permission checks.
Scope
Question contract
Resolve query entities, intent, tenant, allowed relation types, hop limit, and freshness target.
Connect
Bounded traversal
Retrieve candidate paths without crossing forbidden edges or expanding an unbounded neighborhood.
Support
Evidence join
Attach source passages, timestamps, extraction versions, and permission decisions to every candidate claim.
Verify
Claim gate
Reject proposed, stale, contradictory, or inaccessible evidence according to the answer policy.
Explain
Answer or abstain
Generate from the surviving path and cite it, or state which evidence boundary prevented an answer.
Retrieval policy changes what the system may claim
The shortest path is not automatically the safest answer. A path can be structurally complete but contain a proposed edge, an expired source, or evidence the requester cannot access.
Design the graph store around access patterns
Property graphs and RDF graphs offer different modeling and query conventions, but the operational questions are shared:
- Identity: Which IDs remain stable across source systems, merges, and deletions?
- Cardinality: Which nodes can have millions of neighbors, and how will queries cap fan-out?
- Time: Does an edge describe current truth, historical truth, or a validity interval?
- Consistency: Which graph writes must be atomic with the source transaction, and which can arrive through an outbox or stream?
- Indexes: Which labels, properties, vector representations, and edge directions serve the dominant queries?
- Isolation: How are tenant and document permissions enforced before a path becomes model context?
Store documents outside the graph when they are large or have independent retention needs. Keep durable document IDs and source spans on graph claims so retrieval can join structure back to evidence without duplicating whole documents into node properties.
Operate knowledge graphs as changing evidence
Knowledge graphs drift because sources, schemas, organizations, and access policies change. Treat graph quality as an observable production contract.
Identity
Resolution quality
False merges, missed merges, unresolved candidates, and review age
Edges
Claim quality
Acceptance, contradiction, retraction, provenance, and stale-edge rates
Paths
Retrieval quality
Answerable, permission-pruned, over-depth, empty, and unsupported queries
Answers
User outcome
Citation support, abstention precision, freshness, latency, and correction rate
Failure modes to rehearse
- False identity merge: quarantine affected edges, split the canonical entity, replay dependent claims, and notify downstream indexes.
- Schema migration: dual-read or dual-write compatible versions, validate backfills, and retire old relation types only after query migration.
- Source retraction: invalidate every derived claim and cached answer linked to the source rather than deleting provenance first.
- Hot supernode: add typed filters, fan-out budgets, pagination, summaries, and query-cost limits before adding hardware.
- Permission change: revoke path visibility and derived caches promptly; do not assume graph-level authorization covers copied model context.
- Extractor change: shadow the new version, compare claim precision by relation type, and keep extraction version on every proposal.
The strongest graph systems can explain both why an edge exists and why another candidate edge was rejected. Rejection evidence is operational data, not discarded noise.
Production readiness checklist
- Define canonical IDs, alias rules, merge review, and reversible split procedures.
- Version entity types, relation types, constraints, and extraction prompts or models.
- Attach source, timestamp, confidence, extractor version, tenant, and access policy to every claim.
- Bound traversal by relation type, direction, depth, fan-out, time, and authorization.
- Distinguish verified, proposed, disputed, expired, and retracted edges in storage and retrieval.
- Evaluate graph construction separately from answer generation so identity and relation errors remain attributable.
- Load-test dominant path shapes, supernodes, permission filtering, and evidence joins.
- Test source deletion, identity split, schema rollback, graph rebuild, and cache invalidation.