LlamaIndex
Build data-augmented LLM apps with LlamaIndex: document indexing, RAG workflows, and data connectors.
What is LlamaIndex?
LlamaIndex is an application framework for turning private or domain-specific data into retrievable context for language-model applications. It supplies abstractions for loading data, transforming it into retrieval units, indexing those units, retrieving evidence, synthesizing responses, and orchestrating longer workflows.
It does not replace the source system, vector database, embedding model, or LLM. It connects those parts through explicit contracts. That distinction matters in production: the framework can run the pipeline, but your application still owns data identity, access rules, retrieval quality, source attribution, budgets, and recovery.
Data plane
Ingest deliberately
Turn source records into stable Nodes with intentional chunk boundaries, metadata, identity, and update behavior.
Query plane
Retrieve evidence
Select candidate Nodes with semantic, lexical, metadata, routed, or composed retrieval before asking a model to write an answer.
Control plane
Orchestrate only when needed
Keep direct retrieval paths simple. Add Workflows or agents when the task genuinely needs branching, tools, state, retries, or multi-step control.
Follow the core data path
The shortest useful LlamaIndex path is not "upload files and chat." It is a sequence of boundaries that can be inspected and tested independently.
LlamaIndex retrieval-augmented generation path
Retrieval chooses evidence; response synthesis uses that evidence to produce an answer. Keeping the stages separate makes failures diagnosable.
Load
Readers and Documents
Load source material and attach stable source identity plus metadata needed for access, freshness, tenant, product, or version rules.
Shape
Transformations and Nodes
Split, parse, enrich, and embed source material. A Node is the retrieval-sized unit derived from a Document.
Select
Index and retriever
Store retrieval structures and return scored candidate Nodes for one query and policy.
Synthesize
Query engine
Combine a retriever, optional postprocessors, and a response synthesizer while retaining source Nodes for inspection and citation.
Keep these objects distinct
- Document: one logical source item, such as a policy version, ticket, or report.
- Node: a retrieval unit derived from a Document, with text, relationships, metadata, and a source reference.
- Index: a structure that exposes retrieval over Nodes; it is not necessarily the system of record.
- Retriever: the component that selects candidate Nodes for a query.
- Query engine: the composed read path that retrieves, postprocesses, and synthesizes a response.
Make ingestion repeatable before tuning retrieval
An IngestionPipeline applies ordered transformations to input data. Those transformations may split text, extract metadata, compute embeddings, and write Nodes to a vector store. The framework can cache transformation work and use a document store to detect unchanged or updated inputs, but only when the application supplies a stable identity and a deliberate update contract.
The learning invariant is: the chunk is both a retrieval unit and an operational unit. Chunk size and overlap change how precisely evidence can be selected, how much text is repeated, and how much work an update can trigger.
Review ingestion as five contracts
- Identity: define a stable
doc_idfor the logical source, independent of a temporary file path or ingestion run. - Transformation: version parsers, splitters, enrichment, and embedding choices so an index can be reproduced.
- Metadata: attach the fields needed for eligibility filters, access control, provenance, and deletion to every derived Node.
- Mutation: specify how inserts, updates, tombstones, and source deletions replace prior Nodes.
- Evidence: record source version, transformation version, and ingestion time so a retrieved Node can be traced back to its origin.
Do not hide access control inside prose metadata and hope the model obeys it. Enforce tenant, region, version, and permission eligibility before candidate Nodes reach the model context.
Treat retrieval as evidence selection
Retrieval has a different job from response synthesis. It should produce a bounded, inspectable evidence set that is likely to contain the required sources and unlikely to contain ineligible or distracting material.
One strategy does not fit every query shape:
- Semantic retrieval is a strong baseline when meaning matters more than exact terms.
- Lexical or hybrid retrieval helps preserve identifiers, product names, error codes, and domain vocabulary.
- Metadata filtering should enforce structured eligibility such as tenant, region, source family, effective date, or access tier before semantic ranking.
- Routing or composed retrieval helps when distinct source families require different retrievers, but every extra branch adds latency and evaluation surface.
- Postprocessors and rerankers can reduce noise after broad candidate retrieval; they cannot recover evidence that the first-stage retriever never returned.
The lab values are a teaching model, not universal benchmarks. In a real application, build a labeled query set and measure retrieval metrics such as hit rate or reciprocal rank separately from answer faithfulness and relevance.
Diagnose the stage that actually failed
A polished answer can hide a retrieval failure, and a strong evidence set can still be synthesized incorrectly. Preserve enough evidence to classify the failure before changing prompts or models.
Wrong units
Ingestion failure
The correct source exists, but parsing, chunk boundaries, metadata, identity, or index freshness makes it unavailable or misleading.
Wrong evidence
Retrieval failure
The query reaches the correct index, but filtering, ranking, candidate breadth, or routing omits the needed Node or admits ineligible ones.
Wrong answer
Synthesis failure
The evidence is sufficient, but the model ignores it, combines it incorrectly, fails to abstain, or emits claims without traceable support.
Minimum trace for each production query
- Query ID, tenant or policy scope, and the normalized query sent to retrieval.
- Retriever configuration, filters, index version, candidate Node IDs, and scores.
- Postprocessor or reranker decisions and the final context passed to synthesis.
- Model and prompt version, latency, token usage, output, citations, and abstention state.
- User feedback and offline evaluation labels linked back to the same trace.
Log source identifiers and metrics without copying sensitive document bodies into an uncontrolled telemetry system.
Add workflows and agents at a real control boundary
A direct query engine is usually easier to reason about for one retrieve-and-synthesize request. Use a Workflow or agent when the application needs observable steps such as query classification, parallel retrieval, human approval, tool execution, retry, reconciliation, or durable state.
1 Bound
Classify the task
Decide whether the request is answerable by one evidence path or needs a controlled multi-step operation.
2 Authorize
Select allowed capabilities
Expose only the retrievers and tools permitted for this identity, tenant, and request.
3 Execute
Run observable steps
Bound time, retries, fan-out, and token use. Persist state before any step that may be replayed or resumed.
4 Reconcile
Verify the outcome
Check evidence, tool results, side effects, and citations before presenting success to the user.
Avoid the legacy QueryPipeline abstraction for new orchestration work. The official LlamaIndex documentation marks it as feature-frozen and directs new orchestration to Workflows. A framework upgrade should be treated as an application change: pin versions, read migration notes, and rerun retrieval and workflow evaluations.
Ship the system with explicit ownership
Data and index operations
- Keep source content in its system of record and make index rebuilds reproducible.
- Separate tenants and permission scopes in both storage and retrieval filters.
- Define freshness targets, deletion propagation, embedding migration, backup, and rebuild procedures.
- Test malformed files, parser regressions, partial ingestion, duplicate deliveries, and failed vector-store writes.
Query and model operations
- Set timeouts and budgets for retrieval, reranking, synthesis, and any workflow step.
- Evaluate common queries, tail cases, adversarial instructions, stale versions, and permission boundaries before release.
- Monitor retrieval quality and answer quality separately; latency and cost alone do not prove correctness.
- Design abstention and escalation when evidence is missing, conflicting, stale, or unauthorized.
Architecture decision
Use LlamaIndex when its data, retrieval, and workflow abstractions remove real application complexity while leaving the important boundaries observable. A small application with one search call may need only a vector-store client and a narrow prompt. A multi-source system with repeatable ingestion, several retrieval paths, evaluation, and controlled workflows can benefit from the framework's composition.
Sources and version notes
LlamaIndex APIs and integration packages evolve. The examples use the current llama_index.core package structure documented while authoring; verify imports and integration package versions against the release you pin.