Demonstration Data Curation
Master demonstration data curation: active learning, quality assessment, diversity optimization, and human-in-the-loop validation for fine-tuning.
What is demonstration data curation?
Demonstration data is a collection of examples that shows a model how to respond: an input or instruction paired with a desired output. Curation is the controlled process of defining, collecting, reviewing, balancing, versioning, and releasing those examples.
In plain language, a demonstration dataset is not just text to copy. It is a set of behavioral specifications expressed as examples. During supervised or instruction fine-tuning, the model learns patterns from those examples, including useful behavior, accidental shortcuts, blind spots, and inconsistencies.
The invariant to remember
Every accepted example must teach an intended behavior, carry enough provenance to audit its use, and improve a measured coverage need. More rows do not compensate for a vague rubric, missing slices, duplicated patterns, or unsafe authority.
One teaching record
A demonstration
Contains the instruction, optional context, target response, metadata, rubric result, source lineage, and review history needed to understand why it belongs.
One governed artifact
A curated release
Contains a versioned set of records plus a manifest, quality and coverage evidence, exclusions, access policy, evaluation results, and rollback target.
Define the behavior before collecting examples
A team cannot judge an example until it defines the behavior the example should teach. Start with a behavior contract that a reviewer can apply consistently.
- Task boundary: name the requests the model should answer and the requests it should reject, defer, or escalate.
- Output contract: define structure, level of detail, evidence requirements, allowed tools, and forbidden side effects.
- Correctness rubric: distinguish factual correctness, reasoning validity, instruction following, style, and completeness.
- Safety boundary: state privacy, policy, consent, licensing, and high-consequence review requirements.
- Coverage plan: list domains, languages, difficulty levels, formats, important user groups, and failure cases.
- Decision ownership: identify who can accept, quarantine, revise, and promote records and dataset releases.
Preference data asks which response is better; demonstration data provides the response the model should learn to produce. A production program may collect both, but they have different schemas, reviewer tasks, and quality evidence.
1 Contract
Specify behavior
Write the rubric, refusal boundary, coverage matrix, and release gates before requesting examples.
2 Candidate pool
Collect with lineage
Capture source, authoring method, consent, license, tool use, model lineage, and transformation history with every candidate.
3 Accepted set
Review and balance
Validate correctness and safety, remove duplicates, resolve disagreement, and fill measured coverage gaps.
4 Versioned artifact
Evaluate and release
Train a candidate model, evaluate held-out slices, freeze the manifest, approve the release, and retain a rollback target.
Size the curation program, not only the dataset
Collection plans need a candidate funnel because many proposed examples will be rejected, revised, or quarantined. Two simple estimates expose the operational load:
candidate pool = target accepted examples / acceptance rate
expert review hours = candidate pool x expert-audit share x minutes per review / 60
For 12,000 accepted examples at a 62% acceptance rate, the team needs about 19,355 candidates. Auditing 35% of them for four minutes each consumes about 452 expert hours before adding automated screening, adjudication, or rework.
The calculation is not a quality forecast. It forces the team to make source, review, risk, and coverage assumptions explicit enough to test with a pilot.
Loading curation planner
Mix sources for independent evidence
No source is automatically high quality. The useful question is what evidence each source contributes and what failure it can introduce.
Controlled authority
Domain experts
Experts can produce and judge nuanced, high-consequence answers. They are expensive, may share the same institutional assumptions, and still need calibrated rubrics and adjudication.
Breadth at scale
Trained contributors
Crowd or internal contributors can add language, style, and scenario breadth. Qualification tests, hidden gold tasks, agreement monitoring, and clear escalation paths are essential.
Fast expansion
Synthetic candidates
Models can generate variations, adversarial cases, and rare formats cheaply. Their outputs are candidates, not independent truth, and must retain generator and prompt lineage.
Keep train and evaluation evidence independent
- do not let the same generator, rubric model, and judge define both training rows and proof of quality;
- hold out prompts, authors, sources, time windows, and critical slices where leakage would make evaluation optimistic;
- record whether a human wrote, edited, approved, or merely viewed a synthetic example;
- preserve rejected and quarantined reasons so repeated failure patterns become measurable;
- treat licenses, consent, privacy, and retention as acceptance gates rather than cleanup tasks.
Quality is a vector, not one score
An example can be fluent but wrong, correct but unsafe, or useful but duplicated. Keep the dimensions separate long enough to see which one fails.
Correct
Task truth
The response is factually and logically valid for the supplied context.
Aligned
Instruction fit
The output follows the requested scope, format, constraints, and refusal rules.
Safe
Harm boundary
The example respects privacy, policy, authorization, and high-consequence limits.
Needed
Coverage value
The record fills a measured task, language, difficulty, format, or failure gap.
Build a review system that can disagree
- Run deterministic checks for schema, empty fields, secret patterns, impossible metadata, and exact duplicates.
- Use model-assisted checks to route likely defects, not to silently establish truth.
- Assign qualified reviewers using blinded or overlapping review where disagreement matters.
- Send conflicts and policy-sensitive cases to adjudication with a named owner.
- Recalibrate the rubric when disagreement clusters around the same instruction or slice.
Average reviewer agreement is not enough. Measure agreement by rubric dimension, reviewer cohort, language, domain, and risk level so one easy majority slice cannot hide a failing minority slice.
Make every record reproducible
The training row and its evidence should travel together through the pipeline.
From raw candidate to auditable demonstration
Each boundary appends evidence; no stage should erase the source or the reason for acceptance.
Raw proposal
Candidate
Store input, target output, source identity, authoring method, timestamps, consent, license, and generator lineage.
Deterministic evidence
Automated screen
Validate schema, secrets, exact and semantic duplicates, policy indicators, language, and known benchmark overlap.
Judgment
Human review
Record rubric version, dimension scores, reviewer qualification, confidence, edits, and escalation reason.
Accepted version
Curated record
Freeze the accepted content hash, transformations, coverage labels, split assignment, and links to prior versions.
Governed artifact
Release manifest
List record hashes, exclusions, aggregate evidence, access controls, training runs, evaluations, approvers, and rollback lineage.
Spend expert review where it changes the dataset
Random sampling gives an unbiased estimate of average quality and should remain part of the audit plan. Active curation answers a different question: which candidate is most valuable to review next?
A transparent priority model can combine:
- model uncertainty, where a baseline or candidate model behaves inconsistently;
- coverage gap, where a task, language, risk, or user slice is underrepresented;
- harm severity, where an incorrect target could teach dangerous behavior;
- source novelty, where a candidate contributes evidence not already repeated in the set.
The example below ranks a fixed candidate pool under a review budget. The weights are policy, not universal constants, so they should be versioned and validated against real review outcomes.
Active selection can distort prevalence estimates because reviewed examples are no longer a random sample. Keep a separate random audit stream and record each candidate's selection probability when downstream analysis needs unbiased estimates.
Balance coverage without cloning the same pattern
Dataset diversity is useful only when it represents meaningful variation. Surface-level paraphrases can inflate row count while leaving behavior unchanged.
- Build a coverage matrix across task, language, domain, difficulty, format, risk, user group, and expected behavior.
- Set minimums for critical intersections such as difficult non-English refusals, not only one-dimensional totals.
- Detect exact duplicates before semantic near-duplicates; inspect clusters before deleting valid repeated conventions.
- Split related records by source or semantic cluster so paraphrases do not leak across training and evaluation.
- Cap dominant templates, generators, authors, and domains when their repetition crowds out rarer evidence.
- Review long-tail examples for correctness rather than preserving unusual content merely because it is novel.
Watch the effective sample size
Ten thousand near-duplicate examples do not provide ten thousand independent teaching signals. Track cluster concentration, unique source count, template share, and semantic-neighbor density alongside raw row count.
Promote evidence, not a large row count
A dataset release should pass explicit gates before it can train a production candidate. Gates remain separate because an aggregate score can let evaluation lift hide a privacy incident or let high average safety hide a failing critical slice.
The release lab below applies one remediation at a time. Notice that deleting duplicates can improve uniqueness without repairing missing provenance, privacy incidents, or narrow coverage.
Loading release evidence
Connect dataset evidence to model behavior
Dataset-only checks cannot prove that fine-tuning helped. Train a candidate under a pinned configuration and compare it with the current production model on held-out behavioral evaluations.
Before training
Dataset evidence
Measure schema validity, provenance, duplicates, reviewer agreement, quality dimensions, coverage cells, privacy findings, and split leakage.
After training
Model evidence
Measure task success, safety, calibration where relevant, refusal behavior, regressions, memorization, latency, and critical slices against a frozen baseline.
Why it changed
Causal evidence
Run ablations by source, slice, and curation rule. Correlation between a larger dataset and a better model does not identify which examples created the gain.
Can it be owned
Operational evidence
Verify access controls, reproducibility, approval history, retention, incident response, rollback, and the ability to delete affected records and derived artifacts.
Design for curation failures
Reviewer failure
Reward hacking the rubric
Contributors learn surface patterns that score well without satisfying the behavior. Rotate hidden gold tasks, inspect rationales, and measure model outcomes beyond rubric completion.
Source failure
Synthetic feedback loop
A model generates, scores, and filters its own patterns until errors look consistent. Keep independent human and external evidence in both review and evaluation.
Metric failure
Coverage theater
Marginal totals look balanced while critical intersections remain empty. Gate named slice combinations and publish the matrix with the release.
Governance failure
Unrecoverable lineage
The team cannot identify licensed, private, or deleted material after training. Hash and version records, transformations, splits, checkpoints, and derived models.
Prepare a degraded path
- quarantine a candidate source when provenance, privacy, or reviewer behavior becomes uncertain;
- stop promotion when any blocking gate fails, even if aggregate model quality improves;
- continue serving the previous model and dataset bundle while a candidate is investigated;
- remove affected record clusters, rebuild derived splits, retrain, and repeat held-out evaluation;
- preserve incident evidence without exposing sensitive content in routine logs or dashboards.
Operate curation as a feedback system
- Monitor production gaps: collect user outcomes, escalations, refusals, regressions, and drift without treating raw user data as automatically reusable training material.
- Create a gap backlog: convert observed failures into owned behavior, coverage, and risk requirements.
- Collect targeted candidates: use the source best able to provide independent evidence for the gap.
- Review and adjudicate: apply the current rubric, preserve disagreement, and update guidance when failure is systematic.
- Evaluate a candidate release: run dataset gates, train a pinned model, replay held-out slices, and inspect regressions.
- Promote or roll back: publish the complete release bundle only after approval; keep deletion and rollback paths tested.
Keep these mental models
- Demonstrations are executable behavioral specifications, not interchangeable text rows.
- Provenance, consent, licensing, and privacy are part of quality.
- Diversity means measured behavioral coverage, not visual or lexical variety alone.
- Active curation prioritizes scarce review; random audits estimate overall prevalence.
- Dataset evidence and model evidence answer different questions, and production promotion requires both.