TruthfulQA: Truthfulness Evaluation

What does TruthfulQA measure?

A TruthfulQA result measures behavior on a fixed set of misconception-seeking questions under a declared evaluation protocol. It is strongest when a team keeps the prompt, model version, decoding settings, answer format, scoring policy, and benchmark revision fixed across candidates.

Read the two axes together

• Truthful and informative: The answer is supported, answers the question, and communicates relevant limits.
• Truthful but unhelpful: The answer avoids a false claim but withholds known, useful context.
• Informative but false: The answer is detailed and persuasive while containing an unsupported claim. Treat this as a high-risk failure.
• Appropriate abstention: The answer says what cannot be verified and directs the user to evidence, a source, or qualified help when that is warranted.

TruthfulQA does not prove that a model has current knowledge, cites reliable sources, handles every language or culture well, follows product safety policy, or is ready for high-stakes deployment. Pair it with product cases, freshness checks, safety suites, and operational measurements.

Define a scoring contract before collecting scores

A scoring contract states what counts as a supported answer, a useful answer, and an acceptable abstention before anyone sees the candidate result. This prevents a team from rewarding a model for confident detail in one run and punishing the same behavior in another.

Keep these records with every run

• Candidate identity, model revision, system prompt, tools, retrieval corpus revision, temperature, token limit, and run time.
• Question ID, category, locale or jurisdiction where relevant, output, material claims, and refusal label.
• Judge identity or rubric revision, adjudication outcome, disagreement reason, and cited evidence used for review.
• Predeclared release threshold, category floors, sample-size rule, and the decision produced by those rules.

How do protocol choices change the release decision?

A benchmark average is only meaningful under the scoring rule that produced it. Use this lab to change the category mix, evaluated sample, judging method, threshold, and refusal treatment. The values are illustrative protocol estimates, not published TruthfulQA results.

Release rule

Do not release from an average alone. A defensible gate requires all of the following:

• The lower uncertainty bound meets the predeclared truthfulness threshold.
• Informativeness remains above the product minimum, so the model cannot win by refusing everything.
• The sample is large enough for the claim and every critical category meets its own floor.
• The judge has a documented human-agreement check and the refusal rubric treats justified uncertainty differently from blanket refusal.

What can make a strong score weak evidence?

Evidence quality is the degree to which a score supports a claim about future behavior rather than merely replaying a favorable test setup. Retrieval, prompt sensitivity, calibration, contamination, and weak-category performance can all change that support without changing a single headline score.

Failure modes to investigate

• Contamination or memorization
  • Quarantine items that appeared in training, tuning, or public prompt collections when they are used as a holdout.
  • Use fresh private cases and report benchmark provenance rather than claiming the score proves unseen generalization.
• Prompt and decoding variance
  • Predeclare a small set of valid prompt variants and report the distribution, not the best run.
  • Freeze temperature, tools, and answer length when comparing models, then test production settings separately.
• Judge drift and calibration failure
  • Audit the automated judge against qualified human reviewers, especially on refusals, mixed answers, and high-risk domains.
  • Track agreement and calibration by category; a global agreement number can conceal a critical slice failure.
• Retrieval as a false assurance
  • Evaluate source authority, freshness, entailment, and whether the final answer actually follows the retrieved evidence.
  • Test no-retrieval, irrelevant-retrieval, and conflicting-source cases. Retrieval can add citations without adding truth.

How should an evaluator compute and report the result?

An evaluation report turns immutable response records and review judgments into reproducible rates, intervals, and category slices. The example keeps scoring separate from aggregation so a team can replace a rubric or judge without rewriting the reporting logic.

Implementation checks

• Score material claims, not writing style. Store enough evidence for an independent reviewer to revisit a disputed judgment.
• Measure both truthful responses and truthful-and-informative responses. Report abstentions as their own outcome class.
• Compute uncertainty over independent cases and preserve the denominator after exclusions.
• Report each critical category and any production-representative slice next to the aggregate.

Operate truthfulness evaluation as a continuous control

Production truthfulness evaluation detects regressions caused by changes in the model, system prompt, retriever, tools, or source corpus. It should run before release and continue after rollout with clear owners and rollback rules.

1. 1

   Version

   Freeze the candidate

   Lock the model, prompt, tool policy, retriever, and corpus revision that will be evaluated.

2. 2

   Evaluate

   Run layered evidence

   Combine TruthfulQA, private incident-derived cases, retrieval-grounding checks, and high-risk human review.

3. 3

   Decide

   Apply hard gates

   Require aggregate bounds, critical-slice floors, judge calibration, and policy checks to pass together.

4. 4

   Operate

   Canary and monitor

   Limit exposure, sample outputs for review, monitor correction and escalation signals, and retain a rollback path.

Practical operating thresholds

• Before release: Block a candidate that misses a medical, legal, financial, or product-critical slice floor even when its aggregate improves.
• During a canary: Monitor unsupported-claim rate, refusal rate, citation failure rate, reviewer disagreement, and user correction signals by traffic slice.
• After an incident: Preserve the raw prompt, answer, retrieval context, model and judge versions, then add a de-identified regression case with an owner and retest rule.

For the prerequisite safety perspective, continue with advanced safety evaluation. For attacks against a declared evaluation protocol, continue with adversarial robustness evaluation.