Social Bias: CrowS-pairs, BBQ & BOLD

Start with the harm, not the benchmark

A fairness metric is useful only when it represents a plausible product harm. The same model can look acceptable on one metric and fail badly on another because each metric asks a different question.

Before collecting data, write a measurement contract:

1. Name the user-visible harm and the population that could experience it.
2. Declare the metric direction, comparison groups, intersectional slices, and minimum denominator.
3. Freeze prompt construction, decoding, scorer versions, and human-review guidance.
4. Predeclare the fairness threshold and what pass, hold, and block mean for deployment.

Use CrowS-Pairs, BBQ, and BOLD as different lenses

CrowS-Pairs measures paired likelihood preference

Each item contains a stereotypical sentence and a minimally changed anti-stereotypical sentence. A common score is the percentage of valid pairs for which the model assigns greater likelihood to the stereotypical sentence.

• Keep the pair intact so lexical and grammatical changes do not become confounders.
• Report the number of valid pairs, preference rate, paired effect size, and interval by bias type.
• Inspect items manually because a minimal text edit can still alter fluency or meaning.
• Do not read 50% as a universal fairness target; it is a reference point for this paired task.

BBQ separates ambiguity from answerable context

BBQ presents related question-answering cases with either ambiguous or disambiguated evidence. In an ambiguous case, choosing a person based on stereotype is the failure; in a disambiguated case, the model must still use the stated facts correctly.

• Ambiguous context: measure unsupported stereotype-aligned answers and appropriate unknown selections.
• Disambiguated context: measure task accuracy and whether errors differ by group.
• Joint interpretation: a model that always refuses may reduce assumptions while destroying useful accuracy.

BOLD measures a generation distribution

BOLD prompts models to continue text about professions and social groups. One completion is not enough because decoding is stochastic and rare harmful generations matter.

• Generate multiple continuations with a frozen sampling policy.
• Score regard, sentiment, toxicity, and representation, then audit scorer errors by group.
• Compare full distributions and tail rates instead of only average sentiment.
• Treat repeated generations from one prompt as a cluster, not as fully independent prompts.

Turn a concern into comparable evidence

1. 1

   Contract

   Declare the harm

   Name the affected workflow, metric, direction of harm, and consequence of crossing the threshold.

2. 2

   Design

   Construct matched cases

   Hold task meaning constant, include counterfactual substitutions, and declare intersectional slices before seeing results.

3. 3

   Evidence

   Sample and score

   Version the model, prompt, decoding policy, automated scorers, and human-audit rubric.

4. 4

   Decision

   Gate and monitor

   Use slice intervals for release, then watch product outcomes during a bounded canary with rollback thresholds.

Estimate evaluation volume before running it

Suppose a team declares 12 product-relevant slices, selects 250 prompts per slice, and draws 5 continuations per prompt:

• Generations: 12 x 250 x 5 = 15,000.
• At 180 output tokens per generation: about 2.7 million output tokens.
• With two automated scorers: 30,000 scoring jobs, plus a stratified human audit.
• The slice denominator is still the evidence used by that metric: valid pairs for CrowS-Pairs, ambiguous cases for a BBQ assumption rate, or scored prompt-completion samples for BOLD.

More samples narrow random uncertainty, but they do not repair biased prompts, a miscalibrated regard classifier, or missing populations. When several generations share one prompt, use a prompt-cluster bootstrap or hierarchical model rather than pretending every completion is independent.

Build a versioned evaluation and release path

Store raw evidence separately from derived summaries. A report row should be traceable to a suite version, model and policy version, scorer version, prompt ID, slice keys, event label, and review status. Recomputing a threshold must not require regenerating the model outputs.

Mitigation changes more than the bias metric

A mitigation is part of the product behavior, so evaluate its consequences on the same traffic contract. A targeted reranker may reduce harmful generations but add latency and false interventions. A broad refusal rule may close a measured gap while denying benign requests unevenly.

Compare at least these outcomes:

• Harm rate and fairness gap, with denominators and uncertainty.
• Task success or quality retention for the same slices.
• False interventions on benign requests.
• Added generations, scoring work, latency, and serving cost.
• Residual failures that require human review, escalation, or rollback.

The next lab separates evaluation sampling from deployment mitigation. Increasing samples changes confidence and evaluation cost; selecting a mitigation changes user-visible behavior.

Implement a denominator-aware gate

The example below groups binary harm observations, computes Wilson intervals for each group, estimates the gap interval, and returns one of three actions. It keeps the threshold policy separate from data collection so a reviewer can reproduce both the evidence and the decision.

Use the example as a reporting boundary, not as a complete fairness definition:

• Replace the simple gap interval with a prompt-cluster bootstrap when prompts have repeated generations.
• Add scorer-audit uncertainty when labels come from an automated evaluator.
• Keep reference-group choice and metric direction explicit in configuration.
• Require a minimum denominator even when a wide interval happens to fall below the threshold.

Operate fairness as a release and monitoring loop

1. 1

   Before traffic

   Offline gate

   Block clear threshold violations, hold inconclusive slices, and preserve every tested result.

2. 2

   Limited exposure

   Shadow and canary

   Run the candidate on representative traffic, then expose a bounded cohort only after task and fairness gates pass.

3. 3

   Production

   Slice monitoring

   Track product outcomes, abstentions, complaints, and scorer drift with privacy-safe slice keys and minimum-count suppression.

4. 4

   Incident response

   Rollback and learn

   Rollback on declared thresholds, preserve affected evidence, audit root cause, and add a regression slice before re-release.

Deployment decisions are asymmetric