Prompting and its failure modes

Define the coding task, inspect the labels, and revise the instrument

What belongs in a coding prompt and how to find out where it fails: a prompting primer for the LLM coding and screening exercises.

A coding prompt turns your construct definition into instructions for a model. Small changes in those instructions can change the labels you receive. Use this primer with Chapter 6 (§6.1.3) or Chapter 4 (§4.3.1) to build a prompt, test it on source documents, and record what you learn.

The core idea: a prompt is a codebook

When you hand a coding task to a trained human assistant, you give them a codebook: the construct definition, the decision rules, worked examples, and instructions for ambiguous cases. A coding prompt is the same document, written for a machine coder. Chapter 6 calls the collection of these a promptbook.

Evaluate the resulting labels against independent coding and the source text, then preserve the prompt with your research materials.

Anatomy of a coding prompt

The template specifies the construct, decision rules, examples, and response format, including what to return when evidence is missing.

[1. ROLE AND TASK]
You are coding abstracts from academic articles on education policy.
Assign exactly one label to the abstract provided.

[2. CONSTRUCT DEFINITION]
"Equity-oriented" means the abstract frames its problem in terms of unequal
outcomes, access, or treatment across social groups, OR proposes remedies aimed
at reducing such inequality.
"Efficiency-oriented" means the abstract frames its problem in terms of cost,
throughput, optimization, or resource allocation, without reference to
distribution across groups.

[3. DECISION RULES]
- If both framings are present, choose the one that motivates the study's
  stated purpose.
- The word "access" alone is NOT sufficient evidence of equity framing;
  "data access" and "access to computing resources" are not about social groups.
- Judge only the text provided. Do not use knowledge about the authors,
  the journal, or the field.

[4. OUTPUT SCHEMA]
Return JSON only:
{"label": "equity" | "efficiency" | "insufficient_evidence",
 "evidence": "<verbatim quote from the abstract, max 25 words>",
 "confidence": "high" | "low"}

[5. WORKED EXAMPLES]
Abstract: "We evaluate a scalable optimization framework for improving student
throughput in admissions pipelines."
-> {"label": "efficiency", "evidence": "scalable optimization framework for
improving student throughput", "confidence": "high"}

Abstract: "This study examines community-based strategies for reducing barriers
to college access among first-generation students."
-> {"label": "equity", "evidence": "reducing barriers to college access among
first-generation students", "confidence": "high"}

Abstract: "We develop a low-cost tutoring allocation model and test whether it
narrows achievement gaps between low-income and high-income students."
-> {"label": "equity", "evidence": "narrows achievement gaps between low-income
and high-income students", "confidence": "low"}

[6. GUARDRAILS]
If the abstract does not contain enough information to decide, return
"insufficient_evidence". Do not guess. The "evidence" field must be a verbatim
substring of the abstract; never paraphrase or invent it.

In the third example, both framings appear. Decision rule 1 breaks the tie toward the study’s stated purpose; low confidence marks a close call. Check how the model applies this rule to similar cases in your corpus.

Check two parts of each response:

  • The evidence field. Check automatically whether the quoted text occurs in the abstract. Then read whether it supports the assigned label. A model can quote the text correctly and still interpret it incorrectly.
  • The insufficient_evidence option. Give the model a way to flag cases that the available text cannot settle. Inspect those cases and report how you handle them in later analysis.

How a reasonable prompt goes wrong

Test these four flawed variants on the same documents, then repair the prompt (Chapter 6, Exercise 6.1.3). The signatures below are patterns to look for, not guaranteed outcomes.

1. Presupposition

❌ “Identify how this abstract promotes equity.”

The instruction assumes equity framing is present. Signature: more equity labels supported by strained evidence quotes.

✅ “Determine whether this abstract is equity-oriented, efficiency-oriented, or neither, according to the definitions above.”

2. Asymmetric description

❌ A prompt where the equity definition runs 80 words with three examples, and the efficiency definition runs 12 words with none.

Unequal guidance may make one category easier to assign. Signature: a shift toward the more fully described category.

✅ Give each category comparable definitional depth and an equal number of examples.

3. No abstention option

❌ Removing insufficient_evidence from the schema.

Signature: fewer abstentions without better evidence for the assigned labels. Inspect whether ambiguous documents have been forced into a substantive category.

4. Inviting outside knowledge

❌ Omitting “judge only the text provided.”

Stuhler and colleagues document a vivid case: a model asked to extract religion from obituaries repeatedly inferred it from names and nationalities, even when instructed not to. Signature: high apparent accuracy in aggregate, with errors concentrated in exactly the subgroups where the stereotype is wrong (Chapter 6, §6.1.3 validity note).

The stress-test protocol

This is the workflow for Chapter 6, Exercise 6.1.3, steps 3–4:

  1. Run your careful prompt on a subsample of 200–300 documents. Record the label distribution.
  2. Run a deliberately flawed variant on the same documents. Change one thing (add a presupposition, or drop the abstention option), not several.
  3. Compare distributions. How far did the equity share move? Which documents flipped?
  4. Read ten flipped documents, or all of them if fewer changed. Compare each with the coding rules and look for wording that may explain the change.
  5. Repair and re-run. Check the revised labels against the documents and your coding criteria. A return to the earlier distribution alone does not establish correctness.
  6. Keep the biased labels. Chapter 6’s Exercise 6.1.4 (step 6) has you train a classifier on them. Check whether errors introduced by the prompt carry into its predictions.
Cost control

Develop against 20 documents, stress-test against 200, and only then run the full corpus. Inspect the outputs at each stage before committing to a larger batch.

Checking agreement, not accuracy

Once the prompt is stable, hand-code a random sample yourself before looking at the model’s labels for those documents, then compare.

  • Report agreement alongside class frequencies and the kinds of disagreement. Decide in advance which errors would make the labels unsuitable for your research purpose; no single agreement threshold establishes validity.
  • Read every disagreement. Three outcomes: the model was wrong (revise the prompt), you were wrong (revise your understanding), or the case is genuinely ambiguous (revise the definition, and note that the construct has fuzzy edges).
  • Check whether errors cluster by period, venue, or group. Examine how that pattern could change the comparison your study is trying to make (Chapter 5, §5.7).
Agreement is not validity for open-ended tasks

Agreement with another coder is an accuracy estimate only when that coder’s labels can serve as a defensible reference. For open-ended work (discovering themes, judging a whole document), competent human coders legitimately disagree, so there is no single key to score against, and high consistency across prompts or models may reflect shared training bias rather than correctness. Chapter 7 (§7.4) takes this up: when the task is interpretive, validation shifts toward structured expert judgment of output quality.

A second template: relevance screening

For Chapter 4’s screening exercise, the same anatomy applies with a narrower task:

You are screening search results for a systematic literature review.

INCLUSION: empirical studies of civic participation in which participation is
measured at the individual level.
EXCLUSION: theoretical/conceptual pieces with no data; studies where
"participation" refers to patient participation in clinical care or user
participation in sensing systems; non-English abstracts.

Judge only the title and abstract provided.

Return JSON only:
{"decision": "include" | "exclude" | "uncertain",
 "reason": "<one clause, citing the criterion applied>",
 "evidence": "<verbatim phrase from the abstract>"}

If the abstract is too vague to apply the criteria, return "uncertain".
Do not guess.

The exclusions distinguish different uses of the same word, such as participation in civic life and participation in clinical care. Inspect retrieved examples to decide which distinctions your own screening rules need.

Recording what you did

Keep the verbatim prompt, output schema, model name and version, settings, run date, hand-coding agreement rate, and known failure modes in your promptbook (Chapter 2’s kb/codebooks/). When the instrument changes, update the promptbook and log the version, reason, and audit findings in memory/PROMPTLOG.md, so earlier outputs remain traceable.

Chapter 9 (§9.3.3) adds one more requirement. Because you cannot pin a commercial model the way you pin a library version, archive the prompt–output pairs themselves. Cached outputs are the reproducibility boundary for everything downstream.

Where to go next