Chapter 2 — Project templates

The Rules / Memory / Knowledge skeleton, ready to adapt

The Rules / Memory / Knowledge project skeleton from Chapter 2 as adaptable templates, with a screencast of the structure steering a real AI-assisted analysis.

Chapter 2 argues that most disappointing AI-agent performance is an environment problem rather than a model problem, and proposes a three-part structure: rules (how the agent works), memory (where the project stands), and knowledge (what the project knows). This page holds the files that implement it.

What you will produce. A project directory that an agent can work in without being re-briefed every session: rules it reads, a memory it updates, and a knowledge base it draws on. You will populate it across the rest of the book.

Download

project-skeleton.zip

The complete directory structure with all templates in place. Unzip it, rename the folder, and start filling in memory/CURRENT.md.

Or take the files individually:

File Dimension What it does
AGENTS.md Rules Project-wide behavioral expectations: scope of agent authority, file conventions, verification requirements
analysis/AGENTS.md Rules Analysis-specific refinements: where scripts live, class-level metrics, seed discipline
writing/AGENTS.md Rules Manuscript-specific refinements: voice, citation integrity, file handling
memory/CURRENT.md Memory Current direction, priorities, active assumptions, next steps
memory/DECISIONS.md Memory Append-only decision log with rationale and rejected alternatives
memory/PROMPTLOG.md Memory Revision history of prompt-based instruments
kb/README.md Knowledge Map of the knowledge base so an agent knows which file is authoritative
kb/codebooks/abstract-coding-promptbook.md Knowledge The current prompt-based coding instrument, filled in for the worked example (v3)
README.md — Human-facing project overview

The structure at a glance

project/
├── AGENTS.md              # Rules: how the agent works
├── README.md
├── memory/                # Memory: where the project stands
│   ├── CURRENT.md
│   ├── DECISIONS.md
│   ├── PROMPTLOG.md
│   └── runs/              # dated notes for specific analyses
├── kb/                    # Knowledge: what the project knows
│   ├── README.md
│   ├── concepts/
│   ├── literature/{theory,methods,empirical}/
│   ├── data-docs/
│   └── codebooks/         # codebooks AND promptbooks
│       └── abstract-coding-promptbook.md
├── data/{raw,interim,processed,external}/
├── analysis/              # + its own AGENTS.md
│   ├── scripts/ notebooks/ validation/
├── writing/               # + its own AGENTS.md
└── outputs/{figures,tables,memos}/

The minimum viable version

If the full skeleton feels like too much for a project that has not started yet, Chapter 2 (§2.6.1) recommends three files. Create these and nothing else:

  1. AGENTS.md with your file conventions, basic verification expectations, and instructions for handling uncertainty.
  2. memory/CURRENT.md with your current research direction, one or two open questions, and immediate next tasks.
  3. kb/README.md with a brief map of what materials you have and where they are.

Those three let any agent answer the three questions that orient a session: How should I work here? What is the project currently doing? What materials should I use?

What they look like filled in

The downloads above are blank forms. Below is the same set filled in for the book’s running example: a study of how research abstracts frame work as equity-oriented or efficiency-oriented, built on the OpenAlex corpus from Chapter 4 and coded in Chapter 6. The book explains what belongs in each file; these are one project’s answers on one particular day.

The rules file

AGENTS.md at the project root. Each section settles something the agent would otherwise decide for itself:

# Project Rules

## Scope of Agent Authority
- You may create and edit files in analysis/, writing/, and outputs/.
- You may read but not modify files in data/raw/.
- Ask before modifying files in kb/ or memory/.

## File and Folder Conventions
- Follow the existing directory structure.
- Name new scripts with a numeric prefix (e.g., 01_clean_data.py).
- Save intermediate outputs to data/interim/ with a date prefix.

## Verification Requirements
- Save interim outputs before finalizing any analysis.
- After coding tasks, inspect a random sample of at least 20 records.
- Report class-level precision and recall, not aggregate accuracy alone.
- Compare new results against prior versions when they exist.
- Flag ambiguity rather than forcing a conclusion.

## Documentation Requirements
- Update memory/CURRENT.md after any significant direction change.
- Log major decisions in memory/DECISIONS.md with date and rationale.
- Document new variables in the relevant codebook under kb/codebooks/.
- If LLMs are used for coding or extraction, store the current promptbook
  under kb/codebooks/ and log major prompt revisions in memory/PROMPTLOG.md.

## What to Do When Uncertain
- State the uncertainty explicitly.
- Propose two or three options with brief rationale.
- Do not proceed with a default assumption silently.

Specificity is what separates this from a wish. “Be careful with the data” gives an agent nothing it can act on; “do not modify files in data/raw/” tells it exactly where the line is.

Memory, on one particular day

memory/CURRENT.md while the coding dictionary is being revised for the third time:

# Current Project Direction

## Research Focus
Exploring how scientific abstracts frame research as equity-oriented
vs. efficiency-oriented, using the OpenAlex corpus compiled in the data
management chapter.

## Current Priorities
1. Finalize the deductive coding dictionary (v3 in progress).
2. Run a second BERTopic model with revised stop words.
3. Draft the triangulation comparison between dictionary and topic model
   outputs.

## Active Assumptions
- Treating "equity" and "efficiency" as non-mutually-exclusive categories.
- Using abstracts only (not full text) as the unit of analysis.

## Known Risks
- The term "access" appears in both equity and efficiency contexts; the
  dictionary may need disambiguation rules.
- BERTopic results are sensitive to minimum topic size; need stability tests.

## Immediate Next Steps
- Audit 30 high-disagreement cases from the triangulation table.
- Update kb/codebooks/deductive-codebook.md with v3 revisions.

Notice what is absent. There is no account of how the dictionary got to v3, and no definition of equity. The first belongs in memory/DECISIONS.md, the second in kb/concepts/. This file answers one question, what the project is doing right now, which is what keeps it short enough to be read at the start of every session.

One promptlog entry

memory/PROMPTLOG.md. The promptbook in kb/codebooks/ holds the prompt currently in force; this file records why it changed:

## 2026-03-03: Abstract coding prompt v2
- Task: Classify abstracts as equity-oriented, efficiency-oriented, both,
  or neither.
- Promptbook version: kb/codebooks/abstract-coding-promptbook.md (v2)
- Change: Added an instruction to quote textual evidence and return
  "ambiguous" when criteria conflict.
- Why: The first audit showed unsupported inferences from background
  knowledge rather than evidence in the abstract.
- Evidence from audit: In a 20-case review, 6 disagreements involved labels
  that could not be justified from the source text alone.
- Implications: Re-run the affected batch and manually inspect all cases
  flagged as ambiguous.

That entry documents a measurement decision, not a preference. Six of twenty cases failing for one identifiable reason is the kind of evidence a methods section has to be able to report, and it is unreconstructable a month later if nobody wrote it down.

What working with an agent actually looks like

The book describes the working relationship in words: you set the question and the constraints, the agent executes, and you check what comes back. Reading that is not the same as watching it happen. The recording below is the demonstration, in an ordinary VS Code window, on a real study. It goes from an empty folder to a drafted report by building a co-author network from OpenAlex inside a copy of this skeleton, and it runs at the speed the work actually takes.

How Researchers Can Use AI Without Losing Methodological Control, about 50 minutes. If the player does not load, watch it on YouTube.

It sits on this page because the skeleton is what makes that session possible. The agent opens by reading the rules file and the knowledge base instead of asking to have the project re-explained, and the raw-data rule keeps it out of data/raw/ with nobody supervising. If you do not have fifty minutes, these are the moments to jump to:

The rest runs past this chapter into literature gathering, network construction, regression, and the write-up, so it doubles as a preview of where the folders you are about to create end up.

Adapting the templates

  • Rules should be specific enough to act on. “Be careful with the data” is a wish. “Do not modify files in data/raw/; save intermediate outputs to data/interim/ with a date prefix; after any coding task inspect a random sample of at least 20 records” is a configuration.
  • Keep memory selective. It is not a transcript. Record decisions, direction changes, and dead ends, not every exchange.
  • Do not collapse the three. The most common failure is one long file mixing instructions, notes, and source material. An agent then cannot tell which text should constrain its behavior, which is history, and which is evidence.
  • Promptbook vs. promptlog. The promptbook (in kb/codebooks/) is the current instrument. The promptlog (in memory/) is its history. Collapsing them makes it impossible to tell which prompt is authoritative.

Tool compatibility

These are plain Markdown files, readable by any assistant, editor, or human. A root AGENTS.md is read automatically by several current coding agents; where a tool expects a different filename, a one-line file pointing to AGENTS.md usually suffices. The structure outlasts any particular tool, which is the point of keeping it in plain text.

Where to go next