Tools: automated coding

Dictionaries, classifiers, topic models, and LLM coding

The maintained version of the “Useful Tools” list closing Chapter 6. This is the fastest-moving list on the site; last reviewed August 2026.

Dictionary coding

  • quanteda (R): dictionary lookup, keyword-in-context, document-feature matrices. The most complete text-analysis package in R.
  • LIWC: validated psycholinguistic dictionaries; commercial license.
  • Moral Foundations Dictionary: freely available.
  • Plain regular expressions in Python or R remain entirely appropriate for a project-specific dictionary, and are easier to audit than a black-box package.

Supervised classification

  • scikit-learn: logistic regression, SVM, random forests, cross-validation, and the metrics functions the audit in Chapter 6 requires. Start here.
  • Hugging Face Transformers: pretrained models for fine-tuning when a bag-of-words baseline is not enough.
  • setfit: few-shot classification when labels are scarce, a common situation in social science.

Topic modeling

  • BERTopic (Python): modular embedding-based pipeline; good on short texts such as abstracts.
  • Top2Vec (Python): determines topic count from the data, useful for first exploration.
  • stm (R): the standard when you need topic prevalence to vary by covariates.
  • gensim (Python): LDA and Word2Vec.

LLM-based coding

Access. OpenAI, Anthropic, Google AI Studio, OpenRouter (one interface across many models), Ollama (open-weight models on your own machine, no data leaves it).

Structuring calls. instructor and Pydantic enforce a JSON output schema and retry on malformed responses, which removes most of the parsing pain from batch coding. LangChain and LlamaIndex offer fuller pipeline frameworks, though for a coding task a plain loop plus a schema validator is often clearer and easier to audit.

Batch economics. Most providers offer a batch endpoint at a substantial discount for non-urgent jobs, which suits coding a corpus overnight.

Agentic coding assistants (“vibe coding”)

Tools that write and execute analysis code from a natural-language description: Claude Code, OpenAI Codex, Cursor, GitHub Copilot, Google Antigravity, and others. Adoption among quantitative social scientists is now substantial (Chapter 7, §7.1.4).

Audit generated code before trusting it

Chapter 6’s hands-on tip lists the checks that matter most: the train/test split must happen before any feature fitting; metrics must be class-level, not aggregate accuracy; a random seed must be set and logged; and the script must read from your stable corpus file rather than re-deriving data ad hoc. Then spot-check predictions by hand. Generated code fails silently far more often than it crashes.

Validation and audit

  • sklearn.metrics.classification_report and confusion_matrix: class-level precision, recall, F1.
  • seaborn: error-pattern plots across subgroups.
  • irr (R) / sklearn.metrics.cohen_kappa_score: intercoder agreement for the human-agreement checks in Chapters 4 and 6.

Documentation

Jupyter, Quarto, and Git. Promptbooks and promptlogs are plain Markdown; templates are on the Chapter 2 page.