Chapter 10 — Communicating your pipeline

Reproducible reports, dashboards, and uncertainty views

Chapter 10 turns the analysis into something another person can inspect, run, and read. This page carries starter scaffolding for the integrative artifact (§10.3.4) and pointers for the environment-freezing exercise (§10.1.2).

What you will produce. One artifact for an audience outside your field, carrying at least three coordinated views and an explicit uncertainty layer, built as a reproducible report, an interactive dashboard, or an annotated notebook.

Materials for this chapter

Starter scaffolding

Quarto report and Streamlit dashboard skeletons, below on this page.

Communication tools

Reporting, dashboard, and visualization tools, maintained as software changes.

Worked notebooks

Pre-executed notebooks whose committed output shows what a reader can check without rerunning anything, which is the standard this chapter is aiming at.

Studies on communication

Forty-four published studies annotated for how inspectable and reproducible their pipelines are. Useful models before you build your own.

The standards this artifact is judged against (inspectability, reproducibility, interpretability) are developed in Chapter 9.

Freezing the environment (§10.1.2)

# Python
pip freeze > requirements.txt          # exact pins, committed with the project

# R
renv::init(); renv::snapshot()         # writes renv.lock

Then verify the freeze does what you think: create a clean environment, install from the lock file, and rerun the pipeline end to end. An untested lock file is a hope, not a guarantee. Record the random seed for every stochastic step in the same commit.

Choosing the form of the artifact (§10.3.4)

The exercise asks for at least three coordinated views for a non-specialist audience. Three forms qualify, and the right one depends on the audience, not on which is most impressive.

Form Best when Tools
Reproducible report The audience reads linearly and you want code and narrative in one artifact Quarto, R Markdown, Jupyter Book
Interactive dashboard The audience wants to filter and drill into the data themselves Streamlit, Shiny, Dash, Observable
Annotated notebook The audience is technical and wants to see the pipeline Jupyter, Quarto notebook

Reproducible report skeleton (Quarto)

---
title: "Collaboration and topical framing in <your field>"
author: "<you>"
date: today
format:
  html:
    toc: true
    code-fold: true        # code available, but not in the reader's way
execute:
  warning: false
---

code-fold: true is a small but consequential choice: the analysis stays inspectable without making a policy reader scroll through it.

Dashboard skeleton (Streamlit)

import streamlit as st, pandas as pd

st.title("Research collaboration in <field>")
df = pd.read_parquet("data/processed/analysis_table.parquet")

year = st.slider("Publication year", int(df.year.min()), int(df.year.max()),
                 (2015, 2024))                        # one filter drives every view
sub = df[df.year.between(*year)]

st.caption(f"{len(sub):,} works, {year[0]}–{year[1]}. "
           "Excludes records without abstracts (n = ...).")   # document the filtering

col1, col2 = st.columns(2)
col1.subheader("Publications by year"); col1.line_chart(sub.groupby("year").size())
col2.subheader("Topic prevalence");     col2.bar_chart(sub.groupby("topic").size())

Note the caption. Chapter 10 asks you to document the analytical choices behind the interactive surface: if a default filter excludes outliers or a metric is computed on a subset, the reader must be able to see that. A polished interface that hides its exclusions is less honest than a static figure that states them.

The uncertainty layer (§10.3.5)

The community-detection example from the exercise, made concrete:

from collections import Counter
import community as community_louvain          # pip install python-louvain

runs = [community_louvain.best_partition(G, random_state=s) for s in range(20)]
modal_share = {
    n: Counter(r[n] for r in runs).most_common(1)[0][1] / len(runs)
    for n in G.nodes()
}
# shade each node by modal_share: saturated = stable assignment, pale = flips across runs

Nodes that flip across seeds are exactly the bridge cases a single-partition figure would present as settled. Showing them pale is more honest and usually more interesting.

Graceful degradation and maintenance

Provide a static fallback (PDF or PNG of the default views) for readers without JavaScript or using a screen reader, and check keyboard navigation. Then write the maintenance memo the exercise asks for: what breaks this artifact in five years, and which parts of the analysis are portable across whatever replaces today’s tools.

Where to go next