Reproducible research with bayesgrove

This article covers archiving an analysis, verifying its records, and describing it in a methods section. See vignette("concepts") for definitions and vignette("eight-schools") for a worked analysis.

What the bundle contains

bg_bundle() produces a single portable .tar.gz of the project:

library(bayesgrove)

handle <- bg_open("analysis/")
bundle <- bg_bundle(handle, path = "analysis-bundle.tar.gz")

The archive contains these files under .bayesgrove/:

File	Contents
graph/graph.json	The full analysis graph: nodes, edges, params, gates.
decisions/decisions.jsonl	Every recorded decision: prompt, choice, rationale, scope, evidence, timestamps.
workflow/summaries.jsonl	Typed diagnostic evidence (HMC diagnostics, PPC p-values, LOO Pareto-k counts, LOO-PIT calibration, SBC ranks) with severities and execution fingerprints.
runs/jobs.jsonl	The execution history: which node ran when, with what outcome.
cache/	Content-addressed artifacts (large fit objects are excluded unless include_fits = TRUE).
bundle_manifest.json	A reproducibility manifest: R version, platform, package versions, CmdStan version, creation time.

Data attached with bg_set_node_data() are included, even before execution. External files referenced by node params, such as Stan programs, are not copied or relocated. Archive those files separately and update their paths before rerunning on another machine.

Decisions and summaries are appended to JSONL logs with increasing sequence numbers. Entries are not edited in place, so the log retains earlier decisions when a prior or model changes.

Each summary contains its node’s execution fingerprint: a deterministic hash of the params, upstream fingerprints, Stan program contents, and environment. Use it to check whether the evidence cited by a decision matches the reported configuration.

Restoring a bundle on another machine

Untar the bundle and open it:

utils::untar("analysis-bundle.tar.gz", exdir = "restored")
handle <- bg_open("restored/analysis")

bg_open() compares the bundle’s reproducibility manifest against the current session and warns when the R version, platform, or CmdStan version differ. This warning does not invalidate cached artifacts; fingerprints determine what reruns. Matching the recorded environment may be necessary for bit-level reproduction.

Opening a project does not evaluate saved user executor source. Built-in executors restore by reference. Review saved source before calling bg_restore_executors(trust = TRUE), or re-register custom executors with bg_register_node_kind().

Read the decision and summary logs

Both logs are plain JSONL and parse with any JSON reader:

decisions <- lapply(
  readLines("restored/analysis/.bayesgrove/decisions/decisions.jsonl"),
  jsonlite::fromJSON
)
summaries <- lapply(
  readLines("restored/analysis/.bayesgrove/workflow/summaries.jsonl"),
  jsonlite::fromJSON
)

Use the logs to check, for example, whether every hmc_diagnostics summary with severity warning or error is referenced by at least one computation_review decision.

For a human-readable rendering, bg_export_report() produces a Markdown or HTML report with the graph as a Mermaid diagram, the decision history, and the artifact index.

SBC and simulation studies at scale

The wave scheduler dispatches independent nodes concurrently when their inputs are ready. Start mirai daemons before running the graph:

mirai::daemons(8)
bg_run(handle) # sbc / simulation nodes fan out across the daemons
mirai::daemons(0)

The built-in sbc node kind runs the Talts et al. (2018) rank-statistic loop and emits an sbc_result summary containing a rank histogram and a chi-squared uniformity check. The SBC review pack uses this summary to request reviews. Input comes from an upstream generator executor as plain data: list(simulations = list(list(theta = ..., data = ...), ...)). Generator code requires the same explicit trust as other custom executors. The executor thins each fit to a common number bounded by bulk ESS before ranking, and refuses to grade histograms whose expected bin counts are below five. Twenty simulations are a smoke test; use hundreds for a substantive calibration study.

Parallel workers always preserve the content-addressed R fit artifact. Raw sampler CSV paths can point into daemon-local temporary directories, however, so bg_run() warns when a parallel wave contains fit nodes. Use parallel = "never" when durable raw CSV files are required. See ?bg_run for the wave semantics and vignette("simulation-study") for a full design.

Reporting template

A “data and code availability” statement for a bayesgrove-backed analysis:

We used bayesgrove (Scholz, YEAR) to record the model graph, diagnostic evidence, and analysis decisions in machine-readable form. The project bundle is archived at DOI. It contains the decision log with rationales, typed diagnostic summaries with execution fingerprints, and a reproducibility manifest (R X.Y.Z, CmdStan X.Y.Z).

Cite the package and its workflow reference with citation("bayesgrove").