The experimental research interface records candidates, evidence, and why an investigation moved between them. R users, Glade, and agents can use the same operations. The execution graph remains available for running computations.
This example uses explicitly illustrative evidence so it runs without
Stan. For a real fit and predictive check, run
Rscript tools/research-search-demo.R from the repository
root with cmdstanr and CmdStan installed.
Normal candidate
├─ Record a predictive check without creating a revision
├─ Revise P: lognormal candidate
├─ Record its predictive check
└─ Close, compare, then reopen this path
Compare the two candidates, retaining the closed candidate's evidence.Start with a question
library(bayesgrove)
h <- bg_init(tempfile("research-search-"))
human <- list(id = "researcher", type = "human")
agent <- list(id = "assistant", type = "agent")
review_rule <- list(
id = "review-predictions",
actions = "accept",
type = "require_review",
check = "ppc",
message = "Review the predictive evidence before accepting this candidate.",
interpretation = "Inspect the features that matter for the analysis question.",
references = "https://arxiv.org/abs/2209.02439"
)
state <- bg_research_init(
h,
question = "How should we model these positive measurements?",
goal = list(kind = "observable_prediction", focus = "positive support"),
actor = human,
mode = "guide",
rules = list(review_rule)
)A candidate specifies P (joint distribution), with optional A (approximator) and D (training data). Specifications are plain data. The labels below are illustrative; a reproducible analysis should also retain exact model source, data, and settings, not only labels or mutable file paths.
proposal <- bg_research_propose(
h,
action = list(
kind = "create", label = "Normal",
components = list(P = list(likelihood = "normal", prior = "illustrative"))
),
actor = human,
rationale = "Start with a simple candidate and inspect its predictions."
)
state <- bg_research_apply(h, proposal)
normal <- tail(state$history, 1)[[1]]$idRecord evidence and revise
This local helper only shortens the example. Production clients can
inspect the proposal’s findings and allowed
before calling apply. Apply checks them again.
record <- function(action, rationale, actor = human) {
proposal <- bg_research_propose(h, action, actor, rationale)
bg_research_apply(h, proposal)
}
state <- record(
list(kind = "evidence", candidate_id = normal, label = "Illustrative PPC",
check = "ppc", utility = "predictive_performance",
result = list(source = "illustrative", concern = "negative predictions")),
"The example check raises a concern about predictive support."
)
normal_evidence <- tail(state$history, 1)[[1]]$id
state <- record(
list(kind = "revise", candidate_id = normal, label = "Lognormal",
components = list(P = list(likelihood = "lognormal", prior = "illustrative")),
basis = list(evidence_ids = normal_evidence)),
"Investigate a likelihood with positive support."
)
lognormal <- tail(state$history, 1)[[1]]$idA revision replaces the supplied components whole and inherits the others. The parent and its evidence stay unchanged. Changing A or D uses the same operation. A diagnostic or an interpretation adds evidence rather than a revision.
state <- record(
list(kind = "evidence", candidate_id = lognormal, label = "Illustrative PPC",
check = "ppc", utility = "predictive_performance",
result = list(source = "illustrative", concern = "none in this example")),
"Record the alternative's check without treating it as proof of adequacy.",
actor = agent
)
lognormal_evidence <- tail(state$history, 1)[[1]]$id
state <- record(
list(kind = "close", candidate_id = lognormal),
"Pause this path while considering other alternatives."
)
state <- record(
list(kind = "compare", candidate_ids = c(normal, lognormal),
evidence_ids = c(normal_evidence, lognormal_evidence),
criteria = "Predictive support in the illustrative checks",
result = list(preferred_for_support = lognormal)),
"Compare both candidates; the closed path's evidence remains available."
)
comparison <- tail(state$history, 1)[[1]]$id
state <- record(list(kind = "reopen", candidate_id = lognormal,
basis = list(comparison_ids = comparison)),
"Resume this path after the comparison.")Closing a lineage prevents revisions, evidence collection, and acceptance below it until reopening. Closing does not delete anything or automatically reject a candidate. Comparisons can include candidates from any lineage.
Switch intervention without rewriting history
Record mode preserves actions. Guide adds findings and explanations. Enforce also holds actions with unmet rules. Structural checks apply in every mode.
state <- record(list(kind = "configure", mode = "enforce", rules = list(review_rule)),
"Require human review before recording acceptance.")
acceptance <- list(kind = "accept", candidate_id = lognormal,
evidence_ids = lognormal_evidence,
basis = list(comparison_ids = comparison))
proposal <- bg_research_propose(h, acceptance, agent, "Select the alternative.")
proposal$allowed
#> [1] FALSE
proposal$findings[[1]]$message
#> [1] "Review the predictive evidence before accepting this candidate."An agent can record a review, but it cannot satisfy this rule’s human-review requirement. A human review covers only the evidence it cites. New evidence or a new candidate needs its own review.
state <- record(list(kind = "review", candidate_id = lognormal,
evidence_ids = lognormal_evidence),
"For this example, acknowledge the support check and its limited scope.")
proposal <- bg_research_propose(h, acceptance, human,
"Accept for the example's stated goal, subject to further checks.")
state <- bg_research_apply(h, proposal)
state$version
#> [1] 11A proposal becomes stale after any successful research mutation. Read
the state and propose again rather than replaying it. Declared actor
identity is provenance, not authentication; a host must establish
identity for untrusted callers. Enforce applies to this research
interface, not arbitrary R code or legacy bg_run() calls.
Existing workflow packs retain their separate behavior.
Reframe without losing earlier decisions
A declared human actor can correct the question or change the
inferential goal. Earlier records keep their goal_version;
their reviews do not authorize acceptance under the new goal. The
evidence itself remains available.
state <- record(
list(kind = "reframe", question = "How well are the largest measurements predicted?",
goal = list(kind = "observable_prediction", focus = "upper tail")),
"Focus the investigation on extreme observations."
)
proposal <- bg_research_propose(h, acceptance, human,
"Reconsider the candidate for the revised goal.")
proposal$allowed
#> [1] FALSEShare rules as a ruleset
Rules are data. bg_rule() validates one rule: the
actions it governs (on), optional when
conditions, one requirement, and a severity. Enforce holds
an action only for blocking findings; advisory findings guide without
holding. A ruleset collects rules under a dotted id and version, and can
declare the utilities, goal kinds, and checks it uses. This one extends
nothing.
review_ppc <- bg_rule(
id = "review-ppc",
on = "accept",
require = list(type = "require_review", check = "ppc"),
message = "Review the predictive evidence before accepting this candidate."
)
tail_check <- bg_rule(
id = "tail-check",
on = "accept",
when = list(list(type = "goal_kind", values = "observable_prediction")),
require = list(type = "require_evidence", check = "tail_ppc"),
severity = "advisory",
message = "Consider a check focused on the upper tail."
)
lab <- bg_ruleset(
"lab.positive",
"1.0.0",
title = "Positive measurements",
vocabulary = list(
utilities = list(list(id = "predictive_performance")),
goal_kinds = list(list(id = "observable_prediction")),
checks = list(
list(id = "ppc", utilities = "predictive_performance"),
list(id = "tail_ppc", utilities = "predictive_performance")
)
),
rules = list(review_ppc, tail_check)
)
bg_ruleset_check(lab)[c("ok", "rule_count")]
#> $ok
#> [1] TRUE
#>
#> $rule_count
#> [1] 2Registering a ruleset makes it available by reference in this
session. bg_use_rules() replaces the whole policy through a
recorded configure action. The state pins the resolved
rules and vocabulary, so later changes to the registry do not change the
policy in force. bg_rules() returns what an agent should
read before acting.
bg_register_ruleset(lab)
state <- bg_use_rules(h, "lab.positive@1", mode = "enforce", actor = human,
rationale = "Adopt the shared rules for this lab.")
active <- bg_rules(h)
active$ruleset[c("id", "version")]
#> $id
#> [1] "lab.positive"
#>
#> $version
#> [1] "1.0.0"
vapply(active$rules, `[[`, character(1), "id")
#> [1] "lab.positive/review-ppc" "lab.positive/tail-check"
proposal <- bg_research_propose(h, acceptance, human,
"Reconsider the candidate under the shared rules.")
proposal$allowed
#> [1] FALSE
vapply(proposal$findings, `[[`, character(1), "severity")
#> lab.positive/review-ppc lab.positive/tail-check
#> "blocking" "advisory"The review from the previous goal no longer counts, so the blocking
rule holds the acceptance. The advisory finding alone would not. The
vocabulary maps tail_ppc to a single utility, so evidence
for that check can omit utility. After the tail check and a
review under the current goal, no findings remain.
state <- record(
list(kind = "evidence", candidate_id = lognormal, label = "Illustrative tail check",
check = "tail_ppc", result = list(source = "illustrative")),
"Check the upper tail for the revised goal."
)
state$evidence[[tail(state$history, 1)[[1]]$id]]$utility
#> [1] "predictive_performance"
state <- record(list(kind = "review", candidate_id = lognormal,
evidence_ids = lognormal_evidence),
"Review the predictive check again under the revised goal.")
proposal <- bg_research_propose(h, acceptance, human,
"Accept for the revised goal, subject to further checks.")
proposal$allowed
#> [1] TRUE
length(proposal$findings)
#> [1] 0
state <- bg_research_apply(h, proposal)Bind a candidate to a run
So far, every result was asserted by the caller. A candidate can also be bound to a node in the execution graph. The engine then records which execution it describes and derives evidence from that execution’s summaries, instead of trusting a pasted result.
This needs a node kind that declares how its parameters split into P, A, and D. The small kind below stands in for a real fit, so the example runs without Stan. Its executor reports the data input it used and writes one summary.
runs <- bg_init(tempfile("research-runs-"))
bg_register_node_kind(runs, "measurements", executor = function(node, inputs) {
c(2.1, 3.4, 1.8, 2.9, 4.2)
})
bg_register_node_kind(
runs, "lognormal_fit",
executor = function(node, inputs) {
y <- log(inputs[[1]])
list(
result = list(meanlog = mean(y), sdlog = stats::sd(y)),
summaries = list(list(
summary_kind = "posterior_predictive_check", severity = "ok",
passed = TRUE, metrics = list(share_positive = 1)
)),
metadata = list(provenance = list(consumed = list(D = names(inputs))))
)
},
pad = list(P = "likelihood", A = "method")
)
measurements <- bg_add_node(runs, kind = "measurements")
lognormal_fit <- bg_add_node(
runs, kind = "lognormal_fit", inputs = measurements,
params = list(likelihood = "lognormal", method = "moments")
)
bg_run(runs)$status
#> [1] "succeeded"This investigation requires derived predictive evidence before
acceptance. create with node_id binds the
candidate to the execution behind the node’s current artifact. Without
components, the description defaults to that execution’s
P/A/D snapshot.
derived_rule <- bg_rule(
id = "derived-ppc",
on = "accept",
require = list(type = "require_evidence", check = "posterior_predictive_check",
provenance = "derived"),
message = "Derive the predictive check from the run before accepting."
)
run_state <- bg_research_init(
runs, "How should we model these positive measurements?",
goal = list(kind = "observable_prediction"), actor = human,
mode = "enforce", rules = list(derived_rule)
)
run_record <- function(action, rationale) {
bg_research_apply(runs, bg_research_propose(runs, action, human, rationale))
}
run_state <- run_record(
list(kind = "create", label = "Lognormal run", node_id = lognormal_fit),
"Describe the fit that was actually run."
)
bound <- tail(run_state$history, 1)[[1]]$id
names(run_state$candidates[[bound]]$binding)
#> [1] "node_id" "job_id" "artifact_ref" "pad"
#> [5] "research_version"
run_state$candidates[[bound]]$components$P$params
#> $likelihood
#> [1] "lognormal"The binding stores the node, the job that produced its artifact, the
artifact’s content address, and the P/A/D snapshot with digests. These
fields are engine-owned: an action that supplies them is rejected.
Revising with another node_id records which of P, A, and D
changed by comparing digests. A revision without node_id is
unbound, because it may describe a different model.
Evidence with node_id instead of result is
derived. The node here is the bound fit itself, so its own summary is
used. A check node that evaluated the bound fit’s exact output works as
well. check can be omitted because only one summary kind
matches.
run_state <- run_record(
list(kind = "evidence", candidate_id = bound, label = "Positive support",
node_id = lognormal_fit, utility = "predictive_performance"),
"Use the check the run wrote."
)
derived <- tail(run_state$history, 1)[[1]]$id
run_state$evidence[[derived]]$check
#> [1] "posterior_predictive_check"
run_state$evidence[[derived]]$result[c("severity", "passed")]
#> $severity
#> [1] "ok"
#>
#> $passed
#> [1] TRUE
run_state$evidence[[derived]]$provenance$kind
#> [1] "derived"
proposal <- bg_research_propose(
runs, list(kind = "accept", candidate_id = bound, evidence_ids = derived),
human, "Accept on evidence derived from the run."
)
proposal$allowed
#> [1] TRUE
bg_close(runs)Asserted evidence for the same check would not satisfy this rule. Apply derives bindings and evidence again under the research lock. If the node was re-run between proposal and apply, the proposal is stale.
Start from the official ruleset
bayesgrove ships one built-in ruleset,
bayesgrove.workflow. Its vocabulary declares the ten model
utilities and the two inferential goals of Bürkner, Scholz, and Radev
(2023) and maps the built-in summary kinds to the utilities they
inform. Its nine rules are a selected, tree-informed workflow baseline:
a few goal-dependent expectations drawn from the paper’s utility trees,
plus established practice from Bayesian Workflow (2026). They
do not encode the trees in full. Whether fairness applies, which is the
first, conditional gate in both trees, and how to weigh trade-offs
between utilities are left to the analyst or lab; the lab extension
below shows how to add such rules. It is a ruleset like any other: plain
data, built-in predicates only, loaded through the same validation as
lab.positive above.
bg_rulesets()[c("id", "version", "origin")]
#> id version origin
#> 1 bayesgrove.workflow 1.0.0 builtin
#> 2 lab.positive 1.0.0 session
bg_ruleset_check("bayesgrove.workflow@1")[c("ok", "warnings", "rule_count")]
#> $ok
#> [1] TRUE
#>
#> $warnings
#> character(0)
#>
#> $rule_count
#> [1] 9Start an investigation with it by reference. The goal kind must be
one the vocabulary declares, observable_prediction or
latent_inference; any other kind is an error in every
mode.
h2 <- bg_init(tempfile("official-ruleset-"))
official_state <- bg_research_init(
h2,
question = "How large is the treatment effect?",
goal = list(kind = "latent_inference", estimand = "average treatment effect"),
actor = human,
mode = "guide",
rules = "bayesgrove.workflow@1"
)
official <- bg_rules(h2)
vapply(official$vocabulary$utilities, `[[`, character(1), "id")
#> [1] "causal_consistency" "parameter_recoverability"
#> [3] "predictive_performance" "fairness"
#> [5] "structural_faithfulness" "parsimony"
#> [7] "interpretability" "convergence"
#> [9] "estimation_speed" "robustness"
data.frame(
rule = sub("bayesgrove.workflow/", "", vapply(official$rules, `[[`, "", "id"), fixed = TRUE),
on = vapply(official$rules, function(r) paste(unlist(r$on), collapse = ", "), ""),
severity = vapply(official$rules, `[[`, "", "severity")
)
#> rule on severity
#> 1 convergence-derived accept blocking
#> 2 review-hmc-diagnostics accept blocking
#> 3 review-optimizer-diagnostics accept blocking
#> 4 posterior-predictive-check accept advisory
#> 5 observable-loo accept advisory
#> 6 observable-compare-loo compare advisory
#> 7 latent-causal-consistency accept advisory
#> 8 latent-parameter-recoverability accept advisory
#> 9 latent-compare-causal-consistency compare advisoryThree explicit rules are blocking, all on acceptance: convergence diagnostics derived from the candidate’s own run, and a human review of the computation diagnostics that were recorded. The goal-specific expectations are advisory. In Enforce, the implicit vocabulary rule additionally blocks evidence whose check or utility the vocabulary does not declare, or whose utility the check does not inform. Evidence asserted by a caller, like the illustrative record below, never satisfies the derived-convergence rule: it needs evidence the engine derives from a run bound to the candidate, as in “Bind a candidate to a run” above.
record2 <- function(action, rationale, actor = human) {
bg_research_apply(h2, bg_research_propose(h2, action, actor, rationale))
}
official_state <- record2(
list(kind = "create", label = "Adjusted regression",
components = list(P = list(formula = "y ~ treatment + age"))),
"Start from the adjustment set implied by the assumed causal graph."
)
model <- tail(official_state$history, 1)[[1]]$id
official_state <- record2(
list(kind = "evidence", candidate_id = model, label = "Illustrative sampler check",
check = "hmc_diagnostics", result = list(source = "illustrative")),
"Record the sampler diagnostics."
)
hmc <- tail(official_state$history, 1)[[1]]$id
official_state$evidence[[hmc]]$utility
#> [1] "convergence"
proposal <- bg_research_propose(
h2,
list(kind = "accept", candidate_id = model, evidence_ids = hmc),
human,
"Accept the adjusted model."
)
vapply(proposal$findings, `[[`, character(1), "severity")
#> bayesgrove.workflow/convergence-derived
#> "blocking"
#> bayesgrove.workflow/review-hmc-diagnostics
#> "blocking"
#> bayesgrove.workflow/posterior-predictive-check
#> "advisory"
#> bayesgrove.workflow/latent-causal-consistency
#> "advisory"
#> bayesgrove.workflow/latent-parameter-recoverability
#> "advisory"Each finding carries the rule’s explanation, a suggestion, and references to the paper and the book. Evidence and reviews are records: the rules require that a check was recorded and read, never that it passed.
recoverability <- proposal$findings[["bayesgrove.workflow/latent-parameter-recoverability"]]
recoverability$message
#> [1] "For a latent goal, consider recording parameter recoverability evidence, such as simulation-based calibration, before accepting this candidate."
unlist(recoverability$references)
#> [1] "Bürkner, Scholz & Radev (2023), §3.2.3"
#> [2] "Bürkner, Scholz & Radev (2023), §4.4.2 and Figure 14"
#> [3] "Bayesian Workflow (2026), §6.3"
#> [4] "Bayesian Workflow (2026), §14.3"A lab adapts the official rules by extending them rather than copying
them. Here a lab that has decided fairness applies to its work makes the
causal-consistency expectation blocking and adds a fairness audit, with
a check that informs the fairness utility. Inherited rules
keep their source and record who overrode them.
lab_workflow <- bg_ruleset(
"lab.workflow",
"1.0.0",
title = "Lab workflow",
extends = "bayesgrove.workflow@1",
override = list(
"bayesgrove.workflow/latent-causal-consistency" = list(severity = "blocking")
),
vocabulary = list(checks = list(
list(id = "fairness_audit", title = "Fairness audit", utilities = "fairness")
)),
rules = list(bg_rule(
id = "fairness-audit",
on = "accept",
require = list(type = "require_evidence", check = "fairness_audit"),
severity = "advisory",
message = "Record a fairness audit before accepting a model.",
references = "Bürkner, Scholz & Radev (2023), §3.4"
))
)
bg_ruleset_check(lab_workflow)[c("ok", "warnings", "rule_count")]
#> $ok
#> [1] TRUE
#>
#> $warnings
#> character(0)
#>
#> $rule_count
#> [1] 10
bg_register_ruleset(lab_workflow)
official_state <- bg_use_rules(h2, "lab.workflow@1", actor = human,
rationale = "Adopt the lab's extension of the official rules.")
causal <- Filter(
function(r) r$id == "bayesgrove.workflow/latent-causal-consistency",
bg_rules(h2)$rules
)[[1]]
causal[c("severity", "source", "overridden_by")]
#> $severity
#> [1] "blocking"
#>
#> $source
#> [1] "bayesgrove.workflow@1.0.0"
#>
#> $overridden_by
#> $overridden_by[[1]]
#> [1] "lab.workflow@1.0.0"
bg_close(h2)Reopen and share
path <- h@path
bg_close(h)
h <- bg_open(path)
stopifnot(identical(bg_research_state(h), state))
bg_close(h)bg_snapshot(), bg_bundle(), and
bg_export_report() include research state. The research-search
contract lists all action fields, and the rule
contract specifies rules, predicates, rulesets, pinning, bindings,
and derived evidence. There is no built-in universal model score:
comparisons state their criteria and retain their evidence.