Skip to contents

Package replication checklist

Source: vignettes/package-replication-checklist.Rmd
package-replication-checklist.Rmd

Two models

Model Registry Materials
Folder-backed studies/<folder>.yml stub only Study repo: code/, data/, outputs/
Package-backed studies/<folder>.yml stub only Study R package on GitHub

This checklist covers package-backed studies (recommended for multi-table papers).

Package layout 

rep_<doi-slug>/
  DESCRIPTION
  replication.yml              # full metadata + steps: DAG
  inst/replication.yml         # copy for installed package
  R/
    make_figure_*.R             # analysis
    make_table_*.R
    format_*.R                  # display formatting
  inst/replication_code/        # synced copies for Code tab / get_code()
  inst/report/artifacts/        # baked Display outputs (build_study_outputs())
  data/                         # analysis datasets (LazyData)
  tests/
    testthat/                   # smoke tests
    substantive/                # optional published-value benchmarks

Required replication.yml fields

Paper metadata

  • paper.doi — full DOI URL
  • paper.title
  • paper.package — must match DESCRIPTION Package: field
  • paper.package_repo or top-level repo — GitHub slug (org/repo)
  • paper.package_ref — branch/tag (default main)
  • paper.package_folder — optional; sibling folder name for monorepo dev

Maintainer and collections (required for registry sync)

  • maintainer.name and maintainer.email — contact shown as [maintainer] on the Studies tab
  • collections — tags for bibliography filtering (APSR, PED, World Bank, IPI, …)
  • languages — engines used by the package (usually r)
maintainer:
  name: Jane Maintainer
  email: maintainer@example.org

collections:
  - IPI

languages:
  - r

Steps (required) — package-backed studies use the same unified steps: DAG as folder-backed studies. code: names an exported function (not a file path); a type: format child names the display formatter:

steps:
  - id: fig_1
    type: figure
    label: Figure 1
    code: make_figure_1
    data: my_dataset
    outputs:
      - inst/report/artifacts/fig_1.png

  - id: fig_1_format
    type: format
    parent: fig_1
    code: format_figure_1

  - id: tab_1
    type: table
    label: Table 1
    code: make_table_1
    data: my_dataset
    outputs:
      - inst/report/artifacts/tab_1.html

  - id: tab_1_format
    type: format
    parent: tab_1
    code: format_table_1

Rules:

  • id — short slug used by Shiny and replicateEverything::run_replication()
  • type — figure, table, or transform (shared prep); format for display children
  • code — exported analysis function (figure/table) or format function (format child)
  • data — package dataset name(s), not file paths
  • outputs — declared artifact path(s) under inst/report/; the first displayable path is what Shiny Display reads
  • Legacy replications: with make: / format: entry fields is a hard error — use steps: + code: as above

Study package surface

Export the make_* / format_* helpers named in yaml (plus shared helpers and data).

Do not put these in the study package — they live only in replicateEverything:

Function Purpose
list_replications(doi) Entries from study yaml
run_replication(doi, id) Calls package make_* then format_*
load_artifact(doi, id) Display HTML or PNG path
get_code(doi, id) Source for Code tab
check_replication(".") Validate package study

Artifacts (Display tab)

Run once before registering, with the same entrypoint as folder-backed studies:

replicateEverything::build_study_outputs(".", install_deps = TRUE)

This writes:

  • inst/report/artifacts/<id>.png for each figure
  • inst/report/artifacts/<id>.html for each table (must contain a <table>)

(build_report() still exists as a thin package-local convenience alias for CI/local use, but Contribute guidance and this checklist point at build_study_outputs().) The registry stub does not store artifacts.

Prepare and register

Contributor: validate

A quick manual smoke check first — from the package source directory, "local" resolves to the working-directory study without any registry setup:

library(replicateEverything)

setwd("../rep-10.1371-journal.pone.0278337")  # or open its RStudio project
list_replications("local")
describe_study_dag("local")
run_replication("local", "tab_1")             # one light step

Then run the full checklist (path-based, works from anywhere):

check_and_bake_study(
  "../rep-10.1371-journal.pone.0278337",
  build_artifacts = TRUE
)
check_replication("../rep-10.1371-journal.pone.0278337")

check_and_bake_study() validates only — it never writes a registry stub or any file into the study package. A maintainer syncs directly from your replication.yml (next section).

Maintainer: sync into the central registry 

options(replicateEverything.registry_root = "../registry")

sync_study_to_registry(
  "../rep-10.1371-journal.pone.0278337",
  registry_root = "../registry",
  audit = TRUE
)

# After several syncs:
refresh_registry("../registry", audit = TRUE)

Checks only 

check_replication(
  "../rep-10.1371-journal.pone.0278337",
  full_replication = FALSE
)

check_replication(
  "../rep-10.1371-journal.pone.0278337",
  full_replication = TRUE
)

Substantive (published-value) checks

Package-backed studies use the same tests/substantive/<step_id>.R convention as folder-backed repos. Define substantive_check_<step_id>(object) and call it from tests/testthat/. [check_replication()] reports coverage; full_replication = TRUE runs defined checks. [audit_everything()] includes them in the registry audit (substantive = TRUE, default).

See Fearon & Laitin Table 1 in rep-10.1017-S0003055403000534 for a reference substantive check (study-specific benchmarks live in the study repo).

Reference implementation

See rep-10.1371-journal.pone.0278337 (vaccine solidarity paper).