A template/ approach to Reproducible Generative Research

Architecture and Ergonomics from Configuration through Publication

Daniel Ari Friedman

State: unpublished / pending pairing

Pairing: pending — unresolved: - ✗ GitHub release URL: pending - ✗ PDF SHA-256: pending

Field Value
Title A template/ approach to Reproducible Generative Research
Version 2.0.0
Concept DOI 10.5281/zenodo.20419007
GitHub docxology/template_template
Zenodo https://zenodo.org/records/20419007
SHA-256 pending
SHA-512 pending
Integrity QR strip
Integrity QR strip

Structured manifest: ../data/transmission_manifest.json

Publication pairing flow
Publication pairing flow

Stego: off | overlays text | barcodes on | XMP on | manifest on → ./secure_run.sh


1 Abstract

The reproducibility problem in computational research is partly structural: research artifacts are often distributed across editors, notebooks, scripts, and manual publication steps without a shared mechanism that keeps code, data, and manuscript synchronized. Existing tools address important parts of this lifecycle: workflow managers orchestrate computation, literate-programming systems render documents, and data-versioning tools track artifacts. template/ combines those concerns with test and provenance gates by applying Infrastructure as Code to the research lifecycle. Its Two-Layer Architecture separates 31 infrastructure subdirectories (28 importable Python packages, ~1335 modules, validated by ~12,564 tests) from self-contained project workspaces. A YAML-declared pipeline (17 stages; 10 in a default full run) coordinates environment checks, test execution, analysis, Pandoc/XeLaTeX rendering, SHA-256 hashing with optional steganographic watermarking, structural validation, and optional LLM review. The declared quality floors—90% for project source and 60% for infrastructure source—are injected from executable configuration rather than copied into the manuscript. Documentation Duality equips repository directories with human-facing README.md and agent-facing AGENTS.md files; skill-enabled infrastructure packages additionally expose structured SKILL.md descriptors so agents can discover supported capabilities from versioned interfaces.

The generated public exemplar roster (templates/template_active_inference, templates/template_advanced_literature_review, templates/template_autopoiesis, templates/template_autoresearch_project, templates/template_autoscientists, templates/template_code_project, templates/template_data_descriptor, templates/template_eda_notebook, templates/template_experiment_tree, templates/template_formal, templates/template_gold_refinement, templates/template_literature_meta_analysis, templates/template_madlib, templates/template_methods_paper, templates/template_newspaper, templates/template_pitch_deck, templates/template_pools_rules_tools, templates/template_prose_project, templates/template_redacted_report, templates/template_registered_report, templates/template_search_project, templates/template_sia, templates/template_storybook, templates/template_template, templates/template_textbook) supplies heterogeneous control-positive layouts, including optimization (template_code_project, 263 discovered tests), prose (template_prose_project, 165 discovered tests), and AutoResearch readiness (template_autoresearch_project, 318 discovered tests). Each declares the same 90% project-source floor and consumes the shared pipeline without project-to-project imports. This manuscript adds a reflexive artifact: template_template (157 discovered tests) follows the same analysis and render path while injecting repository counters and figures from live introspection. The result is a self-documenting build whose quantitative repository claims are generated artifacts, whose manuscript claims are checked against an evidence registry, and whose output can be validated and optionally watermarked. A versioned comparative matrix situates template/ against peer tools across the capabilities reported in the Results section. Code is released under the Apache License 2.0 at github.com/docxology/template; the work remains open-ended.


2 Introduction

2.1 Research Tools as Epistemic Infrastructure

Scientific research operates through a layered ecology of tools, documents, and practices—each shaping what can be known, communicated, and verified. When these layers are fragmented, manuscripts, data, code, and figures can drift out of alignment, creating gaps that are structural rather than incidental. Survey and economic evidence documents the practical cost of failed reproduction attempts (Baker 2016; Freedman et al. 2015), but reproducibility is only the most visible face of a broader problem. Research software engineering, epistemic integrity, and the coordination between human researchers and AI collaborators all depend on whether the tools that produce research artifacts can themselves be transparent, testable, and self-documenting. Nosek et al. (Nosek et al. 2018) argue that committing to analytical plans before data collection is a necessary structural reform; we extend this logic to the research build pipeline.

One root cause is fragmentation—of attention (in the mind) as well as in the cyberphysical niche (documents, versions, and reproducibility artefacts). A typical research project scatters its artifacts across disconnected tools: LaTeX editors (Lamport 1994) for prose, Jupyter notebooks for analysis, ad-hoc shell scripts for figure generation, and manual copy-paste for integrating results into manuscripts. Each boundary between tools is a potential locus of desynchronization. The version of a figure embedded in a PDF may not match the code that ostensibly generated it, while tests may exercise analysis code without exercising the rendering path. Pimentel et al. (Pimentel et al. 2019) studied a corpus described as 1.4 million Jupyter notebooks; in the valid execution subset, 24.11% ran without errors and 4.03% reproduced the recorded results. Those distinctions matter: execution success is not equivalent to result reproduction. Peng (Peng 2011) argues that reproducibility in computational science requires, at minimum, that the data and code underlying a published result be available for independent verification. Barba (Barba 2018) further documents how “reproducibility,” “replicability,” and “repeatability” carry conflicting definitions across disciplines, complicating cross-field standards.

Gentleman and Temple Lang (Gentleman and Temple Lang 2007) introduced the concept of a research compendium—a single unit of scholarly communication bundling code, data, and narrative. This vision has driven two decades of tooling, which can be grouped into four categories: workflow managers, literate programming systems, containerization approaches, and best-practice frameworks.

2.2.1 Workflow Managers

Snakemake (Köster and Rahmann 2012) uses a rule-based, Python-derived DSL to specify computational workflows as directed acyclic graphs of file-producing steps. It supports containerized execution and an extension architecture for execution and storage backends, while its central scope remains computational pipeline orchestration rather than this repository’s combined manuscript, test, and document-provenance contract.

Nextflow (Di Tommaso et al. 2017) employs a dataflow programming paradigm with native support for container-based execution across heterogeneous computing environments (local, SLURM, AWS). Like Snakemake, Nextflow excels at bioinformatics pipeline parallelism but does not address manuscript production, document integrity, or the testing–publication coupling that characterizes research reproducibility.

CWL (Common Workflow Language) (Amstutz et al. 2016) provides a portable, YAML-based standard for describing computational workflows and their dependencies. Its strength lies in interoperability across execution engines (cwltool, Toil, Arvados), but it requires external tooling for manuscript generation and offers no built-in testing or provenance framework.

2.2.2 Literate Programming and Publication Tools

Knuth’s literate programming (Knuth 1984) established the principle that programs should be authored as documents intended for human comprehension. Schulte et al. (Schulte et al. 2012) extended this to multi-language computing environments (Org-mode), demonstrating that literate programming could span languages and output formats.

Quarto (Allaire et al. 2024) extends the R Markdown tradition to support Python, Julia, and Observable, rendering to PDF, HTML, and Word. Quarto integrates code execution with document rendering, achieving a modern form of literate programming, but it does not enforce testing thresholds, manage multi-project repositories, or provide cryptographic provenance.

Jupyter Book (Kluyver et al. 2016) builds on Jupyter notebooks to produce publication-quality documents via Sphinx. While powerful for interactive exploration, Jupyter’s notebook format introduces execution-order fragility (Pimentel et al. 2019) and does not naturally support the separation of logic from orchestration that characterizes maintainable research software.

R Markdown (Xie et al. 2018) pioneered knitr-based dynamic documents that weave code and prose. Its ecosystem is rich but R-centric, and it lacks the multi-project management, infrastructure testing, and provenance embedding that characterize template/.

Typst (Mädje and Haug 2023) is an emerging markup-based typesetting system with incremental compilation and a programmable scripting layer. While Typst offers faster compilation than LaTeX and a more modern authoring experience, it does not integrate testing, provenance, or multi-project pipeline management.

2.2.3 Containerization and Environment Capture

Docker (Boettiger 2015) addresses reproducibility at the environment level—packaging operating system, libraries, and code into portable containers. While Docker solves the “works on my machine” problem, containerization is complementary to, not a replacement for, the architectural concerns addressed here: Docker does not enforce testing, embed provenance, or manage multi-project manuscript workflows.

2.2.4 Best-Practice Frameworks and Data Standards

Wilson et al. (Wilson et al. 2017) define “good enough” practices for scientific computing, emphasizing version control, testing, and documentation. Sandve et al. (Sandve et al. 2013) propose ten rules for reproducible computational research. Piccolo and Frampton (Piccolo and Frampton 2016) systematically survey tools for computational reproducibility, finding that environment isolation, workflow automation, and documentation generation address complementary but non-overlapping reproducibility concerns—yet no single tool unifies all three. Stodden et al. (Stodden et al. 2016) advocate for enhanced computational method transparency. The FAIR principles (Wilkinson et al. 2016)—Findable, Accessible, Interoperable, Reusable—establish a standard for data stewardship that has been widely adopted by funding agencies and journals. Lamprecht et al. (Lamprecht et al. 2020) formalize “Towards FAIR Principles for Research Software,” providing the conceptual scaffolding that Barker et al. (Barker et al. 2022) would soon operationalize as the FAIR4RS initiative—recognizing that software has execution, composability, and dependency-management requirements that data-centric FAIR does not address. Cohen et al. (Cohen et al. 2021) characterize the four pillars of research software engineering (software sustainability, software quality, community building, and policy advocacy), situating formal testing and provenance practices within a broader RSE governance framework. Garijo et al. (Garijo et al. 2024) operationalize FAIR4RS through the FAIRsoft evaluator, an automated assessment framework that scores research software against 17+ quality indicators including executability, metadata richness, and documentation completeness. Goble et al. (Goble et al. 2020) extend FAIR to computational workflows specifically, arguing that workflow provenance requires first-class treatment in scientific computing infrastructure. Nüst et al. (Nüst et al. 2017) introduce the executable research compendium (ERC), extending Gentleman and Temple Lang’s compendium concept with containerized, interactive reproduction environments. The W3C PROV data model (Moreau and Missier 2013) provides a formal vocabulary for expressing provenance records, while in-toto (Torres-Arias et al. 2019) provides a framework for end-to-end software supply chain integrity verification, and SLSA (Open Source Security Foundation 2023) (Supply-chain Levels for Software Artifacts) extends this to graduated, attestation-based supply-chain security levels for build pipelines. These frameworks articulate what reproducible research requires but do not provide an integrated how—they lack the tooling, enforcement mechanisms, and architectural patterns that translate standards into practice.

2.2.5 The Gap

Despite advances in FAIR4RS principles (Barker et al. 2022; Lamprecht et al. 2020), automated FAIR software assessment (Garijo et al. 2024), FAIR computational workflow standards (Goble et al. 2020), supply-chain attestation frameworks (Torres-Arias et al. 2019; Open Source Security Foundation 2023), and the preregistration revolution (Nosek et al. 2018), no existing system integrates six cross-cutting concerns into a single enforced pipeline: (1) end-to-end pipeline orchestration with testing enforcement, (2) multi-format manuscript rendering, (3) cryptographic provenance embedding, (4) multi-project repository management, (5) FAIR-aligned software stewardship, and (6) AI-agent collaboration via structured documentation. Each existing framework addresses a subset; none provides the unified enforcement mechanism. The detailed tool-by-tool comparison is developed in the Comparison to Existing Tools section; the summary table below captures the gap landscape.

2.2.6 Summary: Gaps Left by Existing Tools

The six concerns identified above map onto existing tool categories as follows:

Gap Partially Addressed By Not Addressed By
Pipeline orchestration Snakemake, Nextflow, CWL Quarto, Jupyter Book, R Markdown, Typst, Overleaf, Prism
Manuscript rendering Quarto, Jupyter Book, R Markdown, Typst, Overleaf, Prism Snakemake, Nextflow, CWL, DVC
Testing enforcement — All existing tools
Cryptographic provenance SLSA (build-level attestation only) All research-focused tools
Multi-project management — All existing tools
AI-agent documentation Overleaf (partial co-author AI), Prism (partial context reasoning) All pipeline/workflow tools
Agentic skill protocol (MCP-aligned) — All existing tools

No existing system addresses all six concerns within a single enforced pipeline.

2.3 template/: An Integrated Solution

template/ was conceived as a structural antidote to this fragmentation. Rather than adding reproducibility as an afterthought—a Docker container wrapping an already-disjointed workflow (Boettiger 2015)—the template enforces integrity at the architectural level. It realizes Gentleman and Temple Lang’s research compendium vision (Gentleman and Temple Lang 2007) at repository scale, bundling code, data, tests, manuscripts, and provenance into a single, pipeline-enforced system with version-controlled infrastructure (Ram 2013). It stands on four primary pillars:

  1. Ergonomic Modularity: A Two-Layer Architecture cleanly separates globally shared infrastructure (logging, rendering, validation, steganography) from project-specific logic (manuscripts, scripts, data). 31 infrastructure subdirectories (28 importable packages) comprising ~1335 Python modules provide reusable services; projects consume them without modification.

  2. Execution Integrity: Pipeline advancement is contingent on test passage. The configured source-coverage floors are 60% for infrastructure and 90% for each project. The static gate rejects prohibited mock-framework imports, while the suites exercise real filesystem and subprocess behavior and use local protocol servers or explicitly enabled live services for network boundaries. ~12,564 infrastructure tests and 6335+ project tests are discovered by the introspection build.

  3. Automated Provenance: Steganographic watermarking and cryptographic hashing are integrated directly into the rendering pipeline. Every generated PDF carries SHA-256 (and, when enabled, SHA-512) fingerprints, an alpha-channel watermark (configurable text or QR overlay), per-page provenance footers and barcode strips, and embedded PDF metadata recording the build timestamp, document identifier, and Git commit hash. Provenance is not asserted by policy; it is enforced by architecture.

  4. AI-Agent Collaboration and Skill-Based Agentic Operations: A three-tier documentation architecture enables AI agents to operate at every level of the system. At the system level, root AGENTS.md and .agents/ define shared constraints and generic workflows; client files are compatibility references. At the structural level, AGENTS.md files in documented directories expose local API surfaces, file inventories, and integration contracts. At the module level, SKILL.md files—written to a discoverable YAML+Markdown schema aligned with the Model Context Protocol (Anthropic 2024)—describe skill-enabled modules and exemplars as reusable capabilities.

An agent invoking infrastructure.rendering does not need to read source code: it reads the rendering/SKILL.md, which declares the module’s name, description, activation metadata in YAML frontmatter and imports and example invocations in its Markdown body. This architecture is the practical realization of the skill-library paradigm established in the agent literature: Yao et al.’s ReAct framework (Yao et al. 2023) demonstrated that interleaving reasoning traces with tool invocations dramatically improves LLM reliability; Schick et al.’s Toolformer (Schick et al. 2023) showed that self-supervised tool use can be bootstrapped from natural language; Wang et al.’s Voyager (Wang et al. 2023) proved that growing skill libraries enable open-ended autonomous exploration in complex environments. template/ instantiates this vision in the domain of scientific research infrastructure: each SKILL.md is a Voyager-style skill, each pipeline stage is a ReAct action, and the full infrastructure layer constitutes a composable, protocol-aligned skill library for scientific computation.

2.4 Scope and Contributions

This paper is itself a product of the template it describes. The metrics populating its tables were computed by the introspection module documented in the Methods; the figures were rendered by the visualization code validated by the test suite described in Quality Assurance; the PDF carrying these words was assembled by the same YAML-declared pipeline whose architecture is the subject of the Results. This self-productive loop—where the system that is described is also the system that produces the description—is not incidental but structural, a concrete demonstration that template/ can sustain the full lifecycle from source code to published artifact within a single, version-controlled, pipeline-enforced repository. Our contributions are:

2.5 Paper Organization

The Methods describe the Two-Layer Architecture, Thin Orchestrator pattern, pipeline stages, and AI collaboration model. Results present quantitative metrics from multi-project execution, coverage analysis, and steganographic benchmarks. The Discussion addresses the Zero-Mock tradeoff, scalability implications, a detailed tool comparison, and future directions. The Infrastructure Module Reference provides detailed documentation for all 31 subdirectories. Security and Provenance describes the steganographic and cryptographic integrity layer. The Appendices provide pipeline, configuration, and comparative references.


3 Methods

The template/ architecture is deliberately bifurcated into a globally shared infrastructure/ layer and project-specific projects/ silos. This section describes the four core design patterns, the YAML-declared pipeline (17 stages; default full 10) pipeline that operationalizes them, and the AI collaboration model that distinguishes this system from conventional research templates.

3.1 The Two-Layer Architecture

The repository is organized into two strictly separated layers:

Infrastructure Layer (infrastructure/): 31 infrastructure subdirectories—28 of them independently-importable Python packages—comprising ~1335 modules and providing reusable services. Each importable package has its own __init__.py, AGENTS.md, and README.md, and exports a well-defined public API (the remaining subdirectories, e.g. config/, hold shared configuration). The infrastructure layer knows nothing about any specific project—it provides generic capabilities (logging, rendering, validation, steganography) that any project may consume.

Project Layer (projects/): Self-contained research workspaces. Each project directory contains:

Directory Purpose
manuscript/ Markdown chapters and config.yaml
scripts/ Thin orchestrator scripts (Stage 02)
src/ Project-specific Python modules
tests/ Project-specific test suite
data/ Input datasets and generated data
output/ Pipeline artifacts: PDF, figures, reports, logs
docs/ Project-specific architecture documentation

The two layers communicate exclusively through Python imports and filesystem paths. No project modifies infrastructure code; no infrastructure module references a specific project by name (except via runtime project discovery).

3.2 The Standalone Project Paradigm

Projects are designed to be completely self-contained. Adding a new project requires no changes to the infrastructure layer, no modifications to pyproject.toml, and no updates to the pipeline orchestrator. A project is automatically discovered if and only if it satisfies two conditions:

  1. It exists as a subdirectory of projects/.
  2. It contains the file manuscript/config.yaml.

This paradigm enables horizontal scaling: N researchers can maintain N independent projects within a single repository, sharing infrastructure without coupling. Each project declares its own testing tolerances, manuscript metadata, LLM review preferences, and rendering configuration in its config.yaml. The system currently hosts its public canonical exemplars under projects/templates/ (templates/template_active_inference, templates/template_advanced_literature_review, templates/template_autopoiesis, templates/template_autoresearch_project, templates/template_autoscientists, templates/template_code_project, templates/template_data_descriptor, templates/template_eda_notebook, templates/template_experiment_tree, templates/template_formal, templates/template_gold_refinement, templates/template_literature_meta_analysis, templates/template_madlib, templates/template_methods_paper, templates/template_newspaper, templates/template_pitch_deck, templates/template_pools_rules_tools, templates/template_prose_project, templates/template_redacted_report, templates/template_registered_report, templates/template_search_project, templates/template_sia, templates/template_storybook, templates/template_template, templates/template_textbook), including this meta-manuscript at projects/templates/template_template/.

3.3 The Thin Orchestrator Pattern

All scripts in scripts/ (both infrastructure-level and project-level) follow the Thin Orchestrator pattern (Gamma et al. 1995):

This pattern ensures that all testable logic lives in src/ where it is subject to the Zero-Mock testing policy, while scripts remain thin enough to be audited by visual inspection. The separation draws on the Mediator pattern from Gamma et al. (Gamma et al. 1995), where scripts mediate between infrastructure services and project-specific code without implementing any logic of their own.

To make this concrete, the following contrasts the anti-pattern with the correct pattern:

# ANTI-PATTERN: domain logic embedded in script
def calculate_average(data):          # ← never put computation here
    return sum(data) / len(data)

result = calculate_average([1, 2, 3])

# CORRECT PATTERN: script imports from src/ and only wires
from projects.my_project.src.statistics import calculate_average

result = calculate_average([1, 2, 3])  # ← scripts wire, never compute

The critical property is that calculate_average in the correct pattern lives in a testable src/ module, is covered by the Zero-Mock test suite, and can be independently imported, tested, and reused—whereas the anti-pattern buries logic in a script that is invisible to coverage tools.


3.4 DAG Pipeline Declared by pipeline.yaml

Single-project pipelines read infrastructure/core/pipeline/pipeline.yaml. scripts/runner/execute_pipeline.py expands the declarative DAG, applies tag filters (--mode core skips llm stages), checkpoints between nodes, then dispatches the declared scripts/pipeline/stage_NN_*.py scripts or builtin methods (_run_clean_outputs).

The YAML declares 17 stages. Tag filtering selects 10 stages for a default full run and 8 for core mode. The overview below covers the default full selection: core stages plus live LLM stages. Telemetry settings are configuration metadata, not additional DAG nodes.

  1. Clean Output Directories — wipes prior projects/<name>/output/ + delivered output/<name>/ paths so stale PDFs cannot satisfy validation.
  2. Environment Setup (scripts/pipeline/stage_00_setup.py) — Python/uv probing, toolchain discovery, scaffolding directories, PYTHONPATH wiring.
  3. Infrastructure Tests (scripts/pipeline/stage_01_test.py, --infra-only, --infra-scope pipeline-smoke) — focused real infrastructure contracts for the project pipeline. The full infrastructure suite and its configured ≥60% coverage floor remain separate verification.
  4. Project Tests (scripts/pipeline/stage_01_test.py --project-only) — isolated per-project suites with each project’s declared floor (≥90% for this exemplar).
  5. Project Analysis (scripts/pipeline/stage_02_analysis.py) — lexicographically ordered projects/<name>/scripts/*.py, each a thin orchestrator (src/ does real work).
  6. PDF Rendering (scripts/pipeline/stage_03_render.py) — Pandoc → XeLaTeX loop, bibliography assembly, injected variables from Stage 02 artefacts.
  7. Output Validation (scripts/pipeline/stage_04_validate.py) — PDF structure, manifests, Markdown hygiene.
  8. LLM Manuscript Review (scripts/pipeline/stage_06_llm_review.py --reviews-only; tags: llm) — executive + quality critiques via local Ollama; allow_skip: true.
  9. LLM Translations (scripts/pipeline/stage_06_llm_review.py --translations-only; tags llm, same dependency edges) — multilingual abstract expansion.
  10. Copy Outputs (scripts/pipeline/stage_05_copy.py) — reproducible snapshots into canonical output/<project>/.

The two LLM nodes share one script module with orthogonal CLI switches and depend on output validation. Full mode includes them; core mode excludes them. When a stage permits a skip, it must retain a fresh policy-bound reason receipt; unavailable required live acceptance remains blocked.

Executive reporting (scripts/pipeline/stage_07_executive_report.py) is a separate multi-project reporting operation, not an additional YAML node inside the single-project executor. execute_multi_project.py writes aggregate summary reports after project execution.

Topological order therefore differs slightly from declaration order (e.g., copy executes after validation even though stage stage_05_copy.py precedes stage_06_llm_review.py lexically).

3.4.1 Stage Highlights

Infrastructure and project tests. Separate invocations isolate the focused infrastructure smoke contract from each project’s nonempty test suite and project-local coverage gate. Required test or coverage failures and unsuccessful runners block pipeline completion; configured failure tolerances cannot turn a nonzero child test result into success. The full infrastructure coverage floor (≥60%) is verified separately. An explicitly skippable stage may accept exit 2 only with a fresh reason receipt bound to the current run, stage, command, and plan. That recorded skip does not satisfy required live acceptance.

Stage 02 illustration. The analysis stage is deliberately concrete rather than a hypothetical diagram factory: each canonical project ships real behaviour at this node. template_autoresearch_project runs readiness validation; template_search_project merges remote literature JSON, generates scripted figures (y_generate_search_figures.py), and writes manifests; template_code_project emits optimization plots; and template_prose_project triggers structural validation scaffolding. The pipeline shape is identical across all four—only the Stage 02 payload differs—which is exactly what lets one orchestrator serve heterogeneous research domains. The live public roster is generated in active_projects.md.

3.4.2 Interactive Orchestration

3.4.2.1 run.sh

Thin wrapper invoking python -m infrastructure.orchestration. Offers:

Selecting d alone after a passing multi-project run exits immediately once summaries print—avoiding repetitive menu redraw.

3.4.2.2 secure_run.sh

Executes Python secure path: standard pipeline artefact reproduction then invokes run_secure_pipeline for steganographic PDF hardening (infrastructure.steganography). Original PDFs stay immutable; hardened companions carry QR overlays plus hash manifests sidecars.


3.5 Documentation Duality and AI Collaboration

The Documentation Duality standard pairs two files in documented source directories; the live inventory reports actual coverage:

This Documentation Duality standard serves two purposes. First, it ensures that both human researchers and AI agents can navigate the codebase efficiently—AGENTS.md files provide the structured context that language models need to make informed code modifications without hallucinating API signatures or violating architectural invariants. Second, it creates a self-documenting feedback loop: as AI agents modify the codebase, they update the corresponding AGENTS.md files, keeping documentation synchronized with implementation. Lau and Guo’s survey of 90 AI coding assistant systems (Lau and Guo 2025) identifies contextual code understanding as a primary bottleneck; the Documentation Duality standard addresses this by providing pre-structured context at every directory level.

Root AGENTS.md defines shared architectural and testing constraints. .agents/ supplies generic skills, roles, workflows, and confined local state; .agents/AGENTS.md documents that catalog. Per-directory AGENTS.md adds local contracts, and README.md supplies human navigation. CLAUDE.md is a client command reference; .cursor/ and .cursorrules are deprecated compatibility bridges to the shared guidance.

3.6 Agentic Skill Architecture

The Documentation Duality standard addresses human and AI navigation at the directory level. A complementary layer operates at the module level: capability-specific SKILL.md descriptors add discoverable instructions and invocation examples to documented modules. A descriptor supplies context; executable tools remain separately registered interfaces.

3.6.1 The Three-Tier Skill Protocol

template/ implements a progression of agent-facing documentation, escalating in specificity from global constraints to module-level API contracts:

Tier File Scope Purpose
1 — System Root AGENTS.md, .agents/ Repository-wide Shared constraints, skills, roles, workflows, and local state
2 — Structure AGENTS.md Documented directories Local file inventories, API surfaces, integration patterns, architectural constraints
3 — Skill SKILL.md Skill-enabled modules and exemplars Machine-parseable skill descriptor: module name, description, key imports, usage pattern

Tier 1 and Tier 2 have direct analogues in the prompt-engineering literature: system prompts and retrieval-augmented context (Lau and Guo 2025). Tier 3 is novel. The SKILL.md files combine YAML discovery metadata with Markdown instructions. These descriptors complement the separately registered tools exposed through the Model Context Protocol (Anthropic 2024). The following presents the metadata from infrastructure/rendering/SKILL.md, with its description folded into equivalent YAML for readability:

---
name: infrastructure-rendering
description: >-
  Skill for the rendering infrastructure module providing multi-format
  output generation including PDF manuscripts, HTML web pages, and
  Beamer/Reveal.js slides. Use when rendering research outputs,
  converting markdown to PDF, generating slides, or configuring LaTeX
  rendering.
---

A skill loader reading this block discovers the module name and its natural-language activation condition. The Markdown body supplies Python imports and usage examples; the frontmatter does not itself register an executable MCP tool. This is the practical implementation of Toolformer-style self-documented tools (Schick et al. 2023)—rather than a language model learning tool APIs from training data, the APIs are encoded directly in version-controlled, co-located skill files that evolve with the codebase.

3.6.2 Module Skill Coverage

discover_infrastructure_modules() reports the actual README.md, AGENTS.md, SKILL.md, and PAI.md presence for each enumerated subdirectory. python -m infrastructure.skills maintains the shared .agents/skill_manifest.json; module and public-exemplar skills participate independently of any editor. PAI.md provides an optional integration narrative.

Promotion policy: new Layer‑1 directories must ship human + machine-readable docs (README.md, AGENTS.md) immediately; Tier‑3 SKILL assets follow once APIs stabilize.

3.6.3 MCP Server Mapping

The opt-in infrastructure.mcp_server exposes the existing operation catalog, pipeline description, discovered skills, and curated CLI invocation through list_operations, describe_pipeline, list_skills, and invoke_cli. CLI invocation is confined to registered infrastructure modules; mutating or external operations require the server’s explicit opt-in. Skill discovery and tool execution therefore remain separate contracts. The server does not automatically expose every Python function, rendered file, or suggested module-to-resource mapping.

An agent can inspect these catalogs and choose supported operations while retaining the pipeline’s declared dependencies and acceptance gates. Discovery alone does not establish successful execution or publication. The SKILL.md files parallel Voyager’s skill library (Wang et al. 2023)—Voyager’s agent accumulates a growing library of executable Minecraft skills represented as JavaScript functions; template/’s agent accumulates a curated library of research pipeline skills represented as YAML-frontmattered SKILL.md files. In both cases, the skill representation is machine-readable, version-controlled, and self-describing. Wang et al.’s LLM agent survey (L. Wang et al. 2024) identifies tool use, planning, and memory as the three fundamental capabilities of autonomous agents; Yao et al.’s ReAct framework (Yao et al. 2023) demonstrates that interleaving reasoning traces with tool actions dramatically improves agent reliability in interactive settings. The template/ skill architecture maps cleanly onto these three capabilities: the SKILL.md descriptors supply the tool-use layer, the declarative DAG of 17 pipeline.yaml stages (a default full run executes 10) supplies the planning scaffold, and the per-criterion checkpoint system supplies the memory layer.


3.7 FAIR Alignment and Research Infrastructure as Code

The template’s design aligns with both the original FAIR principles (Wilkinson et al. 2016) and the FAIR for Research Software (FAIR4RS) principles (Barker et al. 2022) at the repository level. FAIR4RS recognizes that software has requirements distinct from data—executability, composability, and dependency management—and the template addresses each.

3.7.1 Principle-by-Principle Alignment

Findability. Outputs are Findable through standardized directory structures, manifest files, and machine-readable metadata embedded in PDFs. Every project’s config.yaml provides structured metadata (title, authors, DOIs, keywords) in a format parseable by both Pandoc and external indexing services. The metrics.json output provides a machine-readable inventory of all generated artifacts, their locations, and their provenance hashes.

Accessibility. Outputs are Accessible via open-source distribution on GitHub, with metadata embedded in the artifact itself rather than in a separate registry. The steganographic layer embeds provenance information directly in the PDF—including SHA-256 content hashes, build timestamps, and QR-encoded metadata—ensuring accessibility even when the PDF circulates outside the repository.

Interoperability. Interoperability is achieved through standard formats (PDF, JSON, BibTeX, YAML) and well-defined module APIs that enable cross-project composition. The Pandoc rendering pipeline accepts any Markdown-with-LaTeX input conforming to the template’s section numbering conventions, allowing seamless migration of manuscripts from other Pandoc-based workflows.

Reusability. Reusability is ensured by the Standalone Project Paradigm—any project can be extracted and reused independently—and by the Documentation Duality standard, which satisfies FAIRsoft’s inspectability and documentation quality indicators (Garijo et al. 2024). The pipeline’s automated testing and coverage enforcement directly operationalize the FAIR4RS executability requirement: software that cannot pass its own test suite cannot produce publishable output.

3.7.2 Infrastructure as Code for Research

At a higher level of abstraction, template/ applies the DevOps principle of Infrastructure as Code (IaC) to the research lifecycle. In production software engineering, IaC means that server configuration is version-controlled, automatically provisioned, and independently reproducible (Wilson et al. 2017). template/ extends this principle to the research manuscript: the document is not authored in a word processor and emailed to collaborators, but built from version-controlled Markdown sources, tested against formal coverage thresholds, and deployed to a provenance-embedded PDF.

Every component of the research pipeline—the test suite, the analysis scripts, the rendering configuration, and the steganographic watermark—is specified in code, committed to git, and reproducible from a clean checkout. This deterministic build property means that any researcher can clone the repository, run ./run.sh --pipeline, and produce a byte-for-byte identical manuscript (modulo timestamps in the steganographic metadata).

Software Heritage (Di Cosmo et al. 2020) provides persistent SWHIDs (Software Hash Identifiers) for source code snapshots, enabling stable citation of any specific version of template/ as a discrete software artifact—closing the loop from research infrastructure to citable scientific contribution. Combined with Zenodo DOI registration (supported by infrastructure.publishing), this creates a dual-identifier citation chain: SWHID for source provenance, DOI for publication metadata (Katz et al. 2021).


3.8 Quality Assurance

3.8.1 Zero-Mock Testing Policy

The repository policy prohibits mock frameworks such as unittest.mock, MagicMock, and patch decorators (Martin 2008; Meszaros 2007). A static gate rejects those imports, and a separate inventory surfaces semantic dependency-replacement patterns for review. Tests that require external services (Ollama or public networks) use explicit pytest.mark markers for conditional execution; deterministic network tests use real local HTTP servers. The philosophical motivation—analogizing excessive interaction mocking to Simmons et al.’s researcher degrees of freedom (Simmons et al. 2011) and the preregistration remedy (Nosek et al. 2018)—is developed fully in the Zero-Mock Tradeoff discussion.

The following example, drawn from this meta-project’s own suite, illustrates zero-mock compliance:

def test_core_module_present(self):
    # Real filesystem, real YAML parsing — no MagicMock anywhere
    modules = discover_infrastructure_modules(REPO_ROOT)
    names = [m.name for m in modules]
    assert "core" in names, f"'core' not found in {names}"

This test (tests/core/test_meta.py::TestDiscoverInfrastructureModules::test_core_module_present) exercises the real discover_infrastructure_modules function against the real repository root. There are no mock objects substituting for the directory walk, no patched YAML parsers, and no synthetic return values—the test passes only if the infrastructure modules genuinely exist and are discoverable at their expected paths.

3.8.2 Coverage Thresholds

The pipeline enforces two coverage tiers:

Tier Scope Minimum Current Rationale
Project projects/*/src/ 90% Reported by the current test artifact Domain code must be thoroughly validated
Infrastructure infrastructure/ 60% Reported by the current test artifact Broader shared surface

These thresholds are enforced at Stage 01 of the pipeline. A project test run below its declared 90% floor fails before downstream publication stages. This manuscript deliberately does not quote an achieved coverage percentage unless a fresh coverage artifact supplies it.

3.8.3 Test Suite Composition

The repository maintains three test suites:

3.8.4 Visualization Standards

All generated figures must meet accessibility requirements:

The tests/figures/test_architecture_viz.py suite verifies that each real figure generator writes a non-empty PNG and that the comparative matrix has its declared shape and value domain. The font and render-resolution values above are injected from src/template_template/figures/viz_palette.py, preventing prose and implementation from drifting even though visual accessibility still requires human inspection of rendered figures.


4 Results

This section reports repository facts produced by the meta-project’s live introspection pass. Test counts, module inventories, pipeline-stage counts, and figure data are injected from generated artifacts. Runtime and achieved-coverage claims are intentionally omitted when no versioned benchmark or coverage artifact backs them.

4.1 Multi-Project Pipeline Execution

The ./run.sh interactive orchestrator can execute the public roster through the shared DAG. The table below is a structural snapshot, not a timing benchmark: counts come from repository introspection, and the coverage column states each exemplar’s declared policy floor rather than an unversioned observed percentage.

Project Effective core stages¹ Discovered tests Declared project floor
template_code_project 8 263 90%
template_prose_project 8 165 90%
template_autoresearch_project 8 318 90%

¹“Core-only” excludes LLM-tagged and other opt-in stages according to the YAML stage tags. A fresh run must be used to establish completion status or wall-clock performance for a particular machine and dependency set.

4.2 Infrastructure Test Suite

Metric Value
Test files 947+
Total tests ~12,564
Infrastructure coverage gate ≥60% configured floor
Prohibited mock-framework imports Checked by the static no-mocks gate

Exercises Pandoc/XeLaTeX paths, steganography hashing, telemetry, YAML-driven pipeline DAG resolution, HTTPS-bound optional suites (pytest-httpserver), and local Ollama-gated subsets.

4.3 Infrastructure Module Inventory

The introspection module (template_template.core.introspection) emits the authoritative table below—every row reflects discover_infrastructure_modules(REPO_ROOT).

Module Python Files Has AGENTS.md Has README.md Key Exports
autoresearch 11 ✓ ✓ build_autoresearch_plan, readiness validation CLI
benchmark 4 ✓ ✓ Template harness scoring + comparative gates
config 0 ✓ ✓ Repository defaults + hardened templates
core 201 ✓ ✓ get_logger, load_config, TemplateError
docker 0 ✓ ✓ Containerisation scaffolding
doctor 14 ✓ ✓ Checkout diagnose/fix/undo repairs
documentation 23 ✓ ✓ FigureManager, build_api_index, generate_markdown_table
fonds 6 ✓ ✓ —
formal_contracts 5 ✓ ✓ —
llm 57 ✓ ✓ Ollama helpers, sanitization, review + translation pipelines
logrotate.d 0 ✓ ✓ Rotation snippets (documentation-first)
metadata 3 ✓ ✓ normalized_repository_url, generate_metadata_package
methods 10 ✓ ✓ build_methods_orchestration_plan, methods-stage contracts + validation
orchestration 14 ✓ ✓ PipelineRunner, entry point for ./run.sh
project 116 ✓ ✓ discover_projects, workspace management
prose 9 ✓ ✓ Markdown readability + prose tooling
provenance 9 ✓ ✓ —
publishing 85 ✓ ✓ Zenodo, executable bundle, archival targets
reference 16 ✓ ✓ BibTeX models, parsers, converters
rendering 121 ✓ ✓ PDF/HTML/slide rendering, Pandoc filters
reporting 412 ✓ ✓ Coverage parsers, dashboards, executive artefacts
research 3 ✓ ✓ —
rules 6 ✓ ✓ —
scientific 4 ✓ ✓ check_numerical_stability, benchmark_function
search 77 ✓ ✓ infrastructure.search.literature clients + cache
sia 10 ✓ ✓ Self-Improving-AI loop: task validation, harness, metric capture
skills 8 ✓ ✓ discover_skills, SKILL manifest regeneration
steganography 13 ✓ ✓ Watermark overlays + hash manifests
tools 6 ✓ ✓ —
transmission 6 ✓ ✓ —
validation 86 ✓ ✓ PDF + Markdown + integrity CLIs

All 31 enumerated subdirectories carry Tier‑1/README.md and Tier‑2/AGENTS.md coverage wherever the Documentation Duality standard applies; subsets ship Tier‑3 SKILL.md descriptors for MCP routing (infrastructure/skills manifest generation).

4.4 Agentic Skill Documentation Coverage

Layer Role
Shared agent guidance Root AGENTS.md, .agents/ skills, roles, and workflows
Structural AGENTS.md directories
Skills Optional SKILL.md manifests + generated .agents/skill_manifest.json
PAI capsule Repository level PAI.md narratives

378+ Markdown shards under docs/ capture operational knowledge without duplicating auto-generated inventories.

4.5 DAG Reference (Declarative Executor)

The table summarizes the 10 default full-run stages (core plus live LLM nodes) from 17 YAML declarations; core mode selects 8. Separately tagged opt-in extensions are omitted here. Entries follow dependency order rather than numeric script filenames.

Name Typical script / method Responsibility Failure semantics
Clean Output Directories _run_clean_outputs Deletes stale output/ trees Blocking
Environment Setup scripts/pipeline/stage_00_setup.py Validates tooling, PYTHONPATH scaffolding Blocking
Infrastructure Tests scripts/pipeline/stage_01_test.py --infra-only --infra-scope pipeline-smoke Focused infrastructure smoke contracts Required tests must pass
Project Tests scripts/pipeline/stage_01_test.py --project-only Project-local tests + coverage gates Required tests and coverage must pass
Project Analysis scripts/pipeline/stage_02_analysis.py Executes projects/<name>/scripts/*.py Blocking
PDF Rendering scripts/pipeline/stage_03_render.py Pandoc → XeLaTeX manuscripts Blocking
Output Validation scripts/pipeline/stage_04_validate.py Structural PDF/markdown probes Blocking / warnings
LLM Manuscript Review scripts/pipeline/stage_06_llm_review.py --reviews-only Local Ollama reviews Fresh policy-bound skip; required live acceptance remains blocking
LLM Translations scripts/pipeline/stage_06_llm_review.py --translations-only Optional translations Fresh policy-bound skip
Copy Outputs scripts/pipeline/stage_05_copy.py Mirrors deliverables → output/<project>/ Blocking

scripts/pipeline/stage_07_executive_report.py provides separate multi-project executive reporting, not an additional single-project DAG node (execute_pipeline.py). The multi-project executor writes its aggregate summary after project execution.

4.6 Steganographic Performance Boundary

The steganography path exposes metadata injection, SHA-256 hashing, overlay generation, and optional QR-code insertion as separately testable operations. This revision makes no latency claim because the project does not currently generate a versioned benchmark artifact recording hardware, input PDF, warm-up policy, repetitions, and dispersion. A future performance table should be compiled from such an artifact rather than copied from an ad-hoc workstation run.

4.7 Self-Referential Analysis

Rendered via projects/templates/template_template (generate_manuscript_metrics.py → injected tokens such as 31). Architecture figures stem from template_template.figures.architecture_viz—font sizes constrained by §QA.

Infrastructure module cards appear above public-project cards, joined by a pipeline arrow. Each card records the corresponding file counts and documentation badges.
Figure 1: Live rendering of the Two-Layer Architecture from repository introspection: the infrastructure layer (top) holds the 31 enumerated infrastructure directories, each annotated with its Python file count and a four-slot documentation badge—A AGENTS.md, R README.md, S SKILL.md, P PAI.md, with · marking an absent file—so a fully documented module reads [ARSP]. A YAML DAG arrow connects it to the project layer (bottom) of public exemplars labelled with chapter and test-file counts. The takeaway: documentation-duality coverage is near-uniform across the infrastructure, and every box was placed from the same live data the prose cites.
Pipeline stage cards list every YAML declaration in reading order, with stage numbers, complete names and tags. Their order describes declarations rather than a dependency chain.
Figure 2: Declared-stage reference with all 17 YAML stages in reading order, showing complete names and tags (core, LLM, ebook, docxplus, metadata, bundle, archival). Panel order documents declarations; it does not imply a linear dependency chain.
A horizontal bar chart compares Python file counts across infrastructure modules, with the largest modules first and smaller modules forming a long tail.
Figure 3: Horizontal file-count histogram of every infrastructure subdirectory, sorted largest-first.

The long tail of small, single-purpose packages beside a handful of larger ones (core, validation, publishing) is the visual signature of the Unix-philosophy modularity the architecture section argues for—capability concentrated where it compounds, not spread evenly by fiat.

4.8 Comparative Feature Analysis

The comparative feature matrix below summarizes the Appendix F capability matrix.

A heatmap compares tools across core pipeline, quality and security, and ecosystem capabilities. Each cell distinguishes full, partial or plugin-hosted, and unavailable support.
Figure 4: 14 × 10 heatmap annotated in appendix text—green ✓ full native capability, yellow ◐ partial / plugin-hosted, red — unavailable. Rows group Core Pipeline, Quality & Security, then Ecosystem.

¹ Nextflow lineage records exist at workflow scope, not per rendered PDF citation graph.

² DVC: content-addressed artifacts without native prose rendering.

³ DVC: remote object stores (S3, GCS, Azure) without turnkey CI manuscript gates.

4.9 Test Quality Metrics


5 Discussion

5.1 The Zero-Mock Tradeoff

The Zero-Mock testing policy is template/’s most distinctive design decision. By prohibiting all mock objects, we gain confidence that tests exercise real code paths—a pytest run against the template genuinely invokes pandoc, writes to disk, and parses real YAML. The cost is test duration: the full infrastructure test suite (~12,564 tests) runs for minutes rather than the sub-second execution typical of heavily-mocked suites. This manuscript deliberately reports no wall-clock figure, consistent with the Results section’s discipline of declining timing claims without a versioned benchmark artifact.

We argue this tradeoff is strongly favorable for research software. Unlike web applications where millisecond latency and thousands of daily deploys demand fast feedback loops, research pipelines run infrequently (once per manuscript revision) and correctness vastly outweighs speed. A mocked test that passes while the real renderer fails is worse than a slow test that catches the failure. The analogy to statistical methodology is precise: just as Simmons et al.’s researcher degrees of freedom (Simmons et al. 2011) inflate false-positive rates through undisclosed analytical flexibility, mock objects create testing degrees of freedom that make integration failures invisible. The Zero-Mock policy closes this loophole by the same mechanism that pre-registration (Nosek et al. 2018) closes the p-hacking loophole: removing flexibility before the fact. As Peng (Peng 2011) argues, computational reproducibility requires independent verification—and mock-only tests verify assumptions rather than results. Garijo et al.’s FAIRsoft evaluator (Garijo et al. 2024) identifies executability as a primary quality indicator; the Zero-Mock policy operationalizes executability at the unit level.

5.1.1 When Mocks Are Not the Problem

It is important to distinguish the Zero-Mock policy from a naive rejection of all test isolation techniques. Fowler’s classification (Martin 2008) recognizes that stubs and fakes serve legitimate purposes—a test database populated with known data is not a mock, it is a fixture. The policy specifically prohibits mock objects as defined by Meszaros: assertions on indirect outputs (method calls, argument patterns) rather than direct outputs (return values, side effects). The distinction matters because mock-based assertions encode implementation assumptions (“method X must be called with argument Y”) that become invisible coupling between tests and production code, creating the illusion of coverage without testing real behavior.

5.1.2 Practical Implementation

The template enforces zero-mock compliance at three levels:

  1. Code review: AGENTS.md at every directory level explicitly states the prohibition, ensuring both human and AI contributors are aware before writing tests.
  2. Static analysis: scripts/audit/verify_no_mocks.py implements the enforced lexical gate (prohibited mock-framework imports and calls) and an --inventory mode that separately classifies environment isolation from dependency-replacement debt; pre-push runs it automatically.
  3. Cultural norm: template_code_project documents filesystem + YAML + plotting paths while template_autoresearch_project exercises readiness planning; template_search_project reinforces HTTP-realistic literature queries—both serve as onboarding references alongside infra suites.

However, the policy requires careful management of external dependencies. Tests requiring Ollama (the local LLM backend) use @pytest.mark.requires_ollama and are skipped in environments where the service is unavailable. Tests requiring network access use @pytest.mark.network. This marker system preserves the Zero-Mock principle while acknowledging that not all environments provide all services, especially computationally intensive ones. The key distinction is between replacing an external dependency (which mock objects do, hiding failures) and skipping a test when a dependency is absent (which markers do, preserving transparency).


5.2 Scalability: From 1 to N Projects

The Standalone Project Paradigm enables horizontal scaling: a project supplies its manuscript/config, isolated source and tests, thin analysis scripts, and declared dependencies. New projects use existing infrastructure interfaces; admission to the tracked public roster and release matrix remains explicit. The run.sh orchestrator automatically discovers new projects and presents them in its interactive menu.

We have validated scaling with 25 canonical exemplars under projects/templates/—always present for onboarding and tooling—and with this manuscript from projects/templates/template_template (157 tests) as a git-tracked public exemplar in the same automated discovery menus.

Canonical trio:

Meta manuscript (projects/templates/template_template) analyzes the repository via src/template_template/ introspection metrics; it now lives alongside the other public exemplars under projects/templates/.

These workspaces share no project-level code—only Layer 1 (31 infrastructure subdirectories, ~1335 Python files)—validating insulation between domain repos and reusable services.

5.2.1 Multi-Project Orchestration

When the --all-projects flag is passed to run.sh, the pipeline executes each discovered project sequentially, running infrastructure tests once at the start and skipping them for individual projects to avoid redundant validation. After all projects complete, a cross-project executive report aggregates metrics (test counts, coverage percentages, page counts, rendering durations) into a unified dashboard with both JSON and Markdown output formats. This executive reporting stage provides repository-level visibility without requiring any project-specific reporting code.

5.2.2 Scaling Metrics

Metric template_code_project template_prose_project template_autoresearch_project
Source modules 25 8 59
Test files 16 13 23
Test count 263 165 318
Manuscript chapters 9 8 6
Analysis scripts 9 5 5
Figures (auto-generated) 9 5 27

The infrastructure overhead per project is constant regardless of project size: the same 31 modules, the same 17 declared pipeline stages, the same rendering and validation logic. The shared implementation does not grow with the project roster; execution time and output volume still grow with the work performed.


5.3 Comparison to Existing Tools

The gap analysis established that no single tool integrates all six cross-cutting concerns. Here we synthesize the fourteen-dimension comparison into three structural insights. First, the landscape bifurcates: workflow managers (Snakemake (Köster and Rahmann 2012), Nextflow (Di Tommaso et al. 2017), CWL (Amstutz et al. 2016)) provide distributed execution but no manuscript support; publication tools (Quarto (Allaire et al. 2024), Jupyter Book, R Markdown (Xie et al. 2018), Overleaf (Overleaf (Digital Science) 2025), Prism (OpenAI 2026)) author documents but embed no integrity guarantees; and DVC (Iterative, Inc. 2024) versions artifacts without orchestrating pipelines. template/ occupies the intersection, sacrificing distributed execution for unified enforcement of testing, provenance, and documentation. This positioning is complementary—a mature deployment might use Nextflow upstream and template/ for rendering, testing enforcement, and provenance downstream. Typst (Mädje and Haug 2023), with its faster compilation cycle, is not one of the nine compared peers but could serve as an alternative rendering backend if a Pandoc writer were contributed.

Second, the eight enforcement capabilities template/ co-locates—testing enforcement, coverage thresholds, steganographic watermarking, multi-project management, AI-agent documentation, the agentic skill protocol, an interactive TUI, and Zero-Mock policy—are individually straightforward (and several, such as multi-project management and AI-agent documentation, are matched in part by individual peers). Their value here lies in a shared pipeline contract: required test and documentation gates run together, while selected provenance stages carry their own validation. The FAIR4RS principles (Barker et al. 2022; Lamprecht et al. 2020) articulate what research software quality requires; FAIRsoft (Garijo et al. 2024) scores compliance observationally; template/ operationalizes part of this agenda by failing project tests below the declared 90% floor and validating provenance artifacts when those stages are enabled. Cohen et al.’s four pillars of research software engineering (Cohen et al. 2021)—sustainability, quality, community, and policy—are addressed primarily through the first two pillars via quality-gated automation.

Third, the AI-agent documentation dimension reveals an underserved need. Overleaf and Prism provide AI writing assistance, but neither exposes structured documentation for external agents to consume. template/’s AGENTS.md + SKILL.md layer enables an agent entering the repository to discover capabilities, understand API contracts, and invoke modules without prior training (Documentation Duality, AI Collaboration).

5.3.1 FAIR4RS Evolution (2024–2026)

Since the FAIR4RS principles were published (Barker et al. 2022), the community has moved toward operationalization. The RDA Virtual Plenary 24 (April 2025) featured a two-year retrospective review (Honeyman et al. 2024) recommending principle amendments—notably adding reproducibility as an explicit requirement and clarifying “domain-relevant standards”—alongside a leadership refresh and parallel guidance activities. The ReSA Actionable FAIR4RS Task Force (launched December 2024) analyzed the 17 principles into six actionable categories (identifiers, metadata for publication/discovery/reuse, standards, references, and licenses), with a first draft expected by September 2025 (Research Software Alliance 2024). Tools for automated FAIR assessment have also matured: the F-UJI extension for research software evaluation now scores against FRSM-04 through FRSM-17 metrics, complementing Garijo et al.’s FAIRsoft evaluator (Garijo et al. 2024). template/’s pipeline-enforced quality gates—coverage thresholds, documentation completeness checks, and provenance embedding—anticipate this operationalization trend by implementing FAIR4RS not as a post-hoc assessment but as an architectural invariant.

In Gentleman and Temple Lang’s terminology (Gentleman and Temple Lang 2007), template/ is a research compendium scaled to the repository level—bundling not just one study’s code and data but N studies, with shared infrastructure, automated testing, and embedded provenance. Nüst et al.’s executable research compendium (ERC) (Nüst et al. 2017) extends this vision with containerized reproduction environments; template/ complements containerization by adding the testing enforcement, multi-project management, and provenance embedding layers that ERCs do not address.


5.4 The AI Collaboration Model

The Documentation Duality standard and three-tier skill architecture represent an empirical bet: that structured, machine-readable documentation measurably improves AI agent performance in research codebases. This section reports our key observations.

The documentation investment creates a positive feedback loop: as agents produce higher-quality outputs from structured context, developers maintain that documentation, which in turn improves future interactions (Lau and Guo 2025). We observed this concretely during template/ development—capability-specific SKILL.md descriptors were refined through iterative AI-assisted generation, serving as both input prompt and output validator.

The SKILL.md layer, with its MCP-aligned YAML frontmatter (Anthropic 2024), provides a bridge to the agentic software paradigm. Lu et al.’s AI Scientist (Lu et al. 2024) demonstrates end-to-end autonomous research, while OpenHands is evaluated on SWE-Bench Verified (X. Wang et al. 2024; Jimenez et al. 2024). These systems motivate structured, protocol-aligned tool inventories for unfamiliar codebases. An agent navigating template/ reads root AGENTS.md and .agents/AGENTS.md for shared constraints and generic workflows, scans AGENTS.md for local contracts, and can invoke skill-enabled capabilities through SKILL.md descriptors. The live module inventory verifies AGENTS.md and README.md coverage; SKILL.md remains a capability-specific layer rather than a universal claim about every infrastructure directory.

This three-tier model is, to our knowledge, novel in the research software engineering literature. The scale of the investment is substantial: 378 Markdown files under docs/ alone, plus an AGENTS.md/README.md pair in documented directories and a SKILL.md descriptor on skill-enabled infrastructure modules. That count is itself injected from live introspection—the manuscript refuses to quote a documentation total it cannot recompute—and it represents a deliberate commitment to machine-readable context that shrinks the surface on which an agent can hallucinate.

5.5 The Learning Curve

The Thin Orchestrator pattern imposes a cognitive overhead on researchers accustomed to writing monolithic scripts. The requirement to factor all logic into src/ modules and use scripts only as stateless wiring introduces an additional layer of indirection. We mitigate this through:

  1. Template exemplars: template_code_project ships minimal optimization commentary; template_prose_project and template_autoresearch_project broaden narrative + retrieval scaffolding.
  2. Documentation Duality: Every directory has both README.md (for humans) and AGENTS.md (for AI collaborators), reducing the cost of navigation.
  3. Interactive orchestrator: run.sh provides a TUI menu that abstracts pipeline complexity.
  4. Skill-level documentation: The docs/guides/ directory provides a progressive sequence of beginner, intermediate, advanced, and expert guides alongside a comprehensive new-project setup checklist.

5.6 Limitations


5.7 Future Directions

  1. Supply-chain provenance: Integration with software supply chain frameworks such as in-toto (Torres-Arias et al. 2019) and SLSA (Supply-chain Levels for Software Artifacts) (Open Source Security Foundation 2023) to provide end-to-end attestation from source commit through build pipeline to published artifact. SLSA’s four graduated levels of build integrity (from basic provenance to hermetic, reproducible builds) provide a natural extension ladder for the template’s currently document-centric provenance model. The template’s existing steganographic layer embeds document-level provenance; supply-chain frameworks would add build-level provenance, closing the gap between “this PDF was produced by this pipeline” and “this pipeline was executed with this verified source code.”
  2. Decentralized provenance: Integration with IPFS or Arweave for immutable publication records, extending the SHA-256-based tamper detection to content-addressed storage networks.
  3. Digital signatures: GPG or X.509 signing integrated into the steganographic layer, providing cryptographic non-repudiation in addition to tamper detection.
  4. Continuous integration: GitHub Actions workflows that execute the pipeline on every push, with PDF artifacts as release assets and automated DOI registration via Zenodo.
  5. Multi-language support: Extension of the Thin Orchestrator pattern to R, Julia, and Rust projects, enabling polyglot research workflows within the Two-Layer Architecture.
  6. Automated FAIR4RS assessment: Periodic self-scoring against FAIRsoft metrics (Garijo et al. 2024), with quality indicators (executability, documentation completeness, metadata richness) tracked as pipeline artifacts alongside test coverage and rendering status.
  7. Knowledge graph integration: Connecting pipeline outputs to Active Inference Knowledge Base entries for automated meta-analysis and cross-project citation tracking.
  8. Formal verification: Static analysis tooling to enforce the Thin Orchestrator pattern—verifying that scripts contain no algorithmic logic and that src/ modules do not import from scripts/.
  9. Agentic research pipelines via MCP: The SKILL.md descriptors already define the interface contracts for each infrastructure module; the natural next step is to expose them as MCP server endpoints (Anthropic 2024). An MCP server wrapping infrastructure.llm would expose query, review_manuscript, and translate_abstract as protocol-native Tools; an MCP server wrapping infrastructure.publishing would expose publish_to_zenodo and generate_citation_bibtex. A research agent could then compose these Tools to execute the full pipeline—environment setup → test execution → analysis → rendering → validation → LLM review → DOI registration—without any human in the loop. This closes the loop opened by Lu et al.’s AI Scientist (Lu et al. 2024), which demonstrated automated hypothesis generation and experimental iteration but relied on ad hoc laboratory scaffolding. template/’s pipeline, fully exposed as MCP tools, provides that scaffolding in a reproducible, versioned, and certified form. Longer-term, the Agent2Agent (A2A) protocol (Google 2025) enables heterogeneous specialist agents—a statistical analyst, a figure designer, a peer-review simulator—to coordinate via a shared, protocol-mediated workspace built on precisely the kind of modular, well-documented infrastructure that template/ provides.
  10. Research Infrastructure as Code and software citation: The DevOps IaC paradigm, applied to research, means the entire manuscript pipeline is a version-controlled, reproducible artifact in its own right. Software Heritage (Di Cosmo et al. 2020) provides persistent SWHIDs (Software Hash Identifiers) for source-code snapshots, enabling template/ itself to be cited as a software artifact with a DOI-equivalent stable identifier. Combining this with Zenodo DOI registration (already supported by infrastructure.publishing) creates a full citation chain: the paper cites the data (DOI), the data provenance cites the pipeline (SWHID), and the pipeline cites the framework release (Zenodo DOI). This three-link citation chain operationalizes the Katz et al. (Katz et al. 2021) software citation principles at the infrastructure level.

5.8 Conclusion

template/ demonstrates that high-integrity, reproducible research need not be onerous. By embedding provenance, testing, and documentation into the architecture itself—rather than layering them atop a fragmented workflow—the template transforms “best practices” from aspirational guidelines into enforced invariants (Wilson et al. 2017; Sandve et al. 2013; Lamprecht et al. 2020). The Two-Layer Architecture ensures that infrastructure improvements propagate to all projects without coupling. The Zero-Mock policy ensures that tests reflect reality. The steganographic provenance layer ensures that published artifacts carry their own authentication. The comparative analysis distinguishes native, partial, and absent support across its versioned capability inventory. The total comparison inventory includes capabilities that template/ does not fully support, such as distributed execution and multi-language research. It is not the count of fully native template/ capabilities, and the snapshot does not establish an exhaustive claim about all existing tools.

The template is not merely a build tool; it is an epistemological commitment. It asserts that a research paper is not a static document but a build artifact—reproducible, verifiable, and traceable to the code that generated it. As Knuth observed, programs should be written for humans to read and only incidentally for machines to execute (Knuth 1984). We extend this dictum: research manuscripts should be built for verification and only incidentally for reading. In an era of generative AI, AI-native research workspaces, and synthetic media—where the boundary between human-authored and machine-generated text grows increasingly indeterminate (Gruenpeter et al. 2021)—the provenance chain from source code to published PDF is not an administrative convenience. It is the epistemic ground on which scientific trust must be rebuilt. That this manuscript was itself built, tested, and watermarked by the pipeline it describes—its metrics computed from the repository it inhabits, its figures rendered by the code it documents—demonstrates one reproducible artifact path; broader reliability claims require the declared checks and candidate-bound execution evidence.


6 Infrastructure Module Reference

This section inventories every Layer‑1 subdirectory returned by 31 discover_infrastructure_modules(repo_root). File totals use 1335 Python sources across infra + 12,564 infra tests guarding them. Documentation Duality = paired README.md + AGENTS.md; optional SKILL.md manifests feed python -m infrastructure.skills.

Module Python Files Has AGENTS.md Has README.md Key Exports
autoresearch 11 ✓ ✓ build_autoresearch_plan, readiness validation CLI
benchmark 4 ✓ ✓ Template harness scoring + comparative gates
config 0 ✓ ✓ Repository defaults + hardened templates
core 201 ✓ ✓ get_logger, load_config, TemplateError
docker 0 ✓ ✓ Containerisation scaffolding
doctor 14 ✓ ✓ Checkout diagnose/fix/undo repairs
documentation 23 ✓ ✓ FigureManager, build_api_index, generate_markdown_table
fonds 6 ✓ ✓ —
formal_contracts 5 ✓ ✓ —
llm 57 ✓ ✓ Ollama helpers, sanitization, review + translation pipelines
logrotate.d 0 ✓ ✓ Rotation snippets (documentation-first)
metadata 3 ✓ ✓ normalized_repository_url, generate_metadata_package
methods 10 ✓ ✓ build_methods_orchestration_plan, methods-stage contracts + validation
orchestration 14 ✓ ✓ PipelineRunner, entry point for ./run.sh
project 116 ✓ ✓ discover_projects, workspace management
prose 9 ✓ ✓ Markdown readability + prose tooling
provenance 9 ✓ ✓ —
publishing 85 ✓ ✓ Zenodo, executable bundle, archival targets
reference 16 ✓ ✓ BibTeX models, parsers, converters
rendering 121 ✓ ✓ PDF/HTML/slide rendering, Pandoc filters
reporting 412 ✓ ✓ Coverage parsers, dashboards, executive artefacts
research 3 ✓ ✓ —
rules 6 ✓ ✓ —
scientific 4 ✓ ✓ check_numerical_stability, benchmark_function
search 77 ✓ ✓ infrastructure.search.literature clients + cache
sia 10 ✓ ✓ Self-Improving-AI loop: task validation, harness, metric capture
skills 8 ✓ ✓ discover_skills, SKILL manifest regeneration
steganography 13 ✓ ✓ Watermark overlays + hash manifests
tools 6 ✓ ✓ —
transmission 6 ✓ ✓ —
validation 86 ✓ ✓ PDF + Markdown + integrity CLIs

6.1 Alphabetical summaries

Below, ${module_*_python_file_count} placeholders expand per subdirectory at render-time.

6.1.1 infrastructure.autoresearch (11 files)

Readiness planner, validation CLI, and report models for AutoResearch-style project promotion (infrastructure/autoresearch/).

6.1.2 infrastructure.benchmark (4 files)

Template harness scoring and comparative gate helpers exercised in CI smoke paths.

6.1.3 infrastructure/config (non-package subdirectory)

Repository-wide YAML templates and secure manifests (.env.template, hardened defaults referenced by Docker + CLI). config/ carries no __init__.py, so it is a configuration subdirectory rather than an importable package.

6.1.4 infrastructure.core (201 files)

Checkpointing, logging, pipeline YAML parsing, telemetry bridges, filesystem helpers, hardened exceptions. Everything else imports logging + error taxonomy from here first.

6.1.5 infrastructure.doctor (14 files)

Checkout diagnose/fix/undo repairs for broken local workspace states.

6.1.6 infrastructure.docker (0 files)

Pinned images / compose scaffolding for reproducible CI + remote builds.

6.1.7 infrastructure.documentation (23 files)

Figure registries plus glossary tooling feeding manuscript automation.

6.1.8 infrastructure.formal_contracts (5 files)

Typed, machine-checkable manuscript contract layer: frozen block model, evidence-tier policies, monoidal composition, and a fail-closed checker with stable FORMAL.* diagnostics.

6.1.9 infrastructure.fonds (6 files)

Resource pool management for curated fonds (tracked reference datasets, bibliographic collections, and evidence corpora). Fonds mirror projects/templates/ with git-tracked templates/* exemplars and sidecar-linked private lifecycle folders.

6.1.10 infrastructure.llm (57 files)

Ollama integrations, sanitization adapters, templated reviewer flows. Literature ingestion now lives primarily in search/literature + citation helpers in reference/.

6.1.11 infrastructure.metadata (3 files)

Shared publication-metadata leaves: repository-URL normalization (normalized_repository_url) and ebook metadata package generation (ONIX, metadata.json, EPUB OPF). Consumed by publishing, rendering, documentation, and project layers; extracted from publishing by RENDERING-LAYERING-1 so renderers import it without an inverted publishing dependency.

6.1.12 infrastructure.methods (10 files)

Deterministic methods-orchestration contracts (MethodStage, MethodsOrchestrationPlan, MethodsIssue): builds and validates an ordered methods plan for a research project so the manuscript’s “Methods” track stays bound to executable stages.

6.1.13 infrastructure.orchestration (14 files)

python -m infrastructure.orchestration exposes interactive menus, subprocess wiring for thin shell wrappers (run.sh, secure_run.sh), and stubs used in CI for menu parsing tests.

6.1.14 infrastructure.project (116 files)

Canonical discovery (discover_projects) enforcing src/ + tests/, slug validation, nested WIP namespaces.

6.1.15 infrastructure.prose (9 files)

Readability metrics + Markdown tooling for prose-centric manuscripts / CI gates.

6.1.16 infrastructure.provenance (9 files)

Content-addressed provenance DAG. Records artifact lineage (which run produced which file, from which inputs) as a verifiable graph of artifact/run/source/claim nodes connected by produced/consumed/derived-from/supports/refutes edges. Includes a structured Review system with severity (blocking/major/minor/info) and verdict (refutes/supports). Features a CLI and pipeline integration hooks for automatic lineage recording after every stage.

6.1.17 infrastructure.publishing (85 files)

Metadata models, APA/BibTeX/MLA formatters, optional Zenodo clients.

6.1.18 infrastructure.reference (16 files)

Citation/BibTeX parsing + conversion utilities leveraged by manuscripts and retrieval scripts.

6.1.19 infrastructure.rendering (121 files)

Pandoc shim, Unicode/XeLaTeX postprocessors, combined PDF/HTML/slide exporters.

6.1.20 infrastructure.reporting (412 files)

Parses pytest + coverage artefacts for dashboards; pairs with Stage 01 summaries and downstream executive exports.

6.1.21 infrastructure.research (3 files)

Seven-stage research workflow (SCOPE → LITERATURE → REASON → DESIGN → COMPUTE → SYNTHESIZE → WRITE) defined as typed ResearchStage data classes with explicit outputs, skills, and template commands per stage. Includes a full PRISMA-adapted literature review prompt and a research workflow prompt referencing template/ infrastructure commands.

6.1.22 infrastructure.rules (6 files)

Governance rules layer for validating project lifecycle transitions, sidecar sync policies, and pipeline gate orchestration. Rules mirror projects/templates/ with git-tracked templates/* exemplars.

6.1.23 infrastructure.scientific (4 files)

Stability probing, benchmarking hooks—consumed heavily by optimization exemplars (template_code_project scripts).

6.1.24 infrastructure.search (77 files)

Two-tier search architecture: the literature/ client stack (client.py, backends, caches) powers literature search with arXiv, Crossref, local, and Paperclip backends. connectors/ exposes the built-in scientific database adapters through a uniform ConnectorRegistry; OpenAlex, UniProt, PDB, Semantic Scholar, European PMC, bioRxiv, and other registered adapters share list/search CLI commands with HTTP timeout, retry, and TTL caching. monid/ provides an opt-in paid gateway (discover / inspect / run) with offline USD/1k pricing reference in monid/PRICING.md. deep_research/ dispatches long-horizon provider-neutral manuscript research (PAID, opt-in). The live registry, not this prose, is authoritative for connector count.

6.1.25 infrastructure.sia (10 files)

Generic Self-Improving-AI loop utilities: task-layout validation, execution harness, and metric capture reused by template_sia (fixture-replay by default).

6.1.26 infrastructure.skills (8 files)

Discovers SKILL.md frontmatter → .agents/skill_manifest.json.

6.1.27 infrastructure.steganography (13 files)

Watermark overlays, hashing companions triggered by secure pipeline path.

6.1.28 infrastructure.tools (6 files)

Invocable tool definitions registered by resource-pool governance; tools mirror projects/templates/ with git-tracked templates/* exemplars.

6.1.29 infrastructure.transmission (6 files)

Rendered-release transmission leaves: LaTeX single-page bookends (transmission_bookends), page-check validation for Stage 04 (transmission_page_check), barcode strips, figure embedding, and shared models. Consumed by the publishing release workflow; regular package so the documentation module-reference guard resolves nested refs on disk.

6.1.30 infrastructure.validation (86 files)

Markdown + PDF + integrity CLIs underpinning Stage 04 diagnostics.

6.1.31 infrastructure/logrotate.d (0 files)

Operational templates for deployments (documentation-first; intentionally minimal Python footprint).


Documentation maturity: Coverage statements in Results pull from introspection—not hand-maintained denominators—so newly promoted modules automatically flow into manuscripts after generate_manuscript_metrics.py.

FAIR+RSE linkage: MCP-ready SKILL.md artefacts align with evaluator heuristics (executability + metadata richness) emphasized by FAIRsoft guidance (Garijo et al. 2024).


7 Security and Provenance

Research integrity requires more than reproducibility; it requires verifiable authorship. In an era of generative AI, automated scraping, and synthetic media, the ability to prove that a document was produced by a specific pipeline at a specific time is a critical defense against fabrication and misattribution. The W3C PROV data model (Moreau and Missier 2013) establishes a formal vocabulary for expressing provenance records—entities, activities, and agents connected by derivation, generation, and attribution relations. Digital watermarking, pioneered by Cox et al. (Cox et al. 1997) for multimedia integrity verification, provides the foundational signal-processing theory for embedding imperceptible provenance markers within artifacts. template/ implements a domain-specific provenance layer that embeds these relations directly into the PDF artifact itself, using four complementary steganographic and cryptographic mechanisms.

7.1 Threat Model

The steganography subsystem defends against three classes of threats:

  1. Unauthorized redistribution: A manuscript is scraped and republished without attribution. The steganographic watermark survives the redistribution and can be used to prove original authorship.
  2. Content tampering: Figures or text are modified after publication. The SHA-256 hash embedded in the PDF metadata detects any modification to the file contents.
  3. Provenance forgery: An attacker claims to have produced a document they did not author. The build timestamp, commit hash, and pipeline metadata embedded in the watermark provide a verifiable chain of custody.

7.2 Steganographic Layers

The system applies four complementary layers of provenance information:

7.2.1 Layer 1: PDF Metadata Injection

The inject_pdf_metadata function writes structured metadata into both the PDF Info dictionary and an XMP (Extensible Metadata Platform) packet:

7.2.2 Layer 2: Cryptographic Hashing

Before watermarking, SHA-256 (and SHA-512 when configured) digests of the original PDF are computed and stored in:

This enables post-hoc verification: anyone with the manifest can verify that the PDF has not been modified since rendering.

A semi-transparent overlay is merged onto every page of the hardened PDF:

STEG_ID:<document-id>|TITLE:<title>|HASHES:<hash-prefix>

The build timestamp and Git commit hash (short SHA) are recorded in the PDF metadata and the hash manifest rather than in the visible watermark text itself. Together these fields allow a verifier to reconstruct—from the hardened PDF alone—which version of the code, at which moment in time, produced the document.

7.2.4 Layer 4: QR and Barcode Injection

When barcodes are enabled, a bottom-of-page barcode strip (QR code + Code128 label) is merged onto every page, encoding the document identifier and page number (ID:<doc-id-prefix>|P:<page>). In overlay_mode: "qr", a tiled QR overlay covers the full page in place of the text watermark. Both are configurable via the steganography: block in manuscript/config.yaml (or the repo-level secure_config.yaml defaults).

7.3 The secure_run.sh Orchestrator

The steganographic pipeline is orchestrated by secure_run.sh, a Bash script that wraps the standard run.sh pipeline with post-processing steganography:

  1. Execute the standard YAML-declared pipeline (17 stages; default full 10) pipeline for the target project.
  2. The secure_run.sh script invokes SteganographyProcessor.
  3. Apply metadata injection, hashing, text overlay, and QR code injection.
  4. Save the secured PDF alongside the original.
  5. Generate a steganography report in JSON format.

The orchestrator processes either a single specified project or all discovered projects sequentially.

7.4 Tamper Detection

Verification is performed by comparing the stored SHA-256 hash against a freshly computed hash of the distributed PDF. Any modification—even a single bit flip—produces a hash mismatch. The alpha-channel overlay provides a secondary, visual verification channel that does not require access to the original hash.

7.5 Limitations

7.6 Relationship to Software Supply Chain Integrity

The steganographic provenance layer operates at the document level—it certifies the integrity of a specific PDF artifact. A complementary concern is build-level provenance: certifying that the pipeline itself was executed with verified source code and dependencies. Frameworks such as in-toto (Torres-Arias et al. 2019) and SLSA (Supply-chain Levels for Software Artifacts) address this concern by defining attestation chains from source commit through build steps to final artifact. The NTIA’s minimum elements for a Software Bill of Materials (SBOM) (National Telecommunications and Information Administration 2021) further standardize the enumeration of software components and dependencies—essential for establishing the provenance lineage of build environments. SLSA defines four graduated levels of build integrity:

SLSA Level Requirement template/ Status
1 Provenance document exists ✓ SHA-256 manifest + steganographic metadata
2 Version-controlled build scripts ✓ All scripts in git
3 Isolated build environment ~ Docker support exists but not enforced in CI
4 Hermetic, reproducible builds N – future work

Future versions of template/ may generate SLSA-compatible provenance attestations alongside the steganographic watermarks, creating a two-layer provenance model: in-toto attests that the build pipeline was executed with the claimed source code, while the steganographic layer attests that the PDF was produced by that pipeline at a specific time.

7.7 Relationship to FAIR and Formal Provenance Standards

The steganographic layer supports the FAIR for Research Software (FAIR4RS) principles (Barker et al. 2022) at the artifact level. PDFs carry embedded metadata (Findability) in standardized XMP format (Interoperability). The SHA-256 hash manifest enables persistent integrity verification (a prerequisite for Reusability). The Documentation Duality standard ensures that the software producing the artifact is inspectable and well-documented (satisfying FAIRsoft (Garijo et al. 2024) metadata and documentation indicators). Full PROV-compliant provenance traces—capturing the derivation chain from source data through analysis scripts to rendered PDF—are a natural extension and a primary target for future development.

Software Heritage (Di Cosmo et al. 2020) complements this picture at the source-code level: by archiving the template/ repository and assigning a reproducible SWHID (Software Hash Identifier) to each commit, Software Heritage makes the pipeline itself—not just its output—a citable, persistent digital artifact. A published SWHID alongside the PDF DOI creates a complete, two-artifact citation record: the paper’s content is versioned via DOI; the code that generated it is versioned via SWHID. This combination satisfies the Katz et al. (Katz et al. 2021) software citation principles’ requirement that software used in research be independently citable and permanently accessible.


8 Appendices

8.1 Appendix: Pipeline Stage Reference

8.1.1 Preparation and analysis

Stage name Script / method Primary inputs Outputs / artefacts Failure mode
Clean Output Directories _run_clean_outputs prior projects/<name>/output/, mirrored output/<name>/ targets emptied trees Blocking
Environment Setup scripts/pipeline/stage_00_setup.py toolchain probes scaffold dirs, env exports Blocking
Infrastructure Tests scripts/pipeline/stage_01_test.py --infra-only --infra-scope pipeline-smoke tests/infra_tests/ smoke-test results Required smoke tests must pass
Project Tests scripts/pipeline/stage_01_test.py --project-only projects/<name>/tests/ coverage artefacts Nonempty tests, own coverage, verifier, and runner
Project Analysis scripts/pipeline/stage_02_analysis.py thin scripts figures/, data/, reports Blocking

8.1.2 Search, provenance and rendering

Stage name Script / method Primary inputs Outputs / artefacts Failure mode
Connector Search scripts/pipeline/stage_08_connector_search.py manuscript/config.yaml output/data/connector_search/ opt-in (science tag); fresh policy-bound skip if unconfigured
Provenance Record scripts/pipeline/stage_09_provenance_record.py --stage "Connector Search" prior stage outputs .provenance/graph.json opt-in (provenance tag); fresh policy-bound skip if unconfigured
PDF Rendering scripts/pipeline/stage_03_render.py manuscript/, placeholders .pdf/.tex bundles Blocking
Output Validation scripts/pipeline/stage_04_validate.py render tree Markdown + PDF diagnostics JSON Enabled formats, bookends, and provenance block; prose diagnostics advisory

8.1.3 Review and delivery

Stage name Script / method Primary inputs Outputs / artefacts Failure mode
LLM Manuscript Review scripts/pipeline/stage_06_llm_review.py --reviews-only resolved manuscript artefacts textual reviews Fresh policy-bound skip; required live acceptance remains blocking
LLM Translations scripts/pipeline/stage_06_llm_review.py --translations-only abstract metadata multilingual snippets Fresh policy-bound skip; required live acceptance remains blocking
Copy Outputs scripts/pipeline/stage_05_copy.py validated tree mirrored output/<name>/… Blocking

8.1.4 Optional packaging and publication

Stage name Script / method Primary inputs Outputs / artefacts Failure mode
Ebook Generation scripts/pipeline/stage_11_ebook.py rendered combined markdown output/ebook/ (EPUB/MOBI/DOCX) opt-in (ebook tag); blocking when invoked
docxplus Export scripts/pipeline/stage_13_docxplus.py project source tree + metadata output/docxplus/ (.docx, .docxplus) opt-in (docxplus tag); blocking when invoked
Metadata Package scripts/pipeline/stage_12_metadata.py manuscript/config.yaml output/metadata/ (ONIX/JSON/OPF) opt-in (metadata tag); blocking when invoked
Executable Bundle scripts/runner/bundle_executable.py project tree + outputs container bundle manifest opt-in (bundle tag)
Archival Publication scripts/runner/archive_publication.py bundle + deliverables archival deposit manifest opt-in (archival tag)

scripts/pipeline/stage_07_executive_report.py provides separate multi-project executive reporting, not an additional single-project DAG node. execute_multi_project.py writes aggregate summaries after project execution; this does not imply automatic invocation of that separate Stage 07 script.

Opt-in stages also prevent completion when invoked and unsuccessful. An allowed skip requires a fresh policy-bound reason receipt; it is not a successful execution. The executable DAG and its contracts remain authoritative.


8.2 Appendix: Configuration Reference

paper:
  title: "Paper Title"
  subtitle: "Optional Subtitle"
  version: "1.0"
  date: "2026-03-19"

authors:
  - name: "Author Name"
    orcid: "0000-0000-0000-0000"
    email: "author@example.com"
    affiliation: "Institution"
    corresponding: true

publication:
  doi: "10.5281/zenodo.XXXXXX"
  version_doi: "10.5281/zenodo.XXXXXX"
  version_record: "https://zenodo.org/records/XXXXXX"
  github_repository: "owner/repository"
  repository_url: "https://github.com/owner/repository"
  published_artifacts:
    github_pages: "https://owner.github.io/repository/"
    software_heritage: "https://archive.softwareheritage.org/..."
    pypi: "https://test.pypi.org/project/..."
    osf: "https://osf.io/..."
    huggingface_hub: "https://huggingface.co/datasets/owner/repository"
    netlify: "https://..."
    ipfs_pinata: "https://gateway.pinata.cloud/ipfs/..."
  transmission_bookends:
    enabled: false
    max_prior_releases: 5
    show_steganography: true
  journal: "Target Journal"
  volume: "1"
  pages: "1-10"
  year: "2026"

keywords:
  - "keyword1"
  - "keyword2"

metadata:
  license: "Apache License 2.0"
  language: "en"

llm:
  reviews:
    enabled: true
    types: [executive_summary, quality_review]
  translations:
    enabled: false

testing:
  max_test_failures: 0
  max_infra_test_failures: 3
  max_project_test_failures: 0

8.3 Appendix: Repository Directory Structure

template/
|-- infrastructure/
|   |-- config/ docker/ documentation/ llm/
|   |-- orchestration/   # Thin Python entry equal to `./run.sh` backend
|   |-- prose/ reference/ rendering/ reporting/
|   |-- scientific/ search/ skills/ steganography/ validation/
|   |-- project/ core/
|   `-- logrotate.d/      # Operational rotation templates (no Python pkg)
|-- scripts/
|   |-- pipeline/          # stage_00_setup.py … stage_12_metadata.py (canonical, cited by pipeline.yaml)
|   `-- runner/            # execute_pipeline.py execute_multi_project.py bundle_executable.py archive_publication.py
|-- projects/                    # Typed program subfolders (`discover_projects`)
|   |-- templates/               # Public exemplars (git-tracked) — full roster:
|   |   |                        #   docs/_generated/active_projects.md (regenerated by scripts/docgen/active_projects.py)
|   |   `-- template_template/   # Present manuscript (`manuscript/` here)
|   |-- active/                  # Hot-seat rendered set (symlinked, private)
|   |-- working/                 # Non-rendered backburner (symlinked, private)
|   |-- published/               # Non-rendered published (symlinked, private)
|   |-- archive/                 # Non-rendered retired (symlinked, private)
|   `-- other/                   # Non-rendered misc (symlinked, private)
|-- docs/ (20 top-level areas, 378+ markdown files per live counter)
|-- tests/                       # Infra suites (947+ files)
|-- AGENTS.md / README.md / CLAUDE.md / pyproject.toml
|-- run.sh / secure_run.sh
`-- output/ …                    # Mirrors after copy stage

See docs/_generated/active_projects.md for regenerated slugs (uv run python scripts/docgen/active_projects.py).


8.4 Appendix: Exemplar Project Summary

The full public exemplar roster is: templates/template_active_inference, templates/template_advanced_literature_review, templates/template_autopoiesis, templates/template_autoresearch_project, templates/template_autoscientists, templates/template_code_project, templates/template_data_descriptor, templates/template_eda_notebook, templates/template_experiment_tree, templates/template_formal, templates/template_gold_refinement, templates/template_literature_meta_analysis, templates/template_madlib, templates/template_methods_paper, templates/template_newspaper, templates/template_pitch_deck, templates/template_pools_rules_tools, templates/template_prose_project, templates/template_redacted_report, templates/template_registered_report, templates/template_search_project, templates/template_sia, templates/template_storybook, templates/template_template, templates/template_textbook. The three rows below are a representative sample; a future exemplar_summary_table token in build_manuscript_metrics_dict would let this table cover every exemplar without hand-editing.

Project slug Purpose Highlights Tests Figures (Stage 02 hint)
template_code_project Optimization tutorial Convex demo figures, scripted tables 263 discovered tests; 90% declared gate Controlled matplotlib exports
template_prose_project Prose-heavy workflow Validates narrative-only repos 165 Lightweight / optional plots
template_autoresearch_project AutoResearch readiness Planner + validation CLI 318 Readiness reports from Stage 02

Meta manuscript location: introspective study lives in projects/templates/template_template/ beside the public exemplar set. Discovery now follows the typed projects/ layout—projects/templates/** and projects/active/** are discovered/rendered, while projects/working/**, projects/ongoing/**, and projects/archive/** remain non-rendered—see root CLAUDE.md for invocation patterns (resolve_project_root).


8.5 Appendix: Documentation Inventory

The repository maintains documentation at three levels:

Level Files Purpose
Repository root AGENTS.md, .agents/, CLAUDE.md, README.md, docs/RUN_GUIDE.md Global navigation and AI agent context
docs/ directory 378 files across 20 subdirectories User guides, API reference, troubleshooting
Per-directory AGENTS.md + README.md in documented directories Documentation Duality standard
Per-module (Tier 3) SKILL.md on skill-enabled modules and public exemplars Machine-parseable MCP-aligned skill descriptor
Infrastructure-level (PAI) PAI.md at infrastructure/ directory Personal AI Infrastructure integration contract

The docs/ subdirectories cover: core/ (essential docs), guides/ (progressive skill-level guides), architecture/ (system design), usage/ (content authoring), operational/ (build, config, logging, troubleshooting), reference/ (API, FAQ, glossary), modules/ (31 infrastructure modules), development/ (contributing, testing), maintenance/ (long-horizon maintenance: private projects, CI-local, regression, archival, bundle), best-practices/ (version control, migration), plans/ (strategic plans and decision records), rules/ (contributor norms), prompts/ (22 AI prompt templates), security/ (steganography, hashing), streams/ (livestream notes), and audit/ (review reports), plus the machine-generated _generated/ snippets and images/ assets.

Every count in this appendix is injected from live repository introspection rather than hand-maintained: 378 counts every Markdown file beneath docs/ recursively, 20 counts its first-level subdirectories, and 22 counts the workflow subdirectories that each carry a SKILL.md descriptor. This is the same discipline the manuscript argues for throughout—a hand-typed documentation total silently rots as the tree grows, whereas a token re-resolves on every render. A reader onboarding to the repository should start at docs/core/, follow the graduated docs/guides/ skill ladder, and consult the per-directory AGENTS.md/README.md pair nearest to whatever code they are editing; AI agents begin with root AGENTS.md and .agents/AGENTS.md, then read local contracts and available SKILL.md descriptors. The shared skill manifest supplies discovery; client-specific files remain compatibility references.


8.6 Appendix: Comparative Tool Matrix

Symbol key (applies to all cells): Y = full native support · ~ = partial or plugin-based · N = absent. See the comparative feature matrix in the Results section for a colour-coded heatmap rendering of this table.

Capability template/ Snakemake Nextflow CWL Quarto Jupyter Book R Markdown DVC Overleaf OpenAI Prism
Pipeline orchestration Y Y Y Y ~ N N Y N N
Manuscript rendering Y N N N Y Y Y N Y Y
Testing enforcement Y N N N N N N N N N
Coverage thresholds Y N N N N N N N N N
Cryptographic provenance Y N ~¹ N N N N ~² N N
Steganographic watermarking Y N N N N N N N N N
Multi-project management Y N N N N N N N ~ ~
AI-agent documentation Y N N N N N N N ~ ~
Agentic skill protocol (SKILL.md / MCP) Y N N N N N N N N N
Interactive TUI Y N N N N N N N N N
Zero-mock policy Y N N N N N N N N N
Container support ~ Y Y Y N N N N N N
Distributed execution N Y Y Y N N N ~³ N N
Multi-language (R/Julia) N Y N Y Y Y Y Y N N

¹ Nextflow lineage records operate at workflow scope rather than as a rendered-document citation graph. ² DVC provides content-addressed versioning for data artifacts via its object store. ³ DVC integrates with remote storage (S3, GCS, Azure) but does not natively orchestrate distributed compute. This matrix is a versioned manuscript snapshot, not a continuously updated product survey. A future refresh should re-check every external capability against current primary documentation before changing a cell.


Release: v2.0.0 · DOI 10.5281/zenodo.20419007 · SHA-256 pending… · pairing pending

Integrity QR strip
Integrity QR strip

Prior: v1.0.8 · 10.5281/zenodo.20932076 · b9bc5cf3…

Allaire, J. J., Charles Teague, Carlos Scheidegger, Yihui Xie, and Christophe Dervieux. 2024. Quarto. Released. https://doi.org/10.5281/zenodo.5960048.
Amstutz, Peter, Michael R. Crusoe, Nebojša Tijanić, et al. 2016. “Common Workflow Language, V1.0.” Specification, Common Workflow Language Working Group, ahead of print. https://doi.org/10.6084/m9.figshare.3115156.v2.
Anthropic. 2024. Introducing the Model Context Protocol. Anthropic Engineering Blog. https://www.anthropic.com/news/model-context-protocol.
Baker, Monya. 2016. “1,500 Scientists Lift the Lid on Reproducibility.” Nature 533 (7604): 452–54. https://doi.org/10.1038/533452a.
Barba, Lorena A. 2018. “Terminologies for Reproducible Research.” arXiv Preprint arXiv:1802.03311, ahead of print. https://doi.org/10.48550/arXiv.1802.03311.
Barker, Michelle, Neil P. Chue Hong, Daniel S. Katz, et al. 2022. “Introducing the FAIR Principles for Research Software.” Scientific Data 9 (1): 622. https://doi.org/10.1038/s41597-022-01710-x.
Boettiger, Carl. 2015. “An Introduction to Docker for Reproducible Research.” ACM SIGOPS Operating Systems Review 49 (1): 71–79. https://doi.org/10.1145/2723872.2723882.
Cohen, Jeremy, Daniel S. Katz, Michelle Barker, Neil P. Chue Hong, Robert Haines, and Caroline Jay. 2021. “The Four Pillars of Research Software Engineering.” IEEE Software 38 (1): 97–105. https://doi.org/10.1109/MS.2020.2973362.
Cox, Ingemar J., Joe Kilian, F. Thomson Leighton, and Talal Shamoon. 1997. “Secure Spread Spectrum Watermarking for Multimedia.” IEEE Transactions on Image Processing 6 (12): 1673–87. https://doi.org/10.1109/83.650120.
Di Cosmo, Roberto, Morane Gruenpeter, and Stefano Zacchiroli. 2020. “Referencing Source Code Artifacts: A Separate Concern in Software Citation.” Computing in Science & Engineering 22 (2): 33–43. https://doi.org/10.1109/MCSE.2019.2963148.
Di Tommaso, Paolo, Maria Chatzou, Evan W. Floden, Pablo Prieto Barja, Emilio Palumbo, and Cedric Notredame. 2017. “Nextflow Enables Reproducible Computational Workflows.” Nature Biotechnology 35 (4): 316–19. https://doi.org/10.1038/nbt.3820.
Freedman, Leonard P., Iain M. Cockburn, and Timothy S. Simcoe. 2015. “The Economics of Reproducibility in Preclinical Research.” PLOS Biology 13 (6): e1002165. https://doi.org/10.1371/journal.pbio.1002165.
Gamma, Erich, Richard Helm, Ralph Johnson, and John Vlissides. 1995. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley.
Garijo, Daniel, Luis García, Matthias Barker, Rubén Chaves, Oscar Corcho, et al. 2024. “FAIRsoft—a Practical Implementation of FAIR Principles for Research Software.” Bioinformatics 40 (8): btae464. https://doi.org/10.1093/bioinformatics/btae464.
Gentleman, Robert, and Duncan Temple Lang. 2007. “Statistical Analyses and Reproducible Research.” Journal of Computational and Graphical Statistics 16 (1): 1–23. https://doi.org/10.1198/106186007X178663.
Goble, Carole, Sarah Cohen-Boulakia, Stian Soiland-Reyes, et al. 2020. “FAIR Computational Workflows.” Data Intelligence 2 (1–2): 108–21. https://doi.org/10.1162/dint_a_00033.
Google. 2025. Agent2Agent (A2A) Protocol Specification. Open Protocol Specification. https://a2a-protocol.org/latest/specification/.
Gruenpeter, Morane, Daniel S. Katz, Anna-Lena Lamprecht, et al. 2021. “Defining Research Software: A Controversial Discussion.” SSRN Preprint. https://doi.org/10.2139/ssrn.3884311.
Honeyman, Tom, Michelle Barker, Daniel S. Katz, and Neil P. Chue Hong. 2024. The FAIR Principles for Research Software: Three Years on. Research Data Alliance Virtual Plenary 24. https://www.rd-alliance.org/wp-content/uploads/2025/04/RDA_VP24_FAIR4RS_Three_Years_On_Reformatted.pdf.
Iterative, Inc. 2024. DVC: Data Version Control. Https://dvc.org.
Jimenez, Carlos E., John Yang, Alexander Wettig, et al. 2024. “SWE-bench: Can Language Models Resolve Real-World GitHub Issues?” Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2310.06770.
Katz, Daniel S., Neil P. Chue Hong, Tim Clark, et al. 2021. “Recognizing the Value of Software: A Software Citation Guide.” F1000Research 9: 1257. https://doi.org/10.12688/f1000research.26932.2.
Kluyver, Thomas, Benjamin Ragan-Kelley, Fernando Pérez, et al. 2016. “Jupyter Notebooks — a Publishing Format for Reproducible Computational Workflows.” Positioning and Power in Academic Publishing: Players, Agents and Agendas, 87–90. https://doi.org/10.3233/978-1-61499-649-1-87.
Knuth, Donald E. 1984. “Literate Programming.” The Computer Journal 27 (2): 97–111. https://doi.org/10.1093/comjnl/27.2.97.
Köster, Johannes, and Sven Rahmann. 2012. “Snakemake—a Scalable Bioinformatics Workflow Engine.” Bioinformatics 28 (19): 2520–22. https://doi.org/10.1093/bioinformatics/bts480.
Lamport, Leslie. 1994. LaTeX: A Document Preparation System. 2nd ed. Addison-Wesley.
Lamprecht, Anna-Lena, Leyla Garcia, Mateusz Kuzak, et al. 2020. “Towards FAIR Principles for Research Software.” Data Science 3 (1): 37–59. https://doi.org/10.3233/DS-190026.
Lau, Sam, and Philip J. Guo. 2025. “The Design Space of LLM-Based AI Coding Assistants: An Analysis of 90 Systems.” IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). https://pg.ucsd.edu/publications/ai-coding-assistants-design-space_VLHCC-2025.pdf.
Lu, Chris, Jeff Clune, Jakob Foerster, et al. 2024. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv preprint arXiv:2408.06292. https://doi.org/10.48550/arXiv.2408.06292.
Mädje, Laurenz, and Martin Haug. 2023. Typst: A New Markup-Based Typesetting System. Released. https://typst.app.
Martin, Robert C. 2008. Clean Code: A Handbook of Agile Software Craftsmanship. Prentice Hall.
Meszaros, Gerard. 2007. xUnit Test Patterns: Refactoring Test Code. Addison-Wesley.
Moreau, Luc, and Paolo Missier. 2013. PROV-DM: The PROV Data Model. W3C Recommendation. World Wide Web Consortium. https://www.w3.org/TR/prov-dm/.
National Telecommunications and Information Administration. 2021. The Minimum Elements for a Software Bill of Materials (SBOM). U.S. Department of Commerce. https://www.ntia.gov/sites/default/files/publications/sbom_minimum_elements_report_0.pdf.
Nosek, Brian A., Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor. 2018. “The Preregistration Revolution.” Proceedings of the National Academy of Sciences 115 (11): 2600–2606. https://doi.org/10.1073/pnas.1708274114.
Nüst, Daniel, Markus Konkol, Edzer Pebesma, et al. 2017. “Opening the Publication Process with Executable Research Compendia.” D-Lib Magazine 23 (1/2). https://doi.org/10.1045/january2017-nuest.
Open Source Security Foundation. 2023. SLSA: Supply-Chain Levels for Software Artifacts, V1.0. Https://slsa.dev/spec/v1.0/.
OpenAI. 2026. Prism: AI-Assisted Research Writing Environment. Https://openai.com/prism.
Overleaf (Digital Science). 2025. Overleaf: Collaborative Cloud LaTeX Editor. Https://www.overleaf.com.
Peng, Roger D. 2011. “Reproducible Research in Computational Science.” Science 334 (6060): 1226–27. https://doi.org/10.1126/science.1213847.
Piccolo, Stephen R., and Michael B. Frampton. 2016. “Tools and Techniques for Computational Reproducibility.” GigaScience 5 (1): 30. https://doi.org/10.1186/s13742-016-0135-4.
Pimentel, João Felipe, Leonardo Murta, Vanessa Braganholo, and Juliana Freire. 2019. “A Large-Scale Study about Quality and Reproducibility of Jupyter Notebooks.” Proceedings of the 35th IEEE/ACM International Conference on Software Maintenance and Evolution (ICSME), 507–17. https://doi.org/10.1109/ICSME.2019.00080.
Ram, Karthik. 2013. “Git Can Facilitate Greater Reproducibility and Increased Transparency in Science.” Source Code for Biology and Medicine 8 (1): 7. https://doi.org/10.1186/1751-0473-8-7.
Research Software Alliance. 2024. Developing Actionable Guidelines for FAIR Research Software. ReSA Task Force. https://doi.org/10.5281/zenodo.18877929.
Sandve, Geir Kjetil, Anton Nekrutenko, James Taylor, and Eivind Hovig. 2013. “Ten Simple Rules for Reproducible Computational Research.” PLOS Computational Biology 9 (10): e1003285. https://doi.org/10.1371/journal.pcbi.1003285.
Schick, Timo, Jane Dwivedi-Yu, Roberto Dessì, et al. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv preprint arXiv:2302.04761. https://doi.org/10.48550/arXiv.2302.04761.
Schulte, Eric, Dan Davison, Thomas Dye, and Carsten Dominik. 2012. “A Multi-Language Computing Environment for Literate Programming and Reproducible Research.” Journal of Statistical Software 46 (3): 1–24. https://doi.org/10.18637/jss.v046.i03.
Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. 2011. “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.” Psychological Science 22 (11): 1359–66. https://doi.org/10.1177/0956797611417632.
Stodden, Victoria, Marcia McNutt, David H. Bailey, et al. 2016. “Enhancing Reproducibility for Computational Methods.” Science 354 (6317): 1240–41. https://doi.org/10.1126/science.aah6168.
Torres-Arias, Santiago, Hammad Afzali, Trishank Karthik Kuppusamy, Reza Curtmola, and Justin Cappos. 2019. “In-Toto: Providing Farm-to-Table Guarantees for Bits and Bytes.” 28th USENIX Security Symposium, 1393–410. https://www.usenix.org/conference/usenixsecurity19/presentation/torres-arias.
Wang, Guanzhi, Yuqi Xie, Yunfan Jiang, et al. 2023. Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv preprint arXiv:2305.16291. https://doi.org/10.48550/arXiv.2305.16291.
Wang, Lei, Chen Ma, Xueyang Feng, et al. 2024. “A Survey on Large Language Model Based Autonomous Agents.” Frontiers of Computer Science 18 (6): 186345. https://doi.org/10.1007/s11704-024-40231-1.
Wang, Xingyao, Boxuan Li, Yufan Song, et al. 2024. OpenDevin: An Open Platform for AI Software Developers as Generalist Agents. arXiv preprint arXiv:2407.16741; published as OpenHands at ICLR 2025. https://doi.org/10.48550/arXiv.2407.16741.
Wilkinson, Mark D., Michel Dumontier, IJsbrand Jan Aalbersberg, et al. 2016. “The FAIR Guiding Principles for Scientific Data Management and Stewardship.” Scientific Data 3: 160018. https://doi.org/10.1038/sdata.2016.18.
Wilson, Greg, Jennifer Bryan, Karen Cranston, Justin Kitzes, Lex Nederbragt, and Tracy K. Teal. 2017. “Good Enough Practices in Scientific Computing.” PLOS Computational Biology 13 (6): e1005510. https://doi.org/10.1371/journal.pcbi.1005510.
Xie, Yihui, J. J. Allaire, and Garrett Grolemund. 2018. R Markdown: The Definitive Guide. Chapman; Hall/CRC. https://doi.org/10.1201/9781138359444.
Yao, Shunyu, Jeffrey Zhao, Dian Yu, et al. 2023. “ReAct: Synergizing Reasoning and Acting in Language Models.” Proceedings of the Eleventh International Conference on Learning Representations (ICLR 2023). https://arxiv.org/abs/2210.03629.