Architecture and Ergonomics from Configuration through Publication
State: published
Pairing: complete (DOI, GitHub, SHA-256, Zenodo URL)
| Field | Value |
|---|---|
| Title | A template/ approach to Reproducible Generative Research |
| Version | 1.0.9 |
| Concept DOI | 10.5281/zenodo.20419007 |
| Version DOI | 10.5281/zenodo.20976048 |
| GitHub | https://github.com/docxology/template_template/releases/tag/v1.0.9 |
| Zenodo | https://zenodo.org/records/20419007 |
| SHA-256 | 535bd80943d0ae9f… |
| SHA-512 | pending |
../data/transmission_manifest.json.
Structured manifest:
../data/transmission_manifest.json
Stego: off | overlays text | barcodes on | XMP on |
manifest on → ./secure_run.sh
The reproducibility crisis in computational research is fundamentally
structural: research artifacts are scattered across disconnected
tools—LaTeX editors, Jupyter notebooks, ad-hoc shell scripts—with no
enforced mechanism to keep code, data, and manuscript synchronized.
Studies have shown that most published findings are false positives,
replication rates in psychology hover around 36%, and only 24% of 1.4
million Jupyter notebooks can be successfully re-executed. Existing
tools address fragments of this problem: workflow managers (Snakemake,
Nextflow, CWL) orchestrate computation; literate programming systems
(Quarto, Jupyter Book, R Markdown, Overleaf, OpenAI Prism) render
documents; data versioning tools (DVC) track artifacts—but none enforces
cross-cutting quality standards as architectural invariants.
template/ applies the principle of Infrastructure as Code
to the research lifecycle, making the manuscript, test suite, and
provenance chain version-controlled, deterministically buildable, and
independently verifiable. It is built on a Two-Layer Architecture that
separates 23 infrastructure subdirectories (20 importable Python
packages, ~604 modules, validated by ~7,780 tests) from self-contained
project workspaces, connected by a YAML-declared pipeline (12 stages;
default full 10)-based build pipeline progressing from environment
sanitization through test execution (with a Zero-Mock testing policy
enforcing 90% project-level and 60% infrastructure-level coverage via
real filesystem operations and subprocess invocations), analysis script
invocation, Pandoc/XeLaTeX rendering, SHA-256 cryptographic hashing with
steganographic watermarking, structural PDF validation, and LLM-assisted
review. A Documentation Duality standard equips every directory with
both human-readable README.md and machine-readable
AGENTS.md files, while each infrastructure module
additionally carries a SKILL.md—a structured skill
descriptor aligned with the Model Context Protocol—enabling AI agents to
locate and invoke module capabilities without hallucinating API
signatures.
Scalability is demonstrated across the generated public exemplar
roster (templates/template_active_inference,
templates/template_autoresearch_project,
templates/template_autoscientists,
templates/template_code_project,
templates/template_gold_refinement,
templates/template_literature_meta_analysis,
templates/template_madlib,
templates/template_newspaper,
templates/template_prose_project,
templates/template_sia,
templates/template_template,
templates/template_textbook), with representative
heterogeneous cases under projects/templates/: optimization
(template_code_project, 231 tests), prose
(template_prose_project, 120 tests), and AutoResearch
readiness (template_autoresearch_project, 296 tests). These
guarantee control-positive layouts for code-centric, prose-centric, and
retrieval-centric workflows at 90%+ project coverage alongside 60%+
infrastructure gates. All three share identical pipeline stages without
cross-project coupling. This manuscript adds a complementary reflexive
artifact: authored from
projects/templates/template_template (127 tests) as a
public exemplar in the same discovered/rendered tree, using the same
analysis and render path and injecting counters from repository
introspection. The fact that these words, metrics, and figures were
generated by the pipeline they describe demonstrates self-documenting
capacity: rendered through the DAG, validated without mocks, optionally
watermarked. A comparative analysis against nine peer tools across
fourteen dimensions positions template/ as integrating
fourteen distinctive enforcement capabilities—testing thresholds,
cryptographic provenance, steganographic watermarking, multi-project
management, MCP-aligned skill descriptors, Zero-Mock policy,
orchestration through publication—in one repository. Code is released
under the Apache License 2.0 at
github.com/docxology/template; the work remains
open-ended.
Scientific research operates through a layered ecology of tools, documents, and practices—each shaping what can be known, communicated, and verified. When these layers are fragmented, the artifacts of research (manuscripts, data, code, figures) drift out of alignment with one another, creating gaps that are structural rather than incidental. The “reproducibility crisis” is one symptom of this deeper misalignment: a 2016 Nature survey of 1,576 researchers found that 70% had tried and failed to reproduce another scientist’s experiments, and more than half had failed to reproduce their own [baker2016reproducibility]. Freedman et al. estimate that the biomedical industry alone loses $28 billion annually to irreproducible preclinical research [freedman2015economics]. But reproducibility is only the most visible face of a broader problem. Research software engineering, epistemic integrity, and the coordination between human researchers and AI collaborators all depend on the same underlying question: whether the tools that produce research artifacts can themselves be made transparent, testable, and self-documenting. Nosek et al. [nosek2018preregistration] have argued that the preregistration revolution—requiring researchers to commit to analytical plans before data collection—is a necessary structural reform; we extend this logic to the entire research build pipeline.
One root cause is fragmentation—of attention (in the mind) as well as in the cyberphysical niche (documents, versions, and reproducibility artefacts). A typical research project scatters its artifacts across disconnected tools: LaTeX editors [lamport1994latex] for prose, Jupyter notebooks for analysis, ad-hoc shell scripts for figure generation, and manual copy-paste for integrating results into manuscripts. Each boundary between tools is a potential locus of desynchronization. The version of the figure embedded in the PDF may not match the version of the code that ostensibly generated it. The test suite, if it exists at all, likely tests the code in isolation from the rendering pipeline. Pimentel et al. [pimentel2019jupyter] analyzed 1.4 million Jupyter notebooks from GitHub and found that only 24% could be successfully re-executed, with 36% producing different results—quantifying the reproducibility cost of notebook-based workflows. Peng [peng2011reproducible] argues that reproducibility in computational science requires, at minimum, that the data and code underlying a published result be available for independent verification—yet the tools for enforcing this standard remain ad hoc. Indeed, even the terminology is fractured: Barba [barba2018terminologies] documents how “reproducibility,” “replicability,” and “repeatability” carry conflicting definitions across disciplines, undermining cross-field standards.
Gentleman and Temple Lang [gentleman2007research] introduced the concept of a research compendium—a single unit of scholarly communication bundling code, data, and narrative. This vision has driven two decades of tooling, which can be grouped into four categories: workflow managers, literate programming systems, containerization approaches, and best-practice frameworks.
Snakemake [koster2012snakemake] uses a rule-based, Python-derived DSL to specify computational workflows as directed acyclic graphs of file-producing steps. It supports containerized execution via Conda and Singularity environments. Snakemake 9.x (2024–2025) introduced a plugin architecture for extended execution backends and storage providers, yet its scope remains computational pipeline orchestration—it does not integrate manuscript rendering, testing enforcement, or provenance watermarking.
Nextflow [ditommaso2017nextflow] employs a dataflow programming paradigm with native support for container-based execution across heterogeneous computing environments (local, SLURM, AWS). Like Snakemake, Nextflow excels at bioinformatics pipeline parallelism but does not address manuscript production, document integrity, or the testing–publication coupling that characterizes research reproducibility.
CWL (Common Workflow Language) [amstutz2016cwl] provides a portable, YAML-based standard for describing computational workflows and their dependencies. Its strength lies in interoperability across execution engines (cwltool, Toil, Arvados), but it requires external tooling for manuscript generation and offers no built-in testing or provenance framework.
Knuth’s literate programming [knuth1984literate] established the principle that programs should be authored as documents intended for human comprehension. Schulte et al. [schulte2012multilanguage] extended this to multi-language computing environments (Org-mode), demonstrating that literate programming could span languages and output formats.
Quarto [allaire2024quarto] extends the R Markdown tradition to support Python, Julia, and Observable, rendering to PDF, HTML, and Word. Quarto integrates code execution with document rendering, achieving a modern form of literate programming, but it does not enforce testing thresholds, manage multi-project repositories, or provide cryptographic provenance.
Jupyter Book [kluyver2016jupyter] builds on Jupyter notebooks to produce publication-quality documents via Sphinx. While powerful for interactive exploration, Jupyter’s notebook format introduces execution-order fragility [pimentel2019jupyter] and does not naturally support the separation of logic from orchestration that characterizes maintainable research software.
R Markdown [xie2018dynamic] pioneered knitr-based
dynamic documents that weave code and prose. Its ecosystem is rich but
R-centric, and it lacks the multi-project management, infrastructure
testing, and provenance embedding that characterize
template/.
Typst [madje2023typst] is an emerging markup-based typesetting system with incremental compilation and a programmable scripting layer. While Typst offers faster compilation than LaTeX and a more modern authoring experience, it does not integrate testing, provenance, or multi-project pipeline management.
Docker [boettiger2015docker] addresses reproducibility at the environment level—packaging operating system, libraries, and code into portable containers. While Docker solves the “works on my machine” problem, containerization is complementary to, not a replacement for, the architectural concerns addressed here: Docker does not enforce testing, embed provenance, or manage multi-project manuscript workflows.
Wilson et al. [wilson2017good] define “good enough” practices for scientific computing, emphasizing version control, testing, and documentation. Sandve et al. [sandve2013ten] propose ten rules for reproducible computational research. Piccolo and Frampton [piccolo2016tools] systematically survey tools for computational reproducibility, finding that environment isolation, workflow automation, and documentation generation address complementary but non-overlapping reproducibility concerns—yet no single tool unifies all three. Stodden et al. [stodden2016enhancing] advocate for enhanced computational method transparency. The FAIR principles [wilkinson2016fair]—Findable, Accessible, Interoperable, Reusable—establish a standard for data stewardship that has been widely adopted by funding agencies and journals. Lamprecht et al. [lamprecht2020towards] formalize “Towards FAIR Principles for Research Software,” providing the conceptual scaffolding that Barker et al. [barker2022fair4rs] would soon operationalize as the FAIR4RS initiative—recognizing that software has execution, composability, and dependency-management requirements that data-centric FAIR does not address. Cohen et al. [cohen2021four] characterize the four pillars of research software engineering (software sustainability, software quality, community building, and policy advocacy), situating formal testing and provenance practices within a broader RSE governance framework. Garijo et al. [garijo2024fairsoft] operationalize FAIR4RS through the FAIRsoft evaluator, an automated assessment framework that scores research software against 17+ quality indicators including executability, metadata richness, and documentation completeness. Goble et al. [goble2020fair] extend FAIR to computational workflows specifically, arguing that workflow provenance requires first-class treatment in scientific computing infrastructure. Nüst et al. [nust2017containerization] introduce the executable research compendium (ERC), extending Gentleman and Temple Lang’s compendium concept with containerized, interactive reproduction environments. The W3C PROV data model [moreau2013provdm] provides a formal vocabulary for expressing provenance records, while in-toto [torresarias2019intoto] provides a framework for end-to-end software supply chain integrity verification, and SLSA [openssf2023slsa] (Supply-chain Levels for Software Artifacts) extends this to graduated, attestation-based supply-chain security levels for build pipelines. These frameworks articulate what reproducible research requires but do not provide an integrated how—they lack the tooling, enforcement mechanisms, and architectural patterns that translate standards into practice.
Despite advances in FAIR4RS principles [barker2022fair4rs; lamprecht2020towards], automated FAIR software assessment [garijo2024fairsoft], FAIR computational workflow standards [goble2020fair], supply-chain attestation frameworks [torresarias2019intoto; openssf2023slsa], and the preregistration revolution [nosek2018preregistration], no existing system integrates six cross-cutting concerns into a single enforced pipeline: (1) end-to-end pipeline orchestration with testing enforcement, (2) multi-format manuscript rendering, (3) cryptographic provenance embedding, (4) multi-project repository management, (5) FAIR-aligned software stewardship, and (6) AI-agent collaboration via structured documentation. Each existing framework addresses a subset; none provides the unified enforcement mechanism. The detailed tool-by-tool comparison is developed in the Comparison to Existing Tools section; the summary table below captures the gap landscape.
The six concerns identified above map onto existing tool categories as follows:
| Gap | Partially Addressed By | Not Addressed By |
|---|---|---|
| Pipeline orchestration | Snakemake 9.x, Nextflow 25.x, CWL 1.2 | Quarto, Jupyter Book, R Markdown, Typst, Overleaf, Prism |
| Manuscript rendering | Quarto 1.x, Jupyter Book 2.x, R Markdown, Typst, Overleaf (2025), Prism | Snakemake, Nextflow, CWL, DVC |
| Testing enforcement | — | All existing tools |
| Cryptographic provenance | SLSA (build-level attestation only) | All research-focused tools |
| Multi-project management | — | All existing tools |
| AI-agent documentation | Overleaf (partial co-author AI), Prism (partial context reasoning) | All pipeline/workflow tools |
| Agentic skill protocol (MCP-aligned) | — | All existing tools |
No existing system addresses all six concerns within a single enforced pipeline.
template/: An
Integrated Solutiontemplate/ was conceived as a structural antidote to this
fragmentation. Rather than adding reproducibility as an afterthought—a
Docker container wrapping an already-disjointed workflow
[boettiger2015docker]—the template enforces integrity at the
architectural level. It realizes Gentleman and Temple Lang’s research
compendium vision [gentleman2007research] at repository scale, bundling
code, data, tests, manuscripts, and provenance into a single,
pipeline-enforced system with version-controlled infrastructure
[ram2013git]. It stands on four primary pillars:
Ergonomic Modularity: A Two-Layer Architecture cleanly separates globally shared infrastructure (logging, rendering, validation, steganography) from project-specific logic (manuscripts, scripts, data). 23 infrastructure subdirectories (20 importable packages) comprising ~604 Python modules provide reusable services; projects consume them without modification.
Execution Integrity: A Zero-Mock testing policy where pipeline advancement is contingent on test passage. Infrastructure tests must achieve 60% coverage; project tests must achieve 90%. All tests use real filesystem operations, real subprocess calls, and real network connections—no mock objects, no fake services, no synthetic test doubles. ~7,780 infrastructure tests and 2902+ project tests enforce this standard.
Automated Provenance: Steganographic watermarking and cryptographic hashing are integrated directly into the rendering pipeline. Every generated PDF carries a SHA-256 fingerprint, an alpha-channel text overlay encoding the build timestamp and commit hash, and optionally a QR code linking to the repository. Provenance is not asserted by policy; it is enforced by architecture.
AI-Agent Collaboration and Skill-Based Agentic
Operations: A three-tier documentation architecture enables AI
agents to operate at every level of the system. At the system level,
CLAUDE.md asserts global architectural constraints. At the
structural level, AGENTS.md files at every directory expose
local API surfaces, file inventories, and integration contracts. At the
module level, SKILL.md files—written to a discoverable
YAML+Markdown schema aligned with the Model Context Protocol
[anthropic2024mcp]—define each infrastructure module as a reusable,
self-describing tool. An agent invoking
infrastructure.rendering does not need to read source code:
it reads the rendering/SKILL.md, which declares the
module’s name, description, key imports, and example invocations in a
machine-parseable YAML frontmatter block. This architecture is the
practical realization of the skill-library paradigm established in the
agent literature: Yao et al.’s ReAct framework [yao2023react]
demonstrated that interleaving reasoning traces with tool invocations
dramatically improves LLM reliability; Schick et al.’s Toolformer
[schick2023toolformer] showed that self-supervised tool use can be
bootstrapped from natural language; Wang et al.’s Voyager
[wang2023voyager] proved that growing skill libraries enable open-ended
autonomous exploration in complex environments. template/
instantiates this vision in the domain of scientific research
infrastructure: each SKILL.md is a Voyager-style skill,
each pipeline stage is a ReAct action, and the full infrastructure layer
constitutes a composable, protocol-aligned skill library for scientific
computation.
This paper is itself a product of the template it describes. The
metrics populating its tables were computed by the introspection module
documented in the Methods; the
figures were rendered by the visualization code validated by the test
suite described in Quality
Assurance; the PDF carrying these words was assembled by the same
YAML-declared pipeline whose architecture is the subject of the Results. This self-productive
loop—where the system that is described is also the system that produces
the description—is not incidental but structural, a concrete
demonstration that template/ can sustain the full lifecycle
from source code to published artifact within a single,
version-controlled, pipeline-enforced repository. Our contributions
are:
infrastructure/core/pipeline/pipeline.yaml: default full
runs use 10 stages; --core-only runs 8; opt-in bundle and
archival stages via --tags.template/ against
nine peer tools—Snakemake, Nextflow, CWL, Quarto, Jupyter Book, R
Markdown, DVC, Overleaf, OpenAI Prism—across fourteen feature
dimensions, demonstrating that template/ uniquely bundles
the fourteen enforcement capabilities enumerated in §Results.projects/templates/
(templates/template_active_inference,
templates/template_autoresearch_project,
templates/template_autoscientists,
templates/template_code_project,
templates/template_gold_refinement,
templates/template_literature_meta_analysis,
templates/template_madlib,
templates/template_newspaper,
templates/template_prose_project,
templates/template_sia,
templates/template_template,
templates/template_textbook)—including this meta manuscript
from projects/templates/template_template, which exercises
introspection-derived metrics.github.com/docxology/template.The Methods describe the Two-Layer Architecture, Thin Orchestrator pattern, pipeline stages, and AI collaboration model. Results present quantitative metrics from multi-project execution, coverage analysis, and steganographic benchmarks. The Discussion addresses the Zero-Mock tradeoff, scalability implications, a detailed tool comparison, and future directions. The Infrastructure Module Reference provides detailed documentation for all 23 subdirectories. Security and Provenance describes the steganographic and cryptographic integrity layer. The Appendices provide pipeline, configuration, and comparative references.
The template/ architecture is deliberately bifurcated
into a globally shared infrastructure/ layer and
project-specific projects/ silos. This section describes
the four core design patterns, the YAML-declared pipeline (12 stages;
default full 10) pipeline that operationalizes them, and the AI
collaboration model that distinguishes this system from conventional
research templates.
The repository is organized into two strictly separated layers:
Infrastructure Layer (infrastructure/):
23 infrastructure subdirectories—20 of them independently-importable
Python packages—comprising ~604 modules and providing reusable services.
Each importable package has its own __init__.py,
AGENTS.md, and README.md, and exports a
well-defined public API (the remaining subdirectories,
e.g. config/, hold shared configuration). The
infrastructure layer knows nothing about any specific project—it
provides generic capabilities (logging, rendering, validation,
steganography) that any project may consume.
Project Layer (projects/):
Self-contained research workspaces. Each project directory contains:
| Directory | Purpose |
|---|---|
manuscript/ |
Markdown chapters and config.yaml |
scripts/ |
Thin orchestrator scripts (Stage 02) |
src/ |
Project-specific Python modules |
tests/ |
Project-specific test suite |
data/ |
Input datasets and generated data |
output/ |
Pipeline artifacts: PDF, figures, reports, logs |
docs/ |
Project-specific architecture documentation |
The two layers communicate exclusively through Python imports and filesystem paths. No project modifies infrastructure code; no infrastructure module references a specific project by name (except via runtime project discovery).
Projects are designed to be completely self-contained. Adding a new
project requires no changes to the infrastructure layer, no
modifications to pyproject.toml, and no updates to the
pipeline orchestrator. A project is automatically discovered if and only
if it satisfies two conditions:
projects/.manuscript/config.yaml.This paradigm enables horizontal scaling: N researchers can maintain
N independent projects within a single repository, sharing
infrastructure without coupling. Each project declares its own testing
tolerances, manuscript metadata, LLM review preferences, and rendering
configuration in its config.yaml. The system currently
hosts its public canonical exemplars under
projects/templates/
(templates/template_active_inference,
templates/template_autoresearch_project,
templates/template_autoscientists,
templates/template_code_project,
templates/template_gold_refinement,
templates/template_literature_meta_analysis,
templates/template_madlib,
templates/template_newspaper,
templates/template_prose_project,
templates/template_sia,
templates/template_template,
templates/template_textbook), including this
meta-manuscript at
projects/templates/template_template/.
All scripts in scripts/ (both infrastructure-level and
project-level) follow the Thin Orchestrator pattern
[gamma1995design]:
src/ modules and
wire them to infrastructure services.config.yaml, not by hardcoded values.infrastructure.core.logging.utils.get_logger.This pattern ensures that all testable logic lives in
src/ where it is subject to the Zero-Mock testing policy,
while scripts remain thin enough to be audited by visual inspection. The
separation draws on the Mediator pattern from Gamma et
al. [gamma1995design], where scripts mediate between infrastructure
services and project-specific code without implementing any logic of
their own.
To make this concrete, the following contrasts the anti-pattern with the correct pattern:
# ANTI-PATTERN: domain logic embedded in script
def calculate_average(data): # ← never put computation here
return sum(data) / len(data)
result = calculate_average([1, 2, 3])
# CORRECT PATTERN: script imports from src/ and only wires
from projects.my_project.src.statistics import calculate_average
result = calculate_average([1, 2, 3]) # ← scripts wire, never computeThe critical property is that calculate_average in the
correct pattern lives in a testable src/ module, is covered
by the Zero-Mock test suite, and can be independently imported, tested,
and reused—whereas the anti-pattern buries logic in a script that is
invisible to coverage tools.
pipeline.yamlSingle-project pipelines read
infrastructure/core/pipeline/pipeline.yaml.
scripts/execute_pipeline.py expands the declarative DAG,
applies tag filters (--core-only skips llm
stages), checkpoints between nodes, then dispatches numbered scripts
(scripts/NN_*.py) or builtin methods
(_run_clean_outputs).
The default YAML graph contains ten named stages (plus telemetry configuration metadata):
projects/<name>/output/ + delivered
output/<name>/ paths so stale PDFs cannot satisfy
validation.00_setup_environment.py) — Python/uv probing, toolchain
discovery, scaffolding directories, PYTHONPATH wiring.01_run_tests.py --infra-only) — tests/ suite
with infra coverage thresholds (≥60 %).01_run_tests.py --project-only) — per-project suites with
≥90 % coverage mandate.02_run_analysis.py)
— lexicographically ordered
projects/<name>/scripts/*.py, each a thin
orchestrator (src/ does real work).03_render_pdf.py) —
Pandoc → XeLaTeX loop, bibliography assembly, injected variables from
Stage 02 artefacts.04_validate_output.py) — PDF structure, manifests,
Markdown hygiene.06_llm_review.py --reviews-only; tags: llm) —
executive + quality critiques via local Ollama;
allow_skip: true.06_llm_review.py --translations-only; tags
llm, same dependency edges) — multilingual abstract
expansion.05_copy_outputs.py) —
reproducible snapshots into canonical
output/<project>/.Two LLM nodes intentionally share one script module with orthogonal CLI switches; both depend only on validation so they can parallelize logically while remaining optional.
Executive reporting
(scripts/07_generate_executive_report.py) is
not a YAML node inside the single-project executor.
--all-projects / execute_multi_project.py
invokes it once after iterating projects, consolidating cross-project
KPIs dashboards.
Topological order therefore differs slightly from lexical script
numbering (e.g., copy executes after validation even though script
05 precedes 06 lexically).
Infrastructure vs project tests. Splitting pytest
invocations isolates flaky infra regressions
(MAX_TEST_FAILURES knobs) from zero-tolerance gates on
domain code (max_project_test_failures default 0 declared
in YAML front-matter/testing blocks).
Stage 02 illustration. The analysis stage is
deliberately concrete rather than a hypothetical diagram factory: each
canonical project ships real behaviour at this node.
template_autoresearch_project runs readiness validation;
the archived template_search_project merges remote
literature JSON, generates scripted figures
(y_generate_search_figures.py), and writes manifests;
template_code_project emits optimization plots; and
template_prose_project triggers structural validation
scaffolding. The pipeline shape is identical across all four—only the
Stage 02 payload differs—which is exactly what lets one orchestrator
serve heterogeneous research domains.
run.shThin wrapper invoking
python -m infrastructure.orchestration. Offers:
234 shorthand),a–d presets),scripts/README.md.Selecting d alone after a passing
multi-project run exits immediately once summaries print—avoiding
repetitive menu redraw.
secure_run.shExecutes Python secure path: standard pipeline artefact
reproduction then invokes
run_secure_pipeline for steganographic PDF hardening
(infrastructure.steganography). Original PDFs stay
immutable; hardened companions carry QR overlays plus hash manifests
sidecars.
Every directory at every level of the repository hierarchy contains two documentation files:
README.md: Human-readable overview,
quick-start instructions, and directory structure.AGENTS.md: Machine-readable technical
specification optimized for AI coding assistants. Contains API tables,
dependency graphs, implementation patterns, and architectural
constraints.This Documentation Duality standard serves two purposes. First, it
ensures that both human researchers and AI agents can navigate the
codebase efficiently—AGENTS.md files provide the structured
context that language models need to make informed code modifications
without hallucinating API signatures or violating architectural
invariants. Second, it creates a self-documenting feedback loop: as AI
agents modify the codebase, they update the corresponding
AGENTS.md files, keeping documentation synchronized with
implementation. Lau and Guo’s survey of 90 AI coding assistant systems
[lau2025aicoding] identifies contextual code understanding as a primary
bottleneck; the Documentation Duality standard addresses this by
providing pre-structured context at every directory level.
The template additionally includes CLAUDE.md at the
repository root, providing system-level instructions for AI coding
assistants—architectural principles, testing requirements, and naming
conventions that apply globally. This creates a three-tier documentation
architecture: per-directory AGENTS.md for local context,
root README.md and CLAUDE.md for global
constraints, and README.md for human navigation.
The Documentation Duality standard addresses human and AI navigation at the directory level. A complementary layer operates at the module level: every infrastructure subpackage carries two additional machine-readable files that transform it from a passive code library into an active, protocol-aligned skill endpoint.
template/ implements a progression of agent-facing
documentation, escalating in specificity from global constraints to
module-level API contracts:
| Tier | File | Scope | Purpose |
|---|---|---|---|
| 1 — System | README.md |
Repository root | Global architectural principles, Zero-Mock policy, naming conventions |
| 2 — Structure | AGENTS.md |
Every directory | Local file inventories, API surfaces, integration patterns, architectural constraints |
| 3 — Skill | SKILL.md |
Every infrastructure module | Machine-parseable skill descriptor: module name, description, key imports, usage pattern |
Tier 1 and Tier 2 have direct analogues in the prompt-engineering
literature: system prompts and retrieval-augmented context
[lau2025aicoding]. Tier 3 is novel. The SKILL.md files
follow a YAML frontmatter + Markdown instruction format precisely
aligned with the tool-descriptor schemas of the Model Context Protocol
[anthropic2024mcp]. The following is the exact frontmatter from
infrastructure/rendering/SKILL.md:
---
name: rendering
description: >
Multi-format output generation (PDF, HTML, slides).
Use for: Pandoc/XeLaTeX rendering, RenderManager, slide deck generation.
Key imports: RenderManager, RenderingConfig from infrastructure.rendering
---An MCP client reading this block immediately knows the module name, its natural-language activation condition (“use for”), and which Python symbols to import. No source-code inspection is required. This is the practical implementation of Toolformer-style self-documented tools [schick2023toolformer]—rather than a language model learning tool APIs from training data, the APIs are encoded directly in version-controlled, co-located skill files that evolve with the codebase.
Each infrastructure subdirectory surfaced by
discover_infrastructure_modules() carries paired
README.md + AGENTS.md; agent-facing
SKILL.md manifests exist wherever teams enable Cursor / PAI
manifests (regenerated via
python -m infrastructure.skills). Root-level
PAI.md summarizes cross-package obligations.
Promotion policy: new Layer‑1 directories must ship human +
machine-readable docs (README.md, AGENTS.md)
immediately; Tier‑3 SKILL assets follow once APIs stabilize.
The mapping from SKILL.md descriptors to MCP server
endpoints is intentional but not yet automated; it represents the
principal next integration step. In the MCP architecture
[anthropic2024mcp], a server exposes three primitive types:
Tools (executable functions),
Resources (data the model can read), and
Prompts (structured query templates). Each
infrastructure module maps naturally onto this
taxonomy:
infrastructure.llm → MCP Tool
(query, apply_template) + MCP
Prompt (research prompt templates)infrastructure.rendering → MCP Tool
(render_pdf, render_html) + MCP
Resource (rendered PDFs as URI-addressable
resources)infrastructure.validation → MCP Tool
(validate_pdf_rendering,
validate_markdown)infrastructure.publishing → MCP Tool
(publish_to_zenodo, generate_citation_bibtex)
+ MCP Resource (DOI registry)infrastructure.steganography → MCP
Tool (SteganographyProcessor.process) +
MCP Resource (hash manifests)infrastructure.search ·
infrastructure.reference → MCP Tool
wrappers over literature retrieval + BibTeX handling + MCP
Resource exports for corpus JSON /
.bibAn agent orchestrating a full research pipeline could, in principle,
compose these MCP tools to reproduce the declarative DAG
programmatically—discovering capabilities via SKILL.md
frontmatter, executing them via MCP protocol calls, and consuming their
outputs as Resources. The SKILL.md files parallel Voyager’s
skill library [wang2023voyager]—Voyager’s agent accumulates a growing
library of executable Minecraft skills represented as JavaScript
functions; template/’s agent accumulates a curated library
of research pipeline skills represented as YAML-frontmattered
SKILL.md files. In both cases, the skill representation is
machine-readable, version-controlled, and self-describing. Wang et al.’s
LLM agent survey [wang2024llmagents] identifies tool use, planning, and
memory as the three fundamental capabilities of autonomous agents; Yao
et al.’s ReAct framework [yao2023react] demonstrates that interleaving
reasoning traces with tool actions dramatically improves agent
reliability in interactive settings. The template/ skill
architecture maps cleanly onto these three capabilities: the
SKILL.md descriptors supply the tool-use layer, the
declarative DAG of 12 pipeline.yaml stages (a
default full run executes 10) supplies the planning
scaffold, and the per-criterion checkpoint system supplies the memory
layer.
The template’s design aligns with both the original FAIR principles [wilkinson2016fair] and the FAIR for Research Software (FAIR4RS) principles [barker2022fair4rs] at the repository level. FAIR4RS recognizes that software has requirements distinct from data—executability, composability, and dependency management—and the template addresses each.
Findability. Outputs are Findable through
standardized directory structures, manifest files, and machine-readable
metadata embedded in PDFs. Every project’s config.yaml
provides structured metadata (title, authors, DOIs, keywords) in a
format parseable by both Pandoc and external indexing services. The
metrics.json output provides a machine-readable inventory
of all generated artifacts, their locations, and their provenance
hashes.
Accessibility. Outputs are Accessible via open-source distribution on GitHub, with metadata embedded in the artifact itself rather than in a separate registry. The steganographic layer embeds provenance information directly in the PDF—including SHA-256 content hashes, build timestamps, and QR-encoded metadata—ensuring accessibility even when the PDF circulates outside the repository.
Interoperability. Interoperability is achieved through standard formats (PDF, JSON, BibTeX, YAML) and well-defined module APIs that enable cross-project composition. The Pandoc rendering pipeline accepts any Markdown-with-LaTeX input conforming to the template’s section numbering conventions, allowing seamless migration of manuscripts from other Pandoc-based workflows.
Reusability. Reusability is ensured by the Standalone Project Paradigm—any project can be extracted and reused independently—and by the Documentation Duality standard, which satisfies FAIRsoft’s inspectability and documentation quality indicators [garijo2024fairsoft]. The pipeline’s automated testing and coverage enforcement directly operationalize the FAIR4RS executability requirement: software that cannot pass its own test suite cannot produce publishable output.
At a higher level of abstraction, template/ applies the
DevOps principle of Infrastructure as Code (IaC) to the
research lifecycle. In production software engineering, IaC means that
server configuration is version-controlled, automatically provisioned,
and independently reproducible [wilson2017good]. template/
extends this principle to the research manuscript: the document is not
authored in a word processor and emailed to collaborators, but
built from version-controlled Markdown sources, tested
against formal coverage thresholds, and deployed to a
provenance-embedded PDF.
Every component of the research pipeline—the test suite, the analysis
scripts, the rendering configuration, and the steganographic
watermark—is specified in code, committed to git, and reproducible from
a clean checkout. This deterministic build property means that any
researcher can clone the repository, run
./run.sh --pipeline, and produce a byte-for-byte identical
manuscript (modulo timestamps in the steganographic metadata).
Software Heritage [cosmo2020softwareheritage] provides persistent
SWHIDs (Software Hash Identifiers) for source code snapshots, enabling
stable citation of any specific version of template/ as a
discrete software artifact—closing the loop from research infrastructure
to citable scientific contribution. Combined with Zenodo DOI
registration (supported by infrastructure.publishing), this
creates a dual-identifier citation chain: SWHID for source provenance,
DOI for publication metadata [katz2021software].
All tests use real methods exclusively [martin2008clean;
meszaros2007xunit]. No unittest.mock, no
MagicMock, no patch decorators. Tests that
require external services (Ollama, network) use pytest.mark
markers for conditional execution. The philosophical
motivation—analogizing mock objects to Simmons et al.’s researcher
degrees of freedom [simmons2011falsepositive] and the
pre-registration remedy [nosek2018preregistration]—is developed fully in
the Zero-Mock
Tradeoff discussion. To our knowledge, no prior research software
engineering framework has formalized a zero-mock policy as an
architectural invariant enforced by pipeline gates, where mock
usage is not merely discouraged but structurally prevented from passing
the build.
The following example, drawn from the infrastructure test suite, illustrates zero-mock compliance:
def test_discover_infrastructure_modules_returns_nonempty(tmp_path):
# Real filesystem, real YAML parsing — no MagicMock anywhere
modules = discover_infrastructure_modules(REPO_ROOT)
assert len(modules) >= 8 # actual subpackages on disk
assert any(m.name == "core" for m in modules)This test exercises the real
discover_infrastructure_modules function against the real
filesystem. There are no mock objects substituting for the directory
walk, no patched YAML parsers, and no synthetic return values—the test
passes only if the infrastructure modules genuinely exist and are
discoverable at their expected paths.
The pipeline enforces two coverage tiers:
| Tier | Scope | Minimum | Current | Rationale |
|---|---|---|---|---|
| Project | projects/*/src/ |
90% | 90%+ | Domain code must be thoroughly validated |
| Infrastructure | infrastructure/ |
60% | 83%+ | Broader scope, some code unreachable in test |
These thresholds are enforced at Stage 01 of the pipeline. If project test coverage falls below 90%, the pipeline halts and refuses to produce a PDF—ensuring that no published artifact is backed by undertested source code.
The repository maintains three test suites:
tests/): ~7,780
tests validating the 23 infrastructure subdirectories, covering logging,
rendering, validation, steganography, reporting, and LLM
integration.projects/*/tests/):
Per-project suites whose sizes scale with each exemplar’s surface area —
for example 296 tests in template_autoresearch_project and
231 in template_code_project, with several exemplars larger
still. (A true min/max span would require dedicated
project_test_count_min/project_test_count_max
tokens in build_manuscript_metrics_dict; see the
meta-template’s generator backlog.)All generated figures must meet accessibility requirements:
These standards are validated by the
test_architecture_viz.py test suite, which verifies that
generated figures exist, have non-zero file size, and conform to
expected output specifications. The 16pt font floor ensures readability
in both screen and print contexts, while the DPI range balances file
size against reproduction fidelity.
template/ was evaluated through multi-project pipeline
execution, measuring test coverage, pipeline timing, output integrity,
and steganographic performance across the canonical exemplars under
projects/.
Runs used the ./run.sh interactive orchestrator (“all
projects core (fast)” / menu key d)
skipping infrastructure replication and optional LLM stages orchestrated
via python -m infrastructure.orchestration.
Note: lone menu d returns
after success without redrawing the TUI banner.
| Project | Effective core stages¹ | Approx. duration | Tests² | Coverage |
|---|---|---|---|---|
template_code_project |
8 | ~40 s | 231/231 | 90%+ |
template_prose_project |
8 | ~35 s | 120/120 | 90%+ |
template_autoresearch_project |
8 | ~30 s | 296/296 | 90%+ |
¹“Core-only” disables LLM-tagged YAML stages; durations exclude optional network-heavy LLM or long-running retrieval scripts when run with cached fixtures.
²Counts show passing tests versus discovered tests for the sampled configuration.
³template_search_project lives under
projects/archive/ (local-only literature-search exemplar);
it is not part of the public CI roster.
Overall success: 100 % pipeline completion for sampled runs.
Timing illustrative (Apple Silicon workstation, SSD, fixed seeds).
Search-stage overhead dwarfs the optimization exemplar’s runtime—confirming Stage 02 remains the bottleneck for outbound API traffic while retaining Zero-Mock subprocess + filesystem checks downstream.
| Metric | Value |
|---|---|
| Test files | 447+ |
| Total tests | ~7,780 |
| Infrastructure coverage gate | ≥60 % (repo ≥80 %+ during recent audits) |
| Zero-mock imports | Verified via static scanning |
Exercises Pandoc/XeLaTeX paths, steganography hashing, telemetry,
YAML-driven pipeline DAG resolution, HTTPS-bound optional suites
(pytest-httpserver), and local Ollama-gated subsets.
The introspection module
(template_template.introspection) emits the authoritative
table below—every row reflects
discover_infrastructure_modules(REPO_ROOT).
| Module | Python Files | Has AGENTS.md | Has README.md | Key Exports |
|---|---|---|---|---|
autoresearch |
10 | ✓ | ✓ | build_autoresearch_plan, readiness validation CLI |
benchmark |
3 | ✓ | ✓ | Template harness scoring + comparative gates |
config |
0 | ✓ | ✓ | Repository defaults + hardened templates |
core |
109 | ✓ | ✓ | get_logger, load_config,
TemplateError |
docker |
0 | ✓ | ✓ | Containerisation scaffolding |
doctor |
14 | ✓ | ✓ | Checkout diagnose/fix/undo repairs |
documentation |
12 | ✓ | ✓ | FigureManager, generate_glossary |
llm |
54 | ✓ | ✓ | Ollama helpers, sanitization, review + translation pipelines |
logrotate.d |
0 | ✓ | ✓ | Rotation snippets (documentation-first) |
methods |
5 | ✓ | ✓ | build_methods_orchestration_plan, methods-stage
contracts + validation |
orchestration |
8 | ✓ | ✓ | PipelineRunner, entry point for
./run.sh |
project |
27 | ✓ | ✓ | discover_projects, workspace management |
prose |
9 | ✓ | ✓ | Markdown readability + prose tooling |
publishing |
71 | ✓ | ✓ | Zenodo, executable bundle, archival targets |
reference |
16 | ✓ | ✓ | BibTeX models, parsers, converters |
rendering |
50 | ✓ | ✓ | PDF/HTML/slide rendering, Pandoc filters |
reporting |
57 | ✓ | ✓ | Coverage parsers, dashboards, executive artefacts |
scientific |
4 | ✓ | ✓ | check_numerical_stability,
benchmark_function |
search |
44 | ✓ | ✓ | infrastructure.search.literature clients + cache |
sia |
10 | ✓ | ✓ | Self-Improving-AI loop: task validation, harness, metric capture |
skills |
7 | ✓ | ✓ | discover_skills, SKILL manifest regeneration |
steganography |
11 | ✓ | ✓ | Watermark overlays + hash manifests |
validation |
83 | ✓ | ✓ | PDF + Markdown + integrity CLIs |
All 23 enumerated subdirectories carry Tier‑1/README.md
and Tier‑2/AGENTS.md coverage wherever the Documentation
Duality standard applies; subsets ship Tier‑3 SKILL.md
descriptors for MCP routing (infrastructure/skills manifest
generation).
| Layer | Role |
|---|---|
| System prompts | Root CLAUDE.md, README policy |
| Structural | AGENTS.md directories |
| Skills | Optional SKILL.md manifests + generated
.cursor/skill_manifest.json |
| PAI capsule | Repository level PAI.md narratives |
390+ Markdown shards under docs/ capture
operational knowledge without duplicating auto-generated
inventories.
Stages below mirror pipeline.yaml (executor-topological
order—not strict numeric script filenames). Scripts live under
scripts/.
| Name | Typical script / method | Responsibility | Failure semantics |
|---|---|---|---|
| Clean Output Directories | _run_clean_outputs |
Deletes stale output/ trees |
Blocking |
| Environment Setup | 00_setup_environment.py |
Validates tooling, PYTHONPATH scaffolding | Blocking |
| Infrastructure Tests | 01_run_tests.py --infra-only |
Infra pytest + coverage gates | Tunable thresholds |
| Project Tests | 01_run_tests.py --project-only |
Project pytest + coverage gates | Zero failures default |
| Project Analysis | 02_run_analysis.py |
Executes projects/<name>/scripts/*.py |
Blocking |
| PDF Rendering | 03_render_pdf.py |
Pandoc → XeLaTeX manuscripts | Blocking |
| Output Validation | 04_validate_output.py |
Structural PDF/markdown probes | Blocking / warnings |
| LLM Scientific Review | 06_llm_review.py --reviews-only |
Local Ollama reviews | Skippable / exit 2 tolerated |
| LLM Translations | 06_llm_review.py --translations-only |
Optional translations | Skippable |
| Copy Outputs | 05_copy_outputs.py |
Mirrors deliverables → output/<project>/ |
Soft-fail surfaced in logs |
scripts/07_generate_executive_report.py is
multi-project orchestration glue invoked after
iterating active projects—not a tenth DAG node for single-repo runs
(execute_pipeline.py).
| Project | Pages (approx.) | Metadata | SHA-256 | Overlay | QR Code | Total (approx.) |
|---|---|---|---|---|---|---|
template_code_project |
~20 | <0.3 s | <0.05 s | <0.8 s | <0.4 s | <1.5 s |
template_prose_project |
~25 | <0.3 s | <0.05 s | <0.9 s | <0.4 s | <1.6 s |
template_autoresearch_project |
~25 | <0.2 s | <0.04 s | <0.9 s | <0.3 s | <1.5 s |
All measurements use single-threaded execution on Apple Silicon, and the totals are dominated by watermark-overlay complexity rather than by hashing or metadata embedding, which each stay well under one-tenth of a second per document.
Rendered via projects/templates/template_template
(generate_manuscript_metrics.py → injected tokens such as
23). Architecture figures stem from template_template.architecture_viz—font
sizes constrained by §QA.
Figure 1. Live
rendering of the Two-Layer Architecture from repository introspection:
the infrastructure layer (top) holds the 23 reusable
subpackages, each annotated with its Python file count and a four-slot
documentation badge—A AGENTS.md, R README.md,
S SKILL.md, P PAI.md, with ·
marking an absent file—so a fully documented module reads
[ARSP]. A YAML DAG arrow connects it to the project layer
(bottom) of public exemplars labelled with chapter and test counts. The
takeaway: documentation-duality coverage is near-uniform across the
infrastructure, and every box was placed from the same live data the
prose cites.
Figure 2. Pipeline DAG
with 12 YAML-declared stages (core, LLM, bundle, archival tags).
Figure 3.
Horizontal file-count histogram of every infrastructure subdirectory,
sorted largest-first. The long tail of small, single-purpose packages
beside a handful of larger ones (core,
validation, publishing) is the visual
signature of the Unix-philosophy modularity the architecture section
argues for—capability concentrated where it compounds, not spread evenly
by fiat.
Figure 4 summarizes the Appendix F capability matrix.
Figure 4.
14 × 10 heatmap annotated in appendix text—green ✓ full
native capability, yellow ◐ partial / plugin-hosted,
red — unavailable. Rows group Core Pipeline,
Quality & Security, then Ecosystem.
¹ Nextflow 25.04.0: lineage records exist at workflow scope, not per rendered PDF citation graph.
² DVC: content-addressed artifacts without native prose rendering.
³ DVC: remote object stores (S3, GCS, Azure) without turnkey CI manuscript gates.
Zero mocks: repository policy bans
unittest.mock / patching frameworks in tests.
Filesystem + subprocess realism: ephemeral directories + actual CLI binaries.
HTTP realism: infra suites favour
pytest-httpserver; literature tests hit recorded
fixtures.
template_code_project focuses on
numerical reproducibility assertions.
template_autoresearch_project
exercises the AutoResearch readiness planner
(infrastructure/autoresearch/).
template_search_project remains
archive-only for literature-search workflows.
The Zero-Mock
testing policy is template/’s most distinctive design
decision. By prohibiting all mock objects, we gain confidence that tests
exercise real code paths—a pytest run against the template genuinely
invokes pandoc, writes to disk, and parses real YAML. The
cost is test duration: the full infrastructure test suite (~7,780 tests)
takes 2–4 minutes, compared to sub-second execution typical of
heavily-mocked suites.
We argue this tradeoff is strongly favorable for research software. Unlike web applications where millisecond latency and thousands of daily deploys demand fast feedback loops, research pipelines run infrequently (once per manuscript revision) and correctness vastly outweighs speed. A mocked test that passes while the real renderer fails is worse than a slow test that catches the failure. The analogy to statistical methodology is precise: just as Simmons et al.’s researcher degrees of freedom [simmons2011falsepositive] inflate false-positive rates through undisclosed analytical flexibility, mock objects create testing degrees of freedom that make integration failures invisible. The Zero-Mock policy closes this loophole by the same mechanism that pre-registration [nosek2018preregistration] closes the p-hacking loophole: removing flexibility before the fact. As Peng [peng2011reproducible] argues, computational reproducibility requires independent verification—and mock-only tests verify assumptions rather than results. Garijo et al.’s FAIRsoft evaluator [garijo2024fairsoft] identifies executability as a primary quality indicator; the Zero-Mock policy operationalizes executability at the unit level.
It is important to distinguish the Zero-Mock policy from a naive rejection of all test isolation techniques. Fowler’s classification [martin2008clean] recognizes that stubs and fakes serve legitimate purposes—a test database populated with known data is not a mock, it is a fixture. The policy specifically prohibits mock objects as defined by Meszaros: assertions on indirect outputs (method calls, argument patterns) rather than direct outputs (return values, side effects). The distinction matters because mock-based assertions encode implementation assumptions (“method X must be called with argument Y”) that become invisible coupling between tests and production code, creating the illusion of coverage without testing real behavior.
The template enforces zero-mock compliance at three levels:
AGENTS.md at every
directory level explicitly states the prohibition, ensuring both human
and AI contributors are aware before writing tests.grep -rn "MagicMock\|unittest.mock\|@patch" tests/ can be
run as a pre-commit hook to catch violations.template_code_project
documents filesystem + YAML + plotting paths while
template_autoresearch_project exercises readiness planning;
template_search_project (archive-only) reinforces
HTTP-realistic literature queries—both serve as onboarding references
alongside infra suites.However, the policy requires careful management of external
dependencies. Tests requiring Ollama (the local LLM backend) use
@pytest.mark.requires_ollama and are skipped in
environments where the service is unavailable. Tests requiring network
access use @pytest.mark.network. This marker system
preserves the Zero-Mock principle while acknowledging that not all
environments provide all services, especially computationally intensive
ones. The key distinction is between replacing an external
dependency (which mock objects do, hiding failures) and
skipping a test when a dependency is absent (which markers do,
preserving transparency).
The Standalone Project Paradigm enables horizontal scaling: adding a
new project requires creating a directory with
manuscript/config.yaml and nothing else. No infrastructure
code changes, no pyproject.toml modifications, no CI
configuration updates. The run.sh orchestrator
automatically discovers new projects and presents them in its
interactive menu.
We have validated scaling with 12 canonical exemplars under
projects/templates/—always present for onboarding and
tooling—and with this manuscript from
projects/templates/template_template (127 tests) as a
git-tracked public exemplar in the same automated discovery menus.
Canonical trio:
template_code_project: Numerical
optimization example with gradient-descent narration, 231 tests, 90%+
coverage. Minimal footprint: compact src/, scripted
analysis, short manuscript sections.template_prose_project: Prose-heavy
manuscript emphasizing narrative structure and bibliography discipline,
120 tests, 90%+ coverage—tests exercise rendering and Markdown integrity
without heavyweight numerics.template_autoresearch_project:
AutoResearch readiness workflow invoking
projects/template_search_project/scripts/ to run corpus
builders, scripted figures (../figures/), and
manifold-variable injection (the archive-only literature-search
exemplar, restored on demand). Typical Stage 02 workloads include
bibliography fusion, corpus JSON assembly, deep-search aggregates, and
report composition.Meta manuscript
(projects/templates/template_template)
analyzes the repository via src/template_template/
introspection metrics; it now lives alongside the other public exemplars
under projects/templates/.
These workspaces share no project-level code—only Layer 1 (23 infrastructure subdirectories, ~604 Python files)—validating insulation between domain repos and reusable services.
When the --all-projects flag is passed to
run.sh, the pipeline executes each discovered project
sequentially, running infrastructure tests once at the start and
skipping them for individual projects to avoid redundant validation.
After all projects complete, a cross-project executive report aggregates
metrics (test counts, coverage percentages, page counts, rendering
durations) into a unified dashboard with both JSON and Markdown output
formats. This executive reporting stage provides repository-level
visibility without requiring any project-specific reporting code.
| Metric | template_code_project |
template_prose_project |
template_autoresearch_project |
|---|---|---|---|
| Source modules | 25 | 6 | 60 |
| Test files | 11 | 8 | 17 |
| Test count | 231 | 120 | 296 |
| Manuscript chapters | 9 | 8 | 6 |
| Analysis scripts | 6 | 4 | 4 |
| Figures (auto-generated) | 8 | 5 | 27 |
The infrastructure overhead per project is constant regardless of project size: the same 23 modules, the same 11 pipeline stages, the same rendering and validation logic. This O(1) infrastructure cost is the architectural payoff of the Two-Layer separation.
The gap analysis established
that no single tool integrates all six cross-cutting concerns. Here we
synthesize the fourteen-dimension
comparison into three structural insights. First, the landscape
bifurcates: workflow managers (Snakemake [koster2012snakemake], Nextflow
[ditommaso2017nextflow], CWL [amstutz2016cwl]) provide distributed
execution but no manuscript support; publication tools (Quarto
[allaire2024quarto], Jupyter Book, R Markdown [xie2018dynamic], Overleaf
[overleaf2025], Prism [openai2026prism]) author documents but embed no
integrity guarantees; and DVC [iterative2024dvc] versions artifacts
without orchestrating pipelines. template/ occupies the
intersection, sacrificing distributed execution for unified enforcement
of testing, provenance, and documentation. This positioning is
complementary—a mature deployment might use Nextflow upstream and
template/ for rendering, testing enforcement, and
provenance downstream. Typst [madje2023typst], with its faster
compilation cycle, is not one of the nine compared peers but could serve
as an alternative rendering backend if a Pandoc writer were
contributed.
Second, the eight enforcement capabilities template/
co-enforces—testing enforcement, coverage thresholds, steganographic
watermarking, multi-project management, AI-agent documentation, the
agentic skill protocol, an interactive TUI, and Zero-Mock policy—are
individually straightforward (and several, such as multi-project
management and AI-agent documentation, are matched in part by individual
peers); their novelty lies in co-enforcement within a single
pipeline, ensuring that a passing build guarantees documentation
completeness, provenance embedding, and AI-navigability alongside
computational correctness. The FAIR4RS principles [barker2022fair4rs;
lamprecht2020towards] articulate what research software quality
requires; FAIRsoft [garijo2024fairsoft] scores compliance
observationally; template/ operationalizes these standards
by coupling them to pipeline gates that halt the build if coverage drops
below 90% or provenance embedding fails. Cohen et al.’s four pillars of
research software engineering [cohen2021four]—sustainability, quality,
community, and policy—are operationalized by template/
through the first two pillars via quality-gated automation.
Third, the AI-agent documentation dimension reveals an underserved
need. Overleaf and Prism provide AI writing assistance, but
neither exposes structured documentation for external agents to
consume. template/’s AGENTS.md +
SKILL.md layer enables an agent entering the repository to
discover capabilities, understand API contracts, and invoke modules
without prior training (Documentation
Duality, AI
Collaboration).
Since the FAIR4RS principles were published [barker2022fair4rs], the
community has moved toward operationalization. The RDA Virtual Plenary
24 (April 2025) featured a two-year retrospective review
[honeyman2024fair4rs] recommending principle amendments—notably adding
reproducibility as an explicit requirement and clarifying
“domain-relevant standards”—alongside a leadership refresh and parallel
guidance activities. The ReSA Actionable FAIR4RS Task Force (launched
December 2024) analyzed the 17 principles into six actionable categories
(identifiers, metadata for publication/discovery/reuse, standards,
references, and licenses), with a first draft expected by September 2025
[resa2024actionable]. Tools for automated FAIR assessment have also
matured: the F-UJI extension for research software evaluation now scores
against FRSM-04 through FRSM-17 metrics, complementing Garijo et al.’s
FAIRsoft evaluator [garijo2024fairsoft]. template/’s
pipeline-enforced quality gates—coverage thresholds, documentation
completeness checks, and provenance embedding—anticipate this
operationalization trend by implementing FAIR4RS not as a post-hoc
assessment but as an architectural invariant.
In Gentleman and Temple Lang’s terminology [gentleman2007research],
template/ is a research compendium scaled to the
repository level—bundling not just one study’s code and data but N
studies, with shared infrastructure, automated testing, and embedded
provenance. Nüst et al.’s executable research compendium (ERC)
[nust2017containerization] extends this vision with containerized
reproduction environments; template/ complements
containerization by adding the testing enforcement, multi-project
management, and provenance embedding layers that ERCs do not
address.
The Documentation Duality standard and three-tier skill architecture represent an empirical bet: that structured, machine-readable documentation measurably improves AI agent performance in research codebases. This section reports our key observations.
The documentation investment creates a positive feedback loop: as
agents produce higher-quality outputs from structured context,
developers maintain that documentation, which in turn improves future
interactions [lau2025aicoding]. We observed this concretely during
template/ development—each module’s SKILL.md
was refined through iterative AI-assisted generation, serving as both
input prompt and output validator.
The SKILL.md layer, with its MCP-aligned YAML
frontmatter [anthropic2024mcp], provides a critical bridge to the
agentic software paradigm. Lu et al.’s AI Scientist [lu2024aiscientist]
demonstrates end-to-end autonomous research; Wang et al.’s OpenHands
[wang2024opendevin] achieved 53% on SWE-Bench Verified
[jimenez2024swebench]—the first open-source system to exceed 50%. These
systems require structured, protocol-aligned tool inventories to
navigate unfamiliar codebases. An OpenHands-class agent navigating
template/ reads CLAUDE.md for global
constraints, scans AGENTS.md for module surfaces, and
invokes capabilities via SKILL.md without hallucinating
function signatures. All 23 infrastructure modules carry
AGENTS.md and README.md; all additionally
carry SKILL.md, ensuring no documentation blind spots.
This three-tier model is, to our knowledge, novel in the research
software engineering literature. The scale of the investment is
substantial: 390 Markdown files under docs/
alone, plus an AGENTS.md/README.md pair in
every directory and a SKILL.md descriptor on every
infrastructure module. That count is itself injected from live
introspection—the manuscript refuses to quote a documentation total it
cannot recompute—and it represents a deliberate commitment to
machine-readable context that shrinks the surface on which an agent can
hallucinate.
The Thin Orchestrator pattern imposes a cognitive overhead on
researchers accustomed to writing monolithic scripts. The requirement to
factor all logic into src/ modules and use scripts only as
stateless wiring introduces an additional layer of indirection. We
mitigate this through:
template_code_project ships minimal optimization
commentary; template_prose_project and
template_autoresearch_project broaden narrative + retrieval
scaffolding.README.md (for humans) and AGENTS.md (for AI
collaborators), reducing the cost of navigation.run.sh
provides a TUI menu that abstracts pipeline complexity.docs/guides/ directory provides four progressive guides
(Levels 1–3 Beginner, 4–6 Intermediate, 7–9 Advanced, 10–12 Expert)
alongside a comprehensive new-project setup checklist.scientific or
validation modules.qpdf --optimize). QR codes are visible and removable. The
current system provides tamper detection (via SHA-256 hashing)
but not non-repudiation in the cryptographic sense—it lacks
private-key digital signatures. An attacker with access to the source
code could reproduce the watermark without having run the original
pipeline.template/
does not include an integrated AI writing assistant comparable to
Overleaf’s Copilot features or OpenAI Prism’s GPT-5.2 context-aware
editing. The infrastructure.llm module provides LLM review
as a pipeline stage but not as an interactive writing environment.src/ modules do not import from
scripts/.SKILL.md descriptors already define the interface contracts
for each infrastructure module; the natural next step is to expose them
as MCP server endpoints [anthropic2024mcp]. An MCP server wrapping
infrastructure.llm would expose query,
review_manuscript, and translate_abstract as
protocol-native Tools; an MCP server wrapping
infrastructure.publishing would expose
publish_to_zenodo and
generate_citation_bibtex. A research agent could then
compose these Tools to execute the full pipeline—environment setup →
test execution → analysis → rendering → validation → LLM review → DOI
registration—without any human in the loop. This closes the loop opened
by Lu et al.’s AI Scientist [lu2024aiscientist], which demonstrated
automated hypothesis generation and experimental iteration but relied on
ad hoc laboratory scaffolding. template/’s pipeline, fully
exposed as MCP tools, provides that scaffolding in a reproducible,
versioned, and certified form. Longer-term, the Agent2Agent (A2A)
protocol [google2025a2a] enables heterogeneous specialist agents—a
statistical analyst, a figure designer, a peer-review simulator—to
coordinate via a shared, protocol-mediated workspace built on precisely
the kind of modular, well-documented infrastructure that
template/ provides.template/ itself to be cited as a
software artifact with a DOI-equivalent stable identifier. Combining
this with Zenodo DOI registration (already supported by
infrastructure.publishing) creates a full citation chain:
the paper cites the data (DOI), the data provenance cites the pipeline
(SWHID), and the pipeline cites the framework release (Zenodo DOI). This
three-link citation chain operationalizes the Katz et
al. [katz2021software] software citation principles at the
infrastructure level.template/ demonstrates that high-integrity, reproducible
research need not be onerous. By embedding provenance, testing, and
documentation into the architecture itself—rather than layering them
atop a fragmented workflow—the template transforms “best practices” from
aspirational guidelines into enforced invariants [wilson2017good;
sandve2013ten; lamprecht2020towards]. The Two-Layer Architecture ensures
that infrastructure improvements propagate to all projects without
coupling. The Zero-Mock policy ensures that tests reflect
reality. The steganographic provenance layer ensures that published
artifacts carry their own authentication.
The comparative
analysis confirms that no existing tool integrates all eleven
distinctive capabilities—testing enforcement, coverage thresholds,
cryptographic provenance, steganographic watermarking, multi-project
management, AI-agent documentation, the agentic skill protocol,
interactive TUI, Zero-Mock policy, manuscript rendering, and pipeline
orchestration—within a single enforced pipeline.
The template is not merely a build tool; it is an epistemological commitment. It asserts that a research paper is not a static document but a build artifact—reproducible, verifiable, and traceable to the code that generated it. As Knuth observed, programs should be written for humans to read and only incidentally for machines to execute [knuth1984literate]. We extend this dictum: research manuscripts should be built for verification and only incidentally for reading. In an era of generative AI, AI-native research workspaces, and synthetic media—where the boundary between human-authored and machine-generated text grows increasingly indeterminate [gruenpeter2021research]—the provenance chain from source code to published PDF is not an administrative convenience. It is the epistemic ground on which scientific trust must be rebuilt. That this manuscript was itself built, tested, and watermarked by the pipeline it describes—its metrics computed from the repository it inhabits, its figures rendered by the code it documents—is not a rhetorical device but a structural proof: the system works because you are reading its output.
This section inventories every Layer‑1 subdirectory returned by
23 discover_infrastructure_modules(repo_root).
File totals use 604 Python sources across infra +
7,780 infra tests guarding them. Documentation Duality =
paired README.md + AGENTS.md; optional
SKILL.md manifests feed
python -m infrastructure.skills.
| Module | Python Files | Has AGENTS.md | Has README.md | Key Exports |
|---|---|---|---|---|
autoresearch |
10 | ✓ | ✓ | build_autoresearch_plan, readiness validation CLI |
benchmark |
3 | ✓ | ✓ | Template harness scoring + comparative gates |
config |
0 | ✓ | ✓ | Repository defaults + hardened templates |
core |
109 | ✓ | ✓ | get_logger, load_config,
TemplateError |
docker |
0 | ✓ | ✓ | Containerisation scaffolding |
doctor |
14 | ✓ | ✓ | Checkout diagnose/fix/undo repairs |
documentation |
12 | ✓ | ✓ | FigureManager, generate_glossary |
llm |
54 | ✓ | ✓ | Ollama helpers, sanitization, review + translation pipelines |
logrotate.d |
0 | ✓ | ✓ | Rotation snippets (documentation-first) |
methods |
5 | ✓ | ✓ | build_methods_orchestration_plan, methods-stage
contracts + validation |
orchestration |
8 | ✓ | ✓ | PipelineRunner, entry point for
./run.sh |
project |
27 | ✓ | ✓ | discover_projects, workspace management |
prose |
9 | ✓ | ✓ | Markdown readability + prose tooling |
publishing |
71 | ✓ | ✓ | Zenodo, executable bundle, archival targets |
reference |
16 | ✓ | ✓ | BibTeX models, parsers, converters |
rendering |
50 | ✓ | ✓ | PDF/HTML/slide rendering, Pandoc filters |
reporting |
57 | ✓ | ✓ | Coverage parsers, dashboards, executive artefacts |
scientific |
4 | ✓ | ✓ | check_numerical_stability,
benchmark_function |
search |
44 | ✓ | ✓ | infrastructure.search.literature clients + cache |
sia |
10 | ✓ | ✓ | Self-Improving-AI loop: task validation, harness, metric capture |
skills |
7 | ✓ | ✓ | discover_skills, SKILL manifest regeneration |
steganography |
11 | ✓ | ✓ | Watermark overlays + hash manifests |
validation |
83 | ✓ | ✓ | PDF + Markdown + integrity CLIs |
Below, ${module_*_python_file_count} placeholders expand
per subdirectory at render-time.
infrastructure.autoresearch (10 files)Readiness planner, validation CLI, and report models for
AutoResearch-style project promotion
(infrastructure/autoresearch/).
infrastructure.benchmark (3 files)Template harness scoring and comparative gate helpers exercised in CI smoke paths.
infrastructure/config (non-package subdirectory)Repository-wide YAML templates and secure manifests
(.env.template, hardened defaults referenced by Docker +
CLI). config/ carries no __init__.py, so it is
a configuration subdirectory rather than an importable package.
infrastructure.core (109 files)Checkpointing, logging, pipeline YAML parsing, telemetry bridges, filesystem helpers, hardened exceptions. Everything else imports logging + error taxonomy from here first.
infrastructure.doctor (14 files)Checkout diagnose/fix/undo repairs for broken local workspace states.
infrastructure.docker (0 files)Pinned images / compose scaffolding for reproducible CI + remote builds.
infrastructure.documentation (12 files)Figure registries plus glossary tooling feeding manuscript automation.
infrastructure.llm (54 files)Ollama integrations, sanitization adapters, templated reviewer flows.
Literature ingestion now lives primarily in
search/literature + citation helpers in
reference/.
infrastructure.methods (5 files)Deterministic methods-orchestration contracts
(MethodStage, MethodsOrchestrationPlan,
MethodsIssue): builds and validates an ordered methods plan
for a research project so the manuscript’s “Methods” track stays bound
to executable stages.
infrastructure.orchestration (8 files)python -m infrastructure.orchestration exposes
interactive menus, subprocess wiring for thin shell wrappers
(run.sh, secure_run.sh), and stubs used in CI
for menu parsing tests.
infrastructure.project (27 files)Canonical discovery (discover_projects) enforcing
src/ + tests/, slug validation, nested WIP
namespaces.
infrastructure.prose (9 files)Readability metrics + Markdown tooling for prose-centric manuscripts / CI gates.
infrastructure.publishing (71 files)Metadata models, APA/BibTeX/MLA formatters, optional Zenodo clients.
infrastructure.reference (16 files)Citation/BibTeX parsing + conversion utilities leveraged by manuscripts and retrieval scripts.
infrastructure.rendering (50 files)Pandoc shim, Unicode/XeLaTeX postprocessors, combined PDF/HTML/slide exporters.
infrastructure.reporting (57 files)Parses pytest + coverage artefacts for dashboards; pairs with Stage 01 summaries and downstream executive exports.
infrastructure.scientific (4 files)Stability probing, benchmarking hooks—consumed heavily by
optimization exemplars (template_code_project scripts).
infrastructure.search (44 files)literature/ client stack (client.py,
backends, caches) powering archive-only
template_search_project literature workflows when copied
locally from projects/archive/.
infrastructure.sia (10 files)Generic Self-Improving-AI loop utilities: task-layout validation,
execution harness, and metric capture reused by
template_sia (fixture-replay by default).
infrastructure.skills (7 files)Discovers SKILL.md frontmatter →
.cursor/skill_manifest.json.
infrastructure.steganography (11 files)Watermark overlays, hashing companions triggered by secure pipeline path.
infrastructure.validation (83 files)Markdown + PDF + integrity CLIs underpinning Stage 04 diagnostics.
infrastructure/logrotate.d (0 files)Operational templates for deployments (documentation-first; intentionally minimal Python footprint).
Documentation maturity: Coverage statements in
Results pull from introspection—not hand-maintained denominators—so
newly promoted modules automatically flow into manuscripts after
generate_manuscript_metrics.py.
FAIR+RSE linkage: MCP-ready SKILL.md
artefacts align with evaluator heuristics (executability + metadata
richness) emphasized by FAIRsoft guidance [garijo2024fairsoft].
Research integrity requires more than reproducibility; it requires
verifiable authorship. In an era of generative AI, automated scraping,
and synthetic media, the ability to prove that a document was produced
by a specific pipeline at a specific time is a critical defense against
fabrication and misattribution. The W3C PROV data model
[moreau2013provdm] establishes a formal vocabulary for expressing
provenance records—entities, activities, and agents connected by
derivation, generation, and attribution relations. Digital watermarking,
pioneered by Cox et al. [cox1997secure] for multimedia integrity
verification, provides the foundational signal-processing theory for
embedding imperceptible provenance markers within artifacts.
template/ implements a domain-specific provenance layer
that embeds these relations directly into the PDF artifact itself, using
four complementary steganographic and cryptographic mechanisms.
The steganography subsystem defends against three classes of threats:
The system applies four complementary layers of provenance information:
The inject_pdf_metadata function writes structured
metadata into both the PDF Info dictionary and an XMP (Extensible
Metadata Platform) packet:
/Creator: Pipeline identifier/Producer: Module path
(infrastructure.steganography)/CreationDate: UTC timestamp in ISO 8601 format/Author: From config.yaml/Title: From config.yamlBefore watermarking, a SHA-256 hash of the rendered PDF is computed and stored in:
output/manifest.json)/Subject field)output/<name>.sha256)This enables post-hoc verification: anyone with the hash can verify that the PDF has not been modified since rendering.
A semi-transparent text overlay is applied to each page of the PDF, encoding:
The overlay is rendered at low opacity (typically 3–5% alpha) to be invisible during normal viewing but detectable through image analysis. It survives printing (as a faint watermark) and standard PDF operations.
A representative overlay text string takes the following form:
template/ | built: 2026-03-19T14:23:11Z | commit: a4f2c1b | pipeline: v2.0.0 | project: template
This single line, tiled across each page at 3–5% opacity, encodes the complete build provenance chain: the system identifier, ISO 8601 build timestamp, short Git commit hash, pipeline version, and project name. Together these fields allow a verifier to reconstruct—from the watermark alone—which version of the code, at which moment in time, produced the document.
An optional QR code is generated containing a URL pointing to the
repository (e.g., github.com/docxology/template). The QR
code is placed in a configurable position (default: bottom-right corner
of the last page) at a specified size.
secure_run.sh
OrchestratorThe steganographic pipeline is orchestrated by
secure_run.sh, a Bash script that wraps the standard
run.sh pipeline with post-processing steganography:
secure_run.sh script invokes
SteganographyProcessor.The orchestrator processes either a single specified project or all discovered projects sequentially.
Verification is performed by comparing the stored SHA-256 hash against a freshly computed hash of the distributed PDF. Any modification—even a single bit flip—produces a hash mismatch. The alpha-channel overlay provides a secondary, visual verification channel that does not require access to the original hash.
qpdf --optimize).template/’s current
metadata-based provenance to full W3C PROV-compliant traces involves
four steps: (1) entity identification—assigning stable
identifiers (URIs or SWHIDs) to each pipeline input (manuscript files,
data, config); (2) activity logging—recording each pipeline
stage as a PROV Activity with start/end timestamps (already encoded in
the watermark overlay); (3) agent attribution—binding each
Activity to the pipeline version and Git commit hash (already encoded in
the overlay and PDF metadata); (4) PROV-O
serialization—emitting the provenance graph as OWL-RDF (PROV-O) or
text (PROV-N) alongside the PDF. Steps (2) and (3) are already
implemented; steps (1) and (4) are the primary remaining gaps. A future
infrastructure.provenance module would close both gaps
automatically.The steganographic provenance layer operates at the document level—it certifies the integrity of a specific PDF artifact. A complementary concern is build-level provenance: certifying that the pipeline itself was executed with verified source code and dependencies. Frameworks such as in-toto [torresarias2019intoto] and SLSA (Supply-chain Levels for Software Artifacts) address this concern by defining attestation chains from source commit through build steps to final artifact. The NTIA’s minimum elements for a Software Bill of Materials (SBOM) [ntia2021sbom] further standardize the enumeration of software components and dependencies—essential for establishing the provenance lineage of build environments. SLSA defines four graduated levels of build integrity:
| SLSA Level | Requirement | template/ Status |
|---|---|---|
| 1 | Provenance document exists | ✓ SHA-256 manifest + steganographic metadata |
| 2 | Version-controlled build scripts | ✓ All scripts in git |
| 3 | Isolated build environment | ~ Docker support exists but not enforced in CI |
| 4 | Hermetic, reproducible builds | N – future work |
Future versions of template/ may generate
SLSA-compatible provenance attestations alongside the steganographic
watermarks, creating a two-layer provenance model: in-toto attests that
the build pipeline was executed with the claimed source code, while the
steganographic layer attests that the PDF was produced by that pipeline
at a specific time.
The steganographic layer supports the FAIR for Research Software (FAIR4RS) principles [barker2022fair4rs] at the artifact level. PDFs carry embedded metadata (Findability) in standardized XMP format (Interoperability). The SHA-256 hash manifest enables persistent integrity verification (a prerequisite for Reusability). The Documentation Duality standard ensures that the software producing the artifact is inspectable and well-documented (satisfying FAIRsoft [garijo2024fairsoft] metadata and documentation indicators). Full PROV-compliant provenance traces—capturing the derivation chain from source data through analysis scripts to rendered PDF—are a natural extension and a primary target for future development.
Software Heritage [cosmo2020softwareheritage] complements this
picture at the source-code level: by archiving the
template/ repository and assigning a reproducible SWHID
(Software Hash Identifier) to each commit, Software Heritage makes the
pipeline itself—not just its output—a citable, persistent digital
artifact. A published SWHID alongside the PDF DOI creates a complete,
two-artifact citation record: the paper’s content is versioned via DOI;
the code that generated it is versioned via SWHID. This combination
satisfies the Katz et al. [katz2021software] software citation
principles’ requirement that software used in research be independently
citable and permanently accessible.
| Stage name | Script / method | Primary inputs | Outputs / artefacts | Failure mode |
|---|---|---|---|---|
| Clean Output Directories | _run_clean_outputs |
prior projects/<name>/output/, mirrored
output/<name>/ targets |
emptied trees | Blocking |
| Environment Setup | 00_setup_environment.py |
toolchain probes | scaffold dirs, env exports | Blocking |
| Infrastructure Tests | 01_run_tests.py --infra-only --infra-scope pipeline-smoke |
tests/infra_tests/ |
coverage + junit-style logs | tolerant ceilings |
| Project Tests | 01_run_tests.py --project-only |
projects/<name>/tests/ |
coverage artefacts | blocking by default |
| Project Analysis | 02_run_analysis.py |
thin scripts | figures/, data/, reports |
Blocking |
| PDF Rendering | 03_render_pdf.py |
manuscript/, placeholders |
.pdf/.tex bundles |
Blocking |
| Output Validation | 04_validate_output.py |
render tree | Markdown + PDF diagnostics JSON | Blocking / downgrade |
| LLM Scientific Review | 06_llm_review.py --reviews-only |
resolved manuscript artefacts | textual reviews | Optional skip (allow_skip) |
| LLM Translations | 06_llm_review.py --translations-only |
abstract metadata | multilingual snippets | Optional skip (allow_skip) |
| Copy Outputs | 05_copy_outputs.py |
validated tree | mirrored output/<name>/… |
soft fail logged |
| Executable Bundle | 08_executable_bundle.py |
project tree + outputs | container bundle manifest | opt-in (bundle tag) |
| Archival Publication | 09_archive_publication.py |
bundle + deliverables | archival deposit manifest | opt-in (archival tag) |
scripts/07_generate_executive_report.py is invoked
outside this DAG whenever
execute_multi_project.py aggregates pipelines—supplying
cross-project KPI dashboards absent from lone-project checkpoints.
paper:
title: "Paper Title"
subtitle: "Optional Subtitle"
version: "1.0"
date: "2026-03-19"
authors:
- name: "Author Name"
orcid: "0000-0000-0000-0000"
email: "author@example.com"
affiliation: "Institution"
corresponding: true
publication:
doi: "10.5281/zenodo.XXXXXX"
journal: "Target Journal"
volume: "1"
pages: "1-10"
year: "2026"
keywords:
- "keyword1"
- "keyword2"
metadata:
license: "Apache License 2.0"
language: "en"
llm:
reviews:
enabled: true
types: [executive_summary, quality_review]
translations:
enabled: false
testing:
max_test_failures: 0
max_infra_test_failures: 3
max_project_test_failures: 0template/
├── infrastructure/
│ ├── config/ docker/ documentation/ llm/
│ ├── orchestration/ # Thin Python entry equal to `./run.sh` backend
│ ├── prose/ reference/ rendering/ reporting/
│ ├── scientific/ search/ skills/ steganography/ validation/
│ ├── project/ core/
│ └── logrotate.d/ # Operational rotation templates (no Python pkg)
├── scripts/
│ ├── 00_setup_environment.py … 07_generate_executive_report.py
│ ├── execute_pipeline.py execute_multi_project.py
├── projects/ # Typed program subfolders (`discover_projects`)
│ ├── templates/ # Public exemplars (git-tracked)
│ │ ├── template_active_inference/
│ │ ├── template_autoresearch_project/
│ │ ├── template_code_project/
│ │ ├── template_prose_project/
│ │ └── template_template/ # Present manuscript (`manuscript/` here)
│ ├── active/ # Hot-seat rendered set (symlinked, private)
│ ├── working/ # Non-rendered backburner (symlinked, private)
│ ├── published/ # Non-rendered published (symlinked, private)
│ ├── archive/ # Non-rendered retired (symlinked, private)
│ └── other/ # Non-rendered misc (symlinked, private)
├── docs/ (18 top-level areas, 390+ markdown files per live counter)
├── tests/ # Infra suites (447+ files)
├── AGENTS.md / README.md / CLAUDE.md / pyproject.toml
├── run.sh / secure_run.sh
└── output/ … # Mirrors after copy stage
See docs/_generated/active_projects.md for regenerated
slugs
(uv run python scripts/generate_active_projects_doc.py).
The full public exemplar roster is:
templates/template_active_inference,
templates/template_autoresearch_project,
templates/template_autoscientists,
templates/template_code_project,
templates/template_gold_refinement,
templates/template_literature_meta_analysis,
templates/template_madlib,
templates/template_newspaper,
templates/template_prose_project,
templates/template_sia,
templates/template_template,
templates/template_textbook. The three rows below are a
representative sample; a future exemplar_summary_table
token in build_manuscript_metrics_dict would let this table
cover every exemplar without hand-editing.
| Project slug | Purpose | Highlights | Tests | Figures (Stage 02 hint) |
|---|---|---|---|---|
template_code_project |
Optimization tutorial | Convex demo figures, scripted tables | 231 @ 90%+ gate | Controlled matplotlib exports |
template_prose_project |
Prose-heavy workflow | Validates narrative-only repos | 120 | Lightweight / optional plots |
template_autoresearch_project |
AutoResearch readiness | Planner + validation CLI | 296 | Readiness reports from Stage 02 |
Meta manuscript location: introspective study lives
in projects/templates/template_template/ beside the public
exemplar set. Discovery now follows the typed projects/
layout—projects/templates/** and
projects/active/** are discovered/rendered, while
projects/working/**, projects/published/**,
projects/archive/**, and projects/other/**
remain non-rendered—see root CLAUDE.md for invocation
patterns (resolve_project_root).
The repository maintains documentation at three levels:
| Level | Files | Purpose |
|---|---|---|
| Repository root | AGENTS.md,
CLAUDE.md, README.md,
RUN_GUIDE.md |
Global navigation and AI agent context |
docs/ directory |
390 files across 18 subdirectories | User guides, API reference, troubleshooting |
| Per-directory | AGENTS.md +
README.md at every directory |
Documentation Duality standard |
| Per-module (Tier 3) | SKILL.md at every
infrastructure module |
Machine-parseable MCP-aligned skill descriptor |
| Infrastructure-level (PAI) | PAI.md at
infrastructure/ directory |
Personal AI Infrastructure integration contract |
The docs/ subdirectories cover: core/
(essential docs), guides/ (skill levels 1–12),
architecture/ (system design), usage/ (content
authoring), operational/ (build, config, logging,
troubleshooting), reference/ (API, FAQ, glossary),
modules/ (23 infrastructure modules),
development/ (contributing, testing),
best-practices/ (version control, migration),
prompts/ (20 AI prompt templates), security/
(steganography, hashing), and audit/ (review reports).
Every count in this appendix is injected from live repository
introspection rather than hand-maintained: 390 counts every
Markdown file beneath docs/ recursively, 18
counts its first-level subdirectories, and 20 counts the
workflow subdirectories that each carry a SKILL.md
descriptor. This is the same discipline the manuscript argues for
throughout—a hand-typed “90+ files across 12 subdirectories” silently
rots as the tree grows, whereas a token re-resolves on every render. A
reader onboarding to the repository should start at
docs/core/, follow the graduated docs/guides/
skill ladder, and consult the per-directory
AGENTS.md/README.md pair nearest to whatever
code they are editing; AI agents additionally read each module’s
SKILL.md to locate capabilities without guessing API
signatures.
Symbol key (applies to all cells): Y = full native support · ~ = partial or plugin-based · N = absent. See also Figure 4 for a colour-coded heatmap rendering of this table.
| Capability | template/ |
Snakemake 9 | Nextflow 25 | CWL 1.2 | Quarto 1 | Jupyter Book 2 | R Markdown | DVC 3 | Overleaf (2025) | OpenAI Prism |
|---|---|---|---|---|---|---|---|---|---|---|
| Pipeline orchestration | Y | Y | Y | Y | ~ | N | N | Y | N | N |
| Manuscript rendering | Y | N | N | N | Y | Y | Y | N | Y | Y |
| Testing enforcement | Y | N | N | N | N | N | N | N | N | N |
| Coverage thresholds | Y | N | N | N | N | N | N | N | N | N |
| Cryptographic provenance | Y | N | ~¹ | N | N | N | N | ~² | N | N |
| Steganographic watermarking | Y | N | N | N | N | N | N | N | N | N |
| Multi-project management | Y | N | N | N | N | N | N | N | ~ | ~ |
| AI-agent documentation | Y | N | N | N | N | N | N | N | ~ | ~ |
| Agentic skill protocol (SKILL.md / MCP) | Y | N | N | N | N | N | N | N | N | N |
| Interactive TUI | Y | N | N | N | N | N | N | N | N | N |
| Zero-mock policy | Y | N | N | N | N | N | N | N | N | N |
| Container support | N | Y | Y | Y | N | N | N | N | N | N |
| Distributed execution | N | Y | Y | Y | N | N | N | ~³ | N | N |
| Multi-language (R/Julia) | N | Y | N | Y | Y | Y | Y | Y | N | N |
¹ Nextflow 25.04.0 introduced data-lineage provenance tracking (build-level, not document-level). ² DVC provides content-addressed versioning for data artifacts via its object store. ³ DVC integrates with remote storage (S3, GCS, Azure) but does not natively orchestrate distributed compute. ⁴ Overleaf and OpenAI Prism are collaborative cloud LaTeX/AI writing environments; their AI features (GPT-5.2 for Prism, Overleaf Labs AI for Overleaf) are partial/early-stage as of 2025–2026.
Release: v1.0.9 · DOI
10.5281/zenodo.20419007 · SHA-256
535bd80943d0… · pairing complete
Prior: v1.0.7 ·
10.5281/zenodo.20419007 · cc674248… ·
v1.0.8 · 10.5281/zenodo.20932076 ·
b9bc5cf3…