Architecture Decision Records

• 1: [0001] Cross-Environment Topology
• 2: [0002] Tenant Workload Placement Policy
• 3: [0003] Definitions Repository & Tracked-Changes Substrate
• 4: [0004] Rebuild Orchestrator & Phase Model

1 - [0001] Cross-Environment Topology

Parent capability: Self-Hosted Application Platform Addresses requirements: TR-03, TR-17, TR-01, TR-04, TR-18

Context and Problem Statement

TR-03 forces the rebuild’s first phase to establish foundations across both a public-cloud environment and a home-lab environment plus the connectivity between them; single-environment standup is explicitly not a supported outcome. TR-17 requires each tenant to receive compute, persistent storage, internal and external network reachability, identity, backup/DR, and observability — implemented as shared platform offerings. Neither TR names a topology.

The platform therefore needs a decided foundational environment shape: what sits on the public-cloud side, what sits on the home-lab side, and how the two connect. This decision is capability-scoped by confirmed framing — the platform owns its cross-environment topology, and every capability hosted on the platform inherits whatever shape this ADR exposes rather than deciding its own. The topology exposed to end users may differ from the platform-internal topology decided here.

The open question is whether the platform inherits the shape the repository already realizes — an Internet-facing edge (mutual-auth + traffic-control duties) in front of a public-cloud anchor, with a secure tunnel back to a private home lab — or selects a different shape for its cross-environment foundations.

Decision Drivers

• TR-03 — both environments and the link between them are phase-1 foundations, not afterthoughts; a single-environment shape is disqualified outright.
• TR-17 — the shape must provide a clean external reachability tier for tenant applications, cross-environment reachability for platform operation, and must not concentrate backup/DR so narrowly that a single environment’s loss is unrecoverable. (TR-17’s internal reachability — between tenants — is not satisfied by the operations plane; see the amendment note under Decision Outcome.)
• TR-01 / TR-02 (def, rebuild) — the whole topology must be expressible as version-controlled definitions and rebuildable end-to-end within 60 minutes; reusing shapes already realized as reproducible definitions is favored over shapes that must be built from scratch.
• TR-04 (teardown) — each environment and the connectivity between them must be independently teardown-able at a phase checkpoint.
• TR-18 (admissibility) — the edge and tunnel components must allow configuration control, data export, and credential revocation/rotation without vendor cooperation; more vendor surface is more admissibility risk.
• CLAUDE.md house pattern — Internet → Cloudflare (mTLS + DDoS) → Home Lab ↔ GCP (WireGuard) is the repository’s documented inter-environment topology, already realized in cloud/ (mtls/cloudflare-gcp, vpc-network allow-wireguard, network-load-balancer UDP gateway). Departing from it requires explicit justification.
• Capability tiebreaker — reproducibility beats vendor independence beats minimizing operator effort.
• Operator-effort asymmetry between environments (not TR-derived — recorded as an honest driver). Managed public-cloud compute and datastore primitives deliver much of the TR-17 inventory without operator-built machinery, where the home-lab equivalents must be built and operated. Their current low-to-zero cost at this platform’s scale sharpens the asymmetry. No TR ranks hosting cost, so this driver may motivate the option set but cannot by itself justify the outcome — it is subordinate to the TR-anchored drivers above and to the tiebreaker.

Considered Options

Option A — Inherit the three-tier shape (edge → public-cloud anchor → secure tunnel → private home lab)

Keep the shape the repository already realizes: an Internet-facing edge tier carrying mutual-auth and traffic-control (DDoS) duties, a public-cloud environment as the anchor, and a secure tunnel back to the private home lab. Tenant workloads may be placed in either environment (see the placement sub-decision in the outcome below).

• Satisfies TR-03: two environments plus their connectivity, all already foundation-phase concerns.
• Strongest on TR-01/TR-02: the definitions already exist and are already reproducible, so it is the shortest path to a ≤60-minute rebuild.
• Provides the dedicated external-reachability + scrubbing tier that TR-17 external reachability wants, while the tunnel carries cross-environment reachability for platform operation.
• Ranks highest on the reproducibility tiebreaker (reuse of proven definitions).
• Cost: carries the most vendor surface (a dedicated edge vendor), the weakest position on TR-18 and the vendor-independence tiebreaker — the edge vendor must pass the TR-18 admissibility test (config control, export, credential rotation without vendor cooperation).

Option B — Two-environment, edge-less (public-cloud ingress fronts directly)

Keep both environments and the tunnel, but drop the dedicated Internet-facing edge tier; the public-cloud ingress/load balancer terminates external traffic and mutual-auth directly.

• Still satisfies TR-03.
• Better on TR-18 and TR-02 (one fewer vendor, fewer moving parts to rebuild).
• Weakens TR-17 external reachability: loses the edge scrubbing/DDoS tier and pushes all external reachability onto the cloud ingress. Diverges from the CLAUDE.md house pattern and would need that divergence justified.

Option C — Home-lab-primary, cloud-as-thin-edge (inverted anchor)

Invert the anchor: all stateful tenant offerings live home-side; the public cloud shrinks to a minimal always-on relay for external reachability and tunnel termination.

• Satisfies TR-03; ranks highest on the vendor-independence tiebreaker (the cloud becomes a replaceable relay).
• Concentrates blast radius and TR-17 backup/DR on the home lab; external reachability degrades during a home-lab outage; greater distance from the current definitions hurts TR-02 reproducibility speed. Loses on the reproducibility tiebreaker.

Option D — Single-environment (all-cloud or all-home-lab)

• Rejected by TR-03, which explicitly makes single-environment standup an unsupported rebuild outcome. Recorded here so the reason the simplest shape is off the table is auditable.

Decision Outcome

Chosen option: Option A — inherit the three-tier shape, because it satisfies TR-03 directly, is the fastest and most faithful path to the TR-01/TR-02 reproducibility target (its definitions already exist and are proven in cloud/), and wins the capability’s stated reproducibility-first tiebreaker. The TR-18 vendor-surface cost is accepted as a bounded, downstream admissibility check on the edge and tunnel vendors rather than a reason to rebuild the shape from scratch.

The two cross-environment paths are strictly separated planes, and this separation is part of the decision:

• Application data plane (end-user traffic). Deployed tenant applications are reached by end users only through the Internet-facing edge, regardless of which environment hosts them: end user → edge (mTLS + DDoS) → tenant application. This is the external reachability of TR-17. Tenant application traffic never traverses the operations tunnel.
• Operations / maintenance plane. The public-cloud ↔ home-lab tunnel (today WireGuard) exists solely for platform operation and maintenance — the operator’s control of the home-lab environment from the public-cloud side. It carries no tenant application traffic.

Amended 2026-07-18. This sub-decision previously described the operations tunnel as “the internal reachability of TR-17.” That was a misreading, corrected here without changing what the ADR decides. The capability defines TR-17’s internal reachability as reachability between tenants; the tunnel provides platform-internal reachability — the operator reaching the home lab — which is a different thing. The two planes decided above are unchanged and remain correct. What the correction exposes is that tenant-to-tenant reachability across environments has no plane assigned to it: not the tunnel (excluded by rule above), and not the edge (the end-user plane). That gap is now carried as an open question below. It is currently unexercised — no UX in this capability describes tenants calling each other — but it constrains placement, and ADR-0002 depends on it.

Workload placement sub-decision: both environments are valid tenant-hosting targets.

The initial reading of this topology treated the home lab as the sole host for tenant workloads, with the public-cloud anchor limited to edge-facing reachability and the operations plane. That reading is widened here: the public-cloud anchor is also a first-class tenant-hosting environment, because the cloud side already offers managed compute and datastore primitives that satisfy the TR-17 inventory (compute, persistent storage, backup/DR, observability) with materially less operator-built machinery than the home-lab equivalents — which serves the tiebreaker’s third term (minimizing operator effort) without spending anything on the first two.

The consistency rule is what keeps this from fragmenting the topology: placement changes where a workload runs, never how it is reached. A cloud-hosted tenant application is not exposed directly via the cloud provider’s public ingress; it sits behind the same Internet-facing edge as a home-lab-hosted one, so mutual-auth and traffic-control duties stay in exactly one tier and the TR-17 external-reachability story is identical in both environments.

Which environment a given tenant lands in is a placement policy, deliberately not decided here — it is its own decision, scoped to its own ADR.

Offering parity sub-decision: environments may expose unequal offering sets; placement is constrained by tenant need.

TR-17 requires each tenant to receive the full inventory, implemented as shared offerings. It does not require every offering to exist in every environment. This ADR reads it accordingly: an environment is a valid target for a given tenant when it carries every offering that tenant actually needs, not when it mirrors the other environment’s catalog.

The alternative — full parity as a precondition for either environment being a valid target — was rejected because it makes every offering added later a double build before it can ship at all, converting the duplicated-surface cost recorded below from a bounded cost into an unbounded one. Deliberately unequal environments keep that cost proportional to demand: an offering is built in the second environment when a tenant needing it is placed there, not in advance.

What this buys is bounded duplication; what it costs is that placement becomes a matching problem. The platform must therefore know, per environment, which offerings exist, and must refuse a placement whose tenant needs an offering the target environment lacks — a rejection is a correct outcome, not a failure. Making that inventory legible is a tech-design obligation, and the placement-policy ADR consumes it as an input.

Migration sub-decision: placement is fixed at onboarding; cross-environment migration is not a supported platform operation.

The consistent-reachability rule above makes moving a tenant between environments possible by construction — its reachability story does not change when its placement does. This ADR declines to make it supported. Moving a tenant means tearing it down and re-onboarding it in the other environment: an operator-run exercise using the ordinary onboarding path, with no platform-guaranteed data-migration path and no declared downtime budget.

The reason is that first-class migration would obligate every stateful offering, in both environments, forever to expose a matching export/import path — a permanent tax on every future offering, paid to serve a case the platform has no demonstrated demand for. Reachability continuity keeps the door open: if migration is later wanted, this decision is reversible without revisiting the topology, because nothing here bakes placement into how a tenant is reached.

Origin-path sub-decision: the edge→origin hop is mutually authenticated in both environments.

“Reachable only through the edge” is stated above as a property; it is enforced here rather than left as a convention. In both environments the edge→origin hop presents client certificates the origin validates against the edge vendor’s origin-pull trust anchor, and the origin refuses any request that does not present one — so bypassing the edge and reaching an origin directly fails at the TLS layer, not at a firewall rule that could be relaxed.

The public-cloud anchor’s realization of this already exists: cloud/mtls/cloudflare-gcp/ issues the origin keypair, fetches the edge vendor’s authenticated-origin-pull CA trust anchor, and stores both in Secret Manager for the origin to consume. The home-lab side must reach the same posture rather than a weaker one; the two differ in realization, not in strength. Deployment surface that terminates edge traffic without this enforcement is non-conforming to this ADR.

Sequencing sub-decision: the operations tunnel is its own phase-1 checkpoint.

Within rebuild phase 1, the edge and public-cloud anchor stand up first, and the operations tunnel is a distinct checkpoint with its own TR-04 teardown — not one atomic foundations unit. Standing the tunnel up separately follows the plane separation this ADR already draws: the tunnel is operations-plane machinery, and folding it into the same unit as data-plane foundations would couple two things the decision deliberately keeps apart.

The operative benefit is TR-02: a tunnel that fails to come up is torn down and retried against an intact cloud anchor, instead of costing a full foundations teardown and restarting the 60-minute budget. The cost is one more checkpoint boundary to define and keep deterministic. Phase 1 is still not complete until every checkpoint in it has passed — separate checkpoints subdivide the phase, they do not weaken TR-03’s both-environments-plus-connectivity bar.

Home-lab definitions sub-decision: a peer top-level definitions surface, tooling undecided.

The home-lab foundations are expressed as version-controlled definitions in the same tracked-changes repository, as a peer top-level surface alongside cloud/, exposing a teardown entry point per phase. That is the TR-01/TR-04 property this ADR fixes: home-lab state is not a second-class, hand-managed environment, and it is not a separate repository.

Which tooling realizes it is deliberately not decided here. Terraform is the repository’s only definitions precedent, but it is a weak fit for the bare-metal and OS-level state the home lab actually carries, and this ADR is at topology altitude — committing to a tool would pre-empt a component design that is better placed to weigh it. The constraint passed downstream is the property, not the mechanism: same repository, peer surface, deterministic per-phase teardown.

flowchart LR
    user([End user])
    subgraph edge[Internet-facing edge]
        cf[mTLS + DDoS / traffic control]
    end
    subgraph cloud[Public-cloud anchor]
        capp[Tenant applications]
        ops[Operator / platform ops]
    end
    subgraph home[Private home lab]
        happ[Tenant applications]
    end

    user -- application data plane --> cf
    cf -- application data plane --> capp
    cf -- application data plane --> happ
    ops -- operations plane only<br/>secure tunnel --- happ

Both hosting targets sit behind the same edge; no tenant application is reachable by bypassing it.

Vendor admissibility (TR-18)

The shape above commits to two vendor-bearing layers. Both were tested against TR-18 — configuration control through the tracked-changes surface, portable data export, and credential revocation/rotation without vendor cooperation — and both are admissible, with two residual risks recorded rather than waved off.

Edge vendor (today Cloudflare) — admissible.

• Configuration control. Edge configuration is driven by the cloudflare provider from the definitions repository (cloud/mtls/cloudflare-gcp/), so it is already inside the tracked-changes surface TR-01 requires. No dashboard-only step is load-bearing.
• Data export. The edge is a proxy and holds no platform or tenant data of record — the export obligation is close to vacuous by construction. What it does hold, DNS zone state, is expressible as definitions and exportable independently of the vendor.
• Credential revocation/rotation. API tokens are revocable by the operator alone. The origin keypair is generated operator-side (tls_private_key), never by the vendor, so rotation is a local operation and the private key is never vendor-held.

Residual risk 1 — origin certificates are edge-vendor-issued. The origin certificate comes from the edge vendor’s origin CA and is trusted only by that vendor’s edge. Departing the edge vendor therefore means re-issuing origin certificates from another CA and re-pointing DNS. This is migration work, not a lock-in that prevents departure — it fails no limb of TR-18 — but it is the concrete cost of the “largest vendor-surface commitment” recorded below, and it should be sized rather than discovered.

Residual risk 2 — the trust anchor is fetched live from the vendor. The authenticated-origin-pull CA trust anchor is retrieved over HTTP from a vendor-hosted URL at apply time. A rebuild therefore has a runtime dependency on vendor availability, which sits awkwardly against the TR-02 rebuild guarantee. Vendoring the trust anchor into the definitions repository, with a tracked update path, removes the dependency; the tech design owns that change.

Tunnel (today WireGuard) — admissible, and the least vendor-bearing layer.

WireGuard is self-hosted and configuration-file-driven, with keys generated locally: all three TR-18 limbs are satisfied without a vendor in the loop at all. The admissibility question properly attaches to the endpoints carrying it — the public-cloud side (already definitions-driven via cloud/vpc-network/ and cloud/network-load-balancer/) and the home-lab-side network appliance terminating it.

Residual risk 3 — the home-lab tunnel endpoint is configured out-of-band. Today the home-lab side of the tunnel is GUI-configured on the network appliance and documented as prose, which means it is outside the tracked-changes surface and is drift by TR-01’s definition. This does not make the tunnel inadmissible — configuration control is fully available to the operator, which is what TR-18 tests — but it is an open TR-01 gap, and closing it is part of the home-lab definitions surface decided above.

Consequences

• Good, because the foundations phase reuses already-reproducible cloud/ definitions, keeping the TR-02 ≤60-minute rebuild target reachable and honoring the reproducibility tiebreaker.
• Good, because separating the application data plane (edge) from the operations plane (tunnel) gives TR-17 a clean external-reachability tier and a distinct platform-operations tier, and prevents tenant traffic from ever depending on the operations tunnel.
• Bad, because that same separation leaves TR-17’s internal (tenant-to-tenant) reachability unassigned across environments — the tunnel is excluded by rule and the edge is the end-user plane. Two interacting tenants are therefore safely placed only in the same environment until this is decided (see Open Questions), and whole-tenant placement in ADR-0002 is forced rather than merely preferred.
• Good, because routing both environments’ tenant traffic through the one edge keeps mutual-auth and traffic-control duties in a single tier — a tenant’s reachability story does not change when its placement changes, and placement stays a migratable property rather than a baked-in commitment.
• Bad, because the dedicated edge vendor is the largest vendor-surface commitment, making it the weakest point against TR-18 and the vendor-independence tiebreaker; it must clear the TR-18 admissibility test.
• Bad, because two valid hosting environments means TR-17 offerings may have to be realized in both — the requirement is that offerings are shared across tenants, not that they exist once, and a per-environment implementation of each offering is real duplicated surface. The offering-parity sub-decision bounds this cost rather than eliminating it: duplication is paid per offering per environment on demand, not up front for the whole inventory. Divergence between two implementations of the same offering remains a live risk the tech design must contain.
• Bad, because unequal environments make placement fallible in a way full parity would not: a tenant needing an offering the target environment lacks cannot be placed there, and the platform must be able to say so rather than discover it during provisioning. This pushes a per-environment offering inventory into the tech design as a hard requirement.
• Bad, because declining migration support means a placement decision made at onboarding is, in practice, durable for the tenant’s lifetime — a wrong call is corrected by teardown and re-onboarding, with whatever data loss or downtime that entails borne as an operator exercise. Accepted because reachability continuity keeps the decision reversible later without reopening the topology.
• Good, because separating the operations tunnel into its own phase-1 checkpoint keeps a tunnel failure from costing a full foundations teardown, protecting the TR-02 rebuild budget at the price of one additional checkpoint boundary.
• Good, because the edge and tunnel layers were tested against TR-18 and cleared it, converting the accepted vendor-surface cost from an open risk into three named, ownable obligations (see Vendor admissibility above).
• Bad, because hosting tenant workloads on managed cloud primitives deepens exposure to a single cloud provider, pressing on TR-18 (data export, credential rotation without vendor cooperation) more than a home-lab-only placement would. This is accepted because the capability tiebreaker ranks reproducibility above vendor independence, but it is a real concession and each managed primitive admitted as a tenant-hosting offering must pass TR-18 on its own.
• Requires: a downstream component design realizing the home-lab definitions surface decided above — same repository, peer to cloud/, deterministic per-phase teardown. The shape is settled here; the tooling is that design’s to choose, and there is no cloud/ analog to copy.
• Requires: a per-environment offering inventory the placement path can consult, so a placement whose tenant needs an absent offering is refused rather than half-provisioned.
• Requires: the home-lab edge→origin hop to enforce origin-pull client-certificate validation, matching the posture cloud/mtls/cloudflare-gcp/ already gives the cloud anchor.
• Requires: the edge trust anchor to be vendored into the definitions repository with a tracked update path, removing the rebuild-time dependency on vendor availability (residual risk 2).
• Requires: the home-lab tunnel endpoint’s configuration to be brought inside the tracked-changes surface as part of the home-lab definitions surface, closing the TR-01 drift gap (residual risk 3).

Realization

• cloud/mtls/cloudflare-gcp/ — the Internet-facing edge trust; the application data plane’s entry point and the realization of the mutually-authenticated edge→cloud-anchor origin path decided above. Issues the operator-held origin keypair, obtains the origin certificate, and stores both plus the authenticated-origin-pull CA trust anchor in Secret Manager for the origin to validate against.
• cloud/https-load-balancer/, cloud/ip/, cloud/dns/ — external reachability plumbing behind the edge.
• cloud/vpc-network/ (allow-wireguard firewall tag) and cloud/network-load-balancer/ (UDP gateway) — the operations-plane tunnel endpoints on the public-cloud side.
• cloud/rest-api/, cloud/https-load-balancer/, cloud/internal-application-load-balancer/ (Cloud Run backends and their network endpoint groups) and cloud/firestore/ — the public-cloud-side tenant-hosting offerings: managed compute and persistent storage for cloud-placed tenants, fronted by the edge rather than exposed directly.
• Home-lab-side platform offerings (tenant compute/persistent storage per TR-17) — not yet realized; a downstream component design owns the peer definitions surface decided above, its tooling, its per-phase teardown, its edge→origin mutual-auth enforcement, and bringing the home-lab tunnel endpoint’s configuration in-repo. It need not mirror the public-cloud offering set (see the offering-parity sub-decision), but must declare what it does offer so placement can be constrained against it.
• tech-design.md (composed later by plan-tech-design) will fold this shape into the final-state narrative alongside the other accepted ADRs.

Open Questions

One remains, surfaced by the 2026-07-18 amendment. The five realization questions this ADR originally carried are resolved and folded into the sections above; the sixth — tenant placement policy — was handed to its own ADR and has since been decided there.

• Tenant-to-tenant reachability across environments (TR-17). Open. TR-17 requires internal reachability, which the capability defines as reachability between tenants. This ADR’s plane separation assigns it no path across environments: the operations tunnel carries no tenant application traffic, and the edge is the end-user data plane. Two tenants that must reach each other are therefore only safely co-placed in a single environment. Deciding this means choosing between routing tenant-to-tenant traffic through the edge (consistent with “reachable only through the edge”, but sends internal traffic out to the Internet and back), opening a third cross-environment plane distinct from the operations tunnel (preserves the tunnel’s operations-only rule at the cost of a new plane to secure and reproduce), or accepting the constraint as a placement rule and never splitting interacting tenants (free, but silently narrows placement as the tenant set grows). Not decided here, because it is currently unexercised — no UX in this capability describes tenants calling each other — and choosing a plane on a hypothetical would be guessing. The trigger to decide it is the first tenant that declares a dependency on another tenant.

Resolved

• Tenant placement policy (TR-17). → Decided in ADR-0002. This ADR established that both environments are valid hosting targets, that placement is constrained by offering availability, and that the choice is durable; it deliberately left how a tenant’s environment is chosen to a separate decision bearing on the onboarding and modify flows rather than the topology shape. ADR-0002 resolves it: placement is resolved mechanically from the resource needs the tenant already declares, matched against the per-environment offering inventory this ADR required, defaulting to the public-cloud anchor when both environments qualify, with a recorded operator override — and it is an operator-internal detail rather than part of the platform contract.

• Edge/tunnel vendor admissibility (TR-18). → Both layers pass; three residual risks recorded as obligations. The edge vendor clears all three limbs (provider-driven config in the definitions repo, no data of record held, operator-generated keypair and independently revocable tokens); the tunnel is self-hosted with no vendor in the loop. Residual: origin certificates are edge-vendor-issued (departure is migration work, not lock-in), the trust anchor is fetched live from the vendor at apply time, and the home-lab tunnel endpoint is configured out-of-band (Vendor admissibility, Consequences).

• Home-lab definitions surface (TR-01/TR-04). → A peer top-level definitions surface in the same repository as cloud/, with a per-phase teardown entry point — tooling deliberately undecided. The property is fixed here so the home lab cannot become a hand-managed second-class environment; the mechanism stays with a component design better placed to weigh Terraform’s poor fit for bare-metal and OS-level state (Decision Outcome, Realization).

• Ops-plane sequencing within phase 1 (TR-03/TR-04). → Its own checkpoint. The edge and cloud anchor stand up first; the operations tunnel follows as a distinct checkpoint with its own TR-04 teardown. A failed tunnel is retried against an intact anchor instead of costing a full foundations teardown and the TR-02 budget. Phase 1 still completes only when every checkpoint passes (Decision Outcome).

• Cross-environment offering parity (TR-17). → No parity requirement; environments may expose deliberately unequal offering sets, and placement is constrained by what a tenant needs. Full parity would make every future offering a double build before it could ship at all. The cost moved onto placement: the platform must know each environment’s offerings and refuse placements it cannot satisfy (Decision Outcome, Consequences).

• Migration between environments (TR-17). → Not supported; placement is fixed at onboarding. Moving a tenant means teardown and re-onboarding through the ordinary path — an operator exercise with no platform-guaranteed data-migration path or downtime budget. First-class migration would tax every stateful offering in both environments forever for undemonstrated demand. Reachability continuity keeps the decision reversible later without reopening the topology (Decision Outcome, Consequences).

• Edge→cloud-anchor origin path. → Mutually authenticated in both environments, enforced at the TLS layer. The origin validates the edge’s client certificate against the vendor’s origin-pull trust anchor and refuses requests without one, so “reachable only through the edge” holds by enforcement rather than convention. Realized cloud-side today by cloud/mtls/cloudflare-gcp/; the home-lab side must match it (Decision Outcome, Realization).

2 - [0002] Tenant Workload Placement Policy

Parent capability: Self-Hosted Application Platform Addresses requirements: TR-17, TR-09, TR-10, TR-11, TR-19, TR-20, TR-21, TR-01, TR-54

Context and Problem Statement

ADR-0001 established that both the public-cloud anchor and the private home lab are valid tenant-hosting environments, that environments may expose deliberately unequal offering sets, and that a placement is fixed at onboarding because cross-environment migration is not a supported platform operation. It deliberately declined to decide how a given tenant’s environment is chosen, and handed that decision two inputs: the per-environment offering inventory the placement path must consult, and the fact that the choice is durable for the tenant’s lifetime.

This ADR answers what ADR-0001 left open:

• Who decides — the operator, the capability owner, or a rule the platform evaluates mechanically?
• On what basis — declared tenant needs, operator judgment, or a default-with-override?
• What is promised — is placement a visible part of the platform contract a tenant can rely on, or an operator-internal detail?

“When” is not open. ADR-0001’s migration sub-decision already fixed it: placement is chosen at onboarding and is durable; changing it means teardown and re-onboarding through the ordinary path. This ADR does not reopen that, and the consequence it inherits is recorded below — the modify flow (TR-21) never carries a placement delta.

The decision is capability-scoped: it bears on the onboarding and modify engagement flows, not on the topology shape, and every constraint it consumes is internal to this capability.

What the platform already knows about a tenant

The option set is bounded by what a tenant actually tells the platform. TR-09 fixes the declaration set at exactly four items — resource needs (compute, storage, network), the packaged artifact, the identity choice, and acceptance of the platform’s availability characteristics — declared in the tenant’s own tech design rather than at a separate acceptance gate.

No declaration is placement-shaped. There is no data-residency field, no latency or proximity requirement, and no hardware-access declaration anywhere in this capability. Where specialized hardware and regulatory constraints do appear — in the capability’s eviction rule and in the host-a-capability UX’s “the platform will never have GPUs” example — they are grounds for declining to host at all, not inputs that steer a tenant toward one environment. Any option that resolves placement from a declared location preference is therefore proposing a fifth declaration, not reading an existing one.

Decision Drivers

• TR-17 — every tenant receives the full inventory as shared offerings. Combined with ADR-0001’s offering-parity sub-decision, a placement is only valid if the target environment carries every offering that tenant needs; the platform must be able to refuse a placement it cannot satisfy rather than discover the gap during provisioning.
• TR-09 — the declaration set is a hard precondition of the provisioning gate. Growing it is a change to the platform contract, not a free extension.
• TR-11 — when a tenant needs an offering the platform lacks, the flow holds and resumes without a refile. This is the platform’s existing answer to an unsatisfiable need, and a placement policy should route into it rather than invent a parallel failure mode.
• TR-01 — every per-tenant binding must be expressible as a version-controlled definition. Whatever decides placement, the result is tracked state; a placement arrived at by unrecorded reasoning is drift by this requirement’s definition.
• TR-54 — every operator-facing surface is bounded by its share of a 2-hour weekly budget. The host-a-capability UX describes the onboarding review scope as deliberately narrow — exactly two questions — so adding a third is a real cost, not a formality.
• TR-19 / TR-20 / TR-21 — engagement is a single append-only thread per lifecycle event, typed by scope, with modify review restricted to the delta. Any placement negotiation has to land on that thread or it does not exist.
• Capability tiebreaker (full chain) — tenant adoption beats reproducibility beats vendor independence beats minimizing operator effort. Adoption at the top is the one term that argues for giving tenants a placement voice; reproducibility immediately below it argues for a rule in the definitions repository over per-tenant judgment.
• Capability Out of Scope — “Dictating the implementation.” “Homelab”, “Kubernetes”, and any specific stack are named as possible implementations of this capability, not part of its definition. Substrate is explicitly not a tenant-facing concept.
• Onboarding UX step 5 precedent — while the operator provisions, the capability owner does “nothing” and is explicitly “not pinged for DNS choices or secrets.” That is the repository’s archetype of a legitimately operator-internal provisioning decision.
• Counter-precedent — the identity choice. Identity is a tenant declaration, made in the tech design and “not a fresh question at onboarding.” A tenant-declared placement would have a working structural model to copy, so the case for Option B is real rather than straw.
• Anti-snowflake rule — provisioning must run the platform’s existing definitions, not hand-rolled per-tenant configuration; bespoke manual config is a Reproducibility-KPI failure, not a tolerable exception.

Considered Options

Option A — Operator judgment at provisioning time

The operator chooses the environment during step 5 of onboarding, alongside DNS names and secrets. Nothing is declared, no review question is added, and the outcome is recorded as a per-tenant binding.

• Satisfies TR-09 trivially — no schema change — and TR-10 without a new gate, since the choice rides on the authorization signal that already exists.
• Best possible position on TR-54: zero added operator-facing surface, and it fits the step-5 precedent exactly.
• Satisfies TR-01 only for the result. The reasoning is nowhere: why a tenant landed in one environment is unrecoverable, so a later operator cannot tell a deliberate placement from an arbitrary one. Over a tenant set this trends toward per-tenant judgment calls, which is the shape the anti-snowflake rule exists to prevent.
• Weak on TR-17: nothing forces the operator to consult the per-environment offering inventory ADR-0001 requires. A placement into an environment lacking a needed offering fails at provisioning rather than at review — precisely the outcome ADR-0001 called “a rejection is a correct outcome, not a failure.”
• Loses the reproducibility tiebreaker term to Option C, which expresses the same policy as a definition.

Option B — The capability owner declares a target environment

Placement becomes a fifth TR-09 declaration, made in the tenant’s tech design following the identity-choice precedent. The operator honors it or refuses it at review.

• Strongest on the tenant adoption tiebreaker term — the top of the chain. A tenant with a genuine environment-specific need gets a channel to express it instead of being routed to a decline.
• Fails TR-09 as currently written: the declaration set grows from four to five, and because the tech design is the contract acceptance, this is a platform-contract change rather than a form field. With live tenants it would require a TR-24 rollout; today it is cheap only because no tenant exists yet.
• Presses on TR-54: adds a third question to a review scope the UX describes as deliberately narrow.
• Contradicts ADR-0001 in a way that cannot be repaired here. A declared placement is a lever the tenant can pull once and never again — migration is not supported, so a tenant that later wants a different environment must be torn down and re-onboarded. Promising something the platform has no supported path to change is worse than not promising it.
• Collides with the capability’s “Dictating the implementation” Out of Scope entry by making substrate a tenant-facing, contractual concept.

Option C — Mechanical resolution from declared needs, with a default and a recorded override

The platform resolves placement from the resource needs the tenant already declares under TR-09, matched against each environment’s offering inventory: exactly one environment qualifies → that one; both qualify → a declared platform default; neither qualifies → the TR-11 hold or a decline. The operator may override, and the override is recorded with its reason.

• Satisfies TR-09 with no schema change and no contract change — it consumes the resource-needs declaration that already exists.
• Directly discharges TR-17 and ADR-0001’s stated obligation: the offering inventory becomes a load-bearing input, and an unsatisfiable placement is refused at review rather than discovered at provisioning.
• Strongest on TR-01 and the reproducibility tiebreaker term: the resolution rule is itself a version-controlled definition, so the placement and the reasoning that produced it are both tracked, and the same inputs reproduce the same placement on a rebuild.
• Good on TR-54: mechanical resolution costs no operator time per tenant, and it folds into step 2’s existing offering-alignment question rather than adding a third one. The override is an exception path, not routine work.
• Routes an unsatisfiable placement into TR-11’s existing hold-and-resume rather than a new failure mode, which is also what BR-64’s evolve-the-platform default demands.
• Cost: makes the per-environment offering inventory a blocking prerequisite — it must exist and be machine-readable before the first onboarding, where ADR-0001 left its timing open.
• Cost: weaker than Option B on tenant adoption, since a tenant with a real environment preference still has no channel to state it.

Option D — One default environment for every tenant

No per-tenant placement exists. Every tenant lands on the public-cloud anchor; the home lab hosts tenants only once it carries an offering the cloud lacks. A tenant the cloud cannot serve is a TR-11 hold or a decline.

• Best possible position on TR-54, TR-01, and TR-02 — there is no per-tenant placement state to define, record, or reproduce.
• Consistent with ADR-0001’s recorded honest driver: managed cloud primitives deliver much of the TR-17 inventory without operator-built machinery.
• Presses hardest on TR-18 and the vendor-independence tiebreaker term by concentrating every tenant’s data in one cloud provider — a concession ADR-0001 already flagged as real.
• Makes ADR-0001’s “both environments are valid tenant-hosting targets” true on paper but inert in practice, with no written path for the home lab to ever start receiving tenants.

Decision Outcome

Chosen option: Option C — mechanical resolution from declared needs, with a default and a recorded override.

It is the only option that satisfies TR-17’s refuse-what-you-cannot-satisfy obligation without spending anything on TR-09’s declaration set, and the only one that puts the reasoning behind a placement inside the TR-01 tracked-changes surface rather than leaving it in the operator’s head. It folds into the existing narrow review scope, so it costs TR-54 nothing per tenant, and it reuses TR-11’s hold rather than inventing a parallel failure path. Option B was rejected because it promises a tenant something ADR-0001 gives the platform no supported way to change; Option A because it satisfies TR-01 for the result but not the reasoning, trending toward exactly the per-tenant judgment the anti-snowflake rule forbids; Option D because it leaves ADR-0001’s second hosting environment with no written path to ever receive a tenant.

The resolution rule

Placement is resolved as a function of the tenant’s declared resource needs and the per-environment offering inventory:

flowchart TD
    needs[Tenant's TR-09 declared<br/>resource needs]
    inv[Per-environment offering<br/>inventory - ADR-0001]
    match{Which environments carry<br/>every offering this tenant needs?}
    one[Exactly one qualifies]
    both[Both qualify]
    none[Neither qualifies]
    useit[Place there]
    def[Place in the platform default:<br/>public-cloud anchor]
    hold[TR-11 hold — evolve the platform<br/>or 3c decline if out of scope]

    needs --> match
    inv --> match
    match --> one --> useit
    match --> both --> def
    match --> none --> hold

The rule is a version-controlled definition, not operator practice. This is the TR-01 property this ADR fixes: the same declared needs against the same inventory must produce the same placement, on a rebuild as on the original onboarding.

Sub-decision: the default when both environments qualify is the public-cloud anchor

The default has to be some environment, and cloud-first is the only default that is realizable today — ADR-0001’s realization section records the cloud-side tenant-hosting offerings as existing (cloud/rest-api/, cloud/firestore/, and the reachability plumbing) and the home-lab-side offerings as not yet realized. It also matches ADR-0001’s honest operator-effort driver.

This is deliberately the weakest-committed part of the decision. It is a single value in the definitions repository, it binds only tenants for whom both environments are valid — meaning the choice is by construction never the difference between a tenant being hostable and not — and reversing it requires no change to the rule, the declaration set, or the topology. If the home lab later grows enough offering surface to be the better default, flipping it is a one-line change, though per ADR-0001 it applies only to tenants onboarded afterward.

Sub-decision: placement is not part of the platform contract

Placement is an operator-internal detail. A tenant does not declare it, is not told it as a guarantee, and cannot rely on it. This follows the capability’s “Dictating the implementation” Out of Scope entry — substrate is not part of the capability’s definition — and the step-5 precedent, where the operator settles DNS and secrets without pinging the capability owner.

The distinction being drawn is between needs and location: a tenant declares what it needs, and the platform is accountable for meeting those needs (TR-17). Where it meets them is the platform’s business, exactly as the platform’s availability characteristics are something a tenant accepts rather than negotiates.

The concrete consequence is that the resolved placement, though recorded as tracked state under TR-01 and legible to the operator, is not a promise. Nothing in the tenant-facing contract prevents the platform from changing how the rule resolves for future tenants.

Sub-decision: an operator override exists and must be recorded with its reason

The operator may override the resolved placement. Without an escape hatch the rule would become the kind of surface TR-54 warns about — one whose failure mode is unbounded operator work to route around it — and TR-14 makes the operator the only principal on administrative surfaces anyway.

The override is bounded by two conditions that keep it from eroding the decision:

• It may not override a refusal. An override can only choose between environments that both satisfy the tenant’s needs. Forcing a tenant into an environment lacking a needed offering is not an override, it is a TR-17 violation, and the correct outcomes there remain the TR-11 hold or a decline.
• It must be recorded on the engagement thread with its reason, per TR-19 — on the onboarding issue, which is already the channel of record. An unrecorded override is Option A with extra steps, and reintroduces exactly the untracked reasoning that disqualified it.

A recurring override is a signal that the rule or the inventory is wrong, and should be answered by changing the definition rather than by repeating the override.

Sub-decision: the modify flow carries no placement delta

Inherited from ADR-0001, recorded here because it is this ADR’s flows that would otherwise be its home. Since placement is fixed at onboarding and cross-environment migration is unsupported, a TR-21 modify review never surfaces a placement change, and TR-20’s modify issue type needs no placement field.

A modify request can change a tenant’s declared resource needs — and those needs are the rule’s input. This ADR resolves the resulting question explicitly: a modify that would resolve to a different environment does not move the tenant. If the tenant’s new needs can be met in its current environment, they are met there. If they cannot, the outcome is a TR-11 hold to add the missing offering to the environment the tenant is already in — consistent with BR-64’s evolve-the-platform default — or, if that is out of scope, the ordinary decline. Re-resolution on modify would be migration by another name, which ADR-0001 forbids.

Sub-decision: placement is whole-tenant, and this is forced rather than preferred

A tenant’s components are placed together, in one environment. Independent per-component placement is not admissible.

This was originally left open on the grounds that whole-tenant was merely the conservative reading. It is not — ADR-0001 leaves it no alternative. Split placement requires intra-tenant traffic to cross environments, and ADR-0001’s plane separation gives that traffic no lawful path: the operations tunnel carries no tenant application traffic, and the edge is the end-user data plane. Split placement is therefore not a trade-off this ADR declines to make; it is unrealizable under an accepted ADR.

The TR-17 internal-reachability question this raises is broader than placement and is not resolved here. Establishing it exposed that ADR-0001 had mapped TR-17’s internal reachability onto the operations tunnel, when the capability defines internal reachability as reachability between tenants — a different thing. ADR-0001 was amended to correct the mapping and now carries the resulting gap as its own open question. The consequence this ADR inherits: two tenants that must reach each other can only be safely placed in the same environment, and the resolution rule has no way to express that today, because a tenant declares its own needs and not its dependencies on other tenants. This is acceptable while the tenant set is small and no tenant declares such a dependency; the first one that does is the trigger to revisit both ADRs.

Note that the host-a-capability UX reviews offering alignment per component, which is what made per-component placement look plausible. That review granularity is preserved and is not in tension with whole-tenant placement: every component must be satisfiable in the chosen environment, which is exactly the per-component check the UX already describes. Per-component review, whole-tenant placement.

Sub-decision: declared needs are authored machine-readably in the tenant’s own capability docs

The authoritative machine-readable declaration of a tenant’s resource needs lives with the tenant capability’s documentation, authored by the capability owner, and the platform reads it at onboarding.

This keeps BR-13 intact — the tenant declares, in its own design, and the declaration is the contract acceptance — while satisfying TR-01 at no extra cost, because capability documentation and platform definitions already share one tracked-changes repository. There is no transcription step, so the declaration the platform matches against cannot drift from the declaration the tenant made.

The rejected alternatives are recorded for the same reason the main option set is: operator-transcribed into the platform’s definitions was rejected because it makes the operator restate what the tenant declared, straining BR-13 and introducing exactly the divergence risk the shared repository otherwise eliminates; carried on the onboarding issue was rejected because issue state is not a version-controlled definition, so a rebuild could not reproduce the placement from it — a direct TR-01 failure, notwithstanding that TR-19 makes the issue the channel of record for the engagement.

The serialization format is not decided here and remains a tech-design concern. What is fixed is ownership and location: capability-owner-authored, in the tenant’s docs, in this repository. This is a representation obligation, not a change to what TR-09 requires be declared, so it is not a platform-contract change.

Sub-decision: the offering inventory subdivides on demand, starting at TR-17’s inventory

The per-environment offering inventory begins at the granularity TR-17 already names — compute, persistent storage, network reachability, identity, backup/DR, observability — and an entry is subdivided only when a real tenant declaration forces a distinction an environment cannot meet. A tenant needing GPU compute is what splits compute into a general and a GPU-bearing entry; the split does not exist in anticipation of that tenant.

This follows BR-64 — the platform evolves in response to a tenant need rather than ahead of one — and keeps the inventory bounded by TR-54, since every entry is surface the operator maintains. A designed taxonomy fixed up front was rejected as speculative: it grows maintained surface for distinctions no tenant has asked for, against a capability rule that explicitly does not oblige the platform to grow without bound. Letting each environment declare its own offering names at its own grain was rejected because it leaves no shared vocabulary — matching would degrade into string comparison across independently-chosen names, and divergence would be silent rather than caught.

The accepted cost is that the rule stays degenerate longer. At TR-17 granularity both environments either carry an offering or do not, so until a real tenant forces the first subdivision, every placement resolves to the default. That is the same degeneracy already recorded in Consequences, now with a decided mechanism for exiting it: the first tenant whose needs one environment cannot meet is what makes the inventory discriminating, and the TR-11 hold is what absorbs that tenant while the split is made.

Consequences

• Good, because a placement whose target environment lacks a needed offering is refused at review rather than discovered mid-provisioning, discharging the obligation ADR-0001 pushed onto this decision.
• Good, because the resolution rule is a definition rather than operator practice, so both the placement and the reasoning behind it are inside the TR-01 tracked-changes surface and a rebuild reproduces the same placements.
• Good, because it costs the TR-09 declaration set nothing — no fifth declaration, no platform-contract change, no third question added to a review scope the UX deliberately kept narrow.
• Good, because an unsatisfiable placement routes into TR-11’s existing hold-and-resume rather than a new failure mode, keeping BR-64’s evolve-the-platform default as the first response.
• Good, because keeping placement out of the contract avoids promising a property that ADR-0001 leaves the platform no supported way to change.
• Bad, because a tenant with a genuine environment-specific need has no channel to express it. The rule reads needs, not preferences, so such a tenant is routed to a decline that Option B would have avoided — a real cost against the adoption term at the top of the tiebreaker chain. Accepted because the capability currently treats specialized-hardware and regulatory needs as grounds for declining to host at all, so this option removes no channel that exists today; if such a tenant ever appears, that is the trigger to revisit, and BR-64 says the response is to evolve the platform.
• Bad, because the per-environment offering inventory becomes a blocking prerequisite for the first onboarding, where ADR-0001 left its timing open. The placement path cannot resolve anything without it.
• Bad, because today the rule degenerates to its default. No home-lab tenant offerings are realized yet, and “compute, storage, network” is not discriminating enough to separate the environments, so every current tenant resolves to the public-cloud anchor — making this operationally identical to Option D until the home lab carries something the cloud does not. This is accepted knowingly: the rule’s value is that the growth path is decided and written down before the first divergent offering exists, rather than being improvised under the pressure of a tenant that needs it. It also means the rule ships largely unexercised, and its first real test will be its first non-degenerate resolution.
• Bad, because concentrating every current tenant in the public cloud presses on TR-18 and the vendor-independence tiebreaker term, exactly as ADR-0001 recorded. Each managed primitive admitted as a tenant-hosting offering must still pass TR-18 on its own.
• Requires: a per-environment offering inventory, machine-readable and version-controlled, that the resolution rule consults — the input ADR-0001 named and this ADR makes blocking.
• Requires: the tenant’s declared resource needs to be captured in a form the rule can evaluate against that inventory, rather than as prose in a tech design. This is a TR-09 representation obligation on the tech design, not a change to what is declared.
• Requires: the resolved placement to be persisted as a per-tenant binding in the definitions repository (TR-01), so provisioning and any later rebuild consume it rather than re-deriving it.
• Requires: the onboarding engagement thread to carry the override and its reason when one is exercised (TR-19).

Realization

• Definitions repository (top level, alongside cloud/) — the per-environment offering inventory and the resolution rule live here as tracked definitions, not as service code. Placement is decided at onboarding by the operator running the platform’s definitions; it is not a runtime request path, so no service under services/ owns it and no HTTP endpoint, protobuf message, or pkg/errorpb problem type is introduced by this decision.
• Per-tenant bindings — the resolved placement is recorded as part of the tenant’s version-controlled binding, consumed by provisioning and reproduced on rebuild.
• cloud/rest-api/, cloud/firestore/, cloud/https-load-balancer/, cloud/internal-application-load-balancer/, cloud/ip/, cloud/dns/ — the public-cloud anchor’s tenant-hosting offerings, and therefore the cloud side’s entries in the offering inventory the rule reads.
• Home-lab definitions surface (peer to cloud/, tooling undecided per ADR-0001) — must declare what it offers so the rule can resolve against it. Until it declares a tenant-hosting offering, it qualifies for no tenant and the rule resolves to the default for every tenant.
• cloud/mtls/cloudflare-gcp/ — unchanged by this decision, and the reason it is unchanged is load-bearing: ADR-0001’s rule that placement changes where a workload runs and never how it is reached means the edge path is identical for both resolutions.
• tech-design.md (composed later by plan-tech-design) will fold the inventory, the resolution rule, and the per-tenant binding into the onboarding-flow narrative.

Open Questions

None remain in this ADR’s scope. The three questions it opened on 2026-07-18 were resolved the same day and folded into Decision Outcome as sub-decisions.

One question left this ADR’s scope rather than closing: resolving placement granularity established that ADR-0001 had misread TR-17’s internal reachability, and the real gap — tenant-to-tenant reachability across environments — is topology-altitude. ADR-0001 was amended to correct the mapping and now carries that gap as its own open question. What this ADR inherits from it is a constraint, not a question: two tenants that must reach each other are only safely placed in the same environment.

Resolved

• Placement granularity. → Whole-tenant, and forced rather than preferred. Split per-component placement would require intra-tenant traffic to cross environments, which ADR-0001’s plane separation gives no lawful path. Per-component review is preserved; per-component placement is not admissible (sub-decision).
• Representation of declared needs. → Machine-readable, authored by the capability owner, in the tenant capability’s own docs. Keeps BR-13 intact and satisfies TR-01 without a transcription step, because capability docs and platform definitions share one repository. Serialization format stays with the tech design (sub-decision).
• Offering-inventory granularity. → Starts at TR-17’s inventory; subdivides only when a real tenant declaration forces a distinction. Follows BR-64 and stays bounded by TR-54; the accepted cost is that the rule remains degenerate until the first such tenant, absorbed by the TR-11 hold (sub-decision).
• Who decides. → The platform, mechanically, from declarations the tenant already makes. Not the operator’s per-tenant judgment (Option A, rejected for leaving the reasoning untracked) and not the capability owner (Option B, rejected for promising a property ADR-0001 cannot let the platform change).
• On what basis. → Declared resource needs matched against the per-environment offering inventory, with the public-cloud anchor as the default when both environments qualify and an operator override bounded to cases where both already satisfy the tenant.
• What is promised. → Nothing. Placement is an operator-internal detail on the same footing as DNS names and secrets. The tenant declares needs and the platform is accountable for meeting them; where it meets them is not contracted.
• When. → Inherited from ADR-0001, not decided here: at onboarding, durably. This ADR adds the corollary that a modify request changing a tenant’s declared needs does not re-resolve placement — the missing offering is added to the tenant’s current environment, or the request is declined.

3 - [0003] Definitions Repository & Tracked-Changes Substrate

Parent capability: Self-Hosted Application Platform Addresses requirements: TR-01, TR-04, TR-05, TR-02, TR-18, TR-54, TR-06, TR-07

Context and Problem Statement

TR-01 makes a tracked-changes definitions repository the only authoritative surface for platform-modifying writes: every offering, every per-tenant binding, every shared piece of configuration must be expressible there, and anything modifiable outside it is drift. TR-04 requires each rebuild phase to expose a deterministic, definitions-driven teardown. TR-05 requires a preflight drift check against a last-known-good reference that fails closed.

None of the three name a repository, a mechanism, or a form for that reference. This ADR decides all three, because every later offering’s definitions surface sits on top of the answer.

Both accepted siblings have already assumed a substrate without deciding one. ADR-0001 fixed the home-lab foundations as “version-controlled definitions in the same tracked-changes repository, as a peer top-level surface alongside cloud/”. ADR-0002 rested its declared-needs sub-decision on the claim that “capability documentation and platform definitions already share one tracked-changes repository”, using that to eliminate a transcription step. Those assumptions are load-bearing and this ADR must either honor or explicitly reconcile them.

What exists today

The substrate is not greenfield. It is a module / root-module split across two repositories, and the seam is already drawn cleanly at the content level:

Three facts about this arrangement were verified rather than assumed, and each bears on the decision:

• The public repository is clean. cloud/* contains no real domain, IP, or project identifier — every module is fully parameterized and every real value arrives as tfvars from the private repository. The seam is not aspirational; it is holding.
• The repositories are joined by a module reference. The private repository’s root terragrunt.hcl sources modules as git::https://github.com/Zaba505/infra.git//${TF_INFRA_MODULE_PATH}?ref=${TF_INFRA_MODULE_REF}.
• That reference is untracked. Each per-component workflow supplies it as TF_INFRA_MODULE_REF: ${{ inputs.module_ref }} — a workflow_dispatch input typed by hand at dispatch time, per component. It is recorded in no definition, in either repository.

The third fact is the problem this ADR exists to fix. Which version of the platform’s modules gets applied is the most load-bearing value in any apply, and it lives only in a workflow run record. That is TR-01’s definition of drift verbatim, and it breaks TR-02 concretely: the operator whom the standup UX describes as arriving at minute zero with “the definitions repo, pulled fresh” cannot reproduce the platform from it, because the definitions do not contain the refs. They would be re-typing remembered values, per component, with nothing recording which combination was ever known-good.

The decision is capability-scoped: it fixes where this capability’s definitions live and what gate owns mutations to them. It does not decide the engagement channel (TR-19/TR-20), which is its own decision, nor how drift comparison is performed, which is the downstream drift-detection ADR’s.

Decision Drivers

• TR-01 — the definitions repository is the only authoritative surface for platform-modifying writes. Any value that determines what gets applied and lives outside it is drift by definition, not by degree.
• TR-02 — a fresh pull must be sufficient to drive an end-to-end rebuild inside 60 minutes. A substrate that requires remembered out-of-band inputs fails this before the clock starts.
• TR-05 — the preflight check needs a last-known-good reference it can compare against and fail closed on. A reference that is a tuple of independently-moving values is materially weaker than a single one, because no atomic commit binds the members together.
• TR-04 — per-phase teardown must be deterministic and definitions-driven. Teardown depends on applied state, so where that state lives is part of this decision rather than an implementation detail.
• TR-18 — every third-party component must allow config control, portable export, and credential rotation without vendor cooperation. Git scores near-perfectly: every clone is a complete export and the forge is replaceable. Forge-specific state is the residual.
• TR-54 — every operator-facing surface is bounded by its share of a 2-hour weekly budget. Each additional repository is another ruleset, another CI surface, and another thing to keep current.
• TR-14 — administrative surfaces admit only the operator. A private binding repository requires a standing write grant to anyone else who must contribute; a public one does not, because fork-and-PR needs no grant at all.
• ADR-0001 constraint — home-lab definitions are a peer surface, not a second-class hand-managed environment, with a per-phase teardown entry point.
• ADR-0002 constraint — tenant declarations are authored by the capability owner in the tenant capability’s own docs, with no transcription step. Any option that separates declarations from capability docs reintroduces the divergence that sub-decision rejected by name.
• Capability tiebreaker — tenant adoption beats reproducibility beats vendor independence beats minimizing operator effort. Reproducibility is the operative term here and it argues for a single sufficient reference.
• Confidentiality of aggregated operational data (not TR-derived — recorded as an honest driver). See the privacy sub-decision; the honest form of this driver is narrower than it first appears, and the ADR records the narrow form so it does not decay into security-by-obscurity.

Considered Options

Option A — Keep the split; make the module reference a tracked definition

Retain the two repositories and the content seam exactly as they stand. Change one thing: the private repository commits the infra module reference as a definition instead of accepting it as a workflow_dispatch input.

• Discharges the TR-01 violation directly and at its actual location — the untracked value moves inside the tracked-changes surface.
• Satisfies TR-02: a fresh pull of the private repository becomes sufficient, since homelab@X transitively determines infra@Y. Nothing must be remembered.
• Strongest realistic position on TR-05: the last-known-good reference collapses from a tuple to a single commit SHA on one repository.
• Costs TR-54 almost nothing — one committed value, no new repository, no new ruleset, no new CI surface.
• Honors ADR-0002 without amendment (declarations stay in capability docs) and ADR-0001 at the module layer (see the reconciliation note).
• Preserves the confidentiality position for the reasons recorded in the privacy sub-decision.
• Cost: the two repositories still exist, so TR-18’s forge-portability story must be told twice, and a contributor to declarations (public) is a different flow from a contributor to bindings (private).
• Cost: does not by itself close the ~20-month staleness gap between the repositories (recorded under Consequences).

Option B — Status quo: keep the split, keep the reference at dispatch

Change nothing.

• Fails TR-01 on the module reference — the value that determines what is applied is modifiable, and only modifiable, outside the repository.
• Fails TR-02’s pull-fresh-and-rebuild property: the definitions are insufficient, and the shortfall scales with the component count.
• Fails TR-05: there is no reference to compare against, so a preflight check has nothing to fail closed on. The check could not be built on this substrate at all.
• Best possible TR-54 position (zero change), which is the only thing it wins on.
• Recorded so the rejection is auditable rather than implied.

Option C — Vendor the modules into the binding repository

Drop the cross-repository source entirely; copy cloud/* into the private repository so one repository fully determines an apply.

• Strongest possible TR-05 and TR-02 position: a single repository, single clone, no cross-repository reference of any kind.
• Satisfies TR-01 trivially — there is nowhere else for a value to hide.
• Costs the public module surface and its reuse; cloud/* maintenance would be duplicated or abandoned.
• Contradicts ADR-0002. Declarations live in capability docs; if the binding repository is the sole substrate, either declarations move private (reintroducing the transcription step that sub-decision rejected) or the substrate is two repositories again and the option collapses into A.
• Presses on TR-54: vendored modules drift from upstream unless a sync ritual is maintained, and that ritual is operator work with no natural forcing function.

Option D — Collapse everything except docs into one private repository

Modules, declarations, bindings, and workflows all move private; only the published Hugo docs remain public.

• Single reference, single ruleset, single clone — strong on TR-05, TR-02, and TR-54.
• Contradicts ADR-0002 the same way Option C does, and more directly: declarations become private, so either capability owners receive standing write grants on the platform’s binding repository or the operator transcribes their declarations. The first strains TR-14’s posture; the second is exactly what ADR-0002 rejected.
• Loses the public module surface, which is the part of the current arrangement most clearly working.
• Would require an accepted ADR to be amended to proceed, which is a real cost and not merely procedural.

Decision Outcome

Chosen option: Option A — keep the split; make the module reference a tracked definition.

It is the only option that fixes the actual TR-01 violation while changing no content, requiring no new surface against TR-54, and contradicting no accepted sibling. Options C and D both buy a marginally stronger TR-05 reference by breaking ADR-0002’s no-transcription sub-decision — paying an accepted decision’s coherence for a property Option A already obtains by committing one value. Option B is disqualified outright: it does not merely score poorly on TR-05, it makes the required preflight check unbuildable, since there is nothing to compare against.

The substrate

flowchart TD
    subgraph pub["Zaba505/infra — public"]
        mods["cloud/* — reusable modules<br/>fully parameterized, no real values"]
        docs["capability docs<br/>+ tenant TR-09 declarations"]
    end
    subgraph priv["Zaba505/homelab — private"]
        pin["committed module ref<br/>(the tracked pin)"]
        binds["root modules, terragrunt.hcl,<br/>per-env tfvars, apply workflows"]
    end
    state[("Terraform state<br/>GCS, in-environment<br/>derived, never committed")]
    sec[("Secret Manager<br/>values, referenced by name")]

    pin -->|"determines"| mods
    binds --> state
    binds -.->|"by name only"| sec

    priv -->|"main@SHA = last-known-good"| ref((single sufficient<br/>reference))

Sub-decision: the module reference is a tracked definition, not a dispatch input

The private repository commits the infra module reference. TF_INFRA_MODULE_REF ceases to be a workflow_dispatch input supplied by hand and becomes a value read from the repository.

This is the whole point of the ADR. With it, the private repository’s main at a given commit fully determines what an apply produces; without it, no amount of structure elsewhere makes the definitions sufficient. It is also what makes TR-05’s preflight check possible rather than merely unimplemented.

Granularity: a single repository-wide pin, not per-component. The reference is one value covering every component, rather than eleven independently-set ones. Per-component pins would reconstruct precisely the tuple problem this decision exists to collapse — a rebuild could combine refs that were never validated together, and the last-known-good reference would again have no atomic representation.

The accepted cost is that a module change cannot be rolled out component-by-component: bumping the pin moves every component to the new modules at once. Staged rollout is available by pinning a branch of infra that carries only the intended change, which keeps the value single and tracked. Divergence by editing individual components’ refs is a deliberate escape from this sub-decision and, if ever exercised, must be recorded in-repo rather than typed at dispatch.

Sub-decision: the private repository’s main is the authoritative reference; last-known-good is a commit SHA

main in the binding repository — not any branch, working copy, or applied environment — is what the definitions are. The last-known-good reference for TR-05 is a commit SHA on that branch: specifically the SHA at which a rebuild’s canary (TR-07) last went green.

The existing ruleset supplies the immutability BR-51 asks for, at no cost: required signatures, required linear history, no force-push, no deletion. A commit SHA under those rules is a tamper-evident pointer to an exact definitions state, and git’s content addressing means comparison needs no separate manifest format.

What is fixed here is the form of the reference. How the SHA is captured on a green canary, where it is recorded, and how comparison against live infrastructure is performed are the downstream drift-detection ADR’s to decide. This ADR’s obligation was to make a single sufficient reference exist.

Sub-decision: applied state is derived, lives in the target environment, and is never committed

Terraform state is not a definition. It is a derived cache of the mapping between definitions and real resources, and this ADR classifies it as such: TR-01’s “entirely expressible as version-controlled definitions” is satisfied by layers 1–3 of the table above, and state is downstream of them.

State stays where it is — a GCS bucket in the target environment, per-component-prefixed — and is never committed to either repository. Two independent reasons, either sufficient: Terraform state routinely contains secret material in plaintext, and a drill (TR-06) runs the same definitions against different infrastructure, which is only coherent if state is a property of the target rather than of the definitions.

The consequence to carry forward is that TR-04 teardown depends on state durability. A phase’s deterministic teardown is driven by definitions but executed against state; losing state does not merely inconvenience a teardown, it strands resources the definitions can no longer address. State durability is therefore a teardown correctness concern, not an operational nicety.

Sub-decision: the state backend is bootstrapped by a definitions-driven phase 0 that adopts its own state

The sub-decision above creates a chicken-and-egg: the bucket holding applied state cannot be managed by the state it holds. It is resolved here rather than deferred, because the state-location decision is what creates it.

The bootstrap is a definitions-driven phase 0. A bootstrap root module in the binding repository — committed like any other definition, not a hand-run command — creates the state bucket with versioning and retention. It runs against a local backend on its first execution and then migrates its own state into the bucket it just created, after which it is an ordinary component managed exactly like every other. The chicken-and-egg is discharged once, at the only moment it exists.

Committing the bootstrap’s state to the binding repository instead was considered and rejected: a bucket-only state is unlikely to carry secret material, but the never-committed rule above admits no per-case exceptions without becoming a judgment call at every future component.

It is a new phase, and it counts against the TR-02 budget. The standup UX enumerates four phases beginning at foundations and has no provisioning step before the preflight check; phase 0 precedes TR-03’s foundations phase and is a required change to that UX. Excluding it from the 60-minute budget was considered and rejected: the budget is a target rather than a gate — missing it files a follow-up issue and does not stop the platform going into service — so exclusion protects nothing, while it would let the reproducibility KPI be measured against a rebuild that was not actually from scratch.

It is parameterized by target, which is what makes TR-06 literally true. A drill creates its own scratch bucket by running the same phase 0 against a different target. Without this, “drill mode and live mode differ only in the underlying target” would be true of everything except the one resource the drill most needs to not share.

Its TR-04 teardown exists but carries an ordering constraint: phase 0 tears down last, after every phase whose state it holds. Torn down out of order it strands precisely what the state-durability consequence above warns about. “Delete everything provisioned so far and start over” stays viable at every checkpoint; what is fixed here is the order in which that unwinds.

The residual is real and recorded: after adoption, the bootstrap module’s own state lives in the bucket that module manages. Losing the bucket therefore also loses the record of how to rebuild it. It is recoverable — the bucket is re-creatable and re-importable by re-running phase 0 — but recovery is a manual exception rather than the ordinary path.

Sub-decision: secrets live in Secret Manager and are referenced by name in both repositories

No secret value is committed to either repository, including the private one. Definitions carry references; TR-44’s secret-management surface carries values.

Stated explicitly because the private repository’s privacy makes the weaker discipline tempting, and because the privacy sub-decision below leans on secrets being absent from both surfaces rather than on one surface being unreadable.

Sub-decision: mutations are gated by pull request into main, under the existing ruleset

The tracked-changes mechanism is git plus the forge’s pull-request gate, under the ruleset already active on Zaba505/infra and to be matched on Zaba505/homelab: required signatures, required linear history, no force-push, no branch deletion, pull request required with zero required approving reviews — the last of which is what makes the mechanism viable for a platform whose TR-14 posture admits exactly one principal. A gate requiring a second reviewer would be unsatisfiable by construction.

The pull request is what gives TR-05’s fail-closed check, terraform fmt, and any future validation a place to attach before a change becomes authoritative. Direct signed commits to main were considered and rejected for exactly this: they satisfy TR-01 in form — still version-controlled, still signed, still immutable — while leaving no pre-merge point at which anything can be checked, and no proposed-versus-accepted artifact for an engagement thread to reference.

The administrative bypass is retained. The ruleset’s admin bypass (bypass_mode: always) stays in place as an emergency escape hatch, and the discipline of routing changes through the gate is a matter of operator convention rather than enforcement.

This is a deliberate choice and its cost is recorded rather than minimized: the operator can mutate main outside the gate, so TR-01 is enforced by convention rather than by construction, and TR-05’s preflight check cannot distinguish a bypassed change from genuine drift. Every bypass is indistinguishable from the failure mode the check exists to catch. If a bypass is ever exercised, that is the trigger to revisit this sub-decision — either by removing the bypass or by adopting the recorded-override discipline ADR-0002 uses for placement.

Sub-decision: the public/private seam is justified by aggregation, not obscurity

The seam is retained as it stands — modules, docs, and declarations public; bindings, per-environment values, and workflows private. The rationale matters as much as the outcome, because the obvious rationale is wrong.

Obscurity is explicitly not load-bearing. Domain names and public IP addresses are public by construction: Certificate Transparency publishes every hostname for which a publicly-trusted certificate is issued, and CT logs are monitored continuously; full-IPv4 scanning is cheap and constant. A hostname fronted by the edge is discoverable within minutes of certificate issuance whether or not it appears in a repository. An ADR recording “bindings are private so crawlers do not find the hosts” would be false on the day it was written.

Three narrower rationales do survive scrutiny, and they are the recorded basis:

• Aggregation. CT tells a scanner that a hostname exists. The binding repository would tell it the full stack, the wiring, which tenant sits where, and what is behind each name — assembled, with no reconnaissance. Each fact is individually discoverable; the collection is not.
• Pre-disclosure. A merged definition names a host before apply — before its defenses exist. CT cannot leak a certificate that has not been issued.
• Mistake blast radius. The binding layer carries secret references and may one day carry a mistake. In a public repository an accidentally-committed credential is harvested within seconds and is permanently public; in a private one it is a recoverable incident.

The actual control remains architectural, not confidential: per ADR-0001 the origin refuses any request that does not present an edge-issued client certificate, so traffic reaching an origin directly fails at the TLS layer. Discovery buys an attacker noise, not access — conditional on that posture holding, which ADR-0001 requires of both environments and which the home-lab side has not yet realized.

Sub-decision: forge-held non-git state is admissible under TR-18; rulesets are the live gap

Git gives TR-18 a near-vacuous answer for the definitions themselves, but the forge also holds state that is not git. That state is three unlike things, and TR-18’s three limbs must be applied to each separately rather than to “the forge” as a lump.

Rulesets are configuration, and they are the actual gap. TR-18’s limb (a) requires configuration control through the platform’s tracked-changes surface — not merely that the operator can change it. Rulesets are set through the forge’s web UI and recorded in no definition, which is drift by TR-01’s plain reading, the same failure class as ADR-0001’s residual risk 3.

This one is worth stating sharply because it is self-referential: the ruleset that enforces this ADR’s pull-request gate is itself an untracked value. The mechanism the substrate leans on for tamper-evidence is configured the same way the module reference was, and is drift for the same reason. Expressing both repositories’ rulesets as definitions is therefore an obligation of this decision, not a downstream nicety.

Workflow run history is derived, and this ADR is what makes that true. It was load-bearing only because the module reference lived in it and nowhere else — the violation this ADR exists to fix. Once the pin is a tracked definition, run history is operational telemetry downstream of the definitions, in the same class as applied state but without state’s teardown role. It carries no export obligation.

Issues are data of record, and limb (b) passes. The TR-19/TR-20 engagement thread is genuinely platform-held data, and the operator can export it in portable form through a documented API with no vendor approval, negotiation, or plan change. “Without vendor cooperation” means without needing the vendor’s permission — not without using the vendor’s interfaces. The stricter reading is rejected on consistency grounds as much as on merit: it would retroactively disqualify the edge vendor that ADR-0001 already admitted, since that vendor’s configuration is likewise driven through its API. An accepted sibling cannot be invalidated by a reading adopted here in passing.

What limb (b) demands is that the export be routine rather than theoretical. An export path that has never been run is an assumption, not a capability.

Scope discipline. This settles the forge’s admissibility as the substrate’s host — the question this ADR opened. It does not choose the engagement channel, which remains its own decision. What that decision inherits is a narrowed constraint rather than an open question: whatever channel it selects must carry an operator-executable export of the thread, and if it selects this forge it inherits the ruleset obligation above as well.

Reconciliation sequencing: clean plans per component, then one drill

The binding repository’s ~20-month staleness must be closed before TR-05’s preflight check can produce a meaningful result — but closing it is itself a large tracked change, and the check that would validate it does not yet work. The resolution splits the problem in two rather than choosing between evidence and tractability.

Stage 1 — clean-plan reconciliation, component by component. A terraform plan that proposes no changes against live infrastructure is a drift check for that component: it demonstrates the definitions and reality agree. This needs no new machinery and no scratch infrastructure, and it decomposes — each component is a separate tracked change, separately reviewable, rather than one enormous diff whose failure modes cannot be localized. Removal or migration of the legacy terraform/ tree happens here.

The limit is important and is the reason stage 2 exists: a clean plan reconciles the definitions to what is, not to what should be. Drift that is itself wrong gets ratified into the definitions by a process that only ever drives the diff to zero. Each plan’s revealed differences must therefore be inspected and judged, not merely eliminated — the operator is deciding, per difference, whether reality or the definition is correct.

Stage 2 — one full drill establishes the baseline. With every component planning clean, a single TR-06 drill runs the rebuild end-to-end against scratch infrastructure, and the TR-07 canary going green stamps the first last-known-good SHA. This is what makes the initial TR-05 baseline demonstrated rather than asserted. Accepting the reconciliation on inspection alone was rejected for exactly that: it would poison the baseline at the moment it is established, and every later drift check would measure against a reference no one had ever validated.

Ordering. Stage 2 depends on the phase-0 bootstrap being parameterized by target, since the drill needs its own state bucket — which is why the bootstrap sub-decision sequences ahead of reconciliation. Reconciliation is deliberately not gated on the home-lab module surface, which does not exist yet: waiting for it would hold the TR-05 baseline hostage to a build-out that is separate work.

Reconciling ADR-0001’s “peer surface” with the seam

ADR-0001 required home-lab foundations to be “version-controlled definitions in the same tracked-changes repository, as a peer top-level surface alongside cloud/”. That wording predates this ADR naming the seam, and read literally against a two-repository substrate it is ambiguous. This ADR resolves the ambiguity without weakening what ADR-0001 decided:

• Home-lab modules are a peer top-level surface alongside cloud/ in the public repository — same repository, same standing, exactly as ADR-0001 required.
• Home-lab bindings follow the same seam as every other component’s bindings: private repository, per-environment values, apply workflow.

The property ADR-0001 was protecting — that the home lab not become a hand-managed, second-class environment outside the tracked-changes surface — is fully preserved, since both layers are tracked definitions. What changes is only that “the same repository” resolves per layer rather than per environment, which is the seam this ADR decides. ADR-0001’s tooling-undecided caveat is untouched.

This also gives ADR-0001’s residual risk 3 (the home-lab tunnel endpoint configured out-of-band, via GUI, documented as prose) a decided destination: it is drift until it is expressed as a module in the public repository and bound in the private one.

Consequences

• Good, because the module reference — the single most load-bearing value in any apply — moves inside the tracked-changes surface, closing a TR-01 violation at its actual location rather than compensating for it elsewhere.
• Good, because a fresh pull of the binding repository becomes sufficient to rebuild, which is what TR-02’s 60-minute budget presumes and what the standup UX describes the operator as having in hand at minute zero.
• Good, because TR-05’s last-known-good reference collapses from a tuple to a single commit SHA, and the existing ruleset makes that SHA tamper-evident for free — no manifest format, no signing scheme, no new machinery.
• Good, because it changes no content and contradicts no accepted sibling, where the two single-repository options both required an accepted ADR to be amended.
• Good, because git gives TR-18 a near-vacuous answer for the definitions themselves: every clone is a complete export and the forge is replaceable without vendor cooperation.
• Bad, because TR-01 remains enforced by convention rather than by construction. The retained administrative bypass means the operator can mutate main outside the gate, and TR-05’s preflight cannot tell a bypassed change from drift. This is accepted knowingly; the first exercised bypass is the trigger to revisit.
• Bad, because a single repository-wide pin means module changes roll out to every component at once. Staged rollout is available only by pinning a purpose-made branch, which is more ceremony than editing one component’s ref would be.
• Bad, because two repositories mean two rulesets, two CI surfaces, and two contribution flows — a public fork-and-PR path for declarations, a private grant-based path for bindings — all of which is TR-54 surface that a single repository would not spend.
• Bad, because the binding repository is roughly 20 months stale (last pushed 2024-11-16) relative to the modules it pins, and it retains a legacy terraform/ tree predating the Terragrunt layout. The first preflight drift check under TR-05 will therefore run against a large accumulated gap, and reconciling it is real work that this decision creates rather than resolves.
• Bad, because TR-04 teardown now visibly depends on applied-state durability: losing state strands resources the definitions can no longer address. Classifying state as derived is correct, but it does not make teardown independent of it.
• Good, because the forge’s non-git state was tested against TR-18 limb by limb and cleared, converting an open risk into one named obligation (rulesets as definitions) and one narrowed constraint handed to the engagement-channel ADR (a routine thread export).
• Good, because the phase-0 bootstrap is parameterized by target, which makes TR-06’s “drill and live differ only in the underlying target” true of applied state as well — the drill no longer needs a hand-made bucket to be honest.
• Good, because reconciliation decomposes into per-component clean plans rather than one large diff, so the work is separately reviewable and needs no scratch infrastructure until the final baseline drill.
• Bad, because the ruleset enforcing this ADR’s own pull-request gate is untracked configuration — the substrate’s tamper-evidence rests on a value configured the same way the module reference was. Recorded as an obligation rather than resolved, and it compounds the bypass cost above: neither the gate nor its configuration is enforced by construction today.
• Bad, because phase 0 adds a rebuild phase the standup UX does not have, and its teardown carries an ordering constraint the other phases do not — phase 0 must tear down last, so “delete everything and start over” is order-sensitive in a way TR-04’s flat per-phase framing does not anticipate.
• Bad, because after adoption the bootstrap module’s state lives in the bucket it manages, so losing the bucket loses the record of how to rebuild it. Recovery is re-running phase 0 and re-importing — possible, but a manual exception rather than the ordinary path.
• Bad, because clean-plan reconciliation drives the diff to zero against current reality, which ratifies any drift that is itself wrong unless each revealed difference is judged rather than merely eliminated. The stage-2 drill bounds this but does not remove it — the operator’s inspection is load-bearing.
• Requires: the module reference committed to the binding repository, and every per-component workflow changed to read it rather than accept a workflow_dispatch input.
• Requires: a phase-0 bootstrap root module in the binding repository that creates the state bucket, adopts its own state into it, and is parameterized by target; plus a standup-UX change introducing phase 0 ahead of foundations, with its teardown documented as last-out.
• Requires: both repositories’ rulesets expressed as definitions rather than set through the forge’s UI, closing the self-referential TR-01 gap on this ADR’s own enforcement gate.
• Requires: the engagement thread’s export path exercised at least once, so TR-18 limb (b) rests on a demonstrated capability rather than a documented one.
• Requires: the Zaba505/infra ruleset (signatures, linear history, no force-push, no deletion, PR with zero required approvals) replicated on Zaba505/homelab, which has no equivalent today.
• Requires: reconciliation of the binding repository against current cloud/*, including removal or migration of the legacy terraform/ tree, before the first TR-05 preflight check can be meaningful — sequenced as per-component clean plans followed by one baseline drill, per the reconciliation note.
• Requires: a home-lab module surface in the public repository, peer to cloud/, per the reconciliation note — the destination for ADR-0001’s residual risk 3.
• Requires: the first last-known-good SHA to be stamped by a green canary on a full drill, not asserted on inspection — so TR-05’s baseline is demonstrated from the outset.

Realization

• Zaba505/infra (public) — cloud/* reusable modules; capability docs including tenant TR-09 declarations per ADR-0002; pkg/ and services/ unchanged by this decision. Gains a home-lab module surface peer to cloud/.
• Zaba505/homelab (private) — root modules and the Terragrunt layout ({component}/{env}/terragrunt.hcl + terraform.tfvars, with optional {region}/terraform.tfvars), per-component apply workflows, and the committed module pin this ADR introduces. Its main is the platform’s authoritative reference.
• terragrunt.hcl (repository root of the binding repo) — the seam’s mechanism: source = "git::https://github.com/Zaba505/infra.git//${module_path}?ref=${module_ref}", with module_ref becoming a tracked value. Also generates the backend and provider configuration.
• GCS state bucket, project infra-home-lab — applied state, per-component prefixed via path_relative_to_include(). Outside both repositories by decision, and a TR-04 teardown dependency. Created by phase 0 rather than assumed present.
• Phase-0 bootstrap root module (binding repository) — new. Creates the state bucket with versioning and retention, adopts its own state into it on first run, and is parameterized by target so drills bootstrap scratch state. Tears down last.
• Ruleset definitions for both repositories — new. The signatures / linear-history / no-force-push / no-deletion / PR-required configuration expressed as definitions rather than set through the forge’s UI, on Zaba505/infra and Zaba505/homelab alike.
• Secret Manager — secret values, referenced by name from both repositories; already the destination for the origin keypair and trust anchor per cloud/mtls/cloudflare-gcp/.
• cloud/mtls/cloudflare-gcp/ — ADR-0001’s residual risk 2 (trust anchor fetched live from the vendor at apply time) now has a decided home: the anchor belongs vendored in the public module surface with a tracked update path. A vendored copy already exists in the binding repository’s legacy terraform/load-balancer/ tree, on the wrong side of the seam.
• tech-design.md (composed later by plan-tech-design) will fold this substrate into the rebuild-flow narrative alongside the other accepted ADRs.

Open Questions

None remain. The three questions this ADR previously carried are resolved and folded into the sections above. What is handed downstream is not a set of open questions but a set of constraints already decided here:

• The rebuild-orchestrator ADR inherits phase 0 as a decided phase — definitions-driven, target-parameterized, inside the TR-02 budget, torn down last — rather than a chicken-and-egg to solve. It still owns how the phases are sequenced and checkpointed.
• The drift-detection ADR inherits a last-known-good reference whose form is fixed (a commit SHA) and whose first value has a decided provenance (stamped by a green canary on the stage-2 drill). It still owns capture and comparison.
• The engagement-channel ADR inherits a constraint, not a question: whatever channel it picks must carry an operator-executable export of the thread, and if it picks this forge it inherits the ruleset obligation too. It still owns the channel choice.

Resolved

• Which repository holds platform state. → Two, with a decided seam. Public: reusable modules, capability docs, tenant declarations. Private: root modules, per-environment values, apply workflows. The seam falls between reusable and parameterized and specific and bound — not between “platform” and “tenant” (Decision Outcome).
• What tracked-changes mechanism owns mutations. → Git plus a pull-request gate into main, under a ruleset requiring signatures and linear history and forbidding force-push and deletion, with zero required approving reviews so a single-operator platform can satisfy it. Direct-to-main signed commits were rejected for leaving no pre-merge point for validation to attach to (sub-decision).
• The untracked module reference. → Committed as a definition, single repository-wide pin. The value that determines what gets applied moves inside the tracked-changes surface, making a fresh pull sufficient to rebuild. Per-component pins were rejected for reconstructing the tuple this decision collapses (sub-decision).
• Form of TR-05’s last-known-good reference. → A commit SHA on the binding repository’s main — the SHA at which a rebuild’s canary last went green. The ruleset makes it tamper-evident without additional machinery. Capture and comparison remain the drift-detection ADR’s (sub-decision).
• Where applied state lives. → In the target environment, never committed. State is derived, not a definition; it contains plaintext secret material; and drill-versus-live parity under TR-06 only works if state is a property of the target. The cost is that TR-04 teardown depends on state durability (sub-decision).
• Administrative bypass of the gate. → Retained; discipline is convention. Accepted with its cost recorded in full: TR-01 is enforced by convention rather than construction, and TR-05 cannot distinguish a bypass from drift. The first exercised bypass is the trigger to revisit (sub-decision).
• Why the private side is private. → Aggregation, pre-disclosure, and mistake blast radius — explicitly not obscurity. Hostnames and IPs are public by construction via Certificate Transparency and continuous IPv4 scanning; the recorded rationale is narrowed accordingly so it does not decay into security-by-obscurity (sub-decision).
• State-backend bootstrap (TR-02, TR-04). → A definitions-driven phase 0 that adopts its own state, counts against the budget, and tears down last. A committed bootstrap module creates the bucket against a local backend, then migrates its state into the bucket it created. It is parameterized by target, which is what makes TR-06’s drill-versus-live parity true of applied state. Exclusion from the 60-minute budget was rejected because the budget is a target rather than a gate, so exclusion protects nothing while letting the KPI be measured against a rebuild that was not from scratch (sub-decision).
• Forge-held non-git state under TR-18. → Admissible, on a three-way classification. Rulesets are configuration and are the live gap — untracked today, and self-referentially so, since the ruleset enforcing this ADR’s gate is itself undefined in any definition. Workflow run history is derived and carries no export obligation, a classification this ADR’s own pin makes true. Issues are data of record and pass limb (b), because “without vendor cooperation” means without the vendor’s permission, not without the vendor’s interfaces — the strict reading would retroactively disqualify the edge vendor ADR-0001 already admitted (sub-decision).
• Reconciliation sequencing. → Per-component clean plans, then one baseline drill. A no-op terraform plan is itself a per-component drift check, needing no new machinery and decomposing the work into separately reviewable changes; a single TR-06 drill whose TR-07 canary goes green then stamps the first last-known-good SHA. Acceptance on inspection was rejected for poisoning the TR-05 baseline at the moment it is established. The recorded limit is that clean plans reconcile to what is, so each revealed difference must be judged rather than merely zeroed (reconciliation note).
• ADR-0001’s “peer surface alongside cloud/”. → Honored at the module layer; bindings follow the seam. “Same repository” resolves per layer rather than per environment. The property ADR-0001 protected — that the home lab is not a hand-managed second-class environment — is preserved intact (reconciliation note).

4 - [0004] Rebuild Orchestrator & Phase Model

Parent capability: Self-Hosted Application Platform Addresses requirements: TR-02, TR-03, TR-04, TR-06, TR-07

Context and Problem Statement

TR-02 requires a single operator-invocable entry point that drives the rebuild from a fresh pull of the definitions, sequences the phases automatically, and can complete inside 60 minutes. It permits manual checkpoints between phases and forbids manual driving of each step. TR-03 fixes the first phase’s scope: both environments plus the connectivity between them, with single-environment standup not a supported outcome. TR-04 requires every phase to expose a deterministic, definitions-driven teardown callable at every checkpoint, so that “delete everything and start over” is always viable. TR-06 requires the same entry point to run against scratch infrastructure, with drill and live differing only in the underlying target.

None of the four name a tool, an execution location, or a checkpoint mechanism. This ADR decides all three, plus the phase list and its ordering.

What exists today

There is no orchestrator. ADR-0003 records the substrate as per-component Terragrunt root modules in the private Zaba505/homelab, each applied by its own GitHub Actions workflow_dispatch. Nothing sequences them. The ordering between components exists only in the operator’s memory — which is the same failure class ADR-0003 closed for the module reference, one level up: a rebuild today would be the operator dispatching workflows in a remembered order, with nothing recording what that order is or whether it was ever validated end to end.

The public repository contributes reusable modules under cloud/* and nothing that drives them. Confirmed by inspection: there is no Makefile, no Taskfile, no root terragrunt.hcl, and no orchestration workflow — .github/workflows/ carries only docs, CodeQL, and Terraform linting.

What the siblings already fixed

This ADR inherits rather than re-opens:

• ADR-0003 hands over phase 0 as decided — a definitions-driven state-backend bootstrap, parameterized by target, counted inside the TR-02 budget, and torn down last. Its Open Questions section states this ADR “still owns how the phases are sequenced and checkpointed.”
• ADR-0001 subdivides phase 1: the edge and public-cloud anchor stand up first, and the operations tunnel is a distinct checkpoint with its own teardown, so a failed tunnel is retried against an intact cloud anchor rather than costing a full foundations teardown. It also leaves home-lab tooling deliberately undecided, noting Terraform is “a weak fit for the bare-metal and OS-level state the home lab actually carries.”
• The standup UX fixes the operator’s experience: automation provisions, pauses at each phase boundary, prints a summary, and waits for the operator to validate against provider UIs and signal continue. On any failure the operator tears down everything and restarts from the top; partial state is never carried forward.

What is actually undecided

The phase list is therefore largely determined. What is not:

1. What holds the phase model — the ordering, the teardown inverse, and the target parameterization.
2. Where it executes. ADR-0001 leaves home-lab tooling open, so the orchestrator must sequence heterogeneous work: cloud API calls on one side, bare-metal provisioning on the other. These have incompatible execution requirements, and that turns out to constrain the decision more than the phase list does.
3. How the operator’s continue is realized, given that the platform admits exactly one principal (TR-14).

Decision Drivers

• TR-02 — one entry point, automated, 60-minute-capable. An orchestrator that requires the operator to invoke each phase by hand puts the sequence back in their memory and fails the property, even if each phase is itself automated.
• TR-03 — phase 1 spans both environments plus the link. The orchestrator must drive work whose execution requirements differ fundamentally between the two sides, not just work that runs in two places.
• TR-04 — deterministic per-phase teardown at every checkpoint. “Deterministic” is the operative word: a teardown that is a hand-maintained mirror of the apply path drifts from it silently, and the drift is only discovered when teardown is needed most.
• TR-06 — drill and live differ only in target. This is a statement about parameterization, so target selection must be a single input threaded through every phase rather than a set of independently-set values.
• TR-01 — inherited from ADR-0003. The orchestrator and the phase model are themselves platform state, so they are definitions, and so is the toolchain they invoke. A rebuild whose result depends on which Terragrunt version happened to be on the runner is not reproducible from definitions.
• TR-18 — configuration control, portable export, and credential rotation without vendor cooperation. An orchestrator expressed entirely in one forge’s YAML is exportable in the trivial sense and portable in no useful sense: the phase model would have to be rewritten to move.
• TR-54 — the 2-hour weekly operator budget. A bespoke orchestrator is a thing that can rot, and its maintenance is charged here.
• TR-14 — exactly one principal. Any checkpoint mechanism requiring a second party is unsatisfiable by construction, exactly as ADR-0003 found for pull-request approvals.
• ADR-0003 constraint — phase 0 exists, is target-parameterized, is inside the budget, and tears down last. The teardown order is not flat.
• ADR-0001 constraint — the tunnel is its own checkpoint; home-lab tooling is undecided and must not be pre-empted here beyond what sequencing requires.
• Capability tiebreaker — tenant adoption beats reproducibility beats vendor independence beats minimizing operator effort. Reproducibility is again the operative term: it argues for the phase model being testable and the toolchain being pinned, and it outranks the operator-effort cost of building that.

Considered Options

Option A — GitHub Actions caller workflow holds the phase model

One rebuild.yml in the private repository, workflow_dispatch with a target input, one job per phase, needs:-chained. Each job invokes the relevant tool directly. Checkpoints are environment protection rules with the operator as required reviewer.

• Satisfies TR-02 cleanly on the entry-point property: one dispatch drives everything.
• Satisfies TR-06 — the target input threads down through the jobs.
• The checkpoint mechanism as specified is unavailable. GitHub’s documentation is explicit that on Free, Pro, and Team plans, required reviewers, wait timers, and custom deployment protection rules are only available for public repositories. Zaba505/homelab is private, so environment-gated approval requires GitHub Enterprise. This is a plan-tier blocker, not a configuration detail. (Self-approval itself is not the obstacle — “Prevent self-review” defaults off, so a solo operator could approve their own job if the gate were available at all.)
• Weak on TR-04: teardown becomes a second workflow that mirrors the apply path in YAML. Nothing enforces that the mirror stays accurate, and it cannot be tested.
• Weak on TR-18: the phase model is the forge’s YAML. Moving forges means rewriting the decision, not porting it.
• Weak on TR-01: tool versions come from whatever the runner image or a setup action provides, so the toolchain is ambient rather than defined.

Option B — One workflow_dispatch per phase

rebuild-phase.yml taking phase and target. The operator dispatches once per phase; the checkpoint is the decision to dispatch the next one.

• Best possible TR-54 position and the only option needing no new machinery at all. No plan upgrade, no polling, no third-party dependency.
• Fails TR-02’s central property. The sequence lives in the operator’s head — precisely the defect ADR-0003 closed one level down for the module reference. A “single top-level entry point” that must be invoked six times in a remembered order is not one entry point; it is the status quo with better labels.
• Weak on TR-04 for the same reason as Option A, plus the ordering of teardown is also unrecorded.
• Recorded so the rejection is auditable, since it is the cheapest option and its cheapness is real.

Option C — A purpose-built orchestrator in Go in the public repository; the forge is a thin invoker

The phase model, ordering, teardown inverse, and target parameterization live in code in Zaba505/infra. CI supplies the target and hosts the checkpoints; it holds no sequencing logic.

• Strongest TR-04 position available: teardown is a first-class inverse of the phase model rather than a parallel artifact, and it is unit-testable. This is the only option where “deterministic” is a property that can be asserted in a test rather than hoped for.
• Strongest TR-06: target selection is one typed parameter threaded through every phase by construction.
• Strongest TR-18: the phase model survives a forge change intact, and it runs identically on a workstation and in CI — which also makes it debuggable without burning a CI run.
• Honors ADR-0003’s seam without amendment: orchestration logic is reusable and parameterized, so it belongs in the public repository alongside cloud/*; the target definitions that bind it are specific and bound, so they stay private.
• Cost against TR-54: this is the most work of any option, and a bespoke orchestrator is a maintenance obligation with no natural forcing function to keep it current.
• Does not by itself supply a checkpoint mechanism — it still needs one from the layer above.

Option D — A dedicated workflow engine (Argo Workflows, Temporal)

• Real DAG execution, retries, and pause/resume as first-class primitives — nominally the best fit for TR-02 and TR-04.
• Circular and therefore disqualified. The engine needs compute to run on, and that compute is platform state this very flow is rebuilding. Phases 0 and 1 would depend on something phase 2 provisions. There is no ordering that makes this work.
• Enormous TR-54 cost for a single-operator platform even if the circularity were solved.

Option E — Ansible as the single orchestrator across both environments

• Genuinely spans TR-03’s both-sides scope in one tool, has a native pause for checkpoints, and post-iPXE the home lab needs OS-level configuration anyway — so it would fill ADR-0001’s undecided home-lab tooling at the same time.
• Weak on TR-04 relative to graph-derived teardown: Ansible teardown is hand-written, so it carries Option A’s mirror-drift problem in a different language.
• Pre-empts a decision ADR-0001 deliberately deferred. Choosing the home-lab configuration tool as a side effect of choosing the orchestrator is exactly the coupling ADR-0001 declined to make, and it would be made here without the component-design context ADR-0001 said was better placed to weigh it.
• Retains the execution-location problem unchanged — Ansible still needs a control node.

Decision Outcome

Chosen option: Option C — a purpose-built orchestrator in Go in the public repository, realized as a Dagger module, with the forge as a thin invoker.

It is the only option under which TR-04’s determinism is testable rather than aspirational, and the only one that does not bet the phase model on a single forge (TR-18). Options A and B both put sequencing in YAML the operator maintains by hand, and Option B additionally fails TR-02’s entry-point property outright by returning the sequence to the operator’s memory. Option D is circular. Option E buys one-tool coverage of TR-03 by pre-empting ADR-0001’s deferred home-lab tooling decision, which is not this ADR’s to make.

Option A’s environment-protection blocker is worth stating plainly, because it would otherwise look like the obvious design: it is unavailable at this repository’s plan tier, and discovering that during implementation rather than here would have cost a rewrite of the checkpoint layer.

Why Dagger specifically

Dagger is chosen over hand-rolled Go orchestration for three reasons that map onto cited TRs, not on general merit:

• It makes the toolchain a definition (TR-01). Terragrunt, OpenTofu, and every other tool a phase invokes are pinned container images rather than whatever the runner image happens to carry. Without this, a rebuild’s result depends on ambient runner state, and “reproducible from definitions” is false in a way no amount of definition discipline elsewhere repairs.
• It makes the orchestrator genuinely portable (TR-18). The same module runs on a workstation and in CI, which is what turns Option C’s portability claim from an assertion into a property. It also means a forge migration moves an invocation, not a phase model.
• The operator has prior production experience with it, which materially lowers the TR-54 maintenance cost that is Option C’s main liability.

Dagger’s costs are accepted with their consequences recorded below: it is pre-1.0 (v0.21.x, no API-stability guarantee, with a breaking Modules v2 redesign shipped in v0.21), and its engine must run privileged, making the host the security boundary.

The phase model

flowchart TD
    P0["Phase 0 — state backend<br/>ADR-0003 bootstrap, adopts own state"]
    P1a["Phase 1a — cloud anchor + edge<br/>+ iPXE boot endpoint"]
    P1b["Phase 1b — operations tunnel<br/>ADR-0001 distinct checkpoint"]
    P1c["Phase 1c — home-lab base<br/>bare metal via iPXE"]
    P2["Phase 2 — core services<br/>compute, storage, identity"]
    P3["Phase 3 — cross-cutting<br/>backup, observability"]
    P4["Phase 4 — canary tenant<br/>TR-07 binding readiness signal"]

    P0 --> P1a --> P1b --> P1c --> P2 --> P3 --> P4
    P4 --> Ready((Ready to<br/>host tenants))

    P1c -.->|"TR-03: phase 1 incomplete<br/>until 1a+1b+1c all pass"| P1a

Each boundary is an operator checkpoint. Teardown unwinds in reverse, with phase 0 last.

Sub-decision: the tunnel precedes home-lab bare metal, reordering ADR-0001’s phase 1

ADR-0001 fixed that the cloud anchor stands up first and that the tunnel is a distinct checkpoint. It did not fix the tunnel’s order relative to home-lab bare metal. This ADR fixes it: the tunnel comes first.

The reason is a circularity that only appears once execution location is considered. Bare-metal provisioning must be triggered — a machine that is powered off will never boot, whatever the boot infrastructure looks like. Triggering it means reaching the home lab’s LAN, whether by BMC power-on or otherwise. If home-lab bare metal preceded the tunnel, the orchestrator would need a path to the LAN that the rebuild has not yet built.

Ordering the tunnel first discharges this, because the tunnel’s home-lab endpoint does not depend on any home-lab host the rebuild provisions. ADR-0001 records that endpoint as configured out-of-band today (its residual risk 3), which means it lives on always-on network equipment rather than on a rebuilt machine. That property — inherited, not introduced here — is what makes the ordering work.

This does not weaken TR-03. Phase 1 remains incomplete until all three checkpoints pass, and single-environment standup remains unsupported; what changes is only the order in which the three are attempted, and the ADR-0001 benefit of retrying a failed tunnel against an intact cloud anchor is preserved exactly.

Sub-decision: iPXE chainloads from a stable cloud-hosted endpoint

The home lab’s boot artifacts — the iPXE script, kernel, initrd, and OS image — are served over HTTPS from the public-cloud anchor, provisioned as part of phase 1a. The only home-lab-local configuration is the DHCP directive pointing at the chainload URL.

The alternative considered was an always-on LAN appliance hosting boot artifacts and a self-hosted runner. It was rejected because it converts a large amount of platform state into a hand-built device that the rebuild flow depends on but does not build — so a disaster-recovery rebuild would begin with an untracked manual step, and the reproducibility claim would quietly exclude it.

Two consequences follow and are decided here rather than left to implementation:

• The chainload URL must be a stable DNS name, not an address or hostname that phase 1a mints fresh. If the URL changed per rebuild, the router’s DHCP configuration would need hand-editing during every rebuild and drill — reintroducing the manual step this sub-decision exists to avoid, at the worst possible moment. The name is stable; what it resolves to is provisioned by phase 1a.
• Boot artifacts are integrity-critical but not confidential. They are served over HTTPS from the cloud anchor; transport authentication is the control. They carry no secret material, so their public readability is not a finding.

The residual is recorded honestly: the router’s DHCP configuration remains out-of-band GUI state, which is drift under TR-01 in exactly the class of ADR-0001’s residual risk 3. This sub-decision shrinks the home lab’s untracked surface to approximately its minimum — one DHCP directive — but does not eliminate it, and it inherits rather than creates the obligation to express network-equipment configuration as definitions.

Sub-decision: the executor moves from hosted to cloud-anchor runner after phase 1a

Phases 0 and 1a run on GitHub-hosted runners: they touch only cloud APIs over outbound HTTPS, and nothing in them needs LAN reachability.

From phase 1b onward, execution moves to a self-hosted runner on the public-cloud anchor, provisioned by phase 1a. This runner reaches the home lab over the operations tunnel once phase 1b establishes it. Self-hosted runners require only outbound HTTPS to register and poll, so this needs no inbound exposure of the cloud anchor.

The point worth stating is what this arrangement avoids: no runner is required inside the home lab, and none is a precondition of the rebuild. The executor for home-lab-touching work is platform state that the rebuild itself provisions in phase 1a, so it is definitions-driven like everything else. An always-on LAN runner would have been a hand-built precondition, with the same objection as the appliance rejected above.

The accepted cost is a runner handoff in the middle of the rebuild, which is real complexity: phase 1a must provision and register a runner that later phases then target, and a failure to register is a phase-1a failure with a somewhat unintuitive symptom. It is recorded rather than minimized.

Sub-decision: checkpoints are separate invocations gated on the engagement thread

Dagger has no primitive for waiting on human input — Terminal() exists for debugging, not for pipeline-time approval — so the checkpoint cannot live inside the orchestrator. Combined with Option A’s finding that environment protection rules are unavailable on a private repository below Enterprise, the mechanism is fixed as follows:

Each phase is a separate invocation of the orchestrator from a needs:-chained job in a single rebuild.yml. Between phases, a gate step posts the phase summary to a GitHub issue and polls until the operator replies continue.

This satisfies TR-02 on both limbs: one workflow_dispatch drives the whole rebuild, so the entry point is genuinely single, while the operator validates at boundaries without driving steps. The phase model stays in Go regardless — what CI holds is the invocation sequence and the gate, not the ordering logic.

Routing the gate through an issue thread is not merely a workaround for the plan tier. It lands checkpoint acknowledgments on the TR-19/TR-20 engagement channel that is already the platform’s record of operator action, so the rebuild leaves an auditable trail of what was validated and when — which environment approvals would have recorded only in run metadata.

The costs are accepted: a job parked on a poll consumes runner minutes, unlike an environment approval, and hosted jobs cap at six hours. Neither binds a rebuild targeting 60 minutes, but an abandoned rebuild left parked overnight is waste rather than merely idle.

Sub-decision: every side-effecting operation carries a per-run cache-buster

This is the highest-risk item in the design and it is decided as a hard correctness rule, not an optimization.

Dagger caches function calls on module source plus argument values, and caches withExec operations at the BuildKit layer. Critically, the documented cache="never" control on a function does not disable layer caching for the execs inside it. The failure mode this produces is documented in the wild — dagger#7090 and dagger#9607 both record deploy and destroy operations returning cached success while the infrastructure no longer existed.

For this capability that is not a performance bug. A teardown that reports success without executing defeats TR-04 precisely when it matters — the operator believes “delete everything and start over” succeeded, restarts onto state that was never removed, and carries partial state forward, which is the one outcome the standup UX rules out by name. A cached apply is the same defect pointed the other way, and it would make the TR-07 canary’s green signal untrustworthy.

The rule: every exec that mutates infrastructure — apply, destroy, boot-trigger, canary deploy — takes a per-run unique value (the rebuild’s run identifier) as an environment variable used only for cache invalidation. Read-only operations may cache freely.

Enforcing this by convention alone was considered and rejected: the failure is silent, and a missed cache-buster surfaces as a successful-looking rebuild. The obligation recorded below is therefore that side-effecting operations be constructed through a single helper that injects the buster, so that omitting it requires bypassing the helper rather than merely forgetting a parameter.

Sub-decision: the teardown contract

Every phase exposes a teardown callable at every checkpoint, per TR-04. Four properties are fixed:

• Teardown unwinds in reverse phase order, and phase 0 is last — inherited from ADR-0003, whose state bucket holds the state every other phase’s teardown depends on. Torn down out of order it strands exactly the resources the definitions can no longer address.
• Terraform-backed phases derive teardown from the dependency graph rather than a hand-written inverse. Terragrunt reverses the graph on destroy, so dependents are removed before dependencies. This is what makes “deterministic” mean something stronger than “we wrote a second script.”
• Phase membership and teardown order are verifiable before execution. Terragrunt can emit the exact unit set and destroy-order a given scope selects, so a phase’s teardown can be dry-run and asserted in a test rather than trusted. The orchestrator uses this as the definition of a phase’s Terraform-backed membership.
• Teardown must be scoped so it cannot exceed its phase. run --all destroy destroys external dependencies of the selected units by default, which would let one phase’s teardown reach outside itself. Phase scoping must exclude external dependencies explicitly, and this is a correctness requirement of the teardown contract rather than a tuning flag.

Two limits are recorded rather than solved. Terragrunt offers no resume-from-failure and no transactional rollback, so a teardown interrupted partway leaves a partially-destroyed graph that the next attempt must be able to tolerate — re-running teardown must be safe. And phase 1c’s teardown is not resource deletion: tearing down bare metal means powering off and wiping, which is a different determinism story than a cloud API destroy and is the weakest link in the per-phase contract.

Sub-decision: drill-vs-live is a single named target

TR-06 requires drill and live to differ only in the underlying target, so target selection is one parameter naming a target definition, not a set of independently-set values. A target definition binds the GCP project, the state bucket, the DNS zone, the boot endpoint, and the home-lab target in one tracked object. Every phase receives it; no phase reads any of those values from anywhere else.

The single-parameter form is the decision, and its rationale is the same one ADR-0003 used for collapsing the module reference from a tuple to a single pin: independently-set values can be combined into a state that was never validated, and here that state would be a drill pointed at live infrastructure.

Target definitions are specific and bound, so per ADR-0003’s seam they live in the private repository; the orchestrator that consumes them is reusable and parameterized, so it lives in the public one.

Sub-decision: a drill’s home-lab target is virtual hosts on the home-lab LAN

The single-target form above is straightforwardly satisfiable for every cloud phase — a drill target names a different project, bucket, and zone. It is not straightforwardly satisfiable for phase 1c: a drill cannot reprovision the live home lab’s bare metal without destroying the thing it is drilling against, and there is no second set of hardware.

The decision: a drill target’s home-lab target names virtual machines on existing home-lab hypervisor capacity, where a live target names the physical hosts. Phase 1c then runs unchanged — same chainload, same boot artifacts, same OS image, same post-boot definitions — against virtual NICs and virtual disks. TR-06’s “differ only in the underlying target” stays literally true, because the virtualization is a property of the target rather than a branch in the flow.

Why home-lab capacity rather than the cloud. Virtualizing the home-lab side into the cloud anchor was the cheaper-looking variant and is rejected: it would make phase 1b degenerate, with the tunnel terminating on both ends inside one environment. A drill would then no longer span two environments, and TR-03’s both-sides-plus-the-link coverage would be silently narrowed in exactly the mode that exists to verify it. Keeping the virtual hosts on the home-lab LAN preserves the tunnel as a real crossing, which is the property a drill most needs to hold.

BR-50’s “without touching live platform state” is satisfied on the distinction that matters: a drill VM consumes live capacity but produces no live platform state.

The two alternatives are recorded with their rejections. Spare hardware as a standing precondition gives the highest fidelity, and is rejected on the same objection that killed the LAN boot appliance above — it makes the rebuild flow depend on a hand-maintained device the rebuild does not itself build. Cloud-only drills cost nothing, and are rejected because they make drill and live differ in scope as well as target, which lets the reproducibility KPI certify a rebuild that skips a phase.

Two consequences follow and are decided here rather than left to implementation:

• Drill hosts sit on an isolated segment with their own DHCP chainload directive. The stable name decided above points at the live target’s boot endpoint. A drill VM booting on the live LAN’s DHCP scope would therefore chainload live artifacts — a drill that quietly rebuilds from the live target is worse than no drill at all, because it returns a green signal for something it never tested. The drill segment’s directive points at the drill target’s boot endpoint. This grows the out-of-band router surface from one directive to two; it does not change its class.
• What a drill does not prove is named rather than implied. Firmware, BMC power-on, disk controller and layout, and NIC driver behaviour — the failure modes specific to metal — go unexercised. A drill certifies the phase model, the sequencing, the definitions, and the teardown; it does not certify that this hardware boots. That is the honest bound on what the reproducibility KPI demonstrates, and it belongs next to the KPI rather than being discovered after a disaster-recovery event.

Sub-decision: phase 4 exposes an idempotent canary teardown, and a canary that will not tear down is a phase-4 failure

TR-07 has the canary deployed, exercised, and torn down within phase 4, so its teardown is partly intrinsic to the success path. That is not sufficient for TR-04, and the reason is specific: the checkpoint at which phase 4’s teardown is actually reached is the one where the canary failed — the standup UX’s “canary tenant fails to come up” case — and there the intrinsic teardown never ran. A phase whose teardown assumes the success path has no teardown for the case it exists to serve.

Phase 4 therefore exposes the standard TR-04 entry point as an idempotent removal of the canary’s runtime footprint wherever it landed — compute, persistent storage, identity registration, backup enrolment, and observability series, matching the surfaces the standup UX has the canary exercise. Idempotence is the operative property, because the teardown cannot know which of three states it faces: a canary that removed itself cleanly, one that removed itself partway, or one that never deployed. All three must be safe.

What it does not remove is the canary’s definitions. TR-07 maintains the canary alongside the platform definitions, so it is a permanent artifact; teardown removes what a rebuild instantiated, never the canary itself.

The second half of the decision is a claim about readiness: a canary that comes up green but does not tear down cleanly is a phase-4 failure. TR-07 names deployment, exercise, and teardown as one obligation, but the reason is not symmetry — the canary’s teardown is the platform’s tenant-offboarding path exercised for real. A canary that cannot be removed has demonstrated that the platform cannot offboard a tenant, which is a readiness defect exactly as much as one that cannot be onboarded, and the standup UX’s rule that readiness does not bend for a failed canary applies unchanged.

This is also where the cache-buster rule is load-bearing a second time. Canary deploy and canary teardown are both side-effecting and both go through the mandatory helper; a cached canary teardown would report a clean removal that never happened, leaving residue that the next rebuild’s canary then collides with.

Sub-decision: checkpoint wait time counts against the 60-minute budget

The TR-02 budget is measured as wall-clock from invocation to canary-green, including time parked at checkpoints.

Excluding operator validation time was considered and rejected on ADR-0003’s precedent, which refused to exclude phase 0 from the budget on the reasoning that the budget is a target rather than a gate — missing it files a follow-up issue and does not stop the platform going into service, so exclusion protects nothing while letting the KPI be measured against something other than the real rebuild. The same logic applies here, and more directly: the standup UX has the operator record how long the rebuild took, which is unambiguously the elapsed experience rather than the automation’s share of it.

The consequence is that the KPI partly measures operator validation speed, which is honest rather than unfortunate — a rebuild requiring 40 minutes of squinting at provider consoles genuinely is a slow rebuild, and the follow-up issue it generates is pointed at a real problem.

Consequences

• Good, because the phase model, its ordering, and its teardown inverse become testable code rather than a hand-maintained YAML mirror — the only option under which TR-04’s “deterministic” is a property that can be asserted rather than hoped for.
• Good, because the toolchain is pinned into container images, so a rebuild’s result no longer depends on ambient runner state. This closes a TR-01 gap that none of the YAML-held options address.
• Good, because the orchestrator runs identically on a workstation and in CI, which makes TR-18 portability real and lets the phase model be debugged without burning CI runs.
• Good, because target selection collapses to one parameter, so a drill cannot be assembled into a configuration that partly points at live infrastructure.
• Good, because ordering the tunnel ahead of home-lab bare metal discharges a circularity — that bare metal must be triggered over a path the rebuild has not yet built — without weakening TR-03 or losing ADR-0001’s independent-tunnel-retry benefit.
• Good, because no runner and no boot appliance is required inside the home lab; the executor for home-lab-touching phases is provisioned by phase 1a, so it is definitions-driven rather than a hand-built precondition.
• Good, because checkpoint acknowledgments land on the TR-19/TR-20 engagement thread, leaving an auditable record of what the operator validated and when — which environment approvals would have left only in run metadata.
• Good, because drills exercise phase 1c for real rather than skipping it — the home-lab target is virtualized rather than the flow being branched, so TR-06’s “differ only in target” stays literally true and the reproducibility KPI certifies the whole phase model rather than its cloud half.
• Good, because keeping the drill’s virtual hosts on the home-lab LAN keeps phase 1b’s tunnel a genuine cross-environment crossing during a drill, so TR-03’s coverage is not quietly narrowed in the mode that exists to verify it.
• Good, because phase 4’s teardown is idempotent across all three states the canary can be left in, so TR-04 holds at the checkpoint the operator actually reaches most often — the failed canary, where the intrinsic success-path teardown never ran.
• Good, because treating a canary that will not tear down as a phase-4 failure makes each rebuild exercise the tenant-offboarding path and not only the onboarding one.
• Bad, because Dagger’s caching can silently skip side-effecting operations, and the documented cache="never" control does not cover execs. A cached teardown reports success without executing, which produces exactly the carried-forward partial state the standup UX rules out. This is mitigated by a mandatory per-run cache-buster, but the mitigation is the design’s most load-bearing convention and its failure mode is a successful-looking rebuild.
• Bad, because Dagger is pre-1.0 with no API-stability guarantee, and shipped a breaking Modules v2 redesign in the current minor line. The orchestrator inherits an upgrade treadmill charged against TR-54, and version pinning is mandatory rather than prudent.
• Bad, because the Dagger engine must run privileged, so the host is the security boundary. This is acceptable on a dedicated runner and would not be on a shared one.
• Bad, because iPXE cannot run inside Dagger. There is no host-network mode, and DHCP/TFTP is broadcast traffic that a NAT’d container bridge does not reach. Phase 1c’s boot-trigger work runs host-side, outside the Dagger graph — a real seam in an otherwise uniform execution model, and the one place where “the orchestrator holds the phase model” is qualified.
• Bad, because the runner handoff mid-rebuild is genuine complexity: phase 1a provisions and registers the runner that later phases target, and a registration failure is a phase-1a failure with an unintuitive symptom.
• Bad, because the router’s DHCP configuration remains out-of-band GUI state — drift under TR-01, in ADR-0001’s residual-risk-3 class. This decision shrinks the home lab’s untracked surface to roughly one directive but does not reach zero.
• Bad, because a job parked on a checkpoint poll consumes runner minutes, where an environment approval would not, and hosted jobs cap at six hours. Neither binds a 60-minute rebuild; an abandoned one is waste.
• Bad, because Option C is the most work of any option considered, and a bespoke orchestrator has no natural forcing function keeping it current. The operator’s prior Dagger experience lowers this cost but does not remove it.
• Bad, because phase 1c’s teardown is not resource deletion. Powering off and wiping bare metal is a weaker determinism story than a cloud destroy, making it the weakest link in the per-phase teardown contract.
• Bad, because a virtualized drill target does not exercise metal. Firmware, BMC power-on, disk controller and layout, and NIC driver behaviour go unproven, so the reproducibility KPI certifies the phase model and the definitions rather than that this hardware boots. This is recorded as the honest bound on the KPI rather than mitigated, and it is the residual a spare-hardware precondition would have bought out.
• Bad, because a drill’s phase-1c teardown is VM deletion rather than power-off-and-wipe, so drills exercise a teardown path that is both stronger than and different from the live one. The weakest link in the teardown contract is precisely the link drills do not test.
• Bad, because the drill’s home-lab hosts require a second out-of-band DHCP directive on an isolated segment. Without it a drill chainloads the live target’s boot artifacts and returns a green signal for something it never tested. The untracked router surface grows from one directive to two — the same TR-01 class, twice.
• Bad, because drill hosts consume live home-lab capacity. No live platform state is touched, which is what BR-50 tests, but a drill and the live platform contend for the same hypervisor — so drill sizing is bounded by what the home lab can spare while still serving.
• Bad, because phase 4’s teardown must be written defensively rather than derived. Unlike the Terraform-backed phases, the canary’s footprint spans identity, backup, and observability enrolments whose removal has no dependency graph to reverse, so this is the one teardown that is a hand-written inverse — the mirror-drift problem this ADR rejected Option A over, readmitted at one phase and accepted because TR-07’s canary surface is small and fixed.
• Neutral but load-bearing: Terragrunt provides no resume-from-failure and no transactional rollback, so re-running a teardown must be safe against a partially-destroyed graph. This is a property the orchestrator must hold, not one it inherits.
• Requires: a Dagger module in Zaba505/infra holding the phase model, per-phase apply and teardown functions, and the target parameter, with the Dagger version and every tool image pinned.
• Requires: a single helper through which all side-effecting execs are constructed, injecting the per-run cache-buster, so omission requires bypassing the helper rather than forgetting a parameter.
• Requires: rebuild.yml in Zaba505/homelab — one workflow_dispatch taking the target, needs:-chained per-phase jobs invoking the module, and the issue-thread gate step between phases.
• Requires: target definitions in Zaba505/homelab binding project, state bucket, DNS zone, boot endpoint, and home-lab target as one tracked object per target.
• Requires: phase 1a extended to provision the iPXE boot-artifact endpoint behind a stable DNS name, and to provision and register the cloud-anchor self-hosted runner.
• Requires: phase scoping that excludes external dependencies on teardown, plus a test asserting each phase’s membership and destroy order against the dry-run output.
• Requires: a standup-UX update — phase 0 from ADR-0003, the 1a/1b/1c subdivision with the tunnel ahead of bare metal, and the issue-thread checkpoint replacing the implied local continue.
• Requires: empirical verification that outbound TCP from a Dagger container reaches an RFC1918 address over the tunnel, before phases 2 and 3 are built on the assumption. Official documentation does not guarantee it.
• Requires: drill target definitions to name virtual home-lab hosts on an isolated LAN segment, with that segment’s own DHCP chainload directive pointing at the drill target’s boot endpoint — not the live one.
• Requires: home-lab hypervisor capacity sufficient to stand up the drill’s virtual host set alongside live workloads, which is a capacity floor the home-lab definitions surface must account for rather than discover at drill time.
• Requires: phase 4 to expose an idempotent canary teardown covering compute, persistent storage, identity registration, backup enrolment, and observability, safe against a canary that removed itself, removed itself partway, or never deployed — and constructed through the cache-buster helper like every other side-effecting operation.
• Requires: the standup UX update already noted above to additionally record that a canary which will not tear down cleanly fails phase 4, and that drills exercise the home-lab phase virtually.

Realization

• Zaba505/infra (public) — new Dagger module (Go, Modules v2) holding the phase model, per-phase apply/teardown functions, the target parameter type, and the cache-buster helper. Reusable and parameterized, so it sits on the public side of ADR-0003’s seam alongside cloud/*. Unit tests assert teardown ordering and phase membership.
• Zaba505/homelab (private) — new rebuild.yml (single workflow_dispatch, needs:-chained phase jobs, issue-thread gate steps) and new target definitions binding each drill/live target as one object. Existing per-component workflows are superseded by phase invocations.
• cloud/dns/ — the stable DNS name fronting the iPXE chainload URL, which must survive across rebuilds independently of what phase 1a provisions behind it.
• cloud/storage-bucket/ — boot artifacts (iPXE script, kernel, initrd, OS image) served over HTTPS from the cloud anchor.
• cloud/compute-engine/ + cloud/service-account/ — the cloud-anchor self-hosted runner provisioned by phase 1a, plus its registration identity.
• Phase-0 bootstrap root module (Zaba505/homelab) — from ADR-0003; invoked here as phase 0 and torn down last.
• cloud/vpc-network/ and the tunnel module — phase 1b, ADR-0001’s distinct checkpoint, reordered ahead of home-lab bare metal.
• Home-lab module surface (Zaba505/infra, peer to cloud/) — from ADR-0003’s reconciliation note; phase 1c’s definitions land here once ADR-0001’s deferred tooling decision is made. The boot-trigger work runs host-side, outside Dagger. Must address both a physical and a virtual host target, and declare the hypervisor capacity a drill requires.
• Canary tenant definitions (Zaba505/infra) — from TR-07, maintained alongside the platform definitions and never removed by teardown. Phase 4’s apply deploys and exercises it; phase 4’s teardown removes its runtime footprint idempotently across compute, storage, identity, backup, and observability.
• tech-design.md (composed later by plan-tech-design) will fold the phase model into the rebuild-flow narrative alongside the other accepted ADRs.

Open Questions

None remain. Both questions this ADR previously carried are resolved and folded into the sections above.

Resolved

• How the home-lab side of a drill is exercised. → A drill target’s home-lab target names virtual machines on existing home-lab hypervisor capacity, so phase 1c runs unchanged against virtual NICs and disks and TR-06’s “differ only in target” stays literally true. Virtualizing into the cloud anchor was rejected for making phase 1b’s tunnel degenerate and narrowing TR-03 coverage in the mode that exists to verify it; spare hardware was rejected on the LAN-appliance objection; cloud-only drills were rejected for making drill and live differ in scope. Recorded limits: metal-specific failure modes go unproven, drill hosts need their own DHCP chainload directive, and drill teardown is VM deletion rather than the live power-off-and-wipe (sub-decision).

• The canary’s teardown relationship to phase 4. → Phase 4 exposes the standard TR-04 entry point as an idempotent removal of the canary’s runtime footprint, and a canary that will not tear down cleanly fails phase 4. The intrinsic success-path teardown is insufficient because the checkpoint where teardown is reached is the failed canary, where it never ran; idempotence is required because teardown cannot know whether the canary removed itself, removed itself partway, or never deployed. The canary’s definitions are never removed. Failure is a readiness failure because the canary’s teardown is the tenant-offboarding path exercised for real (sub-decision).

• What holds the phase model. → A Dagger module in Go in the public repository. The forge holds the invocation sequence and the checkpoint gate; it holds no ordering logic. Chosen for testable teardown (TR-04), pinned toolchain (TR-01), and forge portability (TR-18) (Decision Outcome).

• Phase ordering. → 0 → 1a cloud anchor + edge → 1b tunnel → 1c home-lab bare metal → 2 core → 3 cross-cutting → 4 canary. The tunnel is reordered ahead of bare metal because bare metal must be triggered over a LAN path the rebuild would not otherwise have built yet. TR-03 is unweakened: phase 1 completes only when all three checkpoints pass (sub-decision).

• Home-lab boot infrastructure. → iPXE chainloads from a stable cloud-hosted endpoint provisioned in phase 1a. An always-on LAN appliance was rejected for making the rebuild depend on a hand-built device it does not itself build. The residual is one out-of-band DHCP directive (sub-decision).

• Execution location. → Hosted runners for phases 0–1a; a cloud-anchor self-hosted runner, provisioned by phase 1a, from 1b onward, reaching the home lab over the tunnel. No runner is required inside the home lab. The cost is a runner handoff mid-rebuild (sub-decision).

• Checkpoint mechanism. → Separate per-phase invocations from one workflow_dispatch, gated on an issue thread. Environment protection rules are unavailable on a private repository below Enterprise, and Dagger has no wait-for-human primitive. Routing the gate through the TR-19/TR-20 engagement channel makes checkpoint acknowledgments auditable (sub-decision).

• Cache correctness. → Every side-effecting exec carries a per-run cache-buster, injected through a single mandatory helper. Dagger’s cache="never" does not cover execs, and cached-success deploys and destroys are documented in the wild. A cached teardown produces the carried-forward partial state the standup UX rules out by name (sub-decision).

• Teardown contract. → Reverse phase order with phase 0 last; graph-derived for Terraform-backed phases; membership and order verifiable by dry-run; scoped to exclude external dependencies. Recorded limits: no resume-from-failure, and bare-metal teardown is power-off-and-wipe rather than resource deletion (sub-decision).

• Drill-vs-live parameterization. → One parameter naming a target definition that binds project, state bucket, DNS zone, boot endpoint, and home-lab target together. Independently-set values were rejected on ADR-0003’s tuple reasoning — here the never-validated combination would be a drill pointed at live (sub-decision).

• What counts against the 60-minute budget. → Wall-clock including checkpoint waits. Excluding operator validation time was rejected on ADR-0003’s precedent that the budget is a target rather than a gate, so exclusion protects nothing while making the KPI measure something other than the real rebuild (sub-decision).