Getting started

• 1: What you need
• 2: A first drill
• 3: Going live

1 - What you need

The rebuild entry point is a Dagger module in this repository. Everything it runs, it runs in containers it puts together, so the machine that calls it needs only two tools, and the module needs to know where the values checkout is.

The Dagger CLI

The CLI has to be the version the module pins. That’s v1.0.0-beta.15, the engineVersion in the repository’s dagger-module.toml. Install it with Dagger’s install script, at that version:

curl -fsSL https://dl.dagger.io/dagger/install.sh \
  | DAGGER_VERSION=1.0.0-beta.15 BIN_DIR="$HOME/.local/bin" sh

Check that the CLI on your PATH is the right one:

dagger version

It should report v1.0.0-beta.15. Dagger 1.0 is in beta, and the module moves to a newer engine only when a pull request moves it. Install a newer CLI then, too.

A container runtime

The CLI runs the Dagger engine as a container, and the engine runs every step of a run in containers of its own. Any runtime the Dagger CLI can start the engine with works, such as Docker or Podman. The CLI starts the engine on the first call and reuses it after that.

The values checkout

The values checkout is a git repository of its own, outside this one. It holds a file per target, targets/<name>.yaml, which says whether the target is live or a drill:

targets/
  live.yaml     kind: live
  drill.yaml    kind: drill

A Dagger function can’t read the machine it’s called from, so the module takes the checkout as an argument. Set it once, from this repository’s checkout, as a user-level Dagger setting:

dagger settings -g rebuild values ../infra-values

-g keeps the setting out of the repository: Dagger writes it to ~/.config/dagger/config.toml, keyed by this repository’s git remote, so nothing about your machine is committed. Every dagger call in a checkout of this repository then uses it. Check it with:

dagger settings rebuild

To work on the module

Go is needed only to change the module’s code, not to run it. The module’s go.mod files name the Go version.

Next: A first drill

2 - A first drill

A drill is a rebuild against a scratch target, which leaves the live platform untouched. Drill and live differ only in the target, so a drill is the same commands a live rebuild runs. Run them from this repository’s checkout, with the values checkout set as What you need says.

Add the drill’s target

Give the drill a file in the values checkout, targets/drill.yaml:

kind: drill

The name of the file, without .yaml, is the target’s name. A drill may run from uncommitted changes, in either checkout, so the file doesn’t need to be committed first. The run’s record says it was.

Check the target

dagger call preflight --target drill

It changes nothing. Against a target that holds nothing yet, it prints:

the preflight check passed against drill.

It fails if the target’s last run didn’t finish, or if a phase that run applied no longer verifies. The preflight check says what each means.

Run it

dagger call up --target drill

up runs the preflight check, then applies and verifies the first phase, and stops at the checkpoint after it. dagger call prints the run’s summary: each phase the call ran, what it made and how long it took, the run’s time so far, and what you can do next. At the checkpoint after phase 0, it reads like this:

run 7QH3W2Y5N4RJXK6VZ2B4TQ3MJE against drill
phase 0: verified, 2m31.4s
  made the state backend, bucket drill-state
the run's time so far: 2m40.2s
the run paused at the checkpoint after phase 0. Continue with up --resume, or tear the target down with down.

Look at what the phase made. Then continue with the next phase:

dagger call up --target drill --resume

Each call runs one phase and stops at the checkpoint after it, until the last phase verifies and the summary says so:

the run finished: every phase verified.

No phase has landed yet, so today the first up finishes at once, with nothing to do:

run JCUPWNYUR4NCJ3NCOSRCBM5DKF against drill
the run took 829.009578ms
the run finished: every phase verified.

Two flags change where a call stops:

• --no-pause runs on from one phase to the next without stopping, so each phase’s verify is the only gate.
• --through <phase> stops after the phase it names, such as --through 0. The run is then unfinished, and --resume continues it.

After a failure

A failed apply or verify stops the run at once. up fails with what failed, followed by the summary, and leaves the target as it is for you to look at:

the run failed at phase 1a's verify: <what failed>
Once the target is fit to resume, continue with up --resume, which applies phase 1a again. Or tear the target down with down.

From there, either:

• resume, once the target is fit to: fix what failed, in either checkout if it needs a change, and run up --resume. It verifies every phase the run already verified, then applies the failed phase again and goes on. If an earlier phase no longer verifies, it stops, changing nothing.
• tear down, with down, below, and start again with a fresh up.

up without --resume refuses to start a new run on top of an unfinished one.

Tear it down

dagger call down --target drill

down tears down every phase in reverse order, phase 0 last, and leaves the target as a rebuild from nothing. It succeeds when there’s nothing left, and it’s safe to run again after it fails partway.

Next: Going live

3 - Going live

Going live is the same rebuild a drill runs, against the live target. The commands are the same; what changes is the target, and two rules that keep a drill from ever reaching it by mistake.

Add the live target

Give the live target a file in the values checkout, targets/live.yaml:

kind: live

Only one target is live: every call refuses to run while a second target’s file says kind: live, as committed or as it stands.

Commit everything first

A run against the live target refuses uncommitted changes, in this repository or in the values checkout, so every change to the live platform is made from history. Commit both, and check that neither has anything left over:

git status
git -C ../infra-values status

Otherwise up stops before it changes anything:

a run against the live target refuses uncommitted changes: the target's values have uncommitted changes

Check the target

dagger call preflight --target live

It changes nothing, and prints that the check passed. If the target’s last run didn’t finish, or a phase that run applied no longer verifies, it fails and says which. See The preflight check.

Run it

dagger call up --target live

As in a drill, up applies and verifies one phase, stops at the checkpoint after it, and prints the run’s summary: what the phase made, how long it took and the run’s time so far. Look at what the phase made, then continue:

dagger call up --target live --resume

Each call runs the next phase, until the summary says the run finished and every phase verified. That run is the target’s last-known-good. A live rebuild is meant to take at most 60 minutes, counting the time spent at checkpoints. The summary marks a run that went over, and it’s never stopped for time.

After a failure

A failed apply or verify stops the run at once, prints what failed with the summary, and leaves the target as it is. Look at what failed, then either:

• resume, once the target is fit to: commit the fix, in whichever checkout needs it, and run up --resume. It verifies every phase the run already verified, then applies the failed phase again and goes on.
• tear down, and start again with a fresh up.

Tearing it down

down against the live target needs the target’s name, to confirm it:

dagger call down --target live --confirm live

Without --confirm live it refuses:

down was not confirmed with the target's name: live is the live target; confirm with "live" to tear it down

It refuses uncommitted changes too, as up does. It tears down every phase in reverse order, phase 0 last, and it’s safe to run again after it fails partway.

Next: Fundamentals