Skip to main content

ADR-0006: Deploy to Northflank Through GitOps

Status: Accepted, implemented (staging 2026-10-26, prod 2026-11-01)
Date: 2026-10-21
Deciders: Team

Context​

The backend (api and the Centrifugo broker) and the observability agent (alloy) run on Northflank, in the projects qubital-staging and qubital-prod. Until October 2026, CI deployed by calling the Northflank API: the staging pipeline ran a Northflank template with the image as a run argument, and the prod promote changed each service's image directly.

  • Three writers, no source of truth. CI, template runs and dashboard edits each silently reverted the others; a template run that put back a stale dashboard value took staging down on 2026-10-16.
  • Secrets in plain sight. 54 values sat in template argument overrides, readable by any token with read access, and every template run kept its own copy.
  • Write tokens in GitHub. Each pipeline held a Northflank token that could change services.
  • Tags, not digests. Services ran re-pointable tags, so what ran was not fixed by its reference.
  • ADR-0004 left open how a commit maps to an environment once branches no longer do it.

Decision​

  • Git is the desired state. The private repo qubital-infra holds one Northflank template file per deploy unit: <env>/project.json, <env>/redis.json, <env>/backend.json, and prod's alloy.json. Each file is linked to a Northflank template by template GitOps, so a merge that touches a file is a deploy.
  • Producers build, qubital-infra declares, Northflank applies. Producer CI pushes an image to GHCR, then commits its digest to the file through the GitHub App qubital-infra-deployer. No producer repo holds a Northflank token.
  • verify checks what runs after every push to qubital-infra's main that touches a template file: the template synced, its run succeeded, every service runs the file's image, and nothing restarted that shouldn't have. Its token is read-only.
  • Staging deploys on every merge to the backend's main. A backend release reaches prod only through the promote, which copies the exact digests staging ran and verified. Alloy deploys from its own dispatch; config changes and reverts are pull requests to qubital-infra.
  • Secret values live only in Northflank secret groups. The repo lists their names.
  • People don't edit templates or services in the Northflank UI. Changes are pull requests to qubital-infra; break-glass belongs to the owner.

The design's 21 decisions and the platform behaviour they rely on are recorded in qubital-infra (architecture, spikes).

Alternatives Considered​

  • Keep CI calling the Northflank API — rejected: it keeps write tokens in every pipeline and three writers on the same services.
  • Keep images and secrets as template run arguments — rejected: any read token reads every run's arguments, and a run puts the whole template back, reverting dashboard edits.
  • One template per environment — rejected: one failure blocks every deploy, and every deploy re-applies Redis.

Consequences​

  • Every change is a reviewed, revertable commit. A rollback is a git revert of the deploy commit, never across a DB migration.
  • The Northflank UI becomes read-mostly for people: logs, shell, restarts and secret groups. An edit made to a service is reverted by the next deploy.
  • Prod runs exactly what staging ran: promotion is by digest, guarded by staging's verify and by the repo's validate rules.
  • A new service needs a deploy path: its node in a file, its secret group, and a producer step that writes its digest.
  • A new failure mode: when Northflank's template sync is down, deploys go red at verify, and a push made during the outage isn't replayed. The deploy runbooks cover it.