107 lines
6.8 KiB
Markdown
107 lines
6.8 KiB
Markdown
# Hetzner single-node overlay
|
|
|
|
This overlay turns the shared platform and `unrip` project bases into a concrete first-node bootstrap target for the Terraform-provisioned k3s VM.
|
|
|
|
The checked-in overlay is the declarative template. For first-cluster bootstrap, `scripts/hetzner/bootstrap.sh` renders a generated overlay under `.state/hetzner/generated-overlay/` and applies that generated copy as the source of truth for the run.
|
|
|
|
## Two ways to use this overlay
|
|
|
|
### 1. Recommended: `scripts/hetzner/bootstrap.sh`
|
|
This is the intended operator workflow for a fresh Hetzner cluster. The bootstrap script renders secret and patch inputs from local env and `pass`, creates imperative registry secrets, and applies a generated Kustomize overlay.
|
|
|
|
That generated overlay now imports the platform resources from `deploy/k8s/platform/base/kustomization.yaml`, so new checked-in platform components such as observability manifests are included automatically during bootstrap instead of being silently skipped by a hard-coded file list.
|
|
|
|
Bootstrap overwrites these operator-worktree files on each run:
|
|
- `deploy/k8s/overlays/hetzner-single-node/secrets/unrip.env`
|
|
- `deploy/k8s/overlays/hetzner-single-node/secrets/forgejo.env`
|
|
- `deploy/k8s/overlays/hetzner-single-node/secrets/observability.env`
|
|
|
|
Bootstrap also renders and applies generated copies of these patch files under `.state/hetzner/generated-overlay/` instead of modifying the checked-in overlay files directly:
|
|
- `ingress-hosts.patch.yaml`
|
|
- `issuer-email.patch.yaml`
|
|
- `storage-class.patch.yaml`
|
|
|
|
Secret/config sources when using bootstrap:
|
|
- from `pass` or direct env overrides via `scripts/hetzner/bootstrap-secrets.env`:
|
|
- `HCLOUD_TOKEN`
|
|
- `TAILSCALE_AUTH_KEY`
|
|
- `CLOUDFLARE_API_TOKEN`
|
|
- `CLOUDFLARE_ZONE_ID`
|
|
- `PORKBUN_API_KEY`
|
|
- `PORKBUN_SECRET_API_KEY`
|
|
- `REGISTRY_PASSWORD`
|
|
- `NEAR_INTENTS_API_KEY`
|
|
- `FORGEJO_ADMIN_PASSWORD`
|
|
- optional `GRAFANA_ADMIN_PASSWORD` (bootstrap generates one if omitted)
|
|
- from plain env/non-secret config in `scripts/hetzner/bootstrap-secrets.env`:
|
|
- `PUBLIC_DOMAIN`, `BASE_DOMAIN`, `FORGEJO_DOMAIN`, `FORGEJO_ROOT_URL`, `REGISTRY_DOMAIN`, `GRAFANA_DOMAIN`, `GRAFANA_ROOT_URL`, `HEADLAMP_DOMAIN`
|
|
- default hostname model under `PUBLIC_DOMAIN`: `git.${PUBLIC_DOMAIN}`, `registry.${PUBLIC_DOMAIN}`, `grafana.${PUBLIC_DOMAIN}`, `headlamp.${PUBLIC_DOMAIN}`
|
|
- `LETSENCRYPT_EMAIL`
|
|
- `REGISTRY_USERNAME`
|
|
- `FORGEJO_ADMIN_USERNAME`, `FORGEJO_ADMIN_EMAIL`
|
|
- optional `GRAFANA_ADMIN_USERNAME` (defaults to `admin`)
|
|
- optional project overrides such as `PROJECT_NAME`, `PROJECT_NAMESPACE`, and `PROJECT_SECRET_ENV_BASENAME`
|
|
|
|
Bootstrap materializes Kubernetes inputs like this:
|
|
- `secrets/unrip.env` gets `NEAR_INTENTS_API_KEY`
|
|
- `secrets/forgejo.env` gets only `root_url` and `domain`
|
|
- `secrets/observability.env` gets `grafana_admin_user`, `grafana_admin_password`, and `grafana_root_url`
|
|
- generated overlay Kustomize secret generators create `observability-secrets` in namespace `observability` alongside the project and Forgejo secrets
|
|
- `registry-secrets` in namespace `registry` is created imperatively from `REGISTRY_USERNAME` and `REGISTRY_PASSWORD`
|
|
- `<project>-registry-creds` image pull secret is created imperatively in the project namespace from the same registry credentials
|
|
|
|
Note: the Forgejo runner no longer reads `runner_registration_token` from `forgejo-secrets`. `scripts/hetzner/bootstrap.sh` generates a one-time runner token in-cluster, registers the runner, and writes `/data/forgejo-runner/.runner` on the shared Forgejo PVC before restarting the runner deployment.
|
|
|
|
### 2. Manual: `kubectl apply -k`
|
|
Use this only if you intentionally want to manage the checked-in overlay inputs yourself. In manual mode, the checked-in overlay remains the source of truth; in bootstrap mode, the generated overlay is the source of truth for what gets applied.
|
|
|
|
Before apply, create or edit real local input files:
|
|
|
|
```bash
|
|
cp deploy/k8s/overlays/hetzner-single-node/secrets/unrip.env.example deploy/k8s/overlays/hetzner-single-node/secrets/unrip.env
|
|
cp deploy/k8s/overlays/hetzner-single-node/secrets/forgejo.env.example deploy/k8s/overlays/hetzner-single-node/secrets/forgejo.env
|
|
cp deploy/k8s/overlays/hetzner-single-node/secrets/observability.env.example deploy/k8s/overlays/hetzner-single-node/secrets/observability.env
|
|
cp deploy/k8s/overlays/hetzner-single-node/secrets/registry.htpasswd.example deploy/k8s/overlays/hetzner-single-node/secrets/registry.htpasswd
|
|
```
|
|
|
|
Then update:
|
|
- ingress hosts in `ingress-hosts.patch.yaml` for Forgejo, Registry, Grafana, and Headlamp
|
|
- ACME email in `issuer-email.patch.yaml`
|
|
- project secret values in `secrets/unrip.env`
|
|
- Forgejo secret values in `secrets/forgejo.env` (`root_url` and `domain` only)
|
|
- observability secret values in `secrets/observability.env` (`grafana_admin_user`, `grafana_admin_password`, `grafana_root_url`)
|
|
|
|
Important manual-mode caveat:
|
|
- `kubectl apply -k deploy/k8s/overlays/hetzner-single-node` creates only the Kustomize-managed secrets from the checked-in files (`unrip-secrets`, `forgejo-secrets`, `observability-secrets`, and `registry-secrets` when `secrets/registry.htpasswd` exists)
|
|
- it does **not** create the project docker-registry pull secret
|
|
- if you skip `scripts/hetzner/bootstrap.sh`, you must create that pull secret separately before expecting image pulls or CI builds to work
|
|
|
|
## Apply
|
|
Bootstrap path:
|
|
```bash
|
|
bash scripts/hetzner/bootstrap.sh
|
|
```
|
|
|
|
Manual path:
|
|
```bash
|
|
kubectl apply -k deploy/k8s/overlays/hetzner-single-node
|
|
```
|
|
|
|
## What gets installed
|
|
- shared platform namespaces for registry, ingress, cert-manager, Forgejo, and observability
|
|
- project namespace `unrip`
|
|
- Redpanda plus a topic bootstrap job inside `unrip`
|
|
- app worker deployments referencing `unrip-secrets`
|
|
- Forgejo and Forgejo runner referencing `forgejo-secrets`
|
|
- private registry workload, which still requires the imperative `registry-secrets` auth secret to be created separately unless you used `scripts/hetzner/bootstrap.sh`
|
|
- nginx ingress and ACME issuers for TLS
|
|
- observability ingress for Grafana and Headlamp, plus local-path PVC overrides for Grafana and Loki
|
|
|
|
## Observability UI exposure policy
|
|
- Grafana and Headlamp are both wired into the Hetzner ingress/domain model.
|
|
- Use `grafana.${PUBLIC_DOMAIN}` / `headlamp.${PUBLIC_DOMAIN}` or explicit `GRAFANA_DOMAIN` / `HEADLAMP_DOMAIN` values.
|
|
- Grafana is the historical log search UI backed by Loki.
|
|
- Headlamp is the Kubernetes cluster UI for workloads, events, and pod logs.
|
|
- Grafana is authenticated through `observability-secrets`; Headlamp is authenticated with the generated Kubernetes service-account token that bootstrap stores in `pass` when `HEADLAMP_ADMIN_TOKEN_PASS` is configured.
|
|
|
|
For future projects, do not reuse `unrip`; create a new project namespace and matching `<project>-config`, `<project>-secrets`, and `<project>-registry-creds` resources.
|