108 lines
3.7 KiB
Markdown
108 lines
3.7 KiB
Markdown
# Hetzner self-hosted CI/CD runbook
|
|
|
|
This is the operator runbook for the handoff from local bootstrap to self-hosted Forgejo-based deployment.
|
|
|
|
## Bootstrap prerequisites
|
|
From your workstation:
|
|
|
|
```bash
|
|
cp scripts/hetzner/bootstrap-secrets.env.example scripts/hetzner/bootstrap-secrets.env
|
|
source scripts/hetzner/bootstrap-secrets.env
|
|
bash scripts/hetzner/bootstrap.sh
|
|
```
|
|
|
|
After that you should have:
|
|
- `.state/hetzner/kubeconfig.yaml`
|
|
- Forgejo reachable at `https://${FORGEJO_DOMAIN}`
|
|
- Registry reachable at `https://${REGISTRY_DOMAIN}`
|
|
- private admin/control-plane access over Tailscale if configured
|
|
|
|
## Verify the cluster
|
|
```bash
|
|
export KUBECONFIG=$PWD/.state/hetzner/kubeconfig.yaml
|
|
kubectl get nodes -o wide
|
|
kubectl get pods -A
|
|
kubectl -n forgejo get deploy,pods,svc,ingress
|
|
kubectl -n registry get deploy,pods,svc,ingress
|
|
kubectl -n unrip get deploy,pods
|
|
```
|
|
|
|
## Seed the repo into Forgejo
|
|
Create the target repo in Forgejo, then from your workstation:
|
|
|
|
```bash
|
|
git remote add forgejo https://${FORGEJO_DOMAIN}/<owner>/<repo>.git
|
|
git push forgejo main
|
|
```
|
|
|
|
## Configure Forgejo Actions secrets and variables
|
|
Create these repository secrets in Forgejo:
|
|
- `KUBECONFIG_B64`
|
|
- `REGISTRY_USERNAME`
|
|
- `REGISTRY_PASSWORD`
|
|
|
|
Create these repository variables:
|
|
- `REGISTRY_HOST=${REGISTRY_DOMAIN}`
|
|
- optional: `PROJECT_NAME=unrip`
|
|
- optional: `PROJECT_NAMESPACE=unrip`
|
|
- optional: `PROJECT_DEPLOYMENTS=near-intents-ingest,dummy-reactor,dummy-executor,dummy-consumer`
|
|
|
|
Generate `KUBECONFIG_B64` from the bootstrap kubeconfig:
|
|
|
|
```bash
|
|
base64 -w0 .state/hetzner/kubeconfig.yaml
|
|
```
|
|
|
|
## Workflow behavior
|
|
The workflow in `.forgejo/workflows/deploy.yml` now:
|
|
1. installs `buildah` and `kubectl` on the Forgejo runner
|
|
2. checks out the repo with the Forgejo job token
|
|
3. loads kubeconfig from `KUBECONFIG_B64`
|
|
4. logs into the private registry
|
|
5. builds `registry.<domain>/<project-name>:${GIT_SHA}` with `buildah`
|
|
6. pushes the image
|
|
7. updates each deployment listed in `PROJECT_DEPLOYMENTS` inside `PROJECT_NAMESPACE`
|
|
8. waits for rollout after each image update
|
|
|
|
Default behavior if you do not set project variables:
|
|
- `PROJECT_NAME=unrip`
|
|
- `PROJECT_NAMESPACE=unrip`
|
|
- `PROJECT_DEPLOYMENTS=near-intents-ingest,dummy-reactor,dummy-executor,dummy-consumer`
|
|
|
|
For a future project, reuse the same workflow by changing only the Forgejo repository variables instead of copying the workflow.
|
|
|
|
The first bootstrap deploy is different from routine CI:
|
|
- bootstrap fetches the real kubeconfig from the node and imports a local bootstrap image directly into k3s
|
|
- routine CI is intended to push versioned images to the private registry
|
|
|
|
## Trigger deploys
|
|
Push to `main` in Forgejo:
|
|
|
|
```bash
|
|
git push forgejo main
|
|
```
|
|
|
|
## Observe deploys
|
|
```bash
|
|
export KUBECONFIG=$PWD/.state/hetzner/kubeconfig.yaml
|
|
kubectl -n unrip rollout status deployment/near-intents-ingest --timeout=300s
|
|
kubectl -n unrip rollout status deployment/dummy-reactor --timeout=300s
|
|
kubectl -n unrip rollout status deployment/dummy-executor --timeout=300s
|
|
kubectl -n unrip rollout status deployment/dummy-consumer --timeout=300s
|
|
kubectl -n unrip get pods -o wide
|
|
kubectl get events -A --sort-by=.lastTimestamp | tail -n 50
|
|
```
|
|
|
|
## DNS and TLS
|
|
If DNS automation was enabled during bootstrap, A records for the base, Forgejo, and registry hosts are already managed from the repo-side bootstrap.
|
|
|
|
Currently supported DNS providers:
|
|
- Cloudflare
|
|
- Porkbun
|
|
|
|
TLS is issued by cert-manager using the rendered Let's Encrypt email and ingress hosts.
|
|
|
|
## Current limitations
|
|
- Forgejo admin bootstrap and repository creation are not yet API-automated.
|
|
- Forgejo repository secrets/variables still need to be populated before the first real deploy run.
|
|
- The runner currently uses host-mode jobs and installs `buildah`/`kubectl` at job start, which is functional but not yet optimized.
|