doran/README.md

near-intents-monitor

Production-shaped first slice of the trading system:

• venue ingest: NEAR Intents solver-bus quote flow
• bus: Redpanda first, Kafka-compatible by design
• reactor: dummy decision engine emitting commands
• executor: dummy execution worker with durable idempotency state
• result consumer: downstream observer of execution outcomes

Canonical repo shape

src/
  apps/
    near-intents-ingest.mjs
    dummy-reactor.mjs
    dummy-executor.mjs
    dummy-consumer.mjs
  bus/
    kafka/
      producer.mjs
      consumer.mjs
  core/
    event-envelope.mjs
    executor-state-store.mjs
    log.mjs
    pair-filter.mjs
    schemas.mjs
  lib/
    config.mjs
    env.mjs
  venues/
    near-intents/
      ingest.mjs
      normalize.mjs
      ws.mjs
compose.yml
Dockerfile
docs/contracts.md

Event flow

NEAR Intents WebSocket
        |
        +--> raw.near_intents.quote
        |
        v
norm.swap_demand
        |
        v
cmd.execute_trade
        |
        v
exec.trade_result

Core rule: services do not call each other directly for trading flow; they communicate through bus topics only.

Contracts

See docs/contracts.md.

Current topics:

• raw.near_intents.quote
• norm.swap_demand
• cmd.execute_trade
• exec.trade_result

Canonical deployment path

The canonical production path is the repo-driven Hetzner + k3s bootstrap flow. Compose still exists for local development and optional single-machine testing, but it is not the primary production story.

Current single-node cluster stack includes:

• unrip workloads in namespace unrip
• Redpanda
• Forgejo
• Forgejo runner
• private registry
• cert-manager
• Traefik via the k3s bundled ingress controller
• Grafana
• Loki
• Promtail
• Headlamp

Bootstrap entrypoint

cp scripts/hetzner/bootstrap-secrets.env.example scripts/hetzner/bootstrap-secrets.env
source scripts/hetzner/bootstrap-secrets.env
bash scripts/hetzner/bootstrap.sh

The bootstrap script now:

1. provisions or updates Hetzner infra with Terraform
2. optionally manages DNS via Cloudflare or Porkbun
3. prefers Tailscale for admin/control-plane access when configured
4. fetches kubeconfig from the node into .state/hetzner/kubeconfig.yaml
5. renders .state/hetzner/generated-overlay/ from repo manifests plus local secrets
6. applies platform and project resources to k3s
7. bootstraps Forgejo admin, runner, repo, and Actions configuration
8. seeds this repo into Forgejo
9. lets Forgejo Actions perform the default build/push/deploy path
10. stores the generated Headlamp login token in pass when HEADLAMP_ADMIN_TOKEN_PASS is configured

Detailed bootstrap and destroy documentation lives in:

• docs/hetzner-k3s-bootstrap.md
• docs/hetzner-self-hosted-ci-runbook.md
• docs/k8s-observability.md
• deploy/hetzner/README.md
• deploy/k8s/README.md
• deploy/k8s/overlays/hetzner-single-node/README.md

Runtime surfaces

• Forgejo: https://git.doran.133011.xyz/
• Registry: https://registry.doran.133011.xyz/
• Grafana: https://grafana.doran.133011.xyz/
• Headlamp: https://headlamp.doran.133011.xyz/

Operator notes

• Ingress is Traefik-based. The old ingress-nginx path is obsolete.
• Grafana is for historical log search.
• Headlamp is for browsing workloads, pods, events, and pod logs.
• Use pass-backed *_PASS variables for secrets whenever possible.

Executor persistence in k3s

The executor is stateful by design because it persists idempotency/execution tracking.

Current persistence boundary:

• app env uses EXECUTOR_STATE_DIR=/var/lib/unrip/executor-state
• in Kubernetes, the executor deployment mounts storage at that path
• the Hetzner single-node overlay pins storage to the k3s local-path storage class

Operational meaning:

• executor state lives on node-backed storage in the single-node k3s environment
• if that PVC or underlying node storage is lost, duplicate-suppression history is lost too
• treat executor persistence as part of the minimal durable state of the cluster

Local development with Compose

Compose remains available for local development and debugging.

npm install
cp .env.example .env
# edit .env

docker compose build
docker compose up -d

Useful commands:

docker compose ps
docker compose logs -f
docker compose logs -f near-intents-ingest dummy-reactor dummy-executor dummy-consumer
docker compose restart dummy-executor
docker compose down
docker compose down -v

Individual services

npm run near-intents:ingest
npm run dummy-reactor
npm run dummy-executor
npm run dummy-consumer

Optional pair filter:

npm run near-intents:ingest -- --pair 'asset_a->asset_b'

Idempotent executor behavior

• every command has a command_id
• commands carry idempotency_key and execution_key
• executor persists state under EXECUTOR_STATE_DIR
• completed commands are skipped after restart or replay

Env

NEAR_INTENTS_API_KEY=your_solver_jwt
NEAR_INTENTS_WS_URL=wss://solver-relay-v2.chaindefuser.com/ws
KAFKA_BROKERS=redpanda:9092
KAFKA_CLIENT_ID=unrip
KAFKA_TOPIC_RAW_NEAR_INTENTS_QUOTE=raw.near_intents.quote
KAFKA_TOPIC_NORM_SWAP_DEMAND=norm.swap_demand
KAFKA_TOPIC_CMD_EXECUTE_TRADE=cmd.execute_trade
KAFKA_TOPIC_EXEC_TRADE_RESULT=exec.trade_result
KAFKA_CONSUMER_GROUP_DUMMY=dummy-reactor-v1
KAFKA_CONSUMER_GROUP_EXECUTOR=dummy-executor-v1
EXECUTOR_STATE_DIR=/var/lib/unrip/executor-state