# near-intents-monitor

Production-shaped first slice of the trading system:

- **venue ingest**: NEAR Intents solver-bus quote flow
- **bus**: Redpanda first, Kafka-compatible by design
- **reactor**: dummy decision engine emitting commands
- **executor**: dummy execution worker with durable idempotency state
- **result consumer**: downstream observer of execution outcomes

## Canonical repo shape

```text
src/
  apps/
    near-intents-ingest.mjs
    dummy-reactor.mjs
    dummy-executor.mjs
    dummy-consumer.mjs
  bus/
    kafka/
      producer.mjs
      consumer.mjs
  core/
    event-envelope.mjs
    executor-state-store.mjs
    log.mjs
    pair-filter.mjs
    schemas.mjs
  lib/
    config.mjs
    env.mjs
  venues/
    near-intents/
      ingest.mjs
      normalize.mjs
      ws.mjs
compose.yml
Dockerfile
docs/contracts.md
```

## Event flow

```text
NEAR Intents WebSocket
        |
        +--> raw.near_intents.quote
        |
        v
norm.swap_demand
        |
        v
cmd.execute_trade
        |
        v
exec.trade_result
```

Core rule: services do not call each other directly for trading flow; they communicate through bus topics only.

## Contracts
See `docs/contracts.md`.

Current topics:
- `raw.near_intents.quote`
- `norm.swap_demand`
- `cmd.execute_trade`
- `exec.trade_result`

## Canonical deployment path

The canonical production path is the repo-driven Hetzner + k3s bootstrap flow.
Compose still exists for local development and optional single-machine testing, but it is not the primary production story.

Current single-node cluster stack includes:
- `unrip` workloads in namespace `unrip`
- Redpanda
- Forgejo
- Forgejo runner
- private registry
- cert-manager
- Traefik via the k3s bundled ingress controller
- Grafana
- Loki
- Promtail
- Headlamp

### Bootstrap entrypoint

```bash
cp scripts/hetzner/bootstrap-secrets.env.example scripts/hetzner/bootstrap-secrets.env
source scripts/hetzner/bootstrap-secrets.env
bash scripts/hetzner/bootstrap.sh
```

The bootstrap script now:
1. provisions or updates Hetzner infra with Terraform
2. optionally manages DNS via Cloudflare or Porkbun
3. prefers Tailscale for admin/control-plane access when configured
4. fetches kubeconfig from the node into `.state/hetzner/kubeconfig.yaml`
5. renders `.state/hetzner/generated-overlay/` from repo manifests plus local secrets
6. applies platform and project resources to k3s
7. bootstraps Forgejo admin, runner, repo, and Actions configuration
8. seeds this repo into Forgejo
9. lets Forgejo Actions perform the default build/push/deploy path
10. stores the generated Headlamp login token in `pass` when `HEADLAMP_ADMIN_TOKEN_PASS` is configured

Detailed bootstrap and destroy documentation lives in:
- `docs/hetzner-k3s-bootstrap.md`
- `docs/hetzner-self-hosted-ci-runbook.md`
- `docs/k8s-observability.md`
- `deploy/hetzner/README.md`
- `deploy/k8s/README.md`
- `deploy/k8s/overlays/hetzner-single-node/README.md`

### Runtime surfaces

- Forgejo: `https://git.doran.133011.xyz/`
- Registry: `https://registry.doran.133011.xyz/`
- Grafana: `https://grafana.doran.133011.xyz/`
- Headlamp: `https://headlamp.doran.133011.xyz/`

### Operator notes

- Ingress is Traefik-based. The old ingress-nginx path is obsolete.
- Grafana is for historical log search.
- Headlamp is for browsing workloads, pods, events, and pod logs.
- Use `pass`-backed `*_PASS` variables for secrets whenever possible.

## Executor persistence in k3s

The executor is stateful by design because it persists idempotency/execution tracking.

Current persistence boundary:
- app env uses `EXECUTOR_STATE_DIR=/var/lib/unrip/executor-state`
- in Kubernetes, the executor deployment mounts storage at that path
- the Hetzner single-node overlay pins storage to the k3s `local-path` storage class

Operational meaning:
- executor state lives on node-backed storage in the single-node k3s environment
- if that PVC or underlying node storage is lost, duplicate-suppression history is lost too
- treat executor persistence as part of the minimal durable state of the cluster

## Local development with Compose

Compose remains available for local development and debugging.

```bash
npm install
cp .env.example .env
# edit .env

docker compose build
docker compose up -d
```

Useful commands:

```bash
docker compose ps
docker compose logs -f
docker compose logs -f near-intents-ingest dummy-reactor dummy-executor dummy-consumer
docker compose restart dummy-executor
docker compose down
docker compose down -v
```

### Individual services
```bash
npm run near-intents:ingest
npm run dummy-reactor
npm run dummy-executor
npm run dummy-consumer
```

Optional pair filter:
```bash
npm run near-intents:ingest -- --pair 'asset_a->asset_b'
```

## Idempotent executor behavior
- every command has a `command_id`
- commands carry `idempotency_key` and `execution_key`
- executor persists state under `EXECUTOR_STATE_DIR`
- completed commands are skipped after restart or replay

## Env

```env
NEAR_INTENTS_API_KEY=your_solver_jwt
NEAR_INTENTS_WS_URL=wss://solver-relay-v2.chaindefuser.com/ws
KAFKA_BROKERS=redpanda:9092
KAFKA_CLIENT_ID=unrip
KAFKA_TOPIC_RAW_NEAR_INTENTS_QUOTE=raw.near_intents.quote
KAFKA_TOPIC_NORM_SWAP_DEMAND=norm.swap_demand
KAFKA_TOPIC_CMD_EXECUTE_TRADE=cmd.execute_trade
KAFKA_TOPIC_EXEC_TRADE_RESULT=exec.trade_result
KAFKA_CONSUMER_GROUP_DUMMY=dummy-reactor-v1
KAFKA_CONSUMER_GROUP_EXECUTOR=dummy-executor-v1
EXECUTOR_STATE_DIR=/var/lib/unrip/executor-state
```