unrip/README.md

# unrip project

This repository contains the unrip trading-system code and its project-specific deployment assets.

## Contents

- `src/` — application code
- `package.json` / `package-lock.json` — Node package manifest
- `Dockerfile` / `.dockerignore` — app container build
- `.env.example` — local app runtime example
- `compose.yml` — local development stack
- `deploy/k8s/base/` — project-specific Kubernetes manifests
- `deploy/redpanda/rpk-topics.txt` — project topic reference
- `docs/` — project-specific design and contract docs

## Local development

```bash
npm install
cp .env.example .env
# edit .env

docker compose up -d --build
```

Useful commands:

```bash
docker compose ps
docker compose logs -f
docker compose logs -f near-intents-ingest dummy-reactor dummy-executor dummy-consumer
npm run near-intents:ingest
npm run dummy-reactor
npm run dummy-executor
npm run dummy-consumer
```

## App image

The app image is now built from this directory.

Examples:

```bash
docker build -t unrip:dev .
```

## Kubernetes manifests

Project manifests live under:

- `deploy/k8s/base/`

The shared cluster/platform resources live in the separate infra repository.

## Deployment

This repo is the app-side deployment repo. The shared Hetzner/k3s bootstrap,
Forgejo runner, registry, and other platform services live in the separate
platform repo.

See `docs/deployment.md` for the full operator path.

### One-time app bootstrap

Bootstrap the app namespace, secrets, and Forgejo repo settings from this repo:

```bash
bash scripts/deploy/bootstrap.sh
```

That bootstrap also refreshes the local `forgejo` remote URL for HTTPS pushes
when it has enough auth material to do so.

By default, the script uses the adjacent platform checkout at `../unrip3` for:
- `kubeconfig.yaml`
- `kubeconfig.incluster.yaml`
- registry credentials
- the `NEAR_INTENTS_API_KEY` fallback from `../unrip3/.env`

If you are not using that local split, provide the values yourself via env vars
such as `KUBECONFIG_PATH`, `CI_KUBECONFIG_PATH`, `REGISTRY_HOST`,
`REGISTRY_USERNAME`, `REGISTRY_PASSWORD`, `NEAR_INTENTS_API_KEY`, and either
`FORGEJO_TOKEN` or `FORGEJO_ADMIN_USERNAME` / `FORGEJO_ADMIN_PASSWORD`.

### Routine deploy

After bootstrap, deployment is just a push to Forgejo `main`:

```bash
git push forgejo main
```

`.forgejo/workflows/deploy.yml` then:
- applies `deploy/k8s/base`
- builds the image from this repo root inside the cluster with Kaniko
- pushes it to the shared registry
- rolls the `unrip` deployments
- uses a fresh temporary runner workspace on each run, so reruns do not require
  manual cleanup on the Forgejo runner

### Observe rollout

```bash
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip get deploy,pods,job
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/near-intents-ingest
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/dummy-reactor
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/dummy-executor
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/dummy-consumer
```

### Auxiliary ops scripts

These scripts default to the same adjacent platform checkout as the deployment
bootstrap: `../unrip3/.state/hetzner/kubeconfig.yaml`. Override with
`KUBECONFIG_PATH`, `KUBECONFIG`, or `PLATFORM_REPO_DIR` if needed.

`scripts/ops/deployment_status.py`
- Shows current deployment readiness, pod uptime, restart counts, and mounted storage usage.
- By default it shows the live deployment pods only.
- Use `--include-completed` to include completed Job pods.
- Use `--include-rootfs` if you also want a probe of `/` for pods without PVC-backed mounts.

```bash
python3 scripts/ops/deployment_status.py
python3 scripts/ops/deployment_status.py --include-completed
```

`scripts/ops/redpanda_storage.py`
- Shows how much data Redpanda is currently storing for the unrip topics.
- Prints per-topic local bytes, total bytes, segment counts, and the overall Redpanda data-path usage.
- Use `--all-topics` to inspect every visible topic, or `--topic` multiple times for a subset.

```bash
python3 scripts/ops/redpanda_storage.py
python3 scripts/ops/redpanda_storage.py --all-topics
python3 scripts/ops/redpanda_storage.py --topic raw.near_intents.quote --topic norm.swap_demand
```

`scripts/ops/live_near_intents.py`
- Live-inspects the raw NEAR quote stream entering Redpanda.
- Defaults to the configured raw topic, `--offset end`, and an unbounded live tail.
- Use `--num` for a bounded sample, `--offset start` to read from the beginning, `--value-only` for payload-only output, or `--rpk-json` for the full record metadata emitted by `rpk`.
- Use `--timeout` when you want the script to stop automatically.

```bash
python3 scripts/ops/live_near_intents.py
python3 scripts/ops/live_near_intents.py --num 10 --offset start
python3 scripts/ops/live_near_intents.py --value-only --timeout 30
python3 scripts/ops/live_near_intents.py --rpk-json --num 5
```

### Near Intents control API

`near-intents-ingest` exposes a small in-process control API on port `8081` by
default. It is meant for ad hoc inspection and runtime filter changes without a
redeploy.

Port-forward the deployment:

```bash
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip port-forward deploy/near-intents-ingest 8081:8081
```

Inspect current state, including the active pair filter and ingest counters:

```bash
curl -s http://127.0.0.1:8081/state
```

Set or disable the runtime pair filter:

```bash
curl -s -X PUT http://127.0.0.1:8081/pair-filter \
  -H 'content-type: application/json' \
  -d '{"pair":"nep141:btc.omft.near->nep141:eth.omft.near"}'

curl -s -X PUT http://127.0.0.1:8081/pair-filter \
  -H 'content-type: application/json' \
  -d '{"pair":null}'
```

Reset the runtime filter back to the configured env/file/default state:

```bash
curl -s -X POST http://127.0.0.1:8081/pair-filter/reset
```