unrip project

This repository contains the unrip trading-system code and its project-specific deployment assets.

Contents

• src/ — application code
• package.json / package-lock.json — Node package manifest
• Dockerfile / .dockerignore — app container build
• .env.example — local app runtime example
• compose.yml — local development stack
• deploy/k8s/base/ — project-specific Kubernetes manifests
• deploy/redpanda/rpk-topics.txt — project topic reference
• docs/ — project-specific design and contract docs

Local development

npm install
cp .env.example .env
# edit .env

docker compose up -d --build

Useful commands:

docker compose ps
docker compose logs -f
docker compose logs -f \
  near-intents-ingest market-reference-ingest liquidity-manager \
  inventory-sync history-writer strategy-engine trade-executor
npm run near-intents:ingest
npm run market-reference:ingest
npm run liquidity:manager
npm run inventory:sync
npm run history:writer
npm run strategy:engine
npm run trade:executor

App image

The app image is now built from this directory.

Examples:

docker build -t unrip:dev .

Kubernetes manifests

Project manifests live under:

• deploy/k8s/base/

The shared cluster/platform resources live in the separate infra repository.

Deployment

This repo is the app-side deployment repo. The shared Hetzner/k3s bootstrap, Forgejo runner, registry, and other platform services live in the separate platform repo.

See docs/deployment.md for the full operator path. See docs/operator-runbook.md for the live turn control and arming sequence.

One-time app bootstrap

Bootstrap the app namespace, secrets, and Forgejo repo settings from this repo:

bash scripts/deploy/bootstrap.sh

That bootstrap also refreshes the local forgejo remote URL for HTTPS pushes when it has enough auth material to do so.

By default, the script uses the adjacent platform checkout at ../unrip3 for:

• kubeconfig.yaml
• kubeconfig.incluster.yaml
• registry credentials
• the NEAR_INTENTS_API_KEY fallback from ../unrip3/.env

If you are not using that local split, provide the values yourself via env vars such as KUBECONFIG_PATH, CI_KUBECONFIG_PATH, REGISTRY_HOST, REGISTRY_USERNAME, REGISTRY_PASSWORD, NEAR_INTENTS_API_KEY, and either FORGEJO_TOKEN or FORGEJO_ADMIN_USERNAME / FORGEJO_ADMIN_PASSWORD.

Routine deploy

After bootstrap, deployment is just a push to Forgejo main:

git push forgejo main

.forgejo/workflows/deploy.yml then:

• applies deploy/k8s/base
• builds the image from this repo root inside the cluster with Kaniko
• pushes it to the shared registry
• rolls the unrip deployments
• uses a fresh temporary runner workspace on each run, so reruns do not require manual cleanup on the Forgejo runner

Observe rollout

KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip get deploy,pods,job
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/near-intents-ingest
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/market-reference-ingest
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/liquidity-manager
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/inventory-sync
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/history-writer
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/strategy-engine
KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip rollout status deploy/trade-executor

Auxiliary ops scripts

These scripts default to the same adjacent platform checkout as the deployment bootstrap: ../unrip3/.state/hetzner/kubeconfig.yaml. Override with KUBECONFIG_PATH, KUBECONFIG, or PLATFORM_REPO_DIR if needed.

scripts/ops/deployment_status.py

• Shows current deployment readiness, pod uptime, restart counts, and mounted storage usage.
• Outputs JSON by default so it can be piped directly into jq.
• By default it shows the live deployment pods only.
• Use --include-completed to include completed Job pods.
• Use --include-rootfs if you also want a probe of / for pods without PVC-backed mounts.
• Use --output table for the human-readable table view.

python3 scripts/ops/deployment_status.py
python3 scripts/ops/deployment_status.py | jq '.pods[] | {name, uptime}'
python3 scripts/ops/deployment_status.py --include-completed
python3 scripts/ops/deployment_status.py --output table

scripts/ops/redpanda_storage.py

• Shows how much data Redpanda is currently storing for the unrip topics.
• Outputs JSON by default so it can be piped directly into jq.
• Includes per-topic local bytes, total bytes, segment counts, and the overall Redpanda data-path usage.
• Use --all-topics to inspect every visible topic, or --topic multiple times for a subset.
• Use --output table for the human-readable table view.

python3 scripts/ops/redpanda_storage.py
python3 scripts/ops/redpanda_storage.py | jq '.totals'
python3 scripts/ops/redpanda_storage.py --all-topics
python3 scripts/ops/redpanda_storage.py --topic raw.near_intents.quote --topic norm.swap_demand
python3 scripts/ops/redpanda_storage.py --output table

scripts/ops/live_near_intents.py

• Reads the raw NEAR quote stream entering Redpanda.
• Outputs clean JSON by default. Bounded reads return a JSON array that can be piped directly into jq.
• --num N means "consume N records starting from the chosen offset". With the default --offset end, that means "wait for N new records from now".
• Use --last N when you want the most recent retained N records.
• Use --offset start to read from the beginning of retained history.
• Use --output text for the human-readable stream view or --output jsonl for one JSON object per line.
• Use --value-only to emit only decoded record values in JSON mode.
• Use --timeout when you want the script to stop automatically.

python3 scripts/ops/live_near_intents.py --last 10
python3 scripts/ops/live_near_intents.py --last 10 | jq '.[].value.payload.message.quote_id'
python3 scripts/ops/live_near_intents.py --num 10 --offset start
python3 scripts/ops/live_near_intents.py --num 10 --timeout 30
python3 scripts/ops/live_near_intents.py --value-only --last 5
python3 scripts/ops/live_near_intents.py --output text

Near Intents control API

near-intents-ingest exposes a small in-process control API on port 8081 by default. It is meant for ad hoc inspection and runtime filter changes without a redeploy.

Port-forward the deployment:

KUBECONFIG=../unrip3/.state/hetzner/kubeconfig.yaml kubectl -n unrip port-forward deploy/near-intents-ingest 8081:8081

Inspect current state, including the active pair filter and ingest counters:

curl -s http://127.0.0.1:8081/state

Set or disable the runtime pair filter:

curl -s -X PUT http://127.0.0.1:8081/pair-filter \
  -H 'content-type: application/json' \
  -d '{"pair":"nep141:btc.omft.near->nep141:eth.omft.near"}'

curl -s -X PUT http://127.0.0.1:8081/pair-filter \
  -H 'content-type: application/json' \
  -d '{"pair":null}'

Reset the runtime filter back to the configured env/file/default state:

curl -s -X POST http://127.0.0.1:8081/pair-filter/reset

Agent workflow

This repo now carries a small tracked workflow layer for Codex or other agents. It is meant to create pressure toward real product progress without introducing heavy orchestration.

The key files are:

• AGENTS.md — hard rules for agent behavior in this repo
• THESIS.md — stable product intent and approval boundaries
• PROOF.md — the active implementation proof
• IMPLEMENTATION.md — the current implementation turn
• research/ACTIVE.md — the active research charter
• BACKLOG.md — parked ideas, bugs, and future turn candidates
• ARCHIVE.md — index of archived turns and planning events
• workflow/REVIEW_PROMPT.md — adversarial review prompt for a separate review-only run

Two important rules shape the workflow:

• quote collection and analytics are first-class from day one
• backlog items do not automatically become active implementation without an explicit turn-opening step

Install the tracked git hook

Install the tracked hook path once per clone:

bash scripts/workflow/install_hooks.sh

That sets core.hooksPath to .githooks.

The commit hook rejects non-merge commits unless the commit message body contains:

• Proof: ...
• Assumptions: ...
• Still fake: ...

Workflow scripts

scripts/workflow/add_backlog.py

• Append a new idea, bug, research item, or ops task to BACKLOG.md.
• Prints the created stable backlog ID.

python3 scripts/workflow/add_backlog.py --lane implementation --summary "Durable sink for normalized events"
python3 scripts/workflow/add_backlog.py --lane research --summary "Test whether quote freshness predicts worse downstream execution"
python3 scripts/workflow/add_backlog.py --lane bug --summary "Replay output drops pair metadata"

scripts/workflow/open_turn.py

• Opens a new implementation or research turn.
• Pulls selected backlog items into the active turn and removes them from BACKLOG.md.
• Refuses to overwrite an already-open turn unless --force is passed.
• Use --commit if you want the planning change committed automatically.

python3 scripts/workflow/open_turn.py \
  --lane implementation \
  --title "bounded replay for durable quote history" \
  --summary "Replay recent history for the configured pair from the durable store." \
  --pick I002 \
  --pick B001

scripts/workflow/close_turn.py

• Archives the current implementation or research turn into archive/.
• Resets the live turn file back to idle.
• Updates ARCHIVE.md.
• Use --commit if you want the archive change committed automatically.

python3 scripts/workflow/close_turn.py \
  --lane implementation \
  --status passed \
  --summary "Durable history landed in cluster storage and replay works for recent windows."

Possible close statuses are:

• passed
• failed
• paused
• abandoned

scripts/workflow/review_diff.sh

• Builds a review bundle consisting of a git diff plus the adversarial review prompt.
• Intended for a separate review-only agent run.

bash scripts/workflow/review_diff.sh HEAD~1
bash scripts/workflow/review_diff.sh main...HEAD

Current workflow state

At the moment, the seeded active implementation proof is in:

• PROOF.md
• IMPLEMENTATION.md

That proof is focused on the first real quote -> reference-price -> decision -> execute loop for the live configured pair, with PostgreSQL as the first durable audit and analytics store behind the Kafka backbone.