unrip/docs/operator-runbook.md
philipp 16e7b79978
All checks were successful
deploy / deploy (push) Successful in 22s
Add durable portfolio metrics
Proof: Persist portfolio value and PnL snapshots from the live inventory and reference-price path so operators can inspect trading performance from repo-controlled data.
Assumptions: The last credited inventory snapshot before the first live command is the correct baseline for trade-driven PnL, and EURe remains explicit 1:1 with EUR.
Still fake: The new portfolio metrics and watch output are implemented and tested locally but are not live until the updated app image is deployed to k3s.
2026-04-03 01:02:27 +02:00

179 lines
7.1 KiB
Markdown

# Operator Runbook
This turn implements the first funded market-maker loop for the active BTC/EURe pair on NEAR Intents.
## Verified venue flow
The implementation follows the official NEAR Intents market-maker path:
1. Funding handles come from the Passive Deposit/Withdrawal Service `deposit_address` RPC for the configured treasury chains.
2. Spendable inventory comes from the Verifier internal ledger on `intents.near` via `mt_batch_balance_of`.
3. Pending deposits remain non-spendable and are tracked from `recent_deposits`.
4. Real market-maker execution is a Solver Relay `quote_response` carrying a signed `token_diff`.
5. Named NEAR accounts need the executor public key registered on `intents.near` via `add_public_key` before live submission will succeed.
The Message Bus settles matched intents on-chain after a user accepts the quote. The executor therefore submits quote responses; it does not bridge or top up inventory on the hot path.
## Required env and secrets
Minimum required runtime values:
- `NEAR_INTENTS_API_KEY`
- `NEAR_INTENTS_ACCOUNT_ID`
- `NEAR_INTENTS_SIGNER_PRIVATE_KEY`
- `POSTGRES_URL`
Before the first live attempt on a named NEAR account, register the executor public key on `intents.near` from that named account:
```bash
near contract call-function as-transaction \
intents.near add_public_key json-args '{
"public_key": "ed25519:<executor-public-key>"
}' prepaid-gas '100.0 Tgas' attached-deposit '1 yoctoNEAR' \
sign-as <ACCOUNT_ID> network-config mainnet sign-with-keychain send
```
The executor stays disarmed by default even after the key is registered.
## Local bring-up
```bash
npm install
cp .env.example .env
# fill NEAR_INTENTS_API_KEY, NEAR_INTENTS_ACCOUNT_ID, NEAR_INTENTS_SIGNER_PRIVATE_KEY
docker compose up -d --build
```
Services:
- `near-intents-ingest`
- `market-reference-ingest`
- `liquidity-manager`
- `inventory-sync`
- `history-writer`
- `strategy-engine`
- `trade-executor`
## Control APIs
Default local ports:
- `8081` `near-intents-ingest`
- `8082` `market-reference-ingest`
- `8083` `inventory-sync`
- `8084` `liquidity-manager`
- `8085` `history-writer`
- `8086` `strategy-engine`
- `8087` `trade-executor`
Common inspection:
```bash
curl -s http://127.0.0.1:8081/healthz
curl -s http://127.0.0.1:8081/state
curl -s http://127.0.0.1:8085/portfolio-metrics
curl -s http://127.0.0.1:8086/state
curl -s http://127.0.0.1:8087/state
```
Live watch:
```bash
python3 scripts/ops/watch_live_mm.py --once
python3 scripts/ops/watch_live_mm.py --backfill 3
```
The watch command tails the live market-maker loop through PostgreSQL history plus
service `/state`:
- matching quotes from `swap_demand_events`
- decisions from `trade_decisions`
- emitted commands from `execute_trade_commands`
- execution results from `trade_execution_results`
- current credited BTC and EURe inventory
- current strategy and executor arming state
Use `--heartbeat-every 30` or a larger value if you want less idle output.
Portfolio metrics:
- `history-writer` now derives durable portfolio metrics from the latest inventory snapshot plus the latest reference price.
- The PnL baseline is the last credited inventory snapshot before the first live `cmd.execute_trade`.
- `trade_pnl_eure` keeps the baseline BTC price fixed to isolate execution PnL from later BTC moves.
- `mark_to_market_pnl_eure` values both the baseline and current books at the latest reference price.
- `price_move_pnl_eure` is the difference between those two, so operators can separate execution edge from subsequent BTC repricing.
Useful controls:
```bash
curl -s -X POST http://127.0.0.1:8082/refresh
curl -s -X POST http://127.0.0.1:8083/refresh
curl -s -X POST http://127.0.0.1:8084/refresh
curl -s -X POST http://127.0.0.1:8084/withdrawal-estimate \
-H 'content-type: application/json' \
-d '{"asset_id":"nep141:gnosis-0x420ca0f9b9b604ce0fd9c18ef134c705e5fa3430.omft.near","amount":"5000000000000000000","destination_address":"0xYourGnosisAddress"}'
curl -s -X POST http://127.0.0.1:8084/freeze-withdrawals \
-H 'content-type: application/json' \
-d '{"frozen":false}'
curl -s -X POST http://127.0.0.1:8084/withdraw \
-H 'content-type: application/json' \
-d '{"asset_id":"nep141:gnosis-0x420ca0f9b9b604ce0fd9c18ef134c705e5fa3430.omft.near","amount":"5000000000000000000","destination_address":"0xYourGnosisAddress"}'
curl -s -X POST http://127.0.0.1:8086/arm
curl -s -X POST http://127.0.0.1:8086/disarm
curl -s -X PUT http://127.0.0.1:8086/limits \
-H 'content-type: application/json' \
-d '{"max_notional_eure":5}'
curl -s -X POST http://127.0.0.1:8087/arm
curl -s -X POST http://127.0.0.1:8087/disarm
```
Track a withdrawal so it stays visible in liquidity and inventory state:
```bash
curl -s -X POST http://127.0.0.1:8084/track-withdrawal \
-H 'content-type: application/json' \
-d '{"withdrawal_hash":"<near-burn-tx-hash>","asset_id":"nep141:btc.omft.near","chain":"btc:mainnet","amount":"1000"}'
```
Notes:
- Deposit addresses are built in. `liquidity-manager` refreshes them from the bridge `deposit_address` RPC and exposes them through `/state`.
- The repo withdrawal action is for external-chain exits on the active assets. It submits `intents.near::ft_withdraw`, using the active OMFT token contract as `receiver_id` and `memo=WITHDRAW_TO:<destination>`, then tracks the returned NEAR transaction hash through bridge `withdrawal_status`.
- `destination_address` can be omitted only when a default withdrawal address is configured for that asset via `TRADING_BTC_WITHDRAW_ADDRESS` or `TRADING_EURE_WITHDRAW_ADDRESS`.
- Withdrawals stay frozen by default. Unfreeze explicitly before calling `/withdraw`, then freeze them again after the operation if you do not want further exits.
## Safe arming sequence
1. Confirm `market-reference-ingest` is publishing fresh BTC/EUR data.
2. Confirm `inventory-sync` shows credited spendable balances on `intents.near`.
3. Confirm `liquidity-manager` shows the expected deposit handle and any pending funding separately from spendable inventory.
4. Confirm `history-writer` has PostgreSQL connectivity.
5. Keep `STRATEGY_MAX_NOTIONAL_EURE=5` for the first live test.
6. Arm `strategy-engine` first.
7. Observe actionable decisions without venue errors.
8. Arm `trade-executor` only when the signer key is registered and funded inventory is already credited.
## What to inspect after a live attempt
- `decision.trade_decision` for the reasoning chain.
- `cmd.execute_trade` for the emitted quote response command.
- `exec.trade_result` for submission outcome.
- PostgreSQL tables:
- `swap_demand_events`
- `market_price_events`
- `intent_inventory_snapshots`
- `portfolio_metrics_snapshots`
- `liquidity_actions`
- `trade_decisions`
- `execute_trade_commands`
- `trade_execution_results`
## Still fake
- Settlement follow-up after user quote acceptance is only visible indirectly through Solver Relay quote-status observations; the repo records the live quote-response attempt, not an end-user acceptance flow it does not control.
- The executor checks signer registration best-effort. If the verifier key-check view surface changes, the live submission itself remains the definitive signal.