unrip/archive/implementation/20260402T230445Z-first-non-mocked-tradeable-loop-for-one-pair-proof.md

# Implementation Proof: first non-mocked tradeable loop for one pair

Status: open
Opened: 2026-04-01

## Target outcome
The active turn is not complete when the repo merely observes demand or emits placeholder commands. It is complete only when the deployed system can make a real Near Intents trade attempt for the active BTC/Gnosis EURe pair using pre-funded inventory already credited inside NEAR Intents, with the full chain stored durably for later analytics and backtesting.

## Hypothesis
`unrip` becomes materially more real once one live NEAR swap-demand event can move through this full chain without mocks:

1. real quote and demand ingestion
2. live external reference pricing
3. synchronized spendable inventory state from NEAR Intents
4. auditable strategy decision
5. inventory and safety gating
6. real Near Intents execution attempt using credited internal inventory
7. durable storage of the full event chain

If any of those links is still dummy, manual-only, or opaque after the fact, the proof is not achieved.

## Active pair and trading rule
- Active pair:
  - `nep141:btc.omft.near`
  - `nep141:gnosis-0x420ca0f9b9b604ce0fd9c18ef134c705e5fa3430.omft.near`
- Both directions must be implemented in the same decision and execution pipeline:
  - BTC -> EURe
  - EURe -> BTC
- Runtime spendable inventory inside NEAR Intents decides which direction can actually fire.
- External Gnosis and BTC wallets are treasury and funding inputs, not the per-trade spend path.
- Pending deposits or withdrawals must not be treated as spendable inventory.
- Initial decision rule:
  - use Kraken plus CoinGecko reference prices
  - make the EURe/EUR pricing assumption explicit in code and decision records
  - require a gross edge of at least 2%
  - require fresh reference pricing
  - require fresh internal inventory state
  - require sufficient spendable source-asset inventory already credited inside NEAR Intents
  - keep strategy and executor disarmed by default on deploy
  - start with a very small max notional for the first live attempt

## Required services

### `near-intents-ingest`
Responsibilities:
- connect to the live NEAR Intents quote feed
- maintain the active pair filter
- publish raw venue quotes
- publish normalized swap-demand events
- expose current ingest state

Must not:
- make trading decisions
- execute trades

### `market-reference-ingest`
Responsibilities:
- maintain current reference pricing for BTC/EUR and mapped BTC/EURe fair value
- use Kraken as the primary fast path
- use CoinGecko as fallback or cross-check
- publish reference-price events
- expose latest price state, freshness, and source health

Must not:
- emit trade commands

### `inventory-sync`
Responsibilities:
- read current spendable inventory from the NEAR Intents internal ledger or equivalent supported inventory surface
- distinguish credited inventory from pending deposits and withdrawals
- publish or serve current internal inventory state
- expose freshness, last successful sync time, and reconciliation status

Must not:
- initiate treasury movements
- make trading decisions

### `liquidity-manager`
Responsibilities:
- request or manage deposit addresses for supported treasury assets and chains
- track funding actions from external treasury wallets into NEAR Intents
- track pending deposits, credited deposits, withdrawals, and rebalance actions
- expose the current funding and treasury state
- publish or record liquidity actions so they are auditable

Must not:
- execute trading strategy
- spend inventory on the trade hot path

### `history-writer`
Responsibilities:
- consume the core Kafka topics
- write append-only records into PostgreSQL
- preserve enough IDs and timestamps to reconstruct causality
- expose writer lag, last committed offsets, and write health

Must not:
- serve as the message bus
- make trading decisions

### `strategy-engine`
Responsibilities:
- consume normalized demand plus fresh reference prices
- consume fresh spendable inventory state
- compute implied rate from the quote
- compare quote rate against external reference pricing
- apply the initial 2% gross edge threshold
- apply freshness, arming, and notional checks
- apply inventory-aware direction gating
- emit auditable decision events
- emit `cmd.execute_trade` only when a decision is actionable

Must not:
- hold private keys
- execute venue actions directly

### `trade-executor`
Responsibilities:
- consume `cmd.execute_trade`
- load the correct Near Intents signing authority at runtime
- perform the actual Near Intents submission against pre-funded internal inventory
- preserve idempotency and duplicate suppression
- publish `exec.trade_result`
- expose current arm state, recent attempts, and last venue error

Must not:
- invent trading logic
- bypass Kafka and manual safety gates
- perform ad hoc bridging or treasury funding on the trade hot path

## Required durable storage
PostgreSQL is the first durable analytics and audit store.

It must store at least:
- raw quotes
- normalized demand
- reference-price snapshots
- internal inventory snapshots
- liquidity and funding actions
- strategy decisions
- trade commands
- execution results
- treasury status and reconciliation metadata

Why PostgreSQL first:
- fast enough for the current streaming volume
- queryable for inspection and analytics
- simple to operate in the current cluster
- good fit for append-only audit plus current-state views

Kafka remains the streaming backbone. PostgreSQL is not the hot transport path.

## Required control surface
Every long-running service in this turn must expose a small HTTP control surface. The point is not a polished UI. The point is operability.

At minimum every service must expose:
- `GET /healthz`
- `GET /state`

Services with runtime controls must also expose the relevant action endpoints:

### `near-intents-ingest`
- inspect:
  - pair filter
  - connection state
  - frames received
  - published counts
- control:
  - update pair filter
  - disable or reset pair filter

### `market-reference-ingest`
- inspect:
  - latest Kraken price
  - latest CoinGecko price
  - derived fair price
  - freshness age
  - source health
- control:
  - pause or resume polling or streaming
  - trigger ad hoc refresh

### `inventory-sync`
- inspect:
  - latest spendable balances by asset
  - pending deposits and withdrawals
  - last sync time
  - reconciliation status
- control:
  - trigger sync
  - pause or resume sync

### `liquidity-manager`
- inspect:
  - active deposit addresses
  - recent funding actions
  - pending deposits
  - credited deposits
  - recent withdrawals
- control:
  - request or rotate deposit address where supported
  - refresh treasury status
  - pause or resume funding trackers
  - disable withdrawals or rebalance actions

### `history-writer`
- inspect:
  - last persisted offsets by topic
  - last write time
  - error count
  - database connectivity
- control:
  - pause writes
  - resume writes
  - drain and stop cleanly

### `strategy-engine`
- inspect:
  - arm state
  - active threshold
  - latest reference snapshot used
  - latest inventory snapshot used
  - latest decisions and reasons
  - skipped counts by reason
- control:
  - arm or disarm
  - pause or resume decisions
  - change threshold
  - set notional cap

### `trade-executor`
- inspect:
  - arm state
  - last command received
  - last venue response
  - last error
  - in-flight and completed command counts
- control:
  - arm or disarm execution
  - pause consumption
  - drain and stop cleanly

Stopping a service must mean a graceful stop or drain, not “kill the pod and hope.”

## Logging and observability requirements
All services must emit structured JSON logs with stable fields.

Stable fields:
- `level`
- `service`
- `component`
- `event`
- `namespace`
- `venue`
- `topic`
- `pair`

High-cardinality IDs belong in the body, not labels:
- `quote_id`
- `command_id`
- `decision_id`
- `execution_id`

What must be logged:
- connection loss and recovery
- source stale state
- inventory-sync stale state
- treasury funding action requested, credited, failed, or stuck
- invalid messages
- decision accepted and decision rejected with reason
- arm and disarm actions
- execution submitted, rejected, failed, recovered, completed
- PostgreSQL disconnect and recovery
- control API failures

What should not be logged:
- per-message noise without state change
- full payload spam by default

## Required edge cases and failure handling
- stale reference prices must block decisions
- stale or missing internal inventory state must block execution
- insufficient credited inventory must block the affected direction only
- pending deposits must not be counted as spendable inventory
- deposit address created but never funded must not be mistaken for liquidity
- external transfer observed but not yet credited in NEAR Intents must stay non-spendable
- credited inventory drift between inventory-sync and executor must hard-fail the command and publish a result
- Kraken down but CoinGecko healthy should degrade, not crash, if policy allows fallback
- both reference sources stale must block decisions
- PostgreSQL down must surface a hard health failure and stop claiming the system is trade-ready
- liquidity-manager failure must block new funding operations but must not silently stop already funded trading
- duplicate `cmd.execute_trade` must not cause duplicate venue submission
- executor restart after a partial submission must preserve idempotency behavior
- Near Intents API or RPC failures must become explicit `trade_result` failure records
- pair inactivity must not be mistaken for pipeline breakage

## Definition of done
- The deployed cluster is running all required services for this loop.
- Live Kraken and CoinGecko reference prices are flowing and inspectable.
- Spendable inventory inside NEAR Intents is flowing and inspectable.
- At least one treasury funding path is documented and verified:
  - deposit address or supported funding path created
  - funding action observed
  - credited inventory visible in internal state
- PostgreSQL contains the full event chain for at least one real quote path.
- Strategy decisions are non-dummy and include explicit reason fields with edge, freshness, and inventory context.
- Both trade directions are implemented in the same decision and execution path.
- At least one direction can be armed based on available credited inventory.
- A real Near Intents execution attempt can be triggered through the repo-controlled path.
- The resulting venue response is captured durably and inspectably.
- Each service exposes the required health and state endpoints, and controlled pause or arm semantics where appropriate.
- Fresh deploys start disarmed by default and require explicit operator arming before side effects.
- Validation includes not only happy path but blocked-path evidence:
  - stale reference blocked
  - insufficient inventory blocked
  - pending-deposit-not-spendable blocked
  - disarmed executor blocked

## Current real
- Real NEAR Intents quote data is flowing into the cluster.
- The pair filter can be configured and changed at runtime.
- Raw and normalized events already exist in Redpanda topics.
- The app is deployable and observable in Kubernetes.
- Kafka already provides the event backbone.

## Current fake or incomplete
- Reference pricing does not exist yet.
- Spendable internal inventory sync does not exist yet.
- Treasury funding or deposit management does not exist yet.
- Strategy decisions are still dummy placeholders.
- Execution is still dummy.
- The current executor is not a real Near Intents adapter.
- PostgreSQL is not yet part of the loop.
- Most services do not yet have their own control surfaces.
- Real signing and treasury secret handling is not yet wired.

## Failure conditions
This proof fails if any of the following is true:
- a trade can only happen via manual shell steps or out-of-band operator intervention
- the system depends on ad hoc bridging or external-wallet spending on the trade hot path
- decision logic is still embedded in a dummy or placeholder service
- executor logic is still simulated
- a service cannot be inspected or paused cleanly at runtime
- the system cannot explain why a quote did or did not become a trade
- the stored history cannot reconstruct quote, pricing, decision, command, and result
- both directions are not implemented
- the system attempts to execute without credited internal source inventory

## Expected validation evidence
- live `ref.market_price` events and current state from the pricing control API
- live internal inventory state from the inventory-sync control API
- PostgreSQL rows linking quote, reference, inventory snapshot, decision, command, and execution result
- a blocked decision caused by stale price or insufficient inventory
- a funding action that becomes credited inventory
- a successful command emission from the strategy service with a recorded edge above 2%
- one real Near Intents execution attempt with stored request and response metadata