# Implementation Turn: first non-mocked tradeable loop for one pair Status: open Opened: 2026-04-01 ## Goal Build the first full live vertical slice that can actually attempt a trade: 1. observe a real NEAR Intents quote 2. enrich it with live external reference pricing 3. synchronize spendable inventory already credited inside NEAR Intents 4. evaluate it in a real strategy service 5. gate it by inventory and arm state 6. submit it through a real Near Intents executor 7. persist the full chain in PostgreSQL This turn is explicitly not a storage side-quest. Storage, control surfaces, analytics support, and treasury funding visibility are included because they are required to make the trade loop trustworthy. ## Selected backlog items - [I001] Hybrid reference-price service using Kraken stream and CoinGecko poll or fallback for the active pair inputs. - [I002] PostgreSQL event store plus history writer for quotes, reference prices, decisions, commands, and execution results. - [I003] Strategy engine that consumes `norm.swap_demand` plus reference prices and emits auditable decisions. - [I004] Real Near Intents executor service using pre-funded internal inventory, with explicit arming and idempotent result reporting. - [I006] Decision-to-command safety gate with explicit arm or disarm state, notional caps, and inventory freshness checks. - [I007] Inventory-aware execution rule: implement both directions, but only fire the side backed by credited internal source-asset inventory. - [I008] Inventory-sync service for NEAR Intents internal balances and pending funding state. - [I009] Liquidity-manager service for deposit addresses, funding actions, and treasury visibility. - [B002] No reference price source exists, so the system cannot estimate edge. - [B003] Dummy reactor and dummy executor prevent a non-mocked trade path. - [B004] The current plan assumed external hot wallets were on the hot trade path instead of pre-funded internal inventory. ## Architectural shape ### Event backbone Kafka or Redpanda remains the backbone between services. Required topic set for this turn: - `raw.near_intents.quote` - `norm.swap_demand` - `ref.market_price` - `state.intent_inventory` - `ops.liquidity_action` - `decision.trade_decision` - `cmd.execute_trade` - `exec.trade_result` ### Durable store PostgreSQL is the first durable analytics and audit layer. Why: - enough write throughput for current scope - strong queryability for replay, inspection, and debugging - simple to operate - better fit than inventing a warehouse right now ### Service split The first real loop should be seven services: - `near-intents-ingest` - `market-reference-ingest` - `inventory-sync` - `liquidity-manager` - `history-writer` - `strategy-engine` - `trade-executor` ## Service-by-service responsibilities ### 1. `near-intents-ingest` Responsibilities: - connect to the NEAR Intents quote stream - filter or classify the active pair - emit raw and normalized events - expose runtime state Inputs: - NEAR Intents websocket - pair filter config and runtime override Outputs: - `raw.near_intents.quote` - `norm.swap_demand` Control surface: - `GET /healthz` - `GET /state` - `GET /pair-filter` - `PUT /pair-filter` - `POST /pair-filter/reset` Important edge cases: - websocket disconnect - reconnect storm - invalid JSON - pair silent but connection healthy ### 2. `market-reference-ingest` Responsibilities: - subscribe to Kraken BTC/EUR pricing - poll CoinGecko for fallback or cross-check pricing - derive fair BTC/EURe price for both trade directions - publish reference-price events - expose latest source state and freshness Inputs: - Kraken stream - CoinGecko HTTP API Outputs: - `ref.market_price` Control surface: - `GET /healthz` - `GET /state` - `POST /refresh` - `POST /pause` - `POST /resume` Required state: - latest Kraken price - latest CoinGecko price - derived fair rate - freshness age - source health flags Important edge cases: - Kraken disconnect - CoinGecko timeout or rate limit - both sources stale - conflicting sources beyond tolerance Required behavior: - if Kraken is down but CoinGecko is fresh, degrade according to policy and record fallback usage - if both are stale, mark the service unhealthy for decisioning - the first implementation must make the EURe/EUR pricing assumption explicit: - either treat EURe as 1:1 with EUR and record that plainly - or use a separate sanity source and record the mapping logic ### 3. `inventory-sync` Responsibilities: - read current credited spendable inventory from NEAR Intents internal state - distinguish credited balances from pending deposits and pending withdrawals - publish current internal inventory state - expose freshness and reconciliation state Inputs: - NEAR Intents inventory or verifier surfaces - liquidity-manager state when relevant Outputs: - `state.intent_inventory` Control surface: - `GET /healthz` - `GET /state` - `POST /refresh` - `POST /pause` - `POST /resume` Required state: - spendable balances by asset - pending inbound funding by asset - pending outbound withdrawal by asset - last sync time - reconciliation status Important edge cases: - internal inventory surface unavailable - external deposit seen but not yet credited internally - credited balance lower than expected after funding - stale inventory snapshot Required behavior: - only credited internal inventory counts as spendable - pending treasury movements must remain non-spendable - stale inventory state must be visible and actionable ### 4. `liquidity-manager` Responsibilities: - request and track deposit addresses or equivalent funding handles for treasury assets - track treasury funding actions from external wallets into NEAR Intents - track withdrawals and rebalance actions - expose current funding pipeline state - publish auditable liquidity action records Inputs: - treasury configuration - external funding wallets - NEAR Intents deposit or withdrawal surfaces Outputs: - `ops.liquidity_action` Control surface: - `GET /healthz` - `GET /state` - `POST /refresh` - `POST /pause` - `POST /resume` - `POST /freeze-withdrawals` Required state: - active deposit addresses or funding handles - recent funding attempts - pending credits - recent withdrawals - rebalance state Important edge cases: - deposit address request failure - external wallet funded but NEAR Intents credit delayed - duplicate funding detection - unsupported asset or chain mapping Required behavior: - treasury actions must remain visible and auditable - funding must be decoupled from the per-trade hot path ### 5. `history-writer` Responsibilities: - consume the core topics - write append-only rows into PostgreSQL - preserve causality across quote, decision, and execution records - expose lag and write state Inputs: - all core Kafka topics Outputs: - PostgreSQL rows Control surface: - `GET /healthz` - `GET /state` - `POST /pause` - `POST /resume` - `POST /drain` Required stored record families: - raw quotes - normalized demand - reference prices - inventory snapshots - liquidity actions - trade decisions - execute commands - execution results Important edge cases: - PostgreSQL unavailable - duplicate deliveries from Kafka - offset commit mismatch after partial write Required behavior: - never claim success before the write is durable - surface last committed offsets and last successful write time ### 6. `strategy-engine` Responsibilities: - join latest demand with latest price and inventory state - compute implied quote rate - compare it to fair rate - apply freshness thresholds - apply the initial 2% gross edge threshold - apply max-notional and inventory checks - emit auditable decision events - emit `cmd.execute_trade` only when all gates pass and the system is armed Inputs: - `norm.swap_demand` - `ref.market_price` - `state.intent_inventory` Outputs: - `decision.trade_decision` - `cmd.execute_trade` Control surface: - `GET /healthz` - `GET /state` - `POST /arm` - `POST /disarm` - `POST /pause` - `POST /resume` - `PUT /threshold` - `PUT /limits` Required decision state: - arm state - current threshold - current notional cap - latest decision - latest rejected decision reason - per-reason skip counters Required decision record fields: - `decision_id` - `quote_id` - `pair` - `direction` - `implied_rate` - `reference_rate` - `gross_edge_pct` - `price_freshness_ms` - `inventory_snapshot` - `decision` - `decision_reason` Important edge cases: - stale prices - no price yet - no inventory yet - insufficient spendable inventory - pending deposit exists but is not yet credited - quote already expired - repeated quote IDs - one direction affordable and the other not - fresh deploy starts armed by mistake ### 7. `trade-executor` Responsibilities: - consume `cmd.execute_trade` - load the correct Near Intents signing authority - perform the real Near Intents submission using pre-funded internal inventory - preserve idempotency across retries and restarts - emit `exec.trade_result` Inputs: - `cmd.execute_trade` - signing secrets - optional latest inventory state Outputs: - `exec.trade_result` Control surface: - `GET /healthz` - `GET /state` - `POST /arm` - `POST /disarm` - `POST /pause` - `POST /resume` - `POST /drain` Required state: - arm state - last command seen - last request sent - last venue response - in-flight command count - completed command count - error counters Important edge cases: - duplicate command delivery - crash after submission but before result publish - venue reject - timeout with unknown outcome - insufficient internal inventory detected late - secret missing or invalid - signer not authorized for the intended NEAR Intents account Required behavior: - never silently swallow a venue error - always publish a result record for attempted commands - make duplicate suppression durable - never try to bridge or top up inventory during trade execution - start disarmed by default on fresh deploy ## Persistence design ### Why PostgreSQL now - current scale does not require a special time-series database - rows are easier to inspect than a custom file format for this phase - joins across decision, command, and result matter more right now than raw ingestion throughput - treasury, inventory, and execution joins matter more than raw bus throughput at this stage ### Minimum tables or equivalent record families - `raw_near_intents_quotes` - `swap_demand_events` - `market_price_events` - `intent_inventory_snapshots` - `liquidity_actions` - `trade_decisions` - `execute_trade_commands` - `trade_execution_results` ### Query requirements Must be able to answer: - what was the latest fair price when this decision was made - what spendable inventory existed when this decision was made - what treasury funding actions were still pending - why was this quote skipped - what command was emitted for this quote - what happened when the executor submitted it - what credited internal inventory existed at the time ## Control and stop semantics Every service must support inspection. State endpoints must be sufficient to answer: - is it healthy - is it connected - what is it currently using as input state - what was the last successful action - why is it blocked, if blocked - whether the state is authoritative or stale Stopping must be explicit: - `pause` means stop taking new work but keep process alive - `drain` means finish in-flight work then stop cleanly - `disarm` means remain live and observable but refuse side effects This matters because the operator must be able to halt strategy or execution without destroying the whole pipeline. ## Logging requirements All services use structured JSON logs. Stable top-level fields: - `level` - `service` - `component` - `event` - `namespace` - `venue` - `topic` - `pair` Additional body fields when relevant: - `quote_id` - `decision_id` - `command_id` - `execution_id` - `inventory_id` - `liquidity_action_id` - `gross_edge_pct` - `price_freshness_ms` Log when: - source connection is lost or reestablished - price source becomes stale - inventory state becomes stale or recovers - funding action is requested, seen, credited, delayed, failed, or frozen - strategy rejects with a meaningful reason - strategy arms or disarms - executor arms or disarms - command is submitted - venue rejects or times out - PostgreSQL disconnects or recovers - control API changes state Do not log every healthy message by default. ## Failure and failover behavior ### Reference pricing - Kraken failure with fresh CoinGecko: - allowed only if fallback policy says yes - decision records must note fallback source use - both sources stale: - strategy blocks all execution ### Inventory state - missing or stale inventory state: - strategy may still emit a non-actionable rejected decision - strategy must not emit `cmd.execute_trade` - pending funding action: - remains non-spendable - must be visible in control state and durable records - treasury action service down: - already funded trading may continue if inventory-sync is fresh - new funding or withdrawal operations must be blocked visibly ### Persistence - PostgreSQL unavailable: - history-writer unhealthy - system must surface that audit history is impaired - if the user wants strict mode, strategy or executor may be blocked until persistence returns ### Execution - timeout with unknown venue outcome: - emit explicit uncertain result - preserve idempotency state for recovery - restart after partial submission: - executor must not blindly resubmit without checking prior state - executor sees lower real inventory than strategy snapshot: - emit explicit failure result - do not attempt fallback treasury movement ## Testing and validation plan ### Unit tests - implied-rate calculation for both directions - explicit EURe/EUR pricing-basis test - threshold checks - stale-price blocking - insufficient-inventory blocking - pending-deposit-not-spendable blocking - command emission only when armed - idempotency transitions in executor ### Integration tests - pricing service emits canonical reference-price events - inventory-sync emits credited vs pending inventory state correctly - liquidity-manager records funding actions and status transitions - strategy consumes demand, pricing, and inventory and emits decision plus command as expected - history-writer persists linked records across topics - executor publishes result records for success and failure paths ### Runtime validation in cluster - inspect each service `/state` - verify live reference pricing updates - verify live internal inventory state - verify one treasury funding path from request or deposit tracking to credited inventory - verify strategy and executor start disarmed on a fresh deploy - observe a rejected decision due to block condition - arm strategy and executor - keep the first live max notional tiny, on the order of a few EURe, before any larger cap - observe one command emission - observe one real Near Intents execution attempt - verify PostgreSQL contains the full linked chain ## Deliberately rejected for this turn - dashboards - broad multi-pair abstractions - ML training infrastructure - broad backtest framework - polished operator UI - warehouse or lakehouse design ## Expected deliverables - `market-reference-ingest` - `inventory-sync` - `liquidity-manager` - `history-writer` - `strategy-engine` - `trade-executor` - PostgreSQL schema and migrations or equivalent setup - updated Kafka topic creation and config - shared control API pattern for all long-running services - tests for strategy, inventory, persistence, and executor behavior - docs for operations, arm or disarm flow, and inspection commands ## Validation target This turn is only complete when the deployed system can, through repo-controlled services alone, take one live active-pair quote, price it, verify credited internal inventory, decide on it, gate it, submit a real Near Intents execution attempt, and preserve the full record chain in PostgreSQL.