unrip/archive/implementation/20260518T211420Z-pair-native-trade-semantics-and-multi-asset-outcome-truth-implementation.md
philipp fb547f24d9
All checks were successful
deploy / deploy (push) Successful in 48s
fix: scope inventory fetch by submission time window to prevent OOM
Instead of fetching up to 50k inventory snapshots (~442MB), scope the
query to only the time range of the current submission batch with a
15-minute buffer. For a typical 1-hour batch this drops from 50k rows
to ~300 rows, well within the 1280Mi pod memory limit.

The coalesced_at_idx on intent_inventory_snapshots covers the BETWEEN
clause so this remains efficient.

Proof: history-writer OOM kills from refreshQuoteOutcomes inventory fetch
Assumptions: 15min buffer covers the attribution window for all submissions
Still fake: heuristic gap outcomes may attribute trades imprecisely
2026-06-16 17:17:19 +02:00

14 KiB

Implementation Turn: Pair-native trade semantics and multi-asset outcome truth

Status: open Opened: 2026-05-18

Goal

Replace hidden BTC/EURe assumptions in shared runtime paths with pair-native asset semantics so maker decisions, taker requests, settlement attribution, valuation visibility, dashboard labels, and alerts work from DB asset/pair config without silently ignoring non-BTC/EURe assets.

Selected backlog items

  • none selected; this turn was opened directly from the operator request on 2026-05-18.

Design Rules

  • DB asset, pair, strategy config, and price route rows are the canonical runtime model.
  • Existing nBTC/EURe behavior and 49 bps edge must survive every refactor.
  • nBTC/Ethereum USDC must be a first regression pair because it exposes the current hidden EURe assumptions.
  • Legacy fields may remain for old stored history and API compatibility, but new runtime logic must use generic pair-native fields.
  • Missing route, stale price, missing decimals, missing inventory, or missing DB config fails closed.
  • Tracked assets without valuation remain visible with explicit reasons.
  • No new live trading is enabled by import alone.
  • No manual deployment or cluster repair is part of this turn.

Problem Statement

The DB-backed configuration turn made assets and pairs durable, but several runtime paths still behave as if there is one preferred BTC/EURe trading path:

  • taker preflight parses amount_eure, requires eure_per_btc, and computes BTC receive from EURe
  • strategy stores and displays eure_notional even when the quote notional is USDC
  • outcome attribution takes btcAsset and eureAsset and ignores other pair assets
  • portfolio valuation special-cases BTC, EURe, and USDC instead of making unvalued assets explicit
  • dashboard request and lifecycle copy still says EURe-to-BTC
  • alerts and health still describe stale truth around one activePair

The implementation should turn the current DB pair model into the actual shared runtime model, not just a dashboard/config layer.

Backend Changes

1. Canonical pair-native trade terms

  • Add or consolidate helpers for resolving a directed pair from DB config:
    • by pair_id
    • by asset_in and asset_out
    • by default taker pair when the operator has not specified a pair
  • Define normalized runtime terms used by strategy, request preflight, commands, and UI:
    • pair_id, venue
    • source_asset_id, destination_asset_id
    • source_symbol, destination_symbol
    • source_decimals, destination_decimals
    • source_amount_units, destination_amount_units
    • notional, notional_asset_id, notional_symbol
    • max_notional, min_notional
    • price_route_id, reference_price_id
  • Keep compatibility alias mapping:
    • amount_eure -> source_amount only for old request callers
    • eure_notional may be emitted temporarily only as a deprecated alias when the notional asset is EURe
    • max_notional_eure may be read from old rows but should not be the primary field
  • Add tests that new decision/request builders expose generic fields for both nBTC/EURe and nBTC/USDC.

2. Generic route-rate adapter

  • Update market price events for current adapters to include generic route fields:
    • base_asset_id
    • quote_asset_id
    • quote_per_base
    • base_per_quote
    • price_route_id
    • reference_pair
  • Preserve old fields such as eure_per_btc and usdc_per_btc for stored-row compatibility while new strategy/request code reads generic fields first.
  • Replace strategy direction names like btc_to_eure and usdc_to_btc in new logic with route-relative direction:
    • base_to_quote
    • quote_to_base
  • Preserve display labels that include actual DB symbols.
  • Add tests for BTC/EUR and BTC/USDC adapter payloads and route-relative direction classification.

3. Strategy migration

  • Rename internal variables away from maxNotionalEure and eure_notional where they are not specifically EURe.
  • Compute fair output/input from the generic route-rate adapter.
  • Store new decision fields:
    • notional
    • notional_asset_id
    • notional_symbol
    • max_notional
    • price_route_id
    • reference_price_id
    • asset decimals
  • Store command fields with enough pair-native data for later attribution:
    • source and destination asset ids
    • proposed source/destination amounts
    • expected inventory delta or enough terms to derive it
  • Keep current pair config version and edge bps persistence unchanged.
  • Add regression tests:
    • nBTC/EURe exact-in and exact-out still produce expected decisions
    • nBTC/USDC decision persists USDC notional as notional_symbol=USDC, not EUR
    • unsupported route fails closed with price_route_missing or unsupported_price_route

4. Pair-native taker preflight

  • Change request preflight input handling:
    • accept pair_id
    • accept asset_in and asset_out as an alternative
    • accept source_amount or amount
    • accept amount_eure only as a legacy alias when no generic amount is supplied
  • Resolve the pair from DB config and require taker mode or both mode.
  • Use DB asset decimals to parse the source amount.
  • Load the latest price for the pair's route, not just latest BTC/EUR price.
  • Compute expected destination amount from generic route fields:
    • source is route base -> destination = source * quote_per_base
    • source is route quote -> destination = source * base_per_quote
  • Apply slippage to produce minimum destination amount.
  • Persist preflight fields generically:
    • pair id/config/version/edge
    • source/destination assets and units
    • route id/reference price id
    • expected/min destination units
    • slippage/deadline/signer/verifier
  • Keep live submit gated behind existing executor controls.
  • Add tests:
    • nBTC/EURe legacy alias still works
    • nBTC/USDC pair preflight does not require amount_eure or eure_per_btc
    • route missing blocks before solver quote
    • stale route blocks before solver quote
    • insufficient source inventory emits source-asset-specific reason

5. Pair-native outcome attribution

  • Refactor quote outcome attribution so the active delta assets are derived from expected command deltas.
  • Refactor request outcome attribution so delta assets are derived from preflight source/destination terms.
  • Remove btcAsset/eureAsset parameters from outcome refresh APIs or make them compatibility fallbacks only.
  • Persist attribution deltas by actual asset id and use DB metadata only for formatting.
  • Replace hardcoded humanized text such as "EURe decrease and BTC increase" with generic source/destination wording.
  • Add tests:
    • maker BTC/USDC command completes when inventory shows BTC decrease and USDC increase, or the correct inverse depending on command side
    • taker USDC/BTC request completes from USDC decrease and BTC increase
    • unrelated third-asset movements do not create false completion
    • ambiguous matching remains ambiguous
    • old BTC/EURe rows still normalize for dashboard history

6. Portfolio and valuation visibility

  • Make portfolio inputs carry all tracked assets from DB.
  • Build valuation assets from available price routes instead of symbol-only checks where practical.
  • For this turn, support:
    • BTC wrappers valued from BTC/EUR route
    • EURe as EUR cash-equivalent
    • USDC valued from BTC/EUR plus BTC/USDC route
  • For any other tracked asset, include an explicit unvalued entry:
    • valuation_status=unvalued
    • valuation_reason=valuation_route_missing
  • Ensure totals include only valued assets and expose unvalued assets separately.
  • Add tests:
    • tracked USDC remains valued
    • a tracked non-BTC/non-EURe/non-USDC asset appears with valuation_route_missing
    • no tracked asset disappears from balance rows

7. Dashboard migration

  • Request creation UI:
    • add pair selection from DB taker-enabled or both-mode pairs
    • label amount as "Spend "
    • submit pair_id and source_amount
    • remove EURe-to-BTC headers and copy
  • Strategy/lifecycle UI:
    • display generic notional with notional_symbol
    • show pair route and config version from DB
    • show settlement deltas from attributed asset metadata
    • keep old rows readable when only eure_notional exists
  • Funds/portfolio UI:
    • show every tracked asset
    • show valuation source or unvalued reason
    • show deposit addresses by chain/assets without implying only BTC/EURe
  • Status bar:
    • replace single "Reference BTC/EUR" tile with route-aware reference status or latest key route labels
    • avoid one active-pair wording when multiple pairs are enabled
  • Add dashboard tests for request form payloads, generic notional labels, unvalued asset rows, and no hardcoded EURe-to-BTC request title.

8. Alerts and runtime health

  • Replace single activePair alert context with active pair and price-route sets from DB config where service behavior is pair-scoped.
  • Reference price stale alerts should identify the route or pair whose price is stale.
  • Runtime health should not mark all trading truth healthy because the old active pair has a price while another enabled tradeable pair has no fresh route.
  • Keep service-level alerts for DB unavailable, ingest disconnected, and inventory stale.
  • Add tests for:
    • stale route alert scoped to the affected pair
    • active pair set shown in service state
    • strategy/executor armed critical truth check still fires when any enabled trading pair has critical stale truth

9. Static cleanup and ops compatibility

  • Update or retire ops scripts that still read removed pair/asset env vars, especially scripts/ops/watch_live_mm.py.
  • Static tests should prevent reintroducing runtime dependencies on TRADING_BTC_ASSET_ID, TRADING_EURE_ASSET_ID, NEAR_INTENTS_PAIR_FILTER, or STRATEGY_GROSS_THRESHOLD_PCT.
  • Keep deployment config free of trading asset/pair/edge env vars.

Data and Persistence

  • Prefer additive payload fields over destructive schema rewrites.
  • Existing stored rows with eure_notional remain readable.
  • New rows should use generic fields first.
  • If a DB schema change is necessary, add it through the tracked schema path and make it idempotent.
  • Preserve raw event payloads and normalized records for replay.

Edge Cases

  • Pair exists but is observe-only: preflight and trading block with pair_not_taker_enabled or pair_not_maker_enabled.
  • Pair has no price route: block with price_route_missing.
  • Price route exists but no fresh price event: block with stale_reference_price or reference_price_unavailable.
  • Asset decimals missing: block with asset_decimals_missing.
  • Source inventory unavailable or stale: block before solver quote or command.
  • Source spend exceeds inventory: block with a source-symbol-specific reason.
  • Valuation route missing: asset remains visible as unvalued; no fake zero-value assumption.
  • Old rows with BTC/EURe-only fields: dashboard remains readable through compatibility normalizers.
  • Multiple enabled pairs: status and alerts show pair/route scope instead of one global active pair.

Concrete Implementation Order

Phase 1. Canonical helpers and route-rate model

  • Add pair/route resolution helpers.
  • Add generic route-rate extraction with compatibility fallback for current price payloads.
  • Add tests for BTC/EUR and BTC/USDC route math.

Phase 2. Strategy generic fields

  • Update strategy decision and command payloads to use generic notional and route-relative direction.
  • Preserve current nBTC/EURe behavior.
  • Add nBTC/USDC strategy regression tests.

Phase 3. Taker preflight

  • Update request controller and request helpers to accept pair-native input.
  • Compute min destination amount from generic route fields.
  • Persist generic preflight fields.
  • Add preflight regression tests.

Phase 4. Outcome attribution

  • Refactor maker and taker outcome attribution around expected deltas.
  • Remove fixed BTC/EURe active-asset assumptions from refresh paths.
  • Add BTC/USDC attribution tests and compatibility tests.

Phase 5. Portfolio valuation visibility

  • Add valued/unvalued tracked-asset model.
  • Preserve current BTC/EURe/USDC valuation.
  • Add tests for explicit unvalued tracked assets.

Phase 6. Dashboard and status surfaces

  • Update request form, lifecycle tables, strategy tables, funds rows, and status bar.
  • Add dashboard tests proving labels come from DB assets and no EURe-to-BTC-only request UI remains.
  • Build dashboard bundle.

Phase 7. Alerts, ops scripts, deployment validation

  • Make runtime health and alerts pair/route-aware.
  • Update stale ops scripts or mark them inactive with tests.
  • Run static deployment tests.
  • Run full test suite.
  • Deploy through repo workflow.
  • Validate live dashboard bootstrap, service state, pair config, and current pair quote behavior.

Test Plan

  • Generic route-rate unit tests.
  • Strategy nBTC/EURe compatibility tests.
  • Strategy nBTC/USDC generic notional tests.
  • Taker preflight pair-native tests.
  • Taker preflight fail-closed tests.
  • Outcome attribution generic expected-delta tests.
  • Portfolio valued/unvalued tracked-asset tests.
  • Dashboard request/lifecycle/funds label tests.
  • Alert/runtime-health pair-set tests.
  • Static deployment/config tests.
  • Full npm test.
  • Dashboard build.

Validation Checklist Against The Proof

  • nBTC/EURe current pair still loads from DB and uses 49 bps unless operator changed DB config.
  • nBTC/USDC pair decisions use notional_symbol=USDC and do not display as EUR notional.
  • Request preflight accepts pair_id plus source_amount.
  • Request preflight for a non-EURe pair does not require amount_eure.
  • Outcome attribution can complete a BTC/USDC row from matching inventory deltas.
  • Dashboard has no EURe-to-BTC-only request title or settlement copy in active UI.
  • Every tracked asset appears in balances with either value or unvalued reason.
  • Alerts and state expose active pair/route scope.
  • Deployment config still contains no trading asset/pair/edge env vars.
  • Services deploy from repo push only.

Known Fakes Allowed At Start Of This Turn

  • Only BTC/EUR and BTC/USDC reference route adapters are real.
  • Arbitrary non-BTC asset valuation remains unavailable unless a supported route is configured.
  • NEAR Intents remains the only live venue.
  • Venue-native terminal fills remain unavailable unless existing relay/status paths already provide them.
  • Fee-complete realized PnL remains unavailable.