unrip/archive/implementation/20260518T211420Z-pair-native-trade-semantics-and-multi-asset-outcome-truth-implementation.md
philipp fb547f24d9
All checks were successful
deploy / deploy (push) Successful in 48s
fix: scope inventory fetch by submission time window to prevent OOM
Instead of fetching up to 50k inventory snapshots (~442MB), scope the
query to only the time range of the current submission batch with a
15-minute buffer. For a typical 1-hour batch this drops from 50k rows
to ~300 rows, well within the 1280Mi pod memory limit.

The coalesced_at_idx on intent_inventory_snapshots covers the BETWEEN
clause so this remains efficient.

Proof: history-writer OOM kills from refreshQuoteOutcomes inventory fetch
Assumptions: 15min buffer covers the attribution window for all submissions
Still fake: heuristic gap outcomes may attribute trades imprecisely
2026-06-16 17:17:19 +02:00

273 lines
14 KiB
Markdown

# Implementation Turn: Pair-native trade semantics and multi-asset outcome truth
Status: open
Opened: 2026-05-18
## Goal
Replace hidden BTC/EURe assumptions in shared runtime paths with pair-native asset semantics so maker decisions, taker requests, settlement attribution, valuation visibility, dashboard labels, and alerts work from DB asset/pair config without silently ignoring non-BTC/EURe assets.
## Selected backlog items
- none selected; this turn was opened directly from the operator request on 2026-05-18.
## Design Rules
- DB asset, pair, strategy config, and price route rows are the canonical runtime model.
- Existing nBTC/EURe behavior and 49 bps edge must survive every refactor.
- nBTC/Ethereum USDC must be a first regression pair because it exposes the current hidden EURe assumptions.
- Legacy fields may remain for old stored history and API compatibility, but new runtime logic must use generic pair-native fields.
- Missing route, stale price, missing decimals, missing inventory, or missing DB config fails closed.
- Tracked assets without valuation remain visible with explicit reasons.
- No new live trading is enabled by import alone.
- No manual deployment or cluster repair is part of this turn.
## Problem Statement
The DB-backed configuration turn made assets and pairs durable, but several runtime paths still behave as if there is one preferred BTC/EURe trading path:
- taker preflight parses `amount_eure`, requires `eure_per_btc`, and computes BTC receive from EURe
- strategy stores and displays `eure_notional` even when the quote notional is USDC
- outcome attribution takes `btcAsset` and `eureAsset` and ignores other pair assets
- portfolio valuation special-cases BTC, EURe, and USDC instead of making unvalued assets explicit
- dashboard request and lifecycle copy still says EURe-to-BTC
- alerts and health still describe stale truth around one `activePair`
The implementation should turn the current DB pair model into the actual shared runtime model, not just a dashboard/config layer.
## Backend Changes
### 1. Canonical pair-native trade terms
- Add or consolidate helpers for resolving a directed pair from DB config:
- by `pair_id`
- by `asset_in` and `asset_out`
- by default taker pair when the operator has not specified a pair
- Define normalized runtime terms used by strategy, request preflight, commands, and UI:
- `pair_id`, `venue`
- `source_asset_id`, `destination_asset_id`
- `source_symbol`, `destination_symbol`
- `source_decimals`, `destination_decimals`
- `source_amount_units`, `destination_amount_units`
- `notional`, `notional_asset_id`, `notional_symbol`
- `max_notional`, `min_notional`
- `price_route_id`, `reference_price_id`
- Keep compatibility alias mapping:
- `amount_eure` -> `source_amount` only for old request callers
- `eure_notional` may be emitted temporarily only as a deprecated alias when the notional asset is EURe
- `max_notional_eure` may be read from old rows but should not be the primary field
- Add tests that new decision/request builders expose generic fields for both nBTC/EURe and nBTC/USDC.
### 2. Generic route-rate adapter
- Update market price events for current adapters to include generic route fields:
- `base_asset_id`
- `quote_asset_id`
- `quote_per_base`
- `base_per_quote`
- `price_route_id`
- `reference_pair`
- Preserve old fields such as `eure_per_btc` and `usdc_per_btc` for stored-row compatibility while new strategy/request code reads generic fields first.
- Replace strategy direction names like `btc_to_eure` and `usdc_to_btc` in new logic with route-relative direction:
- `base_to_quote`
- `quote_to_base`
- Preserve display labels that include actual DB symbols.
- Add tests for BTC/EUR and BTC/USDC adapter payloads and route-relative direction classification.
### 3. Strategy migration
- Rename internal variables away from `maxNotionalEure` and `eure_notional` where they are not specifically EURe.
- Compute fair output/input from the generic route-rate adapter.
- Store new decision fields:
- `notional`
- `notional_asset_id`
- `notional_symbol`
- `max_notional`
- `price_route_id`
- `reference_price_id`
- asset decimals
- Store command fields with enough pair-native data for later attribution:
- source and destination asset ids
- proposed source/destination amounts
- expected inventory delta or enough terms to derive it
- Keep current pair config version and edge bps persistence unchanged.
- Add regression tests:
- nBTC/EURe exact-in and exact-out still produce expected decisions
- nBTC/USDC decision persists USDC notional as `notional_symbol=USDC`, not EUR
- unsupported route fails closed with `price_route_missing` or `unsupported_price_route`
### 4. Pair-native taker preflight
- Change request preflight input handling:
- accept `pair_id`
- accept `asset_in` and `asset_out` as an alternative
- accept `source_amount` or `amount`
- accept `amount_eure` only as a legacy alias when no generic amount is supplied
- Resolve the pair from DB config and require taker mode or both mode.
- Use DB asset decimals to parse the source amount.
- Load the latest price for the pair's route, not just latest BTC/EUR price.
- Compute expected destination amount from generic route fields:
- source is route base -> destination = source * quote_per_base
- source is route quote -> destination = source * base_per_quote
- Apply slippage to produce minimum destination amount.
- Persist preflight fields generically:
- pair id/config/version/edge
- source/destination assets and units
- route id/reference price id
- expected/min destination units
- slippage/deadline/signer/verifier
- Keep live submit gated behind existing executor controls.
- Add tests:
- nBTC/EURe legacy alias still works
- nBTC/USDC pair preflight does not require `amount_eure` or `eure_per_btc`
- route missing blocks before solver quote
- stale route blocks before solver quote
- insufficient source inventory emits source-asset-specific reason
### 5. Pair-native outcome attribution
- Refactor quote outcome attribution so the active delta assets are derived from expected command deltas.
- Refactor request outcome attribution so delta assets are derived from preflight source/destination terms.
- Remove `btcAsset`/`eureAsset` parameters from outcome refresh APIs or make them compatibility fallbacks only.
- Persist attribution deltas by actual asset id and use DB metadata only for formatting.
- Replace hardcoded humanized text such as "EURe decrease and BTC increase" with generic source/destination wording.
- Add tests:
- maker BTC/USDC command completes when inventory shows BTC decrease and USDC increase, or the correct inverse depending on command side
- taker USDC/BTC request completes from USDC decrease and BTC increase
- unrelated third-asset movements do not create false completion
- ambiguous matching remains ambiguous
- old BTC/EURe rows still normalize for dashboard history
### 6. Portfolio and valuation visibility
- Make portfolio inputs carry all tracked assets from DB.
- Build valuation assets from available price routes instead of symbol-only checks where practical.
- For this turn, support:
- BTC wrappers valued from BTC/EUR route
- EURe as EUR cash-equivalent
- USDC valued from BTC/EUR plus BTC/USDC route
- For any other tracked asset, include an explicit unvalued entry:
- `valuation_status=unvalued`
- `valuation_reason=valuation_route_missing`
- Ensure totals include only valued assets and expose unvalued assets separately.
- Add tests:
- tracked USDC remains valued
- a tracked non-BTC/non-EURe/non-USDC asset appears with `valuation_route_missing`
- no tracked asset disappears from balance rows
### 7. Dashboard migration
- Request creation UI:
- add pair selection from DB taker-enabled or both-mode pairs
- label amount as "Spend <symbol>"
- submit `pair_id` and `source_amount`
- remove EURe-to-BTC headers and copy
- Strategy/lifecycle UI:
- display generic notional with `notional_symbol`
- show pair route and config version from DB
- show settlement deltas from attributed asset metadata
- keep old rows readable when only `eure_notional` exists
- Funds/portfolio UI:
- show every tracked asset
- show valuation source or unvalued reason
- show deposit addresses by chain/assets without implying only BTC/EURe
- Status bar:
- replace single "Reference BTC/EUR" tile with route-aware reference status or latest key route labels
- avoid one active-pair wording when multiple pairs are enabled
- Add dashboard tests for request form payloads, generic notional labels, unvalued asset rows, and no hardcoded EURe-to-BTC request title.
### 8. Alerts and runtime health
- Replace single `activePair` alert context with active pair and price-route sets from DB config where service behavior is pair-scoped.
- Reference price stale alerts should identify the route or pair whose price is stale.
- Runtime health should not mark all trading truth healthy because the old active pair has a price while another enabled tradeable pair has no fresh route.
- Keep service-level alerts for DB unavailable, ingest disconnected, and inventory stale.
- Add tests for:
- stale route alert scoped to the affected pair
- active pair set shown in service state
- strategy/executor armed critical truth check still fires when any enabled trading pair has critical stale truth
### 9. Static cleanup and ops compatibility
- Update or retire ops scripts that still read removed pair/asset env vars, especially `scripts/ops/watch_live_mm.py`.
- Static tests should prevent reintroducing runtime dependencies on `TRADING_BTC_ASSET_ID`, `TRADING_EURE_ASSET_ID`, `NEAR_INTENTS_PAIR_FILTER`, or `STRATEGY_GROSS_THRESHOLD_PCT`.
- Keep deployment config free of trading asset/pair/edge env vars.
## Data and Persistence
- Prefer additive payload fields over destructive schema rewrites.
- Existing stored rows with `eure_notional` remain readable.
- New rows should use generic fields first.
- If a DB schema change is necessary, add it through the tracked schema path and make it idempotent.
- Preserve raw event payloads and normalized records for replay.
## Edge Cases
- Pair exists but is observe-only: preflight and trading block with `pair_not_taker_enabled` or `pair_not_maker_enabled`.
- Pair has no price route: block with `price_route_missing`.
- Price route exists but no fresh price event: block with `stale_reference_price` or `reference_price_unavailable`.
- Asset decimals missing: block with `asset_decimals_missing`.
- Source inventory unavailable or stale: block before solver quote or command.
- Source spend exceeds inventory: block with a source-symbol-specific reason.
- Valuation route missing: asset remains visible as unvalued; no fake zero-value assumption.
- Old rows with BTC/EURe-only fields: dashboard remains readable through compatibility normalizers.
- Multiple enabled pairs: status and alerts show pair/route scope instead of one global active pair.
## Concrete Implementation Order
### Phase 1. Canonical helpers and route-rate model
- Add pair/route resolution helpers.
- Add generic route-rate extraction with compatibility fallback for current price payloads.
- Add tests for BTC/EUR and BTC/USDC route math.
### Phase 2. Strategy generic fields
- Update strategy decision and command payloads to use generic notional and route-relative direction.
- Preserve current nBTC/EURe behavior.
- Add nBTC/USDC strategy regression tests.
### Phase 3. Taker preflight
- Update request controller and request helpers to accept pair-native input.
- Compute min destination amount from generic route fields.
- Persist generic preflight fields.
- Add preflight regression tests.
### Phase 4. Outcome attribution
- Refactor maker and taker outcome attribution around expected deltas.
- Remove fixed BTC/EURe active-asset assumptions from refresh paths.
- Add BTC/USDC attribution tests and compatibility tests.
### Phase 5. Portfolio valuation visibility
- Add valued/unvalued tracked-asset model.
- Preserve current BTC/EURe/USDC valuation.
- Add tests for explicit unvalued tracked assets.
### Phase 6. Dashboard and status surfaces
- Update request form, lifecycle tables, strategy tables, funds rows, and status bar.
- Add dashboard tests proving labels come from DB assets and no EURe-to-BTC-only request UI remains.
- Build dashboard bundle.
### Phase 7. Alerts, ops scripts, deployment validation
- Make runtime health and alerts pair/route-aware.
- Update stale ops scripts or mark them inactive with tests.
- Run static deployment tests.
- Run full test suite.
- Deploy through repo workflow.
- Validate live dashboard bootstrap, service state, pair config, and current pair quote behavior.
## Test Plan
- Generic route-rate unit tests.
- Strategy nBTC/EURe compatibility tests.
- Strategy nBTC/USDC generic notional tests.
- Taker preflight pair-native tests.
- Taker preflight fail-closed tests.
- Outcome attribution generic expected-delta tests.
- Portfolio valued/unvalued tracked-asset tests.
- Dashboard request/lifecycle/funds label tests.
- Alert/runtime-health pair-set tests.
- Static deployment/config tests.
- Full `npm test`.
- Dashboard build.
## Validation Checklist Against The Proof
- nBTC/EURe current pair still loads from DB and uses 49 bps unless operator changed DB config.
- nBTC/USDC pair decisions use `notional_symbol=USDC` and do not display as EUR notional.
- Request preflight accepts `pair_id` plus `source_amount`.
- Request preflight for a non-EURe pair does not require `amount_eure`.
- Outcome attribution can complete a BTC/USDC row from matching inventory deltas.
- Dashboard has no EURe-to-BTC-only request title or settlement copy in active UI.
- Every tracked asset appears in balances with either value or unvalued reason.
- Alerts and state expose active pair/route scope.
- Deployment config still contains no trading asset/pair/edge env vars.
- Services deploy from repo push only.
## Known Fakes Allowed At Start Of This Turn
- Only BTC/EUR and BTC/USDC reference route adapters are real.
- Arbitrary non-BTC asset valuation remains unavailable unless a supported route is configured.
- NEAR Intents remains the only live venue.
- Venue-native terminal fills remain unavailable unless existing relay/status paths already provide them.
- Fee-complete realized PnL remains unavailable.