From 8d2ca61fa17e9a69eef54a7b4e1137f610792d96 Mon Sep 17 00:00:00 2001 From: philipp Date: Fri, 12 Jun 2026 17:13:03 +0200 Subject: [PATCH] Plan verifier salt hot-path turn Proof: Archived the quote lifecycle retention turn and opened detailed PROOF.md/IMPLEMENTATION.md for removing verifier salt refresh from quote-response execution hot path. Assumptions: The current retention turn should be paused so the live salt-induced executor queue latency defect can take priority; no backlog item was selected because the turn was opened directly from live operator evidence. Still fake: Planning only; salt cache refactor, executor rejection behavior, health/dashboard surfaces, deployment, and live latency evidence are not implemented yet. --- ARCHIVE.md | 12 + IMPLEMENTATION.md | 313 +++++++++++------- PROOF.md | 209 ++++++------ ...on-and-analytics-rollups-implementation.md | 290 ++++++++++++++++ ...e-retention-and-analytics-rollups-proof.md | 168 ++++++++++ 5 files changed, 776 insertions(+), 216 deletions(-) create mode 100644 archive/implementation/20260612T150844Z-quote-lifecycle-retention-and-analytics-rollups-implementation.md create mode 100644 archive/implementation/20260612T150844Z-quote-lifecycle-retention-and-analytics-rollups-proof.md diff --git a/ARCHIVE.md b/ARCHIVE.md index 4967a17..edbc9fc 100644 --- a/ARCHIVE.md +++ b/ARCHIVE.md @@ -15,6 +15,13 @@ Legacy note: - 2026-04-09: `quote-lifecycle-truth-and-execution-explanation` closed with status `passed`. Live operator surfaces now derive quote lifecycle from durable evidence, distinguish strategy rejection from executor blocking and submission, expose copyable quote traces, and no longer overclaim submission as completion or asset movement. - 2026-04-10: `execution-outcome-truth-and-settled-attribution` closed with status `passed`. Deployed e0dfd24, validated 1 completed heuristic settlement-attributed trade, 6 inferred not-filled outcomes, submitted-only rows without asset deltas, and stable armed strategy/executor at the approved 1.49% threshold. - 2026-04-12: `settlement-aware-strategy-and-inventory-skew-controls` closed with status `paused`. Paused after deploying approved 0.49 percent threshold, validating live quote intake recovery, and fixing NEAR Intents ingest reconnect-on-error; 0.49 percent submission resolved not_filled and full inventory-skew controls remain incomplete. +- 2026-05-12: `near-intents-request-creation-and-eure-to-btc-taker-flow` closed with status `paused`. Paused because the approved scope shifted from the narrow EURe-to-BTC taker flow to DB-backed assets, pairs, and per-pair strategy configuration; repo-owned taker submission remains future work unless reintroduced under the new pair model. +- 2026-05-18: `db-backed-asset-registry-and-multi-pair-strategy-configuration` closed with status `paused`. DB-backed assets, pair config, importer, dashboard controls, and deployment cleanup were implemented and deployed, but shared runtime paths still contain BTC/EURe-specific taker, attribution, valuation, dashboard, and alert assumptions that must be handled in the next turn. + +- 2026-05-18: `pair-native-trade-semantics-and-multi-asset-outcome-truth` closed with status `passed`. Pair-native DB asset/pair runtime semantics, nBTC/USDC regression coverage, latency visibility, and verifier salt-cache optimization were implemented, validated, and deployed; remaining maker competitiveness work is a new turn. + +- 2026-06-06: `maker-response-latency-and-quote-competitiveness` closed with status `passed`. Durable maker timing, relay failure classification, pair-native competitiveness surfaces, dashboard cleanup, and emergency quote-history retention fixes were implemented, validated, and deployed; long-term storage shape is opened as a separate turn. +- 2026-06-12: `quote-lifecycle-retention-and-analytics-rollups` closed with status `paused`. Paused after retention, dashboard, semantic, and notification fixes were deployed; the next urgent proof is removing verifier salt refresh from the quote execution hot path. ## Research Turns ## Planning Events @@ -31,3 +38,8 @@ Legacy note: - 2026-04-09: opened implementation turn `execution-outcome-truth-and-settled-attribution` from backlog items I017, I018. - 2026-04-10: opened implementation turn `settlement-aware-strategy-and-inventory-skew-controls` from backlog items I015. - 2026-04-12: opened implementation turn `near-intents-request-creation-and-eure-to-btc-taker-flow` from backlog items I017. +- 2026-05-12: opened implementation turn `db-backed-asset-registry-and-multi-pair-strategy-configuration` from backlog items none. +- 2026-05-18: opened implementation turn `pair-native-trade-semantics-and-multi-asset-outcome-truth` from backlog items none. +- 2026-05-18: opened implementation turn `maker-response-latency-and-quote-competitiveness` from backlog items none. +- 2026-06-06: opened implementation turn `quote-lifecycle-retention-and-analytics-rollups` from backlog items none. +- 2026-06-12: opened implementation turn `verifier-salt-hot-path-removal-and-executor-queue-guardrails` from backlog items none. diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index 11419ff..6d91a73 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -1,156 +1,235 @@ -# Implementation Turn: NEAR Intents request creation and EURe-to-BTC taker flow +# Implementation Turn: Verifier salt hot-path removal and executor queue guardrails Status: open -Opened: 2026-04-12 +Opened: 2026-06-12 ## Goal -Build a repo-owned path to create our own NEAR Intents EURe-to-BTC swap request from credited internal inventory, submit it only behind explicit gates, and show truthful request-to-settlement evidence in the dashboard. +Remove verifier `current_salt` RPC latency from quote-response execution, fail closed quickly when no fresh signing salt is available, and make salt freshness plus executor queue delay visible to operators. ## Selected backlog items -- [I017] Repo-owned NEAR Intents request creation and EURe-to-BTC taker flow: create, sign, submit, and settle our own swap requests from credited inventory with truthful dashboard evidence. tags=intents,taker,execution,inventory,settlement +- none selected; this turn was opened directly from the live latency defect observed on 2026-06-12. ## Design rules -- Treat request creation as a new taker lifecycle, not as an incoming quote-response decision. -- Preflight must be safe and side-effect-free. -- Live submission must be explicit, idempotent, and durably recorded before the relay call. -- Use only spendable credited EURe. -- Keep all ids copyable: request id, idempotency key, intent hash or relay request id, submission id. -- Submitted or relay-accepted does not mean completed. -- Completed requires inventory delta attribution. +- This turn is execution latency and visibility only. +- Do not change strategy selection, pricing, edge thresholds, notional limits, inventory checks, arming, signer identity, or pair enablement. +- Do not skip quotes because relay errors are likely. +- Do not submit a quote response if signing prerequisites are unavailable. +- Do not add active pair, edge, notional, latency, response-policy, or retention env vars. +- Keep old executor result rows readable. +- Deploy only through repo workflow. ## Problem statement -The current system watches the NEAR Intents market and responds to other users' quotes. That is not enough to intentionally convert our EURe inventory to BTC. The operator wants a repo-owned way to create our own request on the intents market and eventually move EURe into BTC, with the same truth standard as the incoming quote lifecycle. +The quote-response executor currently calls `verifierSaltCache.getFreshSalt()` from inside `handleCommand`. If the cache decides it needs to refresh, the command waits for the NEAR verifier `current_salt` RPC before signing. -The existing code can sign verifier token_diff payloads for quote responses and can submit quote_response over the solver relay websocket. It does not yet know how to publish a taker request, quote request, or swap intent that asks the market to fill our EURe-to-BTC order. +Live evidence showed this can block for seconds: + +- `ae824f64-ddfd-4310-a44f-69c401fc8ad7` + - `current_salt_source=refresh` + - `current_salt_ms=7481.458` + - executor received the command quickly, then blocked inside salt refresh +- following commands waited in the single executor consumer queue: + - `command_to_executor_ms` around `7.5s` + - some later rows around `10-12s` + - most failed with `quote_not_found_or_finished` + +External verifier latency must not sit on the per-quote execution critical path. ## Backend changes -### 1. Verify request protocol -- Identify the supported NEAR Intents method for creating our own request using primary code/docs or live non-mutating behavior. -- Determine whether the method uses the existing solver relay websocket, a JSON-RPC HTTP endpoint, or a verifier contract call. -- Determine exact payload fields for EURe-to-BTC exact-in requests. -- Verify signed intent semantics: outgoing EURe, incoming BTC, deadline, nonce, verifier contract, signer. -- Record a hard blocker if the protocol cannot be verified. +### 1. Inspect current salt and executor paths +- Inspect: + - `src/core/verifier-salt-cache.mjs` + - `src/apps/trade-executor.mjs` + - executor command expiry helpers + - executor timing payload construction + - runtime health and service snapshot summaries + - ops sentinel alert generation + - operator dashboard service/system views + - existing verifier salt cache and executor tests +- Identify every call path that can refresh verifier salt. +- Confirm which call paths are quote-response hot path versus background or operator-triggered work. -### 2. Add request model and schemas -- Add a canonical repo-owned request lifecycle model. -- Add event schemas for at least intent_request_preflight, intent_request_submit_command, intent_request_submission_result, and intent_request_outcome. -- Persist request id, idempotency key, source/destination assets, requested spend, min receive, reference price, slippage, signer account, verifier contract, deadline, nonce, and created/submitted timestamps. -- Keep this model separate from incoming swap_demand, trade_decision, execute_trade, and trade_result events. +### 2. Define salt cache contract +- Add or refine a cache API with separate responsibilities: + - background refresh may perform network I/O + - hot-path read must never perform network I/O +- Candidate API: + - `start()` starts periodic refresh and immediate refresh attempt + - `refreshNow()` performs explicit refresh with in-flight de-duplication + - `getCachedFreshSalt({ now, maxAgeMs })` returns fresh salt or a typed unavailable result without I/O + - `getState()` returns freshness, age, last refresh times, duration, in-flight state, and error +- Preserve existing public behavior where needed by non-hot-path tests, but quote-response execution must use the non-blocking API. +- Validate malformed salt before it can enter the cache. -### 3. Build preflight service logic -- Load latest spendable EURe inventory from current inventory state or durable snapshot. -- Compute request amount from all spendable EURe minus an explicit reserve if configured, while allowing a smaller operator-specified amount for testing. -- Compute expected BTC from reference price. -- Compute minimum BTC receive from explicit slippage basis points. -- Reject if reference price is stale, inventory is stale, amount is zero, amount exceeds spendable EURe, or signer is not registered. -- Return a draft request object without signing or submitting. +### 3. Background refresh behavior +- Ensure startup kicks off refresh without making the executor command handler wait on it. +- Refresh periodically before the salt becomes stale. +- If refresh fails: + - keep a still-fresh cached salt usable until its age limit expires + - expose last error and stale/unavailable state + - do not block command handling +- Record: + - `last_refresh_started_at` + - `last_refresh_completed_at` + - `last_refresh_duration_ms` + - `last_refresh_error` + - `refresh_in_flight` + - `salt_age_ms` + - `salt_fresh` -### 4. Build signing/envelope helpers -- Add a helper for taker request signing/envelope construction once the verified protocol is known. -- Reuse buildIntentNonce only if nonce semantics match the request protocol. -- Add deterministic signer tests for the exact signed payload. -- Keep quote-response signing tests unchanged. +### 4. Trade-executor hot path +- Replace hot-path `await verifierSaltCache.getFreshSalt()` with non-blocking cached salt read. +- If no fresh cached salt is available: + - publish a durable trade execution result + - use status `rejected` + - use explicit result code such as `verifier_salt_unavailable` + - include salt cache state summary in the result payload + - do not call `buildQuoteResponseSubmission` + - do not call the relay +- Keep command expiry before signing so stale queued commands remain rejected as `stale_execute_command`. +- Keep idempotency and executor state-store semantics intact. +- Remove misleading timing for salt refresh from hot-path results; after this turn hot-path timing should show local cache read or unavailable. -### 5. Build gated submit path -- Add a narrow request-executor or extend trade-executor only if responsibilities stay clear. -- Require explicit submit command with preflight id or idempotency key. -- Mark request as submit_requested before the relay/API call. -- Submit with timeout and reconnect-safe client behavior. -- Persist result as submitted, accepted_by_relay, failed, or protocol-specific status. -- Enforce idempotency so duplicate commands cannot submit twice. +### 5. Queue delay guardrails +- Keep recording `command_to_executor_ms` and `quote_age_at_executor_receipt_ms`. +- Add summary state for recent command queue delay, using existing durable result payloads or in-memory rolling metrics. +- Add thresholded runtime health fields: + - recent max command-to-executor + - p90 or bucketed queue delay if practical + - latest command queue delay warning reason +- Do not use queue delay to change strategy selection or pricing in this turn. -### 6. Settlement attribution -- Extend outcome attribution to handle repo-created requests. -- Match settlement from inventory deltas: EURe decrease and BTC increase consistent with submitted request terms. -- Mark unresolved as awaiting_settlement before deadline/grace expires. -- Mark not_filled only after deadline/grace and fresh inventory evidence. -- Mark completed only with durable inventory movement evidence. +### 6. Runtime health and alerts +- Extend trade-executor `/state` and `/health` output with salt cache freshness and queue-delay status. +- Add ops-sentinel warning alerts for: + - verifier salt stale or unavailable + - verifier salt refresh failing + - executor command queue delay above a conservative threshold +- Keep alert scopes executor/service-level unless pair-specific evidence is available. +- Do not disarm automatically unless an existing safety invariant already does so. -### 7. Dashboard and controls -- Add a Funds or Strategy panel for Create BTC request with preflight amount, slippage/minimum BTC receive, dry-run/preflight action, and live submit only after a valid preflight. -- Add a repo-created requests table with request id, created/submitted time, source spend, min receive, state, decisive reason, relay/API response, settlement result, and expandable lifecycle. -- Make request ids and returned intent/relay ids copyable. -- Label live fund movement clearly; no green success label without settlement. +### 7. Operator dashboard +- Surface salt cache state in the system/service view: + - fresh/stale/unavailable + - salt age + - last refresh time + - last refresh duration + - last refresh error +- Surface executor queue delay alongside existing trade-executor health. +- Preserve existing lifecycle table timing fields and old-row compatibility. +- Avoid presenting salt failures as trade PnL or settled outcomes. + +### 8. Tests +- Add verifier salt cache tests: + - cached fresh read returns immediately and does not call the loader + - missing/stale salt returns unavailable without loader call + - background refresh records success duration and freshness + - background refresh records error without destroying still-fresh salt + - malformed salts fail closed +- Add trade-executor tests or refactorable helper tests: + - slow salt loader cannot block quote command handling + - no cached fresh salt publishes `verifier_salt_unavailable` + - no relay call happens when salt is unavailable + - stale command expiry still happens before salt lookup + - successful command uses cached salt and records `current_salt_source=cache` +- Add runtime health and alert tests for salt and queue warnings. +- Add dashboard/static tests for salt and queue visibility. + +### 9. Validation and deploy +- Run targeted tests: + - verifier salt cache + - trade executor + - runtime health / ops sentinel + - dashboard static/operator tests +- Run full `npm test`. +- Build operator dashboard bundle. +- Commit with required workflow metadata. +- Push to `main` so repo workflow deploys. +- Do not manually patch or roll Kubernetes deployments. + +### 10. Live evidence after deploy +- Confirm all repo-owned deployments run the new image. +- Query recent trade execution results: + - `current_salt_source` + - `current_salt_ms` + - `command_to_executor_ms` + - `quote_age_at_executor_receipt_ms` + - `result_code` + - `failure_category` +- Prove no post-deploy quote-response result used `current_salt_source=refresh`. +- Report command-to-executor p50/p90/p99 after rollout. +- Report salt cache health from trade-executor state or health. +- Confirm recent quote flow still writes decisions, commands, and results. ## Data and persistence -- Prefer adding narrow tables or durable event routes for request preflight, submit commands, submission results, and outcomes. -- Preserve raw relay/API responses alongside normalized lifecycle state. -- Include enough fields to replay the request and reason about slippage, deadline, and settlement. -- Do not reuse incoming quote ids as request ids. +- Use existing executor result payloads for per-command salt and queue timing. +- Add fields additively so old rows remain readable. +- Prefer in-memory runtime state for recent queue summaries unless durable aggregation is already available. +- Do not add new long-term storage unless required for dashboard or alert correctness. ## Edge cases -- Missing request API evidence: stop with blocker. -- Signer not registered: block before signing/submission. -- EURe inventory stale or unavailable: block. -- Requested amount exceeds spendable EURe: block. -- Reference price stale: block. -- Slippage missing: block. -- Duplicate idempotency key: return original result, do not submit again. -- Relay timeout: failed or unknown state with durable reason, not completed. -- Relay accepted but no inventory movement: awaiting outcome, then not filled after evidence window. -- Partial fill, if supported: represent explicitly; otherwise block or mark ambiguous until semantics are known. +- Salt cache empty at startup: commands reject quickly until background refresh succeeds. +- Salt cache stale: commands reject quickly, no relay call. +- Salt refresh in flight: commands use still-fresh cached salt or reject; they do not wait. +- Salt refresh fails while cached salt is still fresh: commands may continue using cached salt. +- Salt refresh fails and cache is stale: commands reject with explicit salt-unavailable result. +- Malformed salt from verifier: fail closed and expose error. +- Existing stale command: reject as stale command before salt lookup. +- Executor paused or disarmed: existing behavior remains. +- Old result rows without salt fields remain visible. ## Concrete implementation order -### Phase 1. Protocol verification and model -- Inspect primary NEAR Intents request API evidence and current repo clients. -- Decide endpoint/method and payload shape. -- Add request lifecycle constants and schemas. -- Add tests for semantic invariants independent of live API. +### Phase 1. Audit and contracts +- Inspect current salt cache and executor usage. +- Add salt-cache contract tests for non-blocking cached reads and refresh state. +- Decide exact result code and payload shape for salt-unavailable rejection. -### Phase 2. Preflight path -- Implement pure preflight helper for EURe-to-BTC exact-in requests. -- Wire inventory, reference price, signer registration, amount, slippage, and deadline checks. -- Add tests for success, insufficient inventory, stale price, stale inventory, zero amount, and missing signer. +### Phase 2. Salt cache refactor +- Implement non-blocking cached salt read. +- Implement refresh state fields and in-flight de-duplication. +- Ensure background refresh starts and continues. +- Keep existing cache tests passing with updated contract. -### Phase 3. Signing and submit client -- Implement taker request envelope/signing helper. -- Implement relay/API client method with timeout and error normalization. -- Add deterministic signing tests and mocked client submission tests. +### Phase 3. Executor integration +- Change quote-response hot path to use cached salt only. +- Add salt-unavailable rejection result. +- Add timing/state fields for cache read and unavailable rejection. +- Add regression tests for slow salt loader and no relay call. -### Phase 4. Durable execution path -- Add request submit command/result event routes and persistence. -- Add idempotency guard. -- Add submission failure and duplicate-submit tests. +### Phase 4. Health, alerts, dashboard +- Expose salt freshness and queue delay in trade-executor state/health. +- Add ops-sentinel/runtime alert tests. +- Add dashboard visibility tests and UI fields. -### Phase 5. Outcome truth -- Extend request outcome attribution using inventory deltas. -- Add tests for submitted-not-completed, accepted-not-completed, completed with exact settlement, not-filled after deadline, and ambiguous movement. - -### Phase 6. Operator UI -- Add preflight/submit controls and request lifecycle table. -- Add copy affordances for request and relay ids. -- Add dashboard tests proving no submitted-only request is labeled successful. - -### Phase 7. Deploy and validate -- Run targeted tests plus full npm test. +### Phase 5. Validation and deployment +- Run targeted tests. +- Run full `npm test`. - Build dashboard bundle. -- Commit with required proof body. -- Push to forgejo/main. -- Validate rollout image, service state, dashboard bootstrap, and live dry-run/preflight. -- If operator explicitly submits live, validate durable request row, submission result, and later settlement or no-fill. +- Commit and push. +- Collect live post-deploy evidence. ## Test plan -- Request preflight unit tests. -- Request signing/envelope tests. -- Request client timeout/failure tests. -- Idempotency tests. -- Persistence routing tests. -- Outcome attribution tests. -- Dashboard semantic tests. -- Negative tests: submitted != completed; relay accepted != completed; pending EURe cannot be spent; dry-run cannot submit; duplicate submit does not call relay twice; request lifecycle cannot be confused with incoming quote response lifecycle. +- `test/verifier-salt-cache.test.mjs` +- `test/trade-executor*.test.mjs` or new extracted executor helper tests +- `test/runtime-health.test.mjs` +- `test/ops-sentinel*.test.mjs` +- `test/operator-dashboard*.test.mjs` +- Static config tests proving no new active env knobs are added +- Full `npm test` +- Dashboard build ## Validation checklist against the proof -- /state or dashboard shows request preflight capability and current spendable EURe. -- A dry-run EURe-to-BTC request can be produced without side effects. -- A live submit path exists only behind explicit operator action. -- Durable request lifecycle rows exist and are visible with copyable ids. -- Settlement outcome remains truthful after submission. -- All services deploy from repo push without manual cluster reconciliation. +- Quote-response hot path contains no verifier salt network refresh. +- Slow `current_salt` loader does not cause multi-second command queue delay. +- Missing/stale salt creates durable explicit rejection without relay call. +- Salt freshness and refresh errors are operator-visible. +- Executor queue delay is operator-visible and alertable. +- Strategy behavior, edge, notional, inventory, arming, signer, and pair enablement are unchanged. +- Repo workflow deploys the result. +- Live results show `current_salt_source=cache` or salt-unavailable rejection, not hot-path `refresh`. ## Known fakes allowed at start of this turn -- Request API method is not yet verified. -- Venue-native terminal fill events remain unavailable unless the request API exposes them. +- Venue-native terminal fill ids remain unavailable. - Fee-complete realized PnL remains unavailable. -- Automatic portfolio optimization remains unavailable. +- Executor concurrency remains single-consumer unless proven necessary after salt hot-path removal. diff --git a/PROOF.md b/PROOF.md index 35c87be..0fcae60 100644 --- a/PROOF.md +++ b/PROOF.md @@ -1,133 +1,144 @@ -# Implementation Proof: NEAR Intents request creation and EURe-to-BTC taker flow +# Implementation Proof: Verifier salt hot-path removal and executor queue guardrails Status: open -Opened: 2026-04-12 +Opened: 2026-06-12 ## Target outcome -The repository must be able to create our own NEAR Intents EURe-to-BTC swap request from credited internal inventory, submit it through repo-owned code, and explain the full request lifecycle from preflight through settlement truth. +Quote-response execution must not block on the NEAR verifier `current_salt` RPC. -For each repo-created request the operator must be able to answer: -- which asset and amount we intended to spend -- which asset and minimum amount we expected to receive -- which signer, verifier contract, nonce, deadline, slippage, and idempotency key were used -- whether the request was drafted, blocked, submitted, accepted by relay, failed, awaiting settlement, not filled, or completed -- whether completed means durable inventory movement, not relay acceptance -- how much BTC was received and how much EURe was spent, if settlement is proven +The executor should use an already-fresh cached signing salt in the hot path, reject quote-response commands quickly and explicitly when no fresh salt is available, and expose enough runtime evidence for the operator to see salt freshness and command queue delay before they turn fresh quotes into stale relay failures. ## Why this is the next architecture test -The existing system can observe incoming NEAR Intents quote requests and respond as a solver or maker. That path has not produced fills at 1.49%, 0.99%, or 0.49% in recent live validation, and it cannot intentionally convert our EURe inventory into BTC. +Live maker timing exposed a real hot-path false path: -The next false path would be manually moving funds or adding a button that submits opaque live requests without durable preflight, request, and settlement evidence. This turn is successful only if the system can create requests as a first-class, auditable repo-owned flow instead of treating live fund movement as an off-system action. +- normal command-to-executor latency is usually milliseconds +- at `2026-06-12 14:36 UTC`, one command spent about `7481 ms` refreshing `current_salt` +- the single executor consumer then queued later commands for `7-12s` +- those commands reached the relay after the quote was already gone and failed with `quote_not_found_or_finished` + +The trading system cannot treat an external verifier RPC as part of per-quote execution. Execution must outpace observability, but it also must fail closed when signing prerequisites are unavailable. ## Scope -- [I017] Repo-owned NEAR Intents request creation and EURe-to-BTC taker flow: create, sign, submit, and settle our own swap requests from credited inventory with truthful dashboard evidence. tags=intents,taker,execution,inventory,settlement -- Active venue only: NEAR Intents. -- Active direction: credited internal EURe to BTC. -- Use credited spendable internal inventory only. Pending funding remains non-spendable. -- Build the request path with dry-run/preflight first, then gated live submission. -- Actual live submission requires explicit operator control and must be recorded durably. +- Inspect the current verifier salt cache, trade-executor quote-response hot path, command expiry guard, executor timing payloads, runtime health, ops sentinel alerts, and dashboard service views. +- Add a non-blocking cached-salt API for quote-response execution. +- Move verifier salt refresh responsibility fully to background refresh and explicit refresh state. +- Ensure the quote-response hot path never performs `current_salt` RPC or waits on a salt refresh. +- Reject commands quickly with an explicit durable result when no fresh cached salt is available. +- Preserve existing command expiry behavior so stale commands fail closed before relay submission. +- Surface verifier salt freshness, refresh duration/error, and executor queue delay in runtime health and dashboard/operator surfaces. +- Add runtime alert coverage for stale or unavailable verifier salt and excessive executor queue delay. +- Deploy only through the repo workflow and collect live evidence after rollout. ## Assumptions -- NEAR Intents exposes a supported request or quote publication method through the solver relay or another documented endpoint callable with our existing API key and verifier signer. -- The correct taker intent can be represented as a signed verifier token_diff payload or a documented request envelope whose semantics can be proven before live submission. -- Internal inventory snapshots are fresh enough to enforce spendable EURe and settlement deltas. -- BTC minimum receive can be derived from reference price plus explicit slippage or minimum-out policy. -- The user's intent is to build the capability to move EURe into BTC; live all-EURe execution remains behind an explicit final operator action in the repo-owned UI or API. +- A verifier salt can be reused safely within the existing freshness policy enforced by `verifier-salt-cache`. +- The current salt cache background refresher can be extended rather than replaced. +- A command that cannot be signed because no fresh salt is available should be rejected before relay submission; this is a signing prerequisite failure, not quote skipping to avoid relay errors. +- The executor remains a single quote-response consumer unless implementation evidence shows concurrency is required after salt is removed from the hot path. +- Venue-native terminal fill ids and fee-complete realized PnL remain unavailable. ## Turn-shaping rules -- Do not manually move funds. -- Do not submit an all-EURe live request until repo-owned preflight, durable request logging, and settlement attribution are implemented and the operator explicitly triggers that live request. -- Do not use pending or uncredited EURe. -- Do not label relay acceptance as completed settlement. -- Do not hide slippage, minimum receive, deadline, nonce, signer, or request id. -- If the NEAR Intents request API cannot be verified from primary evidence or live dry-run behavior, stop and record the blocker instead of inventing a protocol. -- Keep maker quote-response logic intact unless the new taker flow proves it should be paused or separated. +- Do not change strategy selection, edge thresholds, notional limits, inventory checks, arming semantics, signer identity, pair enablement, or quote response pricing. +- Do not skip quotes because they may produce relay errors. +- Do not add active pair, edge, notional, latency, response-policy, or retention env vars. +- Do not hide or relabel relay failures as successful outcomes. +- Do not perform manual rollout, image patching, or cluster repair; deployment must be push-driven through the repo workflow. +- If a salt-related bug is fixed, add regression tests. +- Preserve old executor result rows as readable even if they lack new salt fields. ## Non-goals -- No new venue. -- No external wallet or manual bridge action. -- No automatic recurring rebalancer. -- No realized net PnL claim without fee and cost attribution. -- No tax or accounting treatment. -- No broad multi-asset request builder beyond EURe-to-BTC. +- No new trading strategy. +- No executor concurrency change unless required to prove the salt fix. +- No change to quote response economics. +- No new venue or signer. +- No automatic disarming based on salt health unless an existing safety path already does so. +- No realized PnL or fee attribution. ## Required operator behavior -### Request preflight truth -Before submission, the operator must see: -- spendable EURe available -- requested EURe spend amount -- BTC reference price -- slippage or minimum-out rule -- expected BTC receive and minimum BTC receive -- deadline and nonce policy -- exact signer account and verifier contract -- whether this is dry-run only or live-submit capable +### Salt Truth +The operator must be able to see: -### Request lifecycle truth -For each repo-created request, the dashboard must show: -- request id, copyable -- idempotency key, copyable -- intent hash or relay request id if returned, copyable -- created_at, submitted_at, resolved_at if known -- state: draft, blocked, submitted, accepted_by_relay, failed, awaiting_settlement, not_filled, completed -- decisive reason for blocks, failures, or no-fill -- settlement evidence if completed +- whether the verifier salt cache is fresh +- current salt age +- last refresh start and completion time +- last refresh duration +- whether a refresh is in flight +- last refresh error, if any +- count or timestamp of commands rejected because no fresh salt was available -### Settlement truth -Completed requires durable asset movement evidence: -- EURe spend delta and BTC receive delta from inventory snapshots -- attribution method and caveat if heuristic -- submitted-only and relay-accepted rows remain separate from completed -- portfolio-vs-hold movement remains separate from realized trade PnL +### Queue Truth +The operator must be able to see: + +- recent command-to-executor latency +- whether queue delay is warning or critical +- recent executor result timing including salt source and salt lookup duration +- whether quote failures cluster after queue delay rather than relay latency + +### Execution Truth +For each quote-response result: + +- cached salt use should be explicit in `executor_timing.current_salt_source` +- `current_salt_source=refresh` must not appear from the quote-response hot path after this turn +- no relay call should be attempted when a fresh cached salt is unavailable +- missing or stale salt must produce a durable rejected result with an explicit result code ## Semantic invariants -The implementation and tests must enforce at least: -- request submitted != completed -- relay accepted != settled inventory movement -- pending EURe cannot be spent -- insufficient spendable EURe blocks before signing or submission -- slippage and minimum-out must be explicit and persisted -- duplicate idempotency key cannot produce duplicate live submissions -- dry-run request cannot accidentally submit -- completed request rows require durable settlement evidence -- request-creation flow is distinct from incoming quote-response flow +- Quote-response signing prerequisites are fail-closed. +- No fresh cached salt means no relay submission. +- Hot-path salt lookup is local and bounded; it never performs network I/O. +- Background salt refresh failures are visible but do not block already-fresh cached salt usage. +- Stale commands are still rejected before relay submission. +- Queue delay and relay latency remain separate timing concepts. +- Old executor result rows remain readable. ## Definition of done -- The repo contains a verified NEAR Intents request-creation client or a recorded blocker proving the venue path is unavailable. -- A request preflight API computes amount, min-out, deadline, nonce policy, signer, verifier, and inventory checks from durable/current data. -- A gated live submission API exists only after preflight and records durable request, submission, and result events. -- The dashboard shows repo-created request rows separately from incoming quote-response rows, with copyable ids and lifecycle reasons. -- Settlement attribution handles repo-created requests without treating relay acceptance as completion. -- Regression tests cover request signing/envelope semantics, inventory blocks, dry-run safety, idempotency, submission failure, submitted-not-completed, and completed-only-with-settlement. -- The result is deployed through the repo workflow and validated against live service state. +- `verifier-salt-cache` exposes a non-blocking fresh-salt read path for hot execution. +- Background salt refresh maintains cache state and exposes freshness, duration, in-flight state, and errors. +- Trade-executor quote-response handling uses only the non-blocking cached-salt path. +- Trade-executor publishes durable rejected results when no fresh cached salt is available. +- Executor timing payloads distinguish cached salt, unavailable salt, and historical refresh rows. +- Runtime health and dashboard expose salt freshness and executor queue delay. +- Ops sentinel or runtime alert tests cover stale/unavailable salt and excessive queue delay. +- Regression tests prove slow verifier salt RPC cannot block quote-response command handling. +- Full `npm test` passes. +- Dashboard bundle build passes. +- The result is deployed through repo workflow. +- Live post-deploy evidence shows quote flow continues and command-to-executor latency is not dominated by salt refresh. ## Validation evidence required -- Unit tests for request preflight and minimum-out/slippage math. -- Unit tests for signed request or request envelope construction using deterministic signer fixtures. -- Unit tests proving insufficient EURe blocks before signing/submission. -- Unit tests proving submitted/accepted requests do not count as completed without inventory deltas. -- Live dry-run/preflight evidence for current credited EURe inventory. -- If live submission is explicitly triggered, durable DB evidence for request, submission result, and later settlement or no-fill. -- Dashboard bootstrap evidence showing the new request lifecycle row with copyable ids and truthful state. +- Unit tests for non-blocking cached-salt reads. +- Unit tests proving background refresh records duration/error and does not require hot-path waiting. +- Trade-executor tests proving a slow `current_salt` RPC does not delay command rejection or handling. +- Tests proving no relay submission is attempted when cached salt is missing/stale. +- Tests proving stale command expiry still wins before signing. +- Runtime health or ops sentinel tests for stale salt and queue-delay warnings. +- Dashboard/static tests for salt freshness and queue-delay visibility. +- Full `npm test`. +- Dashboard bundle build. +- Live evidence after deployment: + - trade-executor deployment on the new image + - recent executor results with `current_salt_source=cache` or explicit salt-unavailable rejection, not hot-path `refresh` + - command-to-executor p50/p90/p99 after rollout + - salt cache freshness state + - recent quote flow still writing decisions, commands, and results ## Failure conditions -- Live funds are moved outside repo-owned code. -- A request is submitted without durable preflight and request records. -- The UI claims BTC was bought from relay acceptance alone. -- A dry-run path can submit. -- Pending funding is treated as spendable. -- The implementation cannot prove the NEAR Intents request API shape and still proceeds. +- Any quote-response command performs verifier `current_salt` RPC in the hot path. +- A slow salt refresh queues later commands for seconds. +- Missing or stale salt leads to relay submission instead of fail-closed rejection. +- Dashboard or health hides stale salt or queue delay. +- The implementation changes strategy behavior, pricing, inventory policy, or pair enablement. +- Deployment requires manual cluster reconciliation. ## Current real before this turn -- The repo has verifier signing for quote-response token_diff payloads. -- The repo has a solver relay websocket client and quote-response submission path. -- Credited internal BTC/EURe inventory is synced durably. -- Quote, decision, command, submission result, and outcome attribution records exist for incoming quote-response flow. -- The dashboard distinguishes submitted, not-filled, completed, rejected, and blocked rows for incoming quote responses. +- Trade-executor has a verifier salt cache and records executor timing. +- Live evidence showed normal command-to-executor p50 around `11 ms`, but p99 near `9954 ms` during a salt refresh stall. +- The slow rows showed `current_salt_source=refresh` with `current_salt_ms` around `7481 ms` and `5144 ms`. +- Subsequent queued rows reached the executor after `7-12s` and often failed with `quote_not_found_or_finished`. +- Command expiry and maker timing fields already exist, but salt freshness and queue-delay guardrails are not operator-visible enough. ## Deliberately not built by this proof -- General portfolio optimizer. -- Automatic all-in rebalancing loop. +- Executor worker pool or broad concurrency model unless the non-blocking salt path is insufficient. - Fee-complete realized PnL. -- Venue-native terminal fill ingestion unless the request API provides it directly. +- Venue-native terminal fill ids. +- Automatic strategy changes based on salt or queue metrics. diff --git a/archive/implementation/20260612T150844Z-quote-lifecycle-retention-and-analytics-rollups-implementation.md b/archive/implementation/20260612T150844Z-quote-lifecycle-retention-and-analytics-rollups-implementation.md new file mode 100644 index 0000000..e257f29 --- /dev/null +++ b/archive/implementation/20260612T150844Z-quote-lifecycle-retention-and-analytics-rollups-implementation.md @@ -0,0 +1,290 @@ +# Implementation Turn: Quote lifecycle retention and analytics rollups + +Status: open +Opened: 2026-06-06 + +## Goal +Keep durable successful trade evidence while preventing quote firehose and non-success lifecycle rows from filling Postgres by adding explicit retention policy, compact rollups, table-size guardrails, and operator-visible storage truth. + +## Selected backlog items +- none selected; this turn was opened directly from the operator request on 2026-06-06. + +## Design rules +- This turn is storage and analytics only. +- Do not change strategy actionability, quote response policy, edge thresholds, inventory checks, pair enablement, arming, signer checks, or relay submission behavior. +- Do not skip quotes to avoid relay errors. +- DB config and persisted runtime state are canonical; do not add active retention env vars. +- Completed successful trade evidence must survive every cleanup path. +- Non-success quote detail may be short-lived if aggregate rollups are persisted first. +- Old rows must remain readable. +- Deploy only through repo workflow. + +## Problem statement +The previous raw quote retention fix did not prevent Postgres growth because normalized lifecycle tables became the dominant storage users: + +- `trade_decisions` +- `swap_demand_events` +- `execute_trade_commands` +- `trade_execution_results` +- `quote_outcome_attributions` + +Emergency cleanup proved the system can keep completed quote outcomes and linked rows while pruning non-success detail, but that cleanup was operational, not a complete product path. The repository now needs a durable retention and rollup layer so live quote analytics remain useful without unbounded row accumulation. + +## Backend changes + +### 1. Inspect current storage and retention paths +- Inspect: + - `ensureHistorySchema` + - quote lifecycle table creation and indexes + - history-writer batch ingestion + - current raw quote prune + - current non-success lifecycle prune + - dashboard bootstrap queries + - competitiveness aggregation helpers + - runtime health and alert summaries + - deploy/static config tests +- Identify all tables that grow with quote firehose or lifecycle activity. +- Record current row sizes and indexes in implementation notes or test fixtures where useful. + +### 2. Define retention data classes +- Add a shared retention classifier for quote-linked rows: + - `successful_trade_evidence` + - `in_flight_detail` + - `non_success_detail` + - `raw_debug_firehose` + - `aggregate_rollup` +- Preserve successful evidence when: + - `quote_outcome_attributions.outcome_status = 'completed'`, or + - attribution status indicates successful linkage such as `heuristic_match`, `exact_match`, or a locally established equivalent. +- Treat submitted/not_filled/failed/rejected rows as non-success detail once outside the retention window. +- Keep classification pair-native and quote-id based. +- Add tests for classification using current and legacy payload shapes. + +### 3. DB-backed retention policy +- Add DB-owned retention policy storage, either: + - a narrow `retention_policies` table, or + - a versioned config row if an established local config pattern fits better. +- Candidate fields: + - `policy_key` + - `enabled` + - `normal_detail_retention_ms` + - `pressure_detail_retention_ms` + - `raw_retention_ms` + - `rollup_granularity_ms` + - `rollup_retention_ms` + - `preserve_successful_evidence` + - `max_delete_rows_per_pass` + - `pressure_table_bytes` + - `pressure_database_bytes` + - `updated_at` + - `config_version` +- Seed conservative defaults in repo-owned DB seed/migration code. +- Missing or invalid policy must fail closed by skipping prune, not by deleting data. +- No env var fallback. + +### 4. Rollup model +- Add compact quote lifecycle rollups before pruning detail. +- Store rollup rows keyed by: + - rollup window start/end + - pair id + - direction + - request kind + - edge bps bucket or exact configured edge + - notional asset + - notional bucket + - quote-age bucket + - result code + - failure category + - outcome status +- Store compact measures: + - quote count + - decision count + - command count + - executor result count + - relay accepted count + - relay failed count + - not_filled count + - completed count + - p50/p90-compatible timing bucket counts, or fixed timing histogram buckets + - min/max observed timestamps + - source row high-water mark or computed-at watermark +- Prefer bucket histograms over storing raw samples indefinitely. +- Add indexes for dashboard rollup queries. + +### 5. Rollup writer +- Extend history-writer or add a narrow repo-owned maintenance module used by history-writer. +- On a schedule: + - find recent unrolled quote lifecycle rows + - aggregate them into rollup windows + - upsert rollup rows idempotently + - record rollup watermarks +- Rollup must run before detail prune. +- If rollup fails, detail prune for the affected window must not run. +- Add tests for idempotent upsert and no double-counting. + +### 6. Detail retention prune +- Replace ad hoc non-success lifecycle prune with policy-driven retention. +- Prune these quote-linked detail tables only after successful rollup: + - `raw_near_intents_quotes` + - `swap_demand_events` + - `trade_decisions` + - `execute_trade_commands` + - `trade_execution_results` + - `quote_outcome_attributions` +- Preserve: + - all successful quote ids and linked lifecycle rows + - rows within the active retention window + - rows whose outcome attribution is still in flight and younger than the configured outcome grace window +- Delete in bounded batches and expose per-table counts. +- Prefer table/index bloat control using repo-owned `VACUUM`/`REINDEX` guidance or tested maintenance where appropriate. +- Do not use untracked manual SQL as the normal path. + +### 7. Pressure guardrails +- Add DB/table size inspection helpers using Postgres catalog views. +- Add a retention mode decision: + - `normal` + - `pressure` + - `blocked_invalid_policy` + - `blocked_rollup_stale` +- In pressure mode: + - shorten non-success detail retention to the DB policy's pressure value + - keep successful evidence + - keep rollups + - log and expose the pressure reason +- Do not disarm, pause, or alter strategy behavior from storage pressure in this turn unless an existing safety invariant already does so. +- Add tests proving pressure mode does not delete successful evidence. + +### 8. Operator dashboard +- Add a storage/retention section to the operator dashboard. +- Show: + - table sizes and approximate row counts + - retention mode + - configured retention windows + - last rollup time + - last prune time + - rows pruned by table + - successful quote evidence count + - rollup row count and freshness +- Update competitiveness views to use rollups for older windows and recent detail for live drilldown. +- Mark old pruned detail plainly as unavailable rather than empty. +- Keep pair-native labels. +- Add dashboard tests. + +### 9. Runtime health and alerts +- Add runtime health fields for: + - retention mode + - storage pressure + - rollup freshness + - prune failures + - table size thresholds +- Add warning alerts for: + - rollup stale beyond threshold + - retention blocked by invalid policy + - quote lifecycle tables above pressure threshold +- Do not make storage pressure look like a trading result or settled PnL issue. +- Add tests for alert payloads and pair-independent storage warnings. + +### 10. Repo-owned emergency command +- Add or extend an ops script for dry-run and execute retention maintenance. +- Required behavior: + - dry-run default + - show preserve count for successful quote ids + - show delete candidates by table + - refuse to run if preserve query fails + - refuse to run if rollups are stale unless explicitly marked as emergency and documented +- This script is for controlled recovery, not normal operation. + +## Data and persistence +- Use additive schema changes where practical. +- Keep successful lifecycle rows in existing tables so old dashboard drilldowns continue to work. +- Keep rollups compact enough to retain for substantially longer than detail. +- Keep raw JSON payloads only in short-lived detail or successful evidence rows. +- Store original relay failure category/text in rollup dimensions where already normalized. +- Maintain old-row compatibility for missing timing/retention fields. + +## Edge cases +- Missing quote id: do not classify as successful evidence; retain only inside short recent window unless rollup logic can safely count it. +- Missing outcome attribution table or failed preserve query: skip prune. +- Rollup watermark missing: skip prune for affected detail window. +- Clock skew or missing timestamps: bucket as `unknown`, do not invent age. +- Duplicate event ids: rollup idempotency must not double count. +- Completed quote has no raw row because raw was already pruned: preserve available linked lifecycle rows. +- Existing emergency-pruned history cannot be recreated. +- Old rows with legacy BTC/EURe fields remain readable and do not break rollups. + +## Concrete implementation order + +### Phase 1. Storage audit and contracts +- Inspect schema, indexes, history-writer retention, dashboard loaders, and runtime-health paths. +- Add retention classifier helpers and tests. +- Add storage metrics helper tests using SQL/static fixtures. + +### Phase 2. Retention policy +- Add DB-backed retention policy schema/seed. +- Add policy loader and validation. +- Add fail-closed tests for missing/invalid policy. + +### Phase 3. Rollup storage and writer +- Add rollup table/schema. +- Add rollup aggregation helper and SQL upsert path. +- Add idempotency and grouping tests for nBTC/EURe and nBTC/USDC. + +### Phase 4. Prune integration +- Replace hardcoded prune constants with DB policy. +- Ensure rollup-before-prune ordering. +- Preserve completed successful quote ids across all quote lifecycle tables. +- Add bounded batch and pressure-mode tests. + +### Phase 5. Dashboard and health +- Add storage/retention dashboard card or page. +- Wire rollup freshness and table sizes into dashboard bootstrap/WebSocket state. +- Add runtime-health warning fields and alert tests. + +### Phase 6. Validation and deploy +- Run targeted retention/rollup/storage tests. +- Run full `npm test`. +- Build dashboard bundle. +- Deploy through repo workflow only. +- Collect live evidence: + - table sizes before/after at least one prune cycle + - rollup freshness + - retained successful evidence count + - recent quote flow still writing rows + - all deployments healthy + +## Test plan +- Retention classifier tests. +- Successful quote preserve tests. +- Rollup grouping tests: + - pair + - direction + - request kind + - result code + - failure category + - quote-age bucket + - notional bucket + - outcome status +- Rollup idempotency tests. +- Policy validation tests. +- Prune blocked when rollup stale. +- Pressure-mode prune tests. +- Old-row compatibility tests. +- Dashboard storage/retention UI tests. +- Runtime-health/alert tests. +- Static config tests proving no new active retention env vars. +- Full `npm test`. +- Dashboard build. + +## Validation checklist against the proof +- Successful completed quote evidence remains in detail tables after prune. +- Non-success detail tables stay bounded under live quote flow. +- Rollups continue to answer competitiveness questions after old detail is pruned. +- Dashboard shows retention mode, table sizes, rollup freshness, and prune results. +- No strategy behavior changes. +- Repo workflow deploys the result. + +## Known fakes allowed at start of this turn +- Historical non-success detail already pruned cannot be recovered. +- Fee-complete realized PnL is still unavailable. +- Venue-native terminal fill ids remain unavailable. +- Rollups are analytics summaries, not proof of individual trades. diff --git a/archive/implementation/20260612T150844Z-quote-lifecycle-retention-and-analytics-rollups-proof.md b/archive/implementation/20260612T150844Z-quote-lifecycle-retention-and-analytics-rollups-proof.md new file mode 100644 index 0000000..5aad22d --- /dev/null +++ b/archive/implementation/20260612T150844Z-quote-lifecycle-retention-and-analytics-rollups-proof.md @@ -0,0 +1,168 @@ +# Implementation Proof: Quote lifecycle retention and analytics rollups + +Status: open +Opened: 2026-06-06 + +## Target outcome +The system must keep durable successful trade evidence and useful competitiveness analytics without allowing quote firehose or non-success lifecycle detail to fill the Postgres PVC again. + +This turn succeeds when raw quotes, normalized quote demand, rejected decisions, failed executor results, and not-filled outcomes have explicit DB-backed retention and compact rollups, while completed trade evidence remains durable and operator-visible. + +## Why this is the next architecture test +The maker timing turn made response behavior explainable, but live operation exposed a storage false path: + +- raw quote retention alone was not enough +- `trade_decisions` and `swap_demand_events` grew into the largest tables +- emergency cleanup had to prune almost all non-success quote lifecycle rows +- the operator still needs competitiveness truth without storing every quote forever + +The next architecture test is whether `unrip` can keep the right evidence permanently and summarize the rest before deleting it. + +## Scope +- Define quote-history data classes: + - successful settled trade evidence + - in-flight lifecycle detail + - non-success detail + - raw debug firehose + - aggregate analytics +- Keep successful trade evidence permanently: + - completed quote outcomes + - linked normalized demand, decision, command, executor result, and attribution rows + - inventory/settlement evidence needed to explain the trade + - config identifiers and payload fields needed to reproduce the decision +- Add compact rollups before pruning detail: + - pair + - direction + - request kind + - edge bps + - notional bucket + - quote-age bucket + - result code + - failure category + - outcome status + - counts and timing bucket summaries +- Add DB-backed retention policy for quote lifecycle data, not env vars. +- Enforce bounded retention in history-writer or a repo-owned maintenance path. +- Add storage guardrails that detect table/PVC pressure and tighten non-success detail retention without deleting successful trade evidence. +- Surface retention mode, table sizes, retained successful rows, and rollup freshness in the operator dashboard. +- Keep old rows readable and dashboards tolerant of missing rollups. +- Deploy only through repo workflow. + +## Assumptions +- Completed quote outcome attribution is the current durable proof of successful trade settlement. +- Venue-native terminal fill ids and fee-complete realized PnL remain unavailable. +- Non-success quote detail is useful for short live debugging, but long-term analysis can use rollups plus sampled/recent detail. +- Postgres remains the canonical event store for the current system. +- The current single-node cluster has enough disk for compact rollups and successful evidence, but not unbounded quote detail. + +## Turn-shaping rules +- Do not change strategy selection or maker response behavior. +- Do not skip quotes because of storage or relay-error concerns. +- Do not loosen edge, notional, inventory, arming, signer, or pair enablement. +- No active pair, edge, notional, retention, or latency env vars. +- Do not delete completed trade evidence. +- Do not label rollup counts as realized PnL or settled trades. +- Manual emergency pruning may be documented as evidence, but the lasting fix must be repo-owned and deployed from push. +- Destructive cleanup paths must have dry-run/test coverage and must fail closed on missing preserve criteria. + +## Non-goals +- No new trading strategy. +- No response-age or competitiveness-based quote skipping. +- No new venue. +- No broad data warehouse outside Postgres. +- No fee-complete realized PnL. +- No permanent infrastructure spend unless explicitly approved. +- No retroactive recovery of quote detail already pruned during emergency cleanup. + +## Required operator behavior + +### Storage Truth +The operator must be able to see: + +- current size of quote lifecycle tables +- estimated row counts by table family +- retention policy in effect +- last prune time and prune result +- rollup freshness +- count of permanent successful trade rows +- whether the system is in normal or pressure retention mode + +### Analytics Truth +The operator must still be able to answer after detail pruning: + +- how many quotes arrived by pair/direction/request kind +- how many were actionable, rejected, submitted, failed, not filled, or completed +- whether failures cluster by quote-age bucket or notional bucket +- whether `quote_not_found_or_finished` dominates a pair/direction +- how many accepted responses became completed settlement evidence + +### Evidence Truth +For completed trades, the operator must retain per-quote lifecycle detail, not only rollups. + +For non-success cases, old detail may disappear, but the rollup must preserve aggregate counts and bucketed timing truth. + +## Semantic invariants +- Completed settlement evidence is permanent unless the operator explicitly approves an archive/export workflow in a future turn. +- In-flight rows inside the retention window are not pruned before attribution can observe settlement/no-fill. +- Rollups are written before detail deletion. +- Retention policy is DB-owned and visible. +- Missing or invalid retention policy does not delete data. +- Pressure mode may shorten non-success detail retention, but never deletes successful evidence. +- Pair-native labels remain canonical. +- Old rows without new retention metadata remain readable. + +## Definition of done +- Quote lifecycle retention policy exists in DB-owned config or an equivalent repo-owned persisted mechanism. +- Rollup tables or records exist for maker quote lifecycle analytics. +- History-writer or a repo-owned maintenance service writes rollups before pruning detail. +- Non-success detail retention prevents `raw_near_intents_quotes`, `swap_demand_events`, `trade_decisions`, `execute_trade_commands`, `trade_execution_results`, and `quote_outcome_attributions` from unbounded growth. +- Successful completed quote lifecycle rows are preserved across retention. +- Storage guardrails surface pressure and tighten only non-success detail. +- Dashboard exposes storage/retention/rollup status. +- Tests cover rollup generation, retention preserve rules, pressure mode, old-row compatibility, and dashboard visibility. +- Full `npm test` passes. +- Dashboard bundle build passes. +- The result is deployed through repo workflow. +- Live post-deploy evidence shows table sizes stay bounded while quote ingest and responding continue. + +## Validation evidence required +- Unit tests for identifying successful quote ids to preserve. +- Unit tests for rollup grouping by pair, direction, request kind, result code, failure category, quote-age bucket, notional bucket, and outcome status. +- Tests proving rollups are written before detail pruning. +- Tests proving completed successful lifecycle rows survive pruning. +- Tests proving non-success detail older than the configured window is pruned. +- Tests proving missing/invalid retention policy fails closed. +- Tests proving pressure mode cannot delete completed evidence. +- Dashboard/static tests for storage and retention surfaces. +- Static deployment tests proving no retention/pair/strategy env vars are added. +- Full `npm test`. +- Dashboard bundle build. +- Live evidence after deployment: + - table sizes + - latest prune events + - latest rollup timestamps + - successful trade row count + - recent quote flow still writing rows + +## Failure conditions +- The DB can still fill from non-success quote lifecycle rows under normal live quote volume. +- Rollups are missing or stale while detail is pruned. +- Completed settlement evidence is deleted. +- Retention behavior depends on env vars or untracked manual SQL. +- The implementation changes maker strategy behavior or skips quotes. +- The dashboard hides storage pressure or makes retained rollups look like per-quote evidence. +- Old rows become unreadable. + +## Current real before this turn +- Maker timing, relay failure classification, competitiveness summaries, and lifecycle dashboards are deployed. +- Emergency cleanup on 2026-06-05 proved the largest growth was normalized quote/decision history, not raw quotes alone. +- A short-term deployed prune now deletes old non-success lifecycle rows and preserves completed quote outcomes. +- Live evidence after cleanup showed quote tables reduced to MB scale while quote flow resumed. +- The current fix is functional but not yet a full retention product: rollups, storage dashboard truth, policy introspection, and pressure guardrails still need to be made first-class. + +## Deliberately not built by this proof +- Recovery of already-pruned historical non-success quote detail. +- Fee-complete realized PnL. +- Venue-native terminal fill ids. +- External archival warehouse. +- Automatic strategy changes based on rollups.