From 0902cb2c93ccf14518bc986d4290d146042f2155 Mon Sep 17 00:00:00 2001 From: philipp Date: Fri, 12 Jun 2026 19:03:13 +0200 Subject: [PATCH] Open implementation turn: Quote lifecycle investigator Proof: Establish the approved quote lifecycle investigator turn so missed and pending quotes can be followed from durable dashboard evidence. Assumptions: The selected I012 traceability scope is the approved next slice, while broader quote analytics remain future backlog work. Still fake: This is a planning commit only; the dashboard investigator, durable lookup endpoint, tests, deployment, and live evidence are not implemented yet. --- ARCHIVE.md | 1 + BACKLOG.md | 2 +- IMPLEMENTATION.md | 191 +++++++++++++++++++++++++++++++++++++++++++++- PROOF.md | 81 +++++++++++++++++++- 4 files changed, 268 insertions(+), 7 deletions(-) diff --git a/ARCHIVE.md b/ARCHIVE.md index 6d60a3e..7a5a721 100644 --- a/ARCHIVE.md +++ b/ARCHIVE.md @@ -44,3 +44,4 @@ Legacy note: - 2026-05-18: opened implementation turn `maker-response-latency-and-quote-competitiveness` from backlog items none. - 2026-06-06: opened implementation turn `quote-lifecycle-retention-and-analytics-rollups` from backlog items none. - 2026-06-12: opened implementation turn `verifier-salt-hot-path-removal-and-executor-queue-guardrails` from backlog items none. +- 2026-06-12: opened implementation turn `quote-lifecycle-investigator-and-missed-quote-follow-up` from backlog items I012; remaining broad analytics-workbench scope was re-added as I017. diff --git a/BACKLOG.md b/BACKLOG.md index 2df334a..d66732b 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -17,10 +17,10 @@ Rules: - [I007] Inventory-aware execution rule: implement both directions, but only fire the side backed by credited internal source-asset inventory. - [I008] Inventory-sync service for NEAR Intents internal balances and pending funding state. - [I009] Liquidity-manager service for deposit addresses, funding actions, and treasury visibility. -- [I012] Quotes and execution analytics workbench: live quote tape, quote count and volume windows, size-bucket distributions, oracle-deviation views, per-quote decision and execution trace, and matched-only filters. - [I013] Benchmark and PnL analytics: compare trade PnL versus mark-to-market, all-BTC hold, all-EURe hold, passive 50/50 hold, and hindsight trade quality against later benchmarks. - [I016] (soon) Treasury cashflow and fee ledger: durably record deposits, withdrawals, bridge or network costs, and other non-trade cashflows so dashboard profit can move from mark-to-market to true net PnL. tags=pnl,fees,treasury +- [I017] (soon) Remaining quote analytics workbench beyond lifecycle investigation: quote count and volume windows, size-bucket distributions, oracle-deviation views, matched-only filters, and broader execution analytics. tags=analytics,dashboard,quotes ## Research Candidates - [R001] Compare Kraken and CoinGecko drift and freshness for the assets needed to price the active pair. - [R002] Test whether the active pair's implied rate diverges from external reference prices enough to justify execution after a simple 2% gross threshold. diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index c6b72a8..2a8b793 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -1,5 +1,190 @@ -# Implementation Turn +# Implementation Turn: Quote lifecycle investigator and missed quote follow-up -Status: idle +Status: open +Opened: 2026-06-12 -No approved implementation turn is active yet. +## Goal +Improve the dashboard Quote lifecycle view so operators can pin, search, and follow missed or pending quotes from durable evidence even after live rows move on. + +## Selected Backlog Items +- [I012] Quotes and execution analytics workbench: live quote tape, quote count and volume windows, size-bucket distributions, oracle-deviation views, per-quote decision and execution trace, and matched-only filters. + +This turn intentionally implements only the per-quote lifecycle investigation and missed-quote follow-up part of I012. Remaining analytics-workbench work has been re-added to backlog as follow-on scope. + +## Constraints +- Preserve trading behavior: no strategy, pricing, threshold, notional, inventory, arming, signer, pair enablement, quote skip, or relay policy changes. +- Do not add active pair, edge, notional, latency, response-policy, or retention env vars. +- Do not introduce a new storage system or retention policy. +- Preserve existing lifecycle and executor result rows as readable. +- Use repo workflow for deployment only; no manual kubectl rollout or image patching. +- Add regression tests for every bug fix or behavior contract introduced here. + +## Current Shape To Inspect +- `src/operator-dashboard/static/pages/StrategyPage.jsx` + - `QuoteLifecycleSection` + - `QuoteLifecycleTable` + - `LifecycleDetails` + - pause/filter/expanded-row behavior +- `src/operator-dashboard/static/state/dashboardReducer.js` + - `quote_lifecycle.updated` handling +- `src/core/operator-dashboard.mjs` + - `deriveQuoteLifecycleRows` + - `buildLiveQuoteLifecycleRows` + - `buildStrategySummary` + - lifecycle row normalization/enrichment +- `src/lib/postgres.mjs` + - recent quote, decision, command, execution result, and outcome loaders +- `src/apps/operator-dashboard.mjs` + - bootstrap data loading + - REST API routes + - WebSocket update path +- Existing dashboard/static tests under `test/`. + +## Implementation Plan + +### 1. Add a durable lifecycle lookup contract +- Add a backend loader that can find lifecycle evidence by one identifier: + - `quote_id` + - `decision_id` + - `command_id` +- Reuse existing durable tables and normalizers: + - `swap_demand_events` + - `trade_decisions` + - `execute_trade_commands` + - `trade_execution_results` + - quote outcomes table +- Return the same normalized lifecycle row shape produced by `deriveQuoteLifecycleRows`, enriched for UI in the same way as recent rows. +- Include lookup metadata: + - requested identifier + - matched identifier type + - found/not found + - lookup timestamp + - latest stage timestamp +- If no row is found, return a structured not-found response instead of a dashboard error. +- Make missing old fields render as `null`/`Unavailable`; do not require new timing fields to exist. + +### 2. Expose a dashboard API endpoint +- Add an authenticated operator-dashboard route for lifecycle lookup, for example: + - `GET /api/strategy/quote-lifecycle/:identifier` +- The endpoint should: + - validate a non-empty identifier + - call the durable lookup loader + - return `{ ok, lifecycle_row, lookup }` + - return a 404-style payload for not found without crashing the dashboard +- Do not expose secrets or raw relay payloads beyond what existing lifecycle rows already show. + +### 3. Improve frontend state and interaction model +- Replace the current "pause solves investigation" behavior with a selected quote investigator: + - row action: inspect or pin quote + - selected row highlight + - stable investigator panel above the table or directly below controls + - selected quote remains visible even when filters change or live rows move +- Keep pause as table-only: + - paused table keeps row order and row contents stable + - selected investigator can still refresh from durable lookup + - resume table applies latest rolling rows without clearing the selected quote +- Add identifier lookup: + - search input for quote id, decision id, or command id + - loading, found, not-found, and error states + - copyable identifiers remain available +- Add problem-state filters without changing lifecycle data: + - all + - awaiting executor + - approved/no command + - blocked before submit + - submission failed + - quote not found or finished + - strategy rejected + - submitted/awaiting outcome +- Ensure filters do not hide or clear the selected investigator. + +### 4. Make `Awaiting executor` explain the next step +- In the investigator panel, show a concise stage status: + - quote observed + - strategy decision + - command recorded or missing + - executor result recorded or missing + - relay/outcome status when present +- For `command_emitted` / `Awaiting executor` rows: + - show command age using the current clock + - show the command id and command timestamp + - show "waiting for executor result" only when no result is found + - if a later lookup finds an executor result, show the terminal status instead of stale awaiting text +- For relay failures, make `quote_not_found_or_finished` visible as a first-class reason. + +### 5. Keep live updates and lookup refresh coherent +- When `quote_lifecycle.updated` arrives: + - update the rolling table as it does today + - if the selected quote appears in the live payload, refresh the selected panel from that row immediately + - if the selected quote is absent from the live payload, schedule or expose a durable lookup refresh for the selected identifier +- Avoid unbounded request loops: + - debounce lookup refreshes client-side + - do not add new env vars for refresh timing + - keep manual refresh available +- Show whether selected evidence came from: + - current live rows + - durable lookup + - not found + - lookup error + +### 6. Preserve layout quality +- Keep the lifecycle table full-width and scan-friendly. +- Avoid large marketing-style or decorative redesign. +- Keep row heights stable with existing fixed row slots. +- Ensure long quote ids and reason text truncate or wrap intentionally. +- Do not nest cards inside cards; use a panel/detail layout consistent with the existing dashboard. +- Keep old rows with missing fields readable on desktop and mobile widths. + +### 7. Tests +- Add or update core tests for lifecycle lookup and row derivation: + - command-only row classifies as `Awaiting executor` + - later executor result for the same quote updates the lifecycle state + - old rows missing timing fields normalize safely +- Add backend API tests or static route tests: + - lookup by quote id + - lookup by decision id + - lookup by command id + - not-found response +- Add frontend/static tests proving: + - Quote lifecycle has a selected/pinned investigator + - pause language is table-specific + - identifier lookup controls exist + - problem-state filters include awaiting executor and quote-not-found/finished + - selected investigation is not represented only by expanded table rows +- Add a regression test for the described bug: + - start with a selected `Awaiting executor` quote + - update the displayed recent list so that quote is absent + - prove the selected quote detail remains available or can be refreshed from durable lookup + +### 8. Validation Commands +- Run targeted tests first, likely: + - `node --test test/operator-dashboard*.test.mjs test/*lifecycle*.test.mjs` + - add narrower filenames as tests are introduced +- Run full: + - `npm test` +- Build dashboard bundle: + - `npm run operator-dashboard:build` + +### 9. Commit, deploy, and evidence +- Commit implementation with required body: + - `Proof: ...` + - `Assumptions: ...` + - `Still fake: ...` +- Push only if tests and dashboard build pass. +- Let repo workflow deploy; do not manually patch deployments. +- Collect live evidence after deploy: + - deployments on the new image + - dashboard still receives live lifecycle updates + - durable lookup returns at least one recent quote lifecycle row + - an unresolved or missed quote can be inspected by identifier without relying on it being visible in the rolling table + - selected quote UI does not clear on live row movement + +## Assumptions +- Existing durable quote, decision, command, execution result, and outcome rows are sufficient to reconstruct a single quote lifecycle. +- The dashboard can add a small authenticated read endpoint without changing service topology. +- Client-side debounce plus manual refresh is enough to keep pinned quote investigation fresh without adding a new response-policy or retention setting. + +## Still Fake +- Venue-native terminal fill truth remains incomplete where outcome rows only infer not-filled or settlement through existing attribution. +- Fee-aware realized PnL and broader quote analytics workbench views remain out of scope. +- Historical detail that has already been pruned cannot be reconstructed beyond existing rollups. diff --git a/PROOF.md b/PROOF.md index 71baf39..b5ddb12 100644 --- a/PROOF.md +++ b/PROOF.md @@ -1,5 +1,80 @@ -# Implementation Proof +# Implementation Proof: Quote lifecycle investigator and missed quote follow-up -Status: idle +Status: open +Opened: 2026-06-12 -No approved implementation proof is active yet. +## Hypothesis +If the operator dashboard turns the quote lifecycle table into a durable quote investigator, an operator can pause or filter the live quote list, select an `Awaiting executor` or otherwise missed quote, and continue following that exact quote through command, executor result, relay failure, outcome, or unresolved state even after the rolling live table moves on. + +## Problem Evidence +- The current Quote lifecycle page is a rolling live table of recent rows. +- Pausing freezes the displayed table snapshot, including stale `Awaiting executor` evidence. +- When live updates resume, the quote can leave the recent window and become hard to find again. +- The existing expandable row explains the lifecycle only while the row remains in the rendered recent set. +- This prevents practical investigation of missed quotes such as commands that later become `quote_not_found_or_finished`, `verifier_salt_unavailable`, stale command rejects, or successful submissions. + +## Scope +- Improve the existing `Quote lifecycle` dashboard view, not a new dashboard product area. +- Add a durable quote lookup path by `quote_id`, `decision_id`, or `command_id` using existing history tables. +- Add a selected or pinned quote investigator that remains visible independently from the rolling table rows. +- Make pause freeze table ordering only; the selected quote investigation must still refresh from durable evidence. +- Add search or lookup controls so a quote can be found again after it leaves the recent table. +- Add filters for investigation states that matter to missed quotes: + - awaiting executor + - approved but command missing + - blocked before submit + - submission failed + - quote not found or finished + - strategy rejected + - submitted or awaiting outcome +- Improve the selected quote detail so `Awaiting executor` clearly shows: + - command id and command timestamp + - elapsed time since command + - whether an executor result is now stored + - relay result or block reason if present + - next expected stage when still unresolved +- Preserve old executor result and lifecycle rows as readable when newer timing or result fields are missing. +- Keep the implementation read-side and dashboard-only unless a small shared lifecycle normalizer change is required. + +## Non-goals +- Do not change strategy selection, pricing, edge thresholds, notional limits, inventory checks, arming, signer identity, or pair enablement. +- Do not skip quotes or alter executor/relay response behavior. +- Do not add active pair, edge, notional, latency, response-policy, or retention environment variables. +- Do not add a new retention policy or new long-term analytics store in this turn. +- Do not build the full remaining analytics workbench: quote volume windows, size-bucket distributions, oracle-deviation views, and matched-only PnL analytics remain backlog work. +- Do not claim submitted quote responses are completed trades unless durable outcome and settlement evidence support that claim. +- Do not use manual kubectl rollout, image patching, or ad hoc production repair. + +## Definition of Done +- The Quote lifecycle page has a stable selected or pinned quote investigator separate from the rolling table rows. +- Selecting an `Awaiting executor` row lets the operator continue following that quote after the table is paused, resumed, filtered, or updated. +- A selected quote can be refreshed or auto-refreshed from durable evidence even when it is no longer present in `recent_lifecycle_rows`. +- Lookup by `quote_id`, `decision_id`, or `command_id` returns the same normalized lifecycle shape used by the table, including stage timestamps, reason text, timing, command, execution result, and outcome fields when present. +- Problem-state filters make missed quote rows easy to isolate without hiding the selected quote investigation. +- Rows with missing old fields render as unavailable instead of crashing or disappearing. +- Bug regression coverage proves: + - a selected quote remains inspectable after it leaves the recent live list + - an `Awaiting executor` row can later refresh into a terminal executor or relay result + - lookup by quote, decision, and command identifiers uses durable evidence and does not rely only on the current client-side recent rows +- Targeted dashboard tests pass. +- Full `npm test` passes. +- The operator dashboard bundle builds. +- The planning and implementation commits include `Proof:`, `Assumptions:`, and `Still fake:`. +- If tests pass, deploy only through the repo push workflow and collect live evidence: + - repo-owned deployments are on the new image + - Quote lifecycle still receives live rows + - at least one durable quote lookup returns a lifecycle row + - a missed or pending quote can be inspected by identifier after it is not relied on as a visible recent row + +## Failure Conditions +- Pause still freezes the only available evidence for a selected quote. +- A selected quote cannot be found once it leaves the rolling recent table. +- The lookup path searches only the browser's current rows instead of durable history. +- The UI improves labels but still cannot answer what happened after `Awaiting executor`. +- Missed-quote filters hide the selected investigation or make old rows unreadable. +- The implementation changes trading behavior or strategy economics. +- No regression test covers the row-follow-up bug. + +## Real Versus Fake +- Real in scope: dashboard investigation backed by durable quote, decision, command, executor result, and outcome rows already stored by repo-owned services. +- Still fake or incomplete: venue-native final fill truth, fee-aware realized PnL, broader quote analytics workbench, and historical aggregate drilldowns beyond the existing retained lifecycle detail and rollups.