unrip/IMPLEMENTATION.md

Implementation Turn: Persisted quote statistics rollups and live counters

Status: open Opened: 2026-06-12

Goal

Persist durable quote lifecycle statistics for 5-minute, hourly, daily, weekly, monthly, and all-time windows so operators can see unique quote totals and outcome-state counts from the same history used by live lifecycle evidence.

Selected backlog items

• [I017] Persisted quote-state rollups only. The broader quote analytics workbench scope from the original backlog item has been re-added to backlog for a later turn.

Hard Constraints

• Do not change strategy selection, pricing, edge thresholds, notional limits, inventory checks, arming, signer identity, pair enablement, executor behavior, relay behavior, or quote skipping behavior.
• Do not add active pair, edge, notional, latency, response-policy, or retention environment variables.
• Do not add a storage system or retention policy. Use existing PostgreSQL history and repo-owned workflow.
• Preserve old lifecycle and executor-result rows as readable; missing fields become unavailable.
• Bug fixes require regression tests.
• No manual kubectl rollout, image patching, or cluster repair. Deploy only through the repo workflow.
• Do not stage or revert unrelated dirty worktree files.

Implementation Plan

1. Inspect Current Paths

• Read the current quote lifecycle row derivation, durable lookup, reducer, dashboard routes, static dashboard components, existing rollup or retention loaders, migrations/schema setup, and related tests.
• Confirm whether an existing quote lifecycle rollup table can store the requested state buckets. Prefer extending existing rollup machinery when it is compatible; otherwise add one narrow PostgreSQL aggregate table dedicated to quote lifecycle statistics.

2. Classification Contract

• Build a reusable lifecycle statistics classifier on normalized durable lifecycle rows.
• Assign each quote to one latest state bucket per recompute:
  • success: durable completed/success result or settlement evidence exists
  • not_filled: durable terminal not-filled or expired outcome exists
  • submission_failed: executor or relay submission failed
  • submitted_no_reply: command was submitted or relay-accepted, but no durable terminal reply is known
  • awaiting_executor: strategy approved and command/executor evidence is still missing
  • blocked_before_submit: executor or safety gate blocked before venue submission
  • rejected_by_strategy: strategy rejected the quote before execution
  • unavailable: old, partial, or malformed evidence cannot be classified safely
• Terminal states must take precedence over intermediate states when multiple history rows exist for the same quote.
• Count distinct quote_id values. Track missing quote identifiers separately as unavailable evidence; do not fabricate identifiers.
• Use the quote's first durable lifecycle timestamp as its window timestamp. If the first timestamp is missing, use the earliest available durable stage timestamp and mark timestamp provenance as unavailable in testable code.

3. Persistence

• Add or extend a PostgreSQL-backed persisted rollup with:
  • grain: all_time, month, week, day, hour, five_minute
  • UTC window_start and window_end for windowed grains
  • unique quote total
  • missing identifier count
  • state bucket counts
  • computed timestamp
• Keep the storage idempotent with a deterministic key per grain/window and upsert semantics.
• Recompute from existing durable history rather than appending event deltas that can drift.
• Wire recomputation into the existing lifecycle history or dashboard maintenance path so live systems persist updated stats without requiring manual scripts.
• Do not introduce new retention knobs or background services unless an existing repo-owned process already performs the relevant maintenance.

4. Dashboard API

• Add an authenticated operator-dashboard endpoint for quote lifecycle statistics.
• Return all-time totals plus recent windows for the requested grains. Clamp limits server-side and preserve missing or unavailable fields in the response.
• Include enough metadata for the UI to label UTC windows and live/update freshness without implying unavailable truth is known.

5. Dashboard UI

• Add a statistics surface to the Quote lifecycle page or its existing operator context.
• Provide controls for 5m, hour, day, week, month, and all-time.
• Display unique quote totals and state bucket counts for the selected grain.
• Show live counters for current windows and all-time totals that update from the live dashboard stream.
• Pausing the recent lifecycle table must not freeze the statistics surface or live counters.
• Use existing component, style, and reducer patterns; keep controls dense and operational rather than marketing-style.

6. Tests

• Add focused classifier/window tests for state precedence, UTC boundaries, Monday UTC weeks, missing IDs, missing timestamps, and old-field compatibility.
• Add persistence tests proving idempotent upsert/recompute and all-time totals.
• Add route tests proving authentication and persisted response shape.
• Add reducer/UI tests proving live counters update while the recent table is paused.
• Add regression coverage for a quote that begins as submitted_no_reply or awaiting_executor, leaves the live recent table, and later refreshes into submission_failed, not_filled, or success through durable history.

7. Validation And Deployment

• Run targeted tests covering the changed backend and dashboard surfaces.
• Run full npm test.
• Build the dashboard bundle.
• Commit with required Proof:, Assumptions:, and Still fake: body.
• Push only after tests and build pass.
• Let the repo workflow deploy the new image.
• Collect live evidence:
  • deployed image digest or tag includes this commit
  • lifecycle live rows still flow
  • statistics endpoint returns real persisted rows
  • live counters move without requiring the quote to remain visible in the rolling table

Assumptions To Verify While Coding

• Existing lifecycle history has a stable enough quote_id for unique quote counting. Rows without one are counted as unavailable evidence, not invented unique quotes.
• Existing durable history has enough timestamps to assign quotes to UTC windows. Missing timestamps remain unavailable rather than blocking all statistics.
• Current lifecycle lookup and row derivation already encode most state distinctions; the statistics classifier should reuse that truth instead of creating a competing interpretation.

Still Fake Or Deferred

• Fee-aware PnL, treasury cashflow, benchmark markout, oracle-deviation, size distribution, and matched-only analytics are not part of this turn.
• Statistics can only be as complete as the already-retained quote lifecycle/history tables. Already-pruned detail cannot be reconstructed.