All checks were successful
deploy / deploy (push) Successful in 48s
Instead of fetching up to 50k inventory snapshots (~442MB), scope the query to only the time range of the current submission batch with a 15-minute buffer. For a typical 1-hour batch this drops from 50k rows to ~300 rows, well within the 1280Mi pod memory limit. The coalesced_at_idx on intent_inventory_snapshots covers the BETWEEN clause so this remains efficient. Proof: history-writer OOM kills from refreshQuoteOutcomes inventory fetch Assumptions: 15min buffer covers the attribution window for all submissions Still fake: heuristic gap outcomes may attribute trades imprecisely
334 lines
14 KiB
Markdown
334 lines
14 KiB
Markdown
# Implementation Turn: CoW Protocol intent-based venue integration
|
|
|
|
Status: open
|
|
Opened: 2026-04-07
|
|
|
|
## Goal
|
|
Integrate CoW Protocol as a second execution venue in a way that makes the architecture more real, not more decorative.
|
|
|
|
The system must be able to request a CoW quote, build the exact order amounts CoW expects, sign the order with the correct chain-specific domain, submit it to the CoW orderbook, durably track later order states, and surface that truth to operators without inventing a parallel data model or widening live-funds risk by accident.
|
|
|
|
## Selected backlog items
|
|
- [I019] CoW Protocol intent-based venue integration: quote, sign, submit, and durably track CoW orders through the shared Kafka and PostgreSQL truth pipeline without widening live-funds risk beyond explicit approval.
|
|
|
|
## Design rules
|
|
- Keep Kafka as the backbone and PostgreSQL as the durable store.
|
|
- Treat CoW as a venue adapter behind the existing architecture, not as a separate mini-application ecosystem.
|
|
- Do not silently change the already-live NEAR spendable inventory truth.
|
|
- Prefer one real chain and one real pair over scaffolding for many chains or advanced CoW features.
|
|
- Do not build solver, autopilot, conditional-order, hook, or CoW AMM support in this turn.
|
|
|
|
## Assumptions to lock before coding
|
|
- Initial chain: Gnosis Chain.
|
|
- Initial signing path: EIP-712 with an EOA signer. If the user later wants Safe or another smart-contract wallet, that is a separate signing-path expansion.
|
|
- Initial pair: `EURe/WBTC` on Gnosis.
|
|
- Initial token addresses:
|
|
- `EURe`: `0x420CA0f9B9b604cE0fd9C18EF134C705e5Fa3430`
|
|
- `WBTC`: `0x8e5bBbb09Ed1ebdE8674Cda39A0c169401db4252`
|
|
- Planning validation already confirmed that CoW's live Gnosis quote API returns a valid `EURe -> WBTC` quote for a sample sell order.
|
|
- Full passed-proof validation requires an explicitly approved isolated CoW trader identity with funded and approved ERC-20 inventory. Without it, coding can proceed, but the turn cannot claim a passed live-trading proof.
|
|
|
|
## Minimal service changes for this turn
|
|
- `trade-executor` must be refactored from a NEAR-only executor into an executor core with venue adapters.
|
|
- `history-writer` and the PostgreSQL layer must learn the new CoW raw quote and order-lifecycle records.
|
|
- `operator-dashboard` must expose CoW venue state, open orders, and last outcomes from the backend.
|
|
- `strategy-engine` should not be turned into a general multi-venue router in this turn. If a command-source tweak is needed to target CoW, keep it narrow and explicit.
|
|
- `inventory-sync` and `liquidity-manager` must not absorb CoW treasury or approval flows in this turn. At most they may expose read-only isolated CoW wallet facts if needed for operator clarity.
|
|
|
|
## Why CoW changes the execution model
|
|
The current execution path is close to:
|
|
- decision
|
|
- command
|
|
- immediate result
|
|
|
|
CoW requires:
|
|
- quote request
|
|
- fee-aware order construction
|
|
- signature generation
|
|
- order submission returning an order UID
|
|
- later monitoring until the order becomes `Filled`, `Partially-filled`, `Expired`, or `Cancelled`
|
|
|
|
The first coding objective is therefore not a new screen or a new script. It is upgrading the execution model so a stateful orderbook venue fits the shared truth pipeline.
|
|
|
|
## Architecture changes
|
|
|
|
### 1. Add a CoW venue package
|
|
Create a new venue package under `src/venues/cow-protocol/` with focused modules:
|
|
- `chains.mjs`
|
|
- chain metadata such as chain ID, orderbook base URL, settlement contract, and human label
|
|
- `client.mjs`
|
|
- thin REST client for `POST /api/v1/quote`, `POST /api/v1/orders`, `GET /api/v1/orders/{uid}`, `GET /api/v1/trades`, and `DELETE /api/v1/orders/{uid}`
|
|
- `amounts.mjs`
|
|
- quote-to-order amount construction using the documented CoW fee semantics
|
|
- explicitly encode the difference between sell and buy order handling
|
|
- `signing.mjs`
|
|
- CoW order digest creation and EIP-712 signing
|
|
- use the documented `Gnosis Protocol` `v2` domain shape and chain-specific settlement contract
|
|
- `normalization.mjs`
|
|
- map CoW quote, order, trade, and cancellation responses into repo-owned event shapes
|
|
- `errors.mjs`
|
|
- normalize HTTP, validation, and remote API failures into durable reason codes
|
|
|
|
Prefer the official CoW SDK or contracts helpers for digest and order-signing math where that reduces risk, but keep the repo-owned normalization and persistence boundaries explicit.
|
|
|
|
### 2. Extend config for CoW
|
|
Add explicit config entries in `src/lib/config.mjs` and manifests for:
|
|
- CoW enabled flag
|
|
- CoW chain name and chain ID
|
|
- CoW orderbook base URL
|
|
- CoW settlement contract address for the selected chain
|
|
- CoW trader address
|
|
- CoW signer private key or signing-provider hook
|
|
- CoW preferred sell and buy token addresses and decimals for the approved pair
|
|
- CoW order validity default
|
|
- CoW monitoring interval for open orders
|
|
- CoW API key slot if partner-authenticated access is later used
|
|
|
|
Do not store secrets in repo. Only add config plumbing.
|
|
|
|
### 3. Refactor `trade-executor` into executor core plus adapters
|
|
The executor should become:
|
|
- a venue-agnostic command consumer
|
|
- a shared idempotency and state owner
|
|
- a venue adapter dispatcher
|
|
|
|
Adapter plan:
|
|
- extract current NEAR execution code into a `near-intents` adapter without changing behavior
|
|
- add a `cow-protocol` adapter implementing:
|
|
- command validation
|
|
- balance and allowance preflight checks
|
|
- CoW quote request
|
|
- amount construction from the quote
|
|
- order digest generation
|
|
- signature creation
|
|
- order submission
|
|
- submission-time event emission
|
|
- open-order monitoring and later state updates
|
|
- cancellation support if enabled
|
|
|
|
The executor core should no longer assume that an adapter returns only one terminal result.
|
|
|
|
### 4. Introduce canonical order-lifecycle events
|
|
The current `exec.trade_result` topic is not enough for CoW.
|
|
|
|
Add one new canonical topic for non-terminal and terminal order-state updates:
|
|
- `exec.order_update`
|
|
|
|
This topic should carry at least:
|
|
- `command_id`
|
|
- `venue`
|
|
- `venue_order_id`
|
|
- `status`
|
|
- `observed_at`
|
|
- `filled_sell_amount`
|
|
- `filled_buy_amount`
|
|
- `settlement_tx_hash`
|
|
- `trade_ids`
|
|
- `reason_code`
|
|
- `reason_text`
|
|
- `is_terminal`
|
|
|
|
Keep `exec.trade_result` as the terminal summary topic for compatibility, but make it derived from the richer order-update model where needed.
|
|
|
|
Near Intents may continue to emit one immediate terminal update through this new topic as a trivial case. CoW will use multiple updates over time.
|
|
|
|
### 5. Persist CoW raw and normalized truth in PostgreSQL
|
|
Add new durable tables and query helpers in `src/lib/postgres.mjs` and `src/core/history-records.mjs`.
|
|
|
|
Required new durable stores:
|
|
- `raw_cow_quotes`
|
|
- quote request inputs and raw response payload
|
|
- `trade_order_events`
|
|
- canonical order lifecycle records across venues
|
|
- `cow_orderbook_orders`
|
|
- latest known CoW order snapshot keyed by order UID
|
|
- `cow_orderbook_trades`
|
|
- trade records returned by CoW trade history when available
|
|
|
|
Required linkage:
|
|
- `command_id` must join submission, order UID, later state updates, and terminal result
|
|
- order UID must join order snapshot and trade records
|
|
- every later state update must be replayable from PostgreSQL without requiring in-memory executor state
|
|
|
|
### 6. Upgrade history writing and replay
|
|
Extend `history-writer` routing so the new CoW topics and records are first-class:
|
|
- `raw.cow.quote` -> `raw_cow_quotes`
|
|
- `exec.order_update` -> `trade_order_events`
|
|
- optional CoW-specific normalized topics if needed for operator surfaces
|
|
|
|
Replay requirement:
|
|
- on restart, the CoW monitor must be able to recover open orders from durable state and resume monitoring
|
|
- local state files may exist for convenience, but PostgreSQL must remain the durable source of truth
|
|
|
|
### 7. Add a CoW open-order monitor
|
|
CoW order submission is not enough. The repo needs a venue monitor loop.
|
|
|
|
Implementation shape:
|
|
- `trade-executor` maintains a small set of currently open CoW orders
|
|
- on startup, it reloads open CoW orders from PostgreSQL
|
|
- on each monitor tick, it fetches latest order state from `GET /api/v1/orders/{uid}`
|
|
- if the order is partially or fully filled, also fetch trade details from `GET /api/v1/trades`
|
|
- it emits `exec.order_update` for every meaningful state transition
|
|
- it emits a terminal `exec.trade_result` when the order reaches a terminal state
|
|
|
|
Operator rules:
|
|
- open-order age and last refresh time must be visible
|
|
- monitoring failures must produce durable alerts or explicit venue-state errors
|
|
|
|
### 8. Handle CoW-specific preflight checks
|
|
Before submitting a CoW order, the adapter must validate:
|
|
- configured chain and orderbook URL are consistent
|
|
- signer address matches configured trader address
|
|
- token addresses and decimals match the selected pair config
|
|
- wallet balance is sufficient for the order
|
|
- allowance is sufficient for the settlement contract or relayer path expected by CoW
|
|
- order validity window is sane
|
|
|
|
Failures here should become structured submission rejections, not opaque thrown errors.
|
|
|
|
### 9. Keep inventory truth separate
|
|
This turn must not mutate the current shared spendable inventory model for the NEAR live loop.
|
|
|
|
Implementation rules:
|
|
- CoW balances are venue-local facts, not part of current global spendable truth by default
|
|
- do not merge CoW balances into the funded BTC/EURe dashboard PnL cards
|
|
- if the dashboard shows CoW wallet balances, label them as isolated venue inventory
|
|
- do not treat pre-credit or observed cross-chain funds as CoW-spendable
|
|
- if a later turn wants unified cross-venue treasury truth, that is separate work
|
|
|
|
### 10. Command-source strategy for this turn
|
|
Do not expand `strategy-engine` into a venue arbiter.
|
|
|
|
Use one of these narrow command sources:
|
|
- a controlled manual command injection path for validation
|
|
- a small venue-targeted control route on the executor
|
|
- a minimal strategy extension that emits `venue = cow-protocol` only for the approved validation pair
|
|
|
|
Whichever path is chosen, keep it explicit and reversible. The point of this turn is the venue plumbing and lifecycle truth, not smart order routing.
|
|
|
|
### 11. Dashboard and operator surfaces
|
|
Use the existing authenticated dashboard backend and WebSocket surface.
|
|
|
|
Add operator visibility for:
|
|
- CoW venue configuration summary
|
|
- current CoW trader address and chain
|
|
- last successful quote time
|
|
- last submitted order UID
|
|
- open order count
|
|
- per-order status with timestamps
|
|
- last terminal outcome and reason text
|
|
- last settlement transaction hash when available
|
|
- monitoring freshness and last error
|
|
|
|
Safe controls that may be real in this turn:
|
|
- refresh CoW venue state
|
|
- pause or resume CoW monitoring
|
|
- request cancellation for an open CoW order
|
|
|
|
Controls that must stay absent without explicit approval:
|
|
- fund CoW wallet
|
|
- approve token spend
|
|
- bridge assets to CoW chain
|
|
- any generic "trade now" button in the browser
|
|
|
|
### 12. Alerting and health
|
|
Extend alerting only where needed:
|
|
- stale CoW order monitoring
|
|
- repeated quote failure
|
|
- repeated submission failure
|
|
- open order older than expected validity window
|
|
- cancellation requested but order still open after grace period
|
|
|
|
Do not build a new alert subsystem. Feed venue issues into the existing alert path.
|
|
|
|
## Concrete implementation order
|
|
|
|
### Phase 1. Lock the venue contract
|
|
- confirm selected chain and pair
|
|
- confirm CoW quoteability for the pair at the intended notional
|
|
- confirm settlement contract and orderbook base URL
|
|
- confirm signing scheme and wallet model
|
|
|
|
### Phase 2. Add CoW libraries and tests
|
|
- implement amount construction from quotes
|
|
- implement order digest and signature helpers
|
|
- add unit tests from documented sell and buy fee semantics
|
|
- add unit tests for domain separator selection and digest generation
|
|
|
|
### Phase 3. Refactor executor core
|
|
- extract current NEAR execution into an adapter
|
|
- add adapter dispatch by `venue`
|
|
- add canonical `exec.order_update` event handling
|
|
- keep existing NEAR behavior passing
|
|
|
|
### Phase 4. Add CoW submission path
|
|
- quote
|
|
- preflight
|
|
- sign
|
|
- submit
|
|
- persist raw quote and submission acknowledgement
|
|
- emit order update and terminal failure events as needed
|
|
|
|
### Phase 5. Add durable monitoring
|
|
- persist open CoW orders
|
|
- restore them on restart
|
|
- poll order and trade endpoints
|
|
- emit durable state transitions and terminal results
|
|
|
|
### Phase 6. Add operator visibility
|
|
- extend dashboard backend aggregation
|
|
- extend UI pages only enough to expose CoW order state truthfully
|
|
- no fake controls
|
|
|
|
### Phase 7. Run live validation
|
|
- request a real quote
|
|
- if approved wallet is available, submit one real order
|
|
- observe at least one later status change
|
|
- persist evidence in PostgreSQL
|
|
- verify dashboard visibility
|
|
|
|
## Test plan
|
|
- unit tests for quote amount transformation for sell orders
|
|
- unit tests for quote amount transformation for buy orders if buy orders are supported in this turn
|
|
- unit tests for EIP-712 domain generation by chain
|
|
- unit tests for order digest generation against known CoW examples or local fixtures
|
|
- unit tests for lifecycle reducer:
|
|
- submitted -> open
|
|
- open -> partially-filled
|
|
- open -> filled
|
|
- open -> expired
|
|
- open -> cancelled
|
|
- integration tests with a mocked CoW orderbook server for quote, order submission, order fetch, trade fetch, and cancellation
|
|
- PostgreSQL persistence tests for new tables and history routing
|
|
- regression tests proving the existing NEAR executor still emits correct results after adapter extraction
|
|
|
|
No CoW bug fix is complete without a regression test covering the relevant fee, digest, lifecycle, or persistence case.
|
|
|
|
## Validation checklist against the proof
|
|
- real quote acquired and stored
|
|
- correct amounts signed from CoW quote semantics
|
|
- correct domain and signature path used
|
|
- order UID returned and linked to command ID
|
|
- later order state observed and stored
|
|
- operator surface shows venue state and order outcome
|
|
- NEAR live path remains unchanged in spendable truth and arm safety
|
|
|
|
## Failure modes to plan for
|
|
- unsupported token or `NoLiquidity` from CoW quote
|
|
- bad token or unsupported token rejection
|
|
- wrong chain ID or settlement contract causing invalid signatures
|
|
- insufficient allowance or balance
|
|
- order accepted but never transitions before expiry
|
|
- partial fill without terminal fill in the same process lifetime
|
|
- duplicate monitoring after restart
|
|
- cancellation race where an order fills while cancellation is in flight
|
|
- orderbook API timeout or rate limiting
|
|
|
|
Each of these needs an explicit durable reason code or operator-visible error path.
|
|
|
|
## What remains deliberately out of scope after this turn
|
|
- unified cross-venue treasury truth
|
|
- fee-complete realized PnL for CoW fills
|
|
- automatic venue selection between NEAR Intents and CoW
|
|
- Safe or generic `ERC-1271` support
|
|
- advanced CoW order types
|