198 lines
7 KiB
Markdown
198 lines
7 KiB
Markdown
# Minimal product: NEAR Intents demand monitor
|
|
|
|
## Goal
|
|
Build the smallest useful event-driven product for crypto trading research:
|
|
|
|
- read **live user demand** from NEAR Intents
|
|
- publish demand into a **central Kafka/Redpanda-compatible bus**
|
|
- prove downstream consumption with a **dummy reactor**
|
|
- avoid dashboards, execution, wallets, storage, auth workflows beyond the required API key, strategy code, and generic infra beyond the message bus itself
|
|
|
|
## Why this is the right first slice
|
|
From the NEAR Intents docs, there are several possible data surfaces:
|
|
|
|
1. **Message Bus WebSocket `quote` subscription**
|
|
- Endpoint: `wss://solver-relay-v2.chaindefuser.com/ws`
|
|
- Real-time stream for quote requests
|
|
- Subscription request shape:
|
|
```json
|
|
{
|
|
"jsonrpc": "2.0",
|
|
"id": 1,
|
|
"method": "subscribe",
|
|
"params": ["quote"]
|
|
}
|
|
```
|
|
- Expected live frame shape is JSON-RPC-like but should be treated as flexible. The adapter should accept quote payloads when the useful fields appear either:
|
|
- directly under `params`
|
|
- directly under `result`
|
|
- or at the top level of the message body
|
|
- Fields of interest include:
|
|
- `quote_id` (or equivalent request identifier)
|
|
- `defuse_asset_identifier_in`
|
|
- `defuse_asset_identifier_out`
|
|
- `exact_amount_in` or `exact_amount_out`
|
|
- `min_deadline_ms`
|
|
- Subscription acknowledgements may also vary. They may arrive as an `id`-matched JSON-RPC response with a simple `result`, a structured `result`, or other non-quote control frame before the first quote event.
|
|
- This is the closest public signal to **current demand**.
|
|
|
|
2. **Message Bus JSON-RPC `publish_intent` / `get_status`**
|
|
- Endpoint: `https://solver-relay-v2.chaindefuser.com/rpc`
|
|
- Useful for posting intents or checking a known `intent_hash`
|
|
- Not a public firehose of all intents.
|
|
|
|
3. **Explorer API `/api/v0/transactions`**
|
|
- Historical and analytics friendly
|
|
- Requires JWT auth
|
|
- Better for history, not best for a minimal live monitor
|
|
|
|
4. **Verifier contract intent payloads**
|
|
- The on-chain swap expression is usually `token_diff`
|
|
- Important for understanding settlement semantics
|
|
- Not the easiest first live intake path for a lean bus-first system
|
|
|
|
## Product decision
|
|
The minimal product should monitor **WebSocket `quote` events** and route them through a bus-first runtime.
|
|
|
|
### Why
|
|
- closest live signal to user demand
|
|
- directly reflects what users are requesting from solvers
|
|
- enough to answer the first trading question: **what assets are being requested right now?**
|
|
- decouples venue intake from downstream analysis through Kafka-compatible topics
|
|
|
|
### Important implementation note
|
|
Current docs for the market-maker quickstart and live endpoint behavior indicate the Message Bus requires a **partner API key / JWT** in the `Authorization: Bearer ...` header.
|
|
That means the best path is still the quote stream, but live operation is partner-gated.
|
|
|
|
### Important caveat
|
|
A `quote` event is **pre-trade demand**, not guaranteed execution.
|
|
That is fine for v0. The purpose is demand sensing, not settlement accounting.
|
|
|
|
## Runtime shape
|
|
|
|
```text
|
|
NEAR Intents websocket
|
|
|
|
|
v
|
|
src/apps/near-intents-ingest.mjs
|
|
|
|
|
+--> raw.near_intents.quote
|
|
|
|
|
+--> norm.swap_demand
|
|
|
|
|
v
|
|
src/apps/dummy-consumer.mjs
|
|
```
|
|
|
|
### Runtime contracts
|
|
|
|
#### Ingest app
|
|
`src/apps/near-intents-ingest.mjs`:
|
|
- loads env
|
|
- parses optional `--pair 'asset_a->asset_b'`
|
|
- starts the NEAR Intents websocket adapter
|
|
- writes raw and normalized events to the configured broker
|
|
|
|
#### Dummy consumer
|
|
`src/apps/dummy-consumer.mjs`:
|
|
- subscribes to `norm.swap_demand`
|
|
- logs observed pair and quote id
|
|
- exists only to prove a downstream consumer contract
|
|
|
|
#### Bus config
|
|
Default env-driven topics and group ids:
|
|
- `KAFKA_TOPIC_RAW_NEAR_INTENTS_QUOTE=raw.near_intents.quote`
|
|
- `KAFKA_TOPIC_NORM_SWAP_DEMAND=norm.swap_demand`
|
|
- `KAFKA_CONSUMER_GROUP_DUMMY=dummy-reactor-v1`
|
|
|
|
Redpanda is a valid runtime target because the transport is Kafka-compatible.
|
|
|
|
## Internal model
|
|
Normalize each quote event into a thin bus envelope:
|
|
|
|
Top-level envelope fields:
|
|
- `venue`
|
|
- `source`
|
|
- `type`
|
|
- `eventId`
|
|
- `occurredAt`
|
|
- `ingestedAt`
|
|
- `assetIn`
|
|
- `assetOut`
|
|
- `raw`
|
|
- `quote`
|
|
|
|
Nested `quote` fields:
|
|
- `quoteId`
|
|
- `assetIn`
|
|
- `assetOut`
|
|
- `amountIn`
|
|
- `amountOut`
|
|
- `ttlMs`
|
|
|
|
Field extraction must remain tolerant to known upstream aliases, and normalization should continue to operate on the merged `metadata + data` payload shape from the Message Bus event.
|
|
The live adapter now intentionally accepts quote-like payloads from `params`, `result`, or the top-level message body, but only processes frames that actually look like quote data. Subscription acknowledgements and unrelated control frames should still be ignored.
|
|
|
|
## Filtering
|
|
The ingest runtime supports an optional exact-pair filter:
|
|
|
|
```bash
|
|
npm run near-intents:ingest -- --pair 'asset_a->asset_b'
|
|
```
|
|
|
|
The filter is direction-agnostic, so the reversed asset order is also accepted.
|
|
|
|
## Scope boundaries
|
|
### Must do
|
|
- connect to the websocket
|
|
- subscribe to `quote` and tolerate control frames
|
|
- normalize quote events into one compact model
|
|
- publish raw and normalized events to Kafka/Redpanda-compatible topics
|
|
- allow a downstream consumer to react to normalized events
|
|
- reconnect automatically on disconnect
|
|
- document `npm` and `node` entrypoints
|
|
|
|
### Must not do
|
|
- Python packaging or CLI guidance
|
|
- TUI-specific product requirements
|
|
- charts
|
|
- account details
|
|
- pnl
|
|
- routing internals
|
|
- market making controls
|
|
- execution buttons
|
|
- config panels
|
|
- speculative infra beyond the current bus and dummy consumer
|
|
|
|
## Path to success
|
|
1. Connect to WebSocket
|
|
2. Subscribe to `quote`
|
|
3. Normalize incoming events into one compact model
|
|
4. Publish raw envelopes to `raw.near_intents.quote`
|
|
5. Publish normalized envelopes to `norm.swap_demand`
|
|
6. Start a dummy consumer on the normalized topic
|
|
7. Reconnect automatically on disconnect
|
|
8. Only after this works, consider:
|
|
- `quote_status`-specific downstream handling
|
|
- historical replay via Explorer API
|
|
- token metadata enrichment
|
|
- filtering and alerts beyond `--pair`
|
|
|
|
## Packaging alignment
|
|
Current repository packaging and usage should stay aligned around the JavaScript runtime entrypoints:
|
|
|
|
- package scripts:
|
|
- `npm run near-intents:ingest`
|
|
- `npm run dummy-consumer`
|
|
- `npm start` as a compatibility wrapper
|
|
- direct app entrypoints:
|
|
- `node src/apps/near-intents-ingest.mjs`
|
|
- `node src/apps/dummy-consumer.mjs`
|
|
|
|
Documentation should treat the npm scripts and `src/apps/*` node entrypoints as canonical. Older single-file and Python/TUI instructions should remain removed to avoid runtime confusion.
|
|
|
|
## Sources
|
|
- NEAR Intents Message Bus WebSocket docs: `subscribe` with `quote` / `quote_status`
|
|
- NEAR Intents Message Bus RPC docs: `quote`, `publish_intent`, `get_status`
|
|
- Verifier contract docs: `token_diff` intent type
|
|
- Explorer API OpenAPI: authenticated historical transactions
|