philipp/unrip
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix: memory leak in history-writer loading massive payload
All checks were successful
deploy / deploy (push) Successful in 42s
Details

Browse source 
Proof:
History writer was crashing with OOM because the payload of intent_inventory_snapshots is massive and loading 5000 of them into memory takes 200MB. Fixed by modifying the query to only select the spendable and synced_at fields.

Assumptions:
We only need spendable and synced_at fields from inventory payloads.

Still fake:
N/A
This commit is contained in:
philipp 2026-06-16 16:06:04 +02:00
parent f655f1a63a
commit 96a047d676
1 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
src/lib/postgres.mjs
View file

@ -4514,7 +4514,7 @@ export async function refreshQuoteOutcomes(pool, {
ORDER BY COALESCE(observed_at, ingested_at) ASC ORDER BY COALESCE(observed_at, ingested_at) ASC
`, [quoteIds]), `, [quoteIds]),
pool.query(` pool.query(`
SELECT event_id, observed_at, ingested_at, quote_id, payload SELECT event_id, observed_at, ingested_at, quote_id, jsonb_build_object('spendable', payload->'spendable', 'synced_at', payload->'synced_at') AS payload
FROM ( FROM (
SELECT event_id, observed_at, ingested_at, quote_id, payload SELECT event_id, observed_at, ingested_at, quote_id, payload
FROM intent_inventory_snapshots FROM intent_inventory_snapshots
@ -5333,13 +5333,13 @@ async function loadIntentInventorySnapshotsForRequestRefresh(pool, {
return pool.query( return pool.query(
` `
WITH previous_snapshot AS ( WITH previous_snapshot AS (
SELECT event_id, observed_at, ingested_at, quote_id, payload, COALESCE(observed_at, ingested_at) AS sort_at SELECT event_id, observed_at, ingested_at, quote_id, jsonb_build_object('spendable', payload->'spendable', 'synced_at', payload->'synced_at') AS payload, COALESCE(observed_at, ingested_at) AS sort_at
FROM intent_inventory_snapshots FROM intent_inventory_snapshots
WHERE COALESCE(observed_at, ingested_at) < $1 WHERE COALESCE(observed_at, ingested_at) < $1
ORDER BY COALESCE(observed_at, ingested_at) DESC ORDER BY COALESCE(observed_at, ingested_at) DESC
LIMIT 1 LIMIT 1
), following_snapshots AS ( ), following_snapshots AS (
SELECT event_id, observed_at, ingested_at, quote_id, payload, COALESCE(observed_at, ingested_at) AS sort_at SELECT event_id, observed_at, ingested_at, quote_id, jsonb_build_object('spendable', payload->'spendable', 'synced_at', payload->'synced_at') AS payload, COALESCE(observed_at, ingested_at) AS sort_at
FROM intent_inventory_snapshots FROM intent_inventory_snapshots
WHERE COALESCE(observed_at, ingested_at) >= $1 WHERE COALESCE(observed_at, ingested_at) >= $1
ORDER BY COALESCE(observed_at, ingested_at) ASC ORDER BY COALESCE(observed_at, ingested_at) ASC