orderbooks/docs/METHODOLOGY.md
philipp 284e465588
Some checks failed
deploy / deploy (push) Has been cancelled
Prepare Kubernetes orderbooks deployment
2026-04-18 11:23:28 +02:00

3.8 KiB

Methodology

This project uses checkpoint-driven compound engineering. The point is to preserve useful data and operational learning, not to accumulate scaffolding.

Checkpoint Cycle

Every checkpoint follows the same loop:

  1. Define the smallest useful checkpoint.
  2. Build only what is required for that checkpoint.
  3. Validate with real commands and real data when applicable.
  4. Write durable artifacts: code or docs, config or run instructions, manifest/report, and validation evidence.
  5. State PASS, FAIL, or BLOCKED.
  6. Identify the strongest fake-progress risk.
  7. Recommend the next smallest step.
  8. Stop and ask only when a real decision is needed.

Gate States

  • PASS: the checkpoint pass condition is met and evidence is on disk.
  • FAIL: the checkpoint was attempted but did not meet its pass condition.
  • BLOCKED: work cannot continue without a decision, credential, service, or unavailable dependency.
  • PARTIAL: useful artifacts exist, but the checkpoint should not be treated as passed.

Evidence Rules

  • Evidence must be reproducible from files and commands, not just chat.
  • If a command was used to validate behavior, record the command and summarize the result in a report or manifest.
  • If data was collected, preserve raw data and include checksums.
  • If synthetic or sample data is used, label it explicitly.
  • If a claim depends on a public endpoint, record the endpoint, request parameters, response fields, status codes, timestamps, and fetch time.
  • Do not claim reliability from a short sample run. Reliability requires the roadmap soak test.

Machine-Readable Manifest Format

Checkpoint manifests should be JSON and stored under data/manifests/. Use this shape unless a later checkpoint documents a better schema:

{
  "checkpoint_id": 1,
  "checkpoint_name": "Project Scaffold And Methodology",
  "status": "PASS",
  "started_at_utc": "2026-04-14T20:53:49Z",
  "ended_at_utc": "2026-04-14T20:53:49Z",
  "scope": "Durable project rules and roadmap only; no collector implementation.",
  "artifacts": [
    {
      "path": "AGENTS.md",
      "kind": "project_rules",
      "status": "valid"
    }
  ],
  "validation": {
    "commands": [
      {
        "command": "git status --short",
        "result": "completed"
      }
    ],
    "summary": "Required files exist and contain checkpoint rules."
  },
  "decisions": [],
  "assumptions": [],
  "fake_progress_risk": "Most progress is documentation until public Polymarket endpoint behavior is proven.",
  "next_step": "Run Checkpoint 2 public source probe."
}

Markdown Checkpoint Report Format

Checkpoint reports should be stored under reports/checkpoints/ and include:

  • active checkpoint
  • scope
  • files created or changed
  • validation commands and results
  • project rules or operational lessons added
  • pass/fail/gate
  • strongest fake-progress risk
  • next smallest step

Deprecated Or Misleading Artifacts

Do not delete mistakes. Preserve the original artifact and label it.

Preferred labels:

  • Add a manifest entry with status: "deprecated" or status: "invalid".
  • Add a sibling note named <artifact>.deprecated.md or <artifact>.invalid.md when a human explanation is useful.
  • Include why the artifact is wrong, when it was labeled, who labeled it, and what replaces it.

If an artifact is dangerous because it contains secrets, stop and ask the user. Do not spread or copy the secret into reports.

Anti-Fake-Progress Rules

  • No dashboard before 24h data reliability.
  • No database before plain compressed files become painful.
  • No strategy, backtest, optimizer, or trading bot code.
  • No private-key or signing code.
  • No generic multi-market abstraction before a second market exists.
  • No "production-ready" claim before a 24h soak test.
  • No endpoint assumptions without probe evidence.
  • No normalized dataset that cannot trace back to raw records.