orderflow_backtest/docs/API.md

API Documentation (Current Implementation)

Overview

This document describes the public interfaces of the current system: SQLite streaming, OHLC/depth aggregation, JSON-based IPC, and the Dash visualizer. Metrics (OBI/CVD), repository/storage layers, and strategy APIs are not part of the current implementation.

Input Database Schema (Required)

book table

CREATE TABLE book (
    id INTEGER PRIMARY KEY,
    instrument TEXT,
    bids TEXT NOT NULL,        -- Python-literal: [[price, size, ...], ...]
    asks TEXT NOT NULL,        -- Python-literal: [[price, size, ...], ...]
    timestamp TEXT NOT NULL
);

trades table

CREATE TABLE trades (
    id INTEGER PRIMARY KEY,
    instrument TEXT,
    trade_id TEXT,
    price REAL NOT NULL,
    size REAL NOT NULL,
    side TEXT NOT NULL,        -- "buy" or "sell"
    timestamp TEXT NOT NULL
);

Data Access: db_interpreter.py

Classes

• OrderbookLevel (dataclass): represents a price level.
• OrderbookUpdate: windowed book update with bids, asks, timestamp, end_timestamp.

DBInterpreter

class DBInterpreter:
    def __init__(self, db_path: Path): ...

    def stream(self) -> Iterator[tuple[OrderbookUpdate, list[tuple]]]:
        """
        Stream orderbook rows with one-row lookahead and trades in timestamp order.
        Yields pairs of (OrderbookUpdate, trades_in_window), where each trade tuple is:
        (id, trade_id, price, size, side, timestamp_ms) and timestamp_ms ∈ [timestamp, end_timestamp).
        """

• Read-only SQLite connection with PRAGMA tuning (immutable, query_only, mmap, cache).
• Batch sizes: BOOK_BATCH = 2048, TRADE_BATCH = 4096.

Processing: ohlc_processor.py

OHLCProcessor

class OHLCProcessor:
    def __init__(self, window_seconds: int = 60, depth_levels_per_side: int = 50): ...

    def process_trades(self, trades: list[tuple]) -> None:
        """Aggregate trades into OHLC bars per window; throttled upserts for UI responsiveness."""

    def update_orderbook(self, ob_update: OrderbookUpdate) -> None:
        """Maintain in-memory price→size maps, apply partial updates, and emit top-N depth snapshots periodically."""

    def finalize(self) -> None:
        """Emit the last OHLC bar if present."""

• Internal helpers for parsing levels from JSON or Python-literal strings and for applying deletions (size==0).

Inter-Process Communication: viz_io.py

Files

• ohlc_data.json: rolling array of OHLC bars (max 1000).
• depth_data.json: latest depth snapshot (bids/asks), top-N per side.
• metrics_data.json: rolling array of OBI OHLC bars (max 1000).

Functions

def add_ohlc_bar(timestamp: int, open_price: float, high_price: float, low_price: float, close_price: float, volume: float = 0.0) -> None: ...

def upsert_ohlc_bar(timestamp: int, open_price: float, high_price: float, low_price: float, close_price: float, volume: float = 0.0) -> None: ...

def clear_data() -> None: ...
 
def add_metric_bar(timestamp: int, obi_open: float, obi_high: float, obi_low: float, obi_close: float) -> None: ...
                   
def upsert_metric_bar(timestamp: int, obi_open: float, obi_high: float, obi_low: float, obi_close: float) -> None: ...
                      
def clear_metrics() -> None: ...

• Atomic writes via temp file replace to prevent partial reads.

Visualization: app.py (Dash)

• Three visuals: OHLC+Volume and Depth (cumulative) with Plotly dark theme, plus an OBI candlestick subplot beneath Volume.
• Polling interval: 500 ms. Tolerates JSON decode races using cached last values.

Callback Contract

@app.callback(
    [Output('ohlc-chart', 'figure'), Output('depth-chart', 'figure')],
    [Input('interval-update', 'n_intervals')]
)

• Reads ohlc_data.json (list of [ts, open, high, low, close, volume]).
• Reads depth_data.json ({"bids": [[price, size], ...], "asks": [[price, size], ...]}).
• Reads metrics_data.json (list of [ts, obi_o, obi_h, obi_l, obi_c]).

CLI Orchestration: main.py

Typer Entry Point

def main(instrument: str, start_date: str, end_date: str, window_seconds: int = 60) -> None:
    """Stream DBs, process OHLC/depth, and launch Dash visualizer in a separate process."""

• Discovers databases under ../data/OKX matching the instrument and date range.
• Launches UI: uv run python app.py.

Usage Examples

Run processing + UI

uv run python main.py BTC-USDT 2025-07-01 2025-08-01 --window-seconds 60
# Open http://localhost:8050

Process trades and update depth in a loop (conceptual)

from db_interpreter import DBInterpreter
from ohlc_processor import OHLCProcessor

processor = OHLCProcessor(window_seconds=60)
for ob_update, trades in DBInterpreter(db_path).stream():
    processor.process_trades(trades)
    processor.update_orderbook(ob_update)
processor.finalize()

Error Handling

• Reader/Writer coordination via atomic JSON prevents partial reads.
• Visualizer caches last valid data if JSON decoding fails mid-write; logs warnings.
• Visualizer start failures do not stop processing; logs error and continues.

Notes

• Metrics computation includes simplified OBI (Order Book Imbalance) calculated as bid_total - ask_total. Repository/storage layers and strategy APIs are intentionally kept minimal.