152 lines
5.3 KiB
Markdown
152 lines
5.3 KiB
Markdown
# API Documentation (Current Implementation)
|
|
|
|
## Overview
|
|
|
|
This document describes the public interfaces of the current system: SQLite streaming, OHLC/depth aggregation, JSON-based IPC, and the Dash visualizer. Metrics (OBI/CVD), repository/storage layers, and strategy APIs are not part of the current implementation.
|
|
|
|
## Input Database Schema (Required)
|
|
|
|
### book table
|
|
```sql
|
|
CREATE TABLE book (
|
|
id INTEGER PRIMARY KEY,
|
|
instrument TEXT,
|
|
bids TEXT NOT NULL, -- Python-literal: [[price, size, ...], ...]
|
|
asks TEXT NOT NULL, -- Python-literal: [[price, size, ...], ...]
|
|
timestamp TEXT NOT NULL
|
|
);
|
|
```
|
|
|
|
### trades table
|
|
```sql
|
|
CREATE TABLE trades (
|
|
id INTEGER PRIMARY KEY,
|
|
instrument TEXT,
|
|
trade_id TEXT,
|
|
price REAL NOT NULL,
|
|
size REAL NOT NULL,
|
|
side TEXT NOT NULL, -- "buy" or "sell"
|
|
timestamp TEXT NOT NULL
|
|
);
|
|
```
|
|
|
|
## Data Access: db_interpreter.py
|
|
|
|
### Classes
|
|
- `OrderbookLevel` (dataclass): represents a price level.
|
|
- `OrderbookUpdate`: windowed book update with `bids`, `asks`, `timestamp`, `end_timestamp`.
|
|
|
|
### DBInterpreter
|
|
```python
|
|
class DBInterpreter:
|
|
def __init__(self, db_path: Path): ...
|
|
|
|
def stream(self) -> Iterator[tuple[OrderbookUpdate, list[tuple]]]:
|
|
"""
|
|
Stream orderbook rows with one-row lookahead and trades in timestamp order.
|
|
Yields pairs of (OrderbookUpdate, trades_in_window), where each trade tuple is:
|
|
(id, trade_id, price, size, side, timestamp_ms) and timestamp_ms ∈ [timestamp, end_timestamp).
|
|
"""
|
|
```
|
|
|
|
- Read-only SQLite connection with PRAGMA tuning (immutable, query_only, mmap, cache).
|
|
- Batch sizes: `BOOK_BATCH = 2048`, `TRADE_BATCH = 4096`.
|
|
|
|
## Processing: ohlc_processor.py
|
|
|
|
### OHLCProcessor
|
|
```python
|
|
class OHLCProcessor:
|
|
def __init__(self, window_seconds: int = 60, depth_levels_per_side: int = 50): ...
|
|
|
|
def process_trades(self, trades: list[tuple]) -> None:
|
|
"""Aggregate trades into OHLC bars per window; throttled upserts for UI responsiveness."""
|
|
|
|
def update_orderbook(self, ob_update: OrderbookUpdate) -> None:
|
|
"""Maintain in-memory price→size maps, apply partial updates, and emit top-N depth snapshots periodically."""
|
|
|
|
def finalize(self) -> None:
|
|
"""Emit the last OHLC bar if present."""
|
|
```
|
|
|
|
- Internal helpers for parsing levels from JSON or Python-literal strings and for applying deletions (size==0).
|
|
|
|
## Inter-Process Communication: viz_io.py
|
|
|
|
### Files
|
|
- `ohlc_data.json`: rolling array of OHLC bars (max 1000).
|
|
- `depth_data.json`: latest depth snapshot (bids/asks), top-N per side.
|
|
- `metrics_data.json`: rolling array of OBI OHLC bars (max 1000).
|
|
|
|
### Functions
|
|
```python
|
|
def add_ohlc_bar(timestamp: int, open_price: float, high_price: float, low_price: float, close_price: float, volume: float = 0.0) -> None: ...
|
|
|
|
def upsert_ohlc_bar(timestamp: int, open_price: float, high_price: float, low_price: float, close_price: float, volume: float = 0.0) -> None: ...
|
|
|
|
def clear_data() -> None: ...
|
|
|
|
def add_metric_bar(timestamp: int, obi_open: float, obi_high: float, obi_low: float, obi_close: float) -> None: ...
|
|
|
|
def upsert_metric_bar(timestamp: int, obi_open: float, obi_high: float, obi_low: float, obi_close: float) -> None: ...
|
|
|
|
def clear_metrics() -> None: ...
|
|
```
|
|
|
|
- Atomic writes via temp file replace to prevent partial reads.
|
|
|
|
## Visualization: app.py (Dash)
|
|
|
|
- Three visuals: OHLC+Volume and Depth (cumulative) with Plotly dark theme, plus an OBI candlestick subplot beneath Volume.
|
|
- Polling interval: 500 ms. Tolerates JSON decode races using cached last values.
|
|
|
|
### Callback Contract
|
|
```python
|
|
@app.callback(
|
|
[Output('ohlc-chart', 'figure'), Output('depth-chart', 'figure')],
|
|
[Input('interval-update', 'n_intervals')]
|
|
)
|
|
```
|
|
- Reads `ohlc_data.json` (list of `[ts, open, high, low, close, volume]`).
|
|
- Reads `depth_data.json` (`{"bids": [[price, size], ...], "asks": [[price, size], ...]}`).
|
|
- Reads `metrics_data.json` (list of `[ts, obi_o, obi_h, obi_l, obi_c]`).
|
|
|
|
## CLI Orchestration: main.py
|
|
|
|
### Typer Entry Point
|
|
```python
|
|
def main(instrument: str, start_date: str, end_date: str, window_seconds: int = 60) -> None:
|
|
"""Stream DBs, process OHLC/depth, and launch Dash visualizer in a separate process."""
|
|
```
|
|
|
|
- Discovers databases under `../data/OKX` matching the instrument and date range.
|
|
- Launches UI: `uv run python app.py`.
|
|
|
|
## Usage Examples
|
|
|
|
### Run processing + UI
|
|
```bash
|
|
uv run python main.py BTC-USDT 2025-07-01 2025-08-01 --window-seconds 60
|
|
# Open http://localhost:8050
|
|
```
|
|
|
|
### Process trades and update depth in a loop (conceptual)
|
|
```python
|
|
from db_interpreter import DBInterpreter
|
|
from ohlc_processor import OHLCProcessor
|
|
|
|
processor = OHLCProcessor(window_seconds=60)
|
|
for ob_update, trades in DBInterpreter(db_path).stream():
|
|
processor.process_trades(trades)
|
|
processor.update_orderbook(ob_update)
|
|
processor.finalize()
|
|
```
|
|
|
|
## Error Handling
|
|
- Reader/Writer coordination via atomic JSON prevents partial reads.
|
|
- Visualizer caches last valid data if JSON decoding fails mid-write; logs warnings.
|
|
- Visualizer start failures do not stop processing; logs error and continues.
|
|
|
|
## Notes
|
|
- Metrics computation includes simplified OBI (Order Book Imbalance) calculated as bid_total - ask_total. Repository/storage layers and strategy APIs are intentionally kept minimal.
|