Add initial implementation of the Orderflow Backtest System with OBI and CVD metrics integration, including core modules for storage, strategies, and visualization. Introduced persistent metrics storage in SQLite, optimized memory usage, and enhanced documentation.

2025-08-26 17:22:07 +08:00
parent 63f723820a
commit fa6df78c1e
52 changed files with 7039 additions and 1 deletions
--- a/docs/modules/metrics.md
+++ b/docs/modules/metrics.md
@@ -0,0 +1,302 @@
+# Module: Metrics Calculation System
+
+## Purpose
+
+The metrics calculation system provides high-performance computation of Order Book Imbalance (OBI) and Cumulative Volume Delta (CVD) indicators for cryptocurrency trading analysis. It processes orderbook snapshots and trade data to generate financial metrics with per-snapshot granularity.
+
+## Public Interface
+
+### Classes
+
+#### `Metric` (dataclass)
+Represents calculated metrics for a single orderbook snapshot.
+
+```python
+@dataclass(slots=True)
+class Metric:
+    snapshot_id: int        # Reference to source snapshot
+    timestamp: int          # Unix timestamp
+    obi: float             # Order Book Imbalance [-1, 1]
+    cvd: float             # Cumulative Volume Delta
+    best_bid: float | None # Best bid price
+    best_ask: float | None # Best ask price
+```
+
+#### `MetricCalculator` (static class)
+Provides calculation methods for financial metrics.
+
+```python
+class MetricCalculator:
+    @staticmethod
+    def calculate_obi(snapshot: BookSnapshot) -> float
+    
+    @staticmethod
+    def calculate_volume_delta(trades: List[Trade]) -> float
+    
+    @staticmethod  
+    def calculate_cvd(previous_cvd: float, volume_delta: float) -> float
+    
+    @staticmethod
+    def get_best_bid_ask(snapshot: BookSnapshot) -> tuple[float | None, float | None]
+```
+
+### Functions
+
+#### Order Book Imbalance (OBI) Calculation
+```python
+def calculate_obi(snapshot: BookSnapshot) -> float:
+    """
+    Calculate Order Book Imbalance using the standard formula.
+    
+    Formula: OBI = (Vb - Va) / (Vb + Va)
+    Where:
+        Vb = Total volume on bid side
+        Va = Total volume on ask side
+    
+    Args:
+        snapshot: BookSnapshot containing bids and asks data
+        
+    Returns:
+        float: OBI value between -1 and 1, or 0.0 if no volume
+        
+    Example:
+        >>> snapshot = BookSnapshot(bids={50000.0: OrderbookLevel(...)}, ...)
+        >>> obi = MetricCalculator.calculate_obi(snapshot)
+        >>> print(f"OBI: {obi:.3f}")
+        OBI: 0.333
+    """
+```
+
+#### Volume Delta Calculation
+```python
+def calculate_volume_delta(trades: List[Trade]) -> float:
+    """
+    Calculate Volume Delta for a list of trades.
+    
+    Volume Delta = Buy Volume - Sell Volume
+    - Buy trades (side = "buy"): positive contribution
+    - Sell trades (side = "sell"): negative contribution
+    
+    Args:
+        trades: List of Trade objects for specific timestamp
+        
+    Returns:
+        float: Net volume delta (positive = buy pressure, negative = sell pressure)
+        
+    Example:
+        >>> trades = [
+        ...     Trade(side="buy", size=10.0, ...),
+        ...     Trade(side="sell", size=3.0, ...)
+        ... ]
+        >>> vd = MetricCalculator.calculate_volume_delta(trades)
+        >>> print(f"Volume Delta: {vd}")
+        Volume Delta: 7.0
+    """
+```
+
+#### Cumulative Volume Delta (CVD) Calculation
+```python
+def calculate_cvd(previous_cvd: float, volume_delta: float) -> float:
+    """
+    Calculate Cumulative Volume Delta with incremental support.
+    
+    Formula: CVD_t = CVD_{t-1} + Volume_Delta_t
+    
+    Args:
+        previous_cvd: Previous CVD value (use 0.0 for reset)
+        volume_delta: Current volume delta to add
+        
+    Returns:
+        float: New cumulative volume delta value
+        
+    Example:
+        >>> cvd = 0.0  # Starting value
+        >>> cvd = MetricCalculator.calculate_cvd(cvd, 10.0)  # First trade
+        >>> cvd = MetricCalculator.calculate_cvd(cvd, -5.0)  # Second trade
+        >>> print(f"CVD: {cvd}")
+        CVD: 5.0
+    """
+```
+
+## Usage Examples
+
+### Basic OBI Calculation
+```python
+from models import MetricCalculator, BookSnapshot, OrderbookLevel
+
+# Create sample orderbook snapshot
+snapshot = BookSnapshot(
+    id=1,
+    timestamp=1640995200,
+    bids={
+        50000.0: OrderbookLevel(price=50000.0, size=10.0, liquidation_count=0, order_count=1),
+        49999.0: OrderbookLevel(price=49999.0, size=5.0, liquidation_count=0, order_count=1),
+    },
+    asks={
+        50001.0: OrderbookLevel(price=50001.0, size=3.0, liquidation_count=0, order_count=1),
+        50002.0: OrderbookLevel(price=50002.0, size=2.0, liquidation_count=0, order_count=1),
+    }
+)
+
+# Calculate OBI
+obi = MetricCalculator.calculate_obi(snapshot)
+print(f"OBI: {obi:.3f}")  # Output: OBI: 0.500
+# Explanation: (15 - 5) / (15 + 5) = 10/20 = 0.5
+```
+
+### CVD Calculation with Reset
+```python
+from models import MetricCalculator, Trade
+
+# Simulate trading session
+cvd = 0.0  # Reset CVD at session start
+
+# Process trades for first timestamp
+trades_t1 = [
+    Trade(id=1, trade_id=1.0, price=50000.0, size=8.0, side="buy", timestamp=1000),
+    Trade(id=2, trade_id=2.0, price=50001.0, size=3.0, side="sell", timestamp=1000),
+]
+
+vd_t1 = MetricCalculator.calculate_volume_delta(trades_t1)  # 8.0 - 3.0 = 5.0
+cvd = MetricCalculator.calculate_cvd(cvd, vd_t1)           # 0.0 + 5.0 = 5.0
+
+# Process trades for second timestamp
+trades_t2 = [
+    Trade(id=3, trade_id=3.0, price=49999.0, size=2.0, side="buy", timestamp=1001),
+    Trade(id=4, trade_id=4.0, price=50000.0, size=7.0, side="sell", timestamp=1001),
+]
+
+vd_t2 = MetricCalculator.calculate_volume_delta(trades_t2)  # 2.0 - 7.0 = -5.0
+cvd = MetricCalculator.calculate_cvd(cvd, vd_t2)           # 5.0 + (-5.0) = 0.0
+
+print(f"Final CVD: {cvd}")  # Output: Final CVD: 0.0
+```
+
+### Complete Metrics Processing
+```python
+from models import MetricCalculator, Metric
+
+def process_snapshot_metrics(snapshot, trades, previous_cvd=0.0):
+    """Process complete metrics for a single snapshot."""
+    
+    # Calculate OBI
+    obi = MetricCalculator.calculate_obi(snapshot)
+    
+    # Calculate volume delta and CVD
+    volume_delta = MetricCalculator.calculate_volume_delta(trades)
+    cvd = MetricCalculator.calculate_cvd(previous_cvd, volume_delta)
+    
+    # Extract best bid/ask
+    best_bid, best_ask = MetricCalculator.get_best_bid_ask(snapshot)
+    
+    # Create metric record
+    metric = Metric(
+        snapshot_id=snapshot.id,
+        timestamp=snapshot.timestamp,
+        obi=obi,
+        cvd=cvd,
+        best_bid=best_bid,
+        best_ask=best_ask
+    )
+    
+    return metric, cvd
+
+# Usage in processing loop
+current_cvd = 0.0
+for snapshot, trades in snapshot_trade_pairs:
+    metric, current_cvd = process_snapshot_metrics(snapshot, trades, current_cvd)
+    # Store metric to database...
+```
+
+## Dependencies
+
+### Internal
+- `models.BookSnapshot`: Orderbook state data
+- `models.Trade`: Individual trade execution data
+- `models.OrderbookLevel`: Price level information
+
+### External
+- **Python Standard Library**: `typing` for type hints
+- **No external packages required**
+
+## Performance Characteristics
+
+### Computational Complexity
+- **OBI Calculation**: O(n) where n = number of price levels
+- **Volume Delta**: O(m) where m = number of trades
+- **CVD Calculation**: O(1) - simple addition
+- **Best Bid/Ask**: O(n) for min/max operations
+
+### Memory Usage
+- **Static Methods**: No instance state, minimal memory overhead
+- **Calculations**: Process data in-place without copying
+- **Results**: Lightweight `Metric` objects with slots optimization
+
+### Typical Performance
+```python
+# Benchmark results (approximate)
+Snapshot with 50 price levels:     ~0.1ms per OBI calculation
+Timestamp with 20 trades:          ~0.05ms per volume delta
+CVD update:                        ~0.001ms per calculation
+Complete metric processing:        ~0.2ms per snapshot
+```
+
+## Error Handling
+
+### Edge Cases Handled
+```python
+# Empty orderbook
+empty_snapshot = BookSnapshot(bids={}, asks={})
+obi = MetricCalculator.calculate_obi(empty_snapshot)  # Returns 0.0
+
+# No trades
+empty_trades = []
+vd = MetricCalculator.calculate_volume_delta(empty_trades)  # Returns 0.0
+
+# Zero volume scenario
+zero_vol_snapshot = BookSnapshot(
+    bids={50000.0: OrderbookLevel(price=50000.0, size=0.0, ...)},
+    asks={50001.0: OrderbookLevel(price=50001.0, size=0.0, ...)}
+)
+obi = MetricCalculator.calculate_obi(zero_vol_snapshot)  # Returns 0.0
+```
+
+### Validation
+- **OBI Range**: Results automatically bounded to [-1, 1]
+- **Division by Zero**: Handled gracefully with 0.0 return
+- **Invalid Data**: Empty collections handled without errors
+
+## Testing
+
+### Test Coverage
+- **Unit Tests**: `tests/test_metric_calculator.py`
+- **Integration Tests**: Included in storage and strategy tests
+- **Edge Cases**: Empty data, zero volume, boundary conditions
+
+### Running Tests
+```bash
+# Run metric calculator tests specifically
+uv run pytest tests/test_metric_calculator.py -v
+
+# Run all tests with metrics
+uv run pytest -k "metric" -v
+
+# Performance tests
+uv run pytest tests/test_metric_calculator.py::test_calculate_obi_performance
+```
+
+## Known Issues
+
+### Current Limitations
+- **Precision**: Floating-point arithmetic limitations for very small numbers
+- **Scale**: No optimization for extremely large orderbooks (>10k levels)
+- **Currency**: No multi-currency support (assumes single denomination)
+
+### Planned Enhancements
+- **Decimal Precision**: Consider `decimal.Decimal` for high-precision calculations
+- **Vectorization**: NumPy integration for batch calculations
+- **Additional Metrics**: Volume Profile, Liquidity metrics, Delta Flow
+
+---
+
+The metrics calculation system provides a robust foundation for financial analysis with clean interfaces, comprehensive error handling, and optimal performance for high-frequency trading data.