Add initial implementation of the Orderflow Backtest System with OBI and CVD metrics integration, including core modules for storage, strategies, and visualization. Introduced persistent metrics storage in SQLite, optimized memory usage, and enhanced documentation.

This commit is contained in:
Simon Moisy
2025-08-26 17:22:07 +08:00
parent 63f723820a
commit fa6df78c1e
52 changed files with 7039 additions and 1 deletions

View File

@@ -0,0 +1,66 @@
# Tasks: OBI and CVD Metrics Integration
Based on the PRD for integrating Order Book Imbalance (OBI) and Cumulative Volume Delta (CVD) calculations into the orderflow backtest system.
## Relevant Files
- `repositories/sqlite_repository.py` - Extend to support metrics table operations and batch insertions
- `repositories/test_metrics_repository.py` - Unit tests for metrics repository functionality
- `models.py` - Add new data models for metrics and update Book class
- `tests/test_models_metrics.py` - Unit tests for new metric models
- `storage.py` - Modify to integrate metric calculations during snapshot processing
- `tests/test_storage_metrics.py` - Unit tests for storage metric integration
- `strategies.py` - Enhance DefaultStrategy to calculate OBI and CVD metrics
- `tests/test_strategies_metrics.py` - Unit tests for strategy metric calculations
- `visualizer.py` - Extend to plot OBI and CVD curves beneath volume graphs
- `tests/test_visualizer_metrics.py` - Unit tests for metric visualization
- `parsers/metric_calculator.py` - New utility class for OBI and CVD calculations
- `tests/test_metric_calculator.py` - Unit tests for metric calculation logic
### Notes
- Unit tests should be placed alongside the code files they are testing
- Use `uv run pytest [optional/path/to/test/file]` to run tests following project standards
- Database schema changes require migration considerations for existing databases
## Tasks
- [ ] 1.0 Database Schema and Repository Updates
- [ ] 1.1 Create metrics table schema with proper indexes and foreign key constraints
- [ ] 1.2 Add metrics table creation method to SQLiteOrderflowRepository
- [ ] 1.3 Implement metrics insertion methods with batch support for performance
- [ ] 1.4 Add metrics querying methods (by timestamp range, snapshot_id)
- [ ] 1.5 Create database migration utility to add metrics table to existing databases
- [ ] 1.6 Add proper error handling and transaction management for metrics operations
- [ ] 2.0 Metric Calculation Engine
- [ ] 2.1 Create MetricCalculator class with OBI calculation method
- [ ] 2.2 Implement CVD calculation with incremental support and reset functionality
- [ ] 2.3 Add volume delta calculation for individual timestamps
- [ ] 2.4 Implement best bid/ask extraction from orderbook snapshots
- [ ] 2.5 Add edge case handling (empty orderbook, no trades, zero volume)
- [ ] 2.6 Create validation methods to ensure OBI values are within [-1, 1] range
- [ ] 3.0 Storage System Integration
- [ ] 3.1 Modify Storage.build_booktick_from_db to integrate metric calculations
- [ ] 3.2 Update _create_snapshots_from_rows to calculate and store metrics per snapshot
- [ ] 3.3 Implement memory optimization by removing full snapshot retention
- [ ] 3.4 Add metric persistence during snapshot processing
- [ ] 3.5 Update Book model to store only essential data (metrics + best bid/ask)
- [ ] 3.6 Add progress reporting for metric calculation during processing
- [ ] 4.0 Strategy Enhancement
- [ ] 4.1 Update DefaultStrategy to use MetricCalculator for OBI and CVD
- [ ] 4.2 Modify compute_OBI method to work with new metric calculation system
- [ ] 4.3 Add CVD computation method to DefaultStrategy
- [ ] 4.4 Return time-series data structures compatible with visualizer
- [ ] 4.5 Integrate metric calculation into on_booktick workflow
- [ ] 4.6 Add configuration options for CVD reset points and calculation parameters
- [ ] 5.0 Visualization Implementation
- [ ] 5.1 Extend Visualizer to load metrics data from database
- [ ] 5.2 Add OBI and CVD plotting methods beneath volume graphs
- [ ] 5.3 Implement shared X-axis time alignment across all charts (OHLC, volume, OBI, CVD)
- [ ] 5.4 Add 6-hour bar aggregation support for metrics visualization
- [ ] 5.5 Implement standard line styling for OBI and CVD curves
- [ ] 5.6 Make time resolution configurable for future flexibility