144 lines
5.9 KiB
Markdown
144 lines
5.9 KiB
Markdown
# Changelog
|
|
|
|
All notable changes to the Orderflow Backtest System are documented in this file.
|
|
|
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
|
## [2.0.0] - 2024-Current
|
|
|
|
### Added
|
|
- **OBI Metrics Calculation**: Order Book Imbalance calculation with formula `(Vb - Va) / (Vb + Va)`
|
|
- **CVD Metrics Calculation**: Cumulative Volume Delta with incremental calculation and reset functionality
|
|
- **Persistent Metrics Storage**: SQLite-based storage for calculated metrics to avoid recalculation
|
|
- **Memory Optimization**: >70% reduction in peak memory usage through streaming processing
|
|
- **Enhanced Visualization**: Multi-subplot charts with OHLC, Volume, OBI, and CVD displays
|
|
- **Metrics Repository**: `SQLiteMetricsRepository` for write-enabled database operations
|
|
- **MetricCalculator Class**: Static methods for financial metrics computation
|
|
- **Batch Processing**: High-performance batch inserts (1000 records per operation)
|
|
- **Time-Range Queries**: Efficient metrics retrieval for specified time periods
|
|
- **Strategy Enhancement**: Metrics analysis capabilities in `DefaultStrategy`
|
|
- **Comprehensive Testing**: 27 tests across 6 test files with full integration coverage
|
|
|
|
### Changed
|
|
- **Storage Architecture**: Modified `Storage.build_booktick_from_db()` to integrate metrics calculation
|
|
- **Visualization Separation**: Moved visualization from strategy to main application for better separation of concerns
|
|
- **Strategy Interface**: Simplified `DefaultStrategy` constructor (removed `enable_visualization` parameter)
|
|
- **Main Application Flow**: Enhanced orchestration with per-database visualization updates
|
|
- **Database Schema**: Auto-creation of metrics table with proper indexes and foreign key constraints
|
|
- **Memory Management**: Stream processing instead of keeping full snapshot history
|
|
|
|
### Improved
|
|
- **Performance**: Batch database operations and optimized SQLite PRAGMAs
|
|
- **Scalability**: Support for months to years of high-frequency trading data
|
|
- **Code Quality**: All functions <50 lines, all files <250 lines
|
|
- **Documentation**: Comprehensive module and API documentation
|
|
- **Error Handling**: Graceful degradation and comprehensive logging
|
|
- **Type Safety**: Full type annotations throughout codebase
|
|
|
|
### Technical Details
|
|
- **New Tables**: `metrics` table with indexes on timestamp and snapshot_id
|
|
- **New Models**: `Metric` dataclass for calculated values
|
|
- **Processing Pipeline**: Snapshot → Calculate → Store → Discard workflow
|
|
- **Query Interface**: Time-range based metrics retrieval
|
|
- **Visualization Layout**: 4-subplot layout with shared time axis
|
|
|
|
## [1.0.0] - Previous Version
|
|
|
|
### Features
|
|
- **Orderbook Reconstruction**: Build complete orderbooks from SQLite database files
|
|
- **Data Models**: Core structures for `OrderbookLevel`, `Trade`, `BookSnapshot`, `Book`
|
|
- **SQLite Repository**: Read-only data access for orderbook and trades data
|
|
- **Orderbook Parser**: Text parsing with price caching optimization
|
|
- **Storage Orchestration**: High-level facade for book building
|
|
- **Basic Visualization**: OHLC candlestick charts with Qt5Agg backend
|
|
- **Strategy Framework**: Basic strategy pattern with `DefaultStrategy`
|
|
- **CLI Interface**: Command-line application for date range processing
|
|
- **Test Suite**: Unit and integration tests
|
|
|
|
### Architecture
|
|
- **Repository Pattern**: Clean separation of data access logic
|
|
- **Dataclass Models**: Lightweight, type-safe data structures
|
|
- **Parser Optimization**: Price caching for performance
|
|
- **Modular Design**: Clear separation between components
|
|
|
|
---
|
|
|
|
## Migration Guide
|
|
|
|
### Upgrading from v1.0.0 to v2.0.0
|
|
|
|
#### Code Changes Required
|
|
|
|
1. **Strategy Constructor**
|
|
```python
|
|
# Before (v1.0.0)
|
|
strategy = DefaultStrategy("BTC-USDT", enable_visualization=True)
|
|
|
|
# After (v2.0.0)
|
|
strategy = DefaultStrategy("BTC-USDT")
|
|
visualizer = Visualizer(window_seconds=60, max_bars=500)
|
|
```
|
|
|
|
2. **Main Application Flow**
|
|
```python
|
|
# Before (v1.0.0)
|
|
strategy = DefaultStrategy(instrument, enable_visualization=True)
|
|
storage.build_booktick_from_db(db_path, db_date)
|
|
strategy.on_booktick(storage.book)
|
|
|
|
# After (v2.0.0)
|
|
strategy = DefaultStrategy(instrument)
|
|
visualizer = Visualizer(window_seconds=60, max_bars=500)
|
|
|
|
strategy.set_db_path(db_path)
|
|
visualizer.set_db_path(db_path)
|
|
storage.build_booktick_from_db(db_path, db_date)
|
|
strategy.on_booktick(storage.book)
|
|
visualizer.update_from_book(storage.book)
|
|
```
|
|
|
|
#### Database Migration
|
|
- **Automatic**: Metrics table created automatically on first run
|
|
- **No Data Loss**: Existing orderbook and trades data unchanged
|
|
- **Schema Addition**: New `metrics` table with indexes added to existing databases
|
|
|
|
#### Benefits of Upgrading
|
|
- **Memory Efficiency**: >70% reduction in memory usage
|
|
- **Performance**: Faster processing through persistent metrics storage
|
|
- **Enhanced Analysis**: Access to OBI and CVD financial indicators
|
|
- **Better Visualization**: Multi-chart display with synchronized time axis
|
|
- **Improved Architecture**: Cleaner separation of concerns
|
|
|
|
#### Testing Migration
|
|
```bash
|
|
# Verify upgrade compatibility
|
|
uv run pytest tests/test_main_integration.py -v
|
|
|
|
# Test new metrics functionality
|
|
uv run pytest tests/test_storage_metrics.py -v
|
|
|
|
# Validate visualization separation
|
|
uv run pytest tests/test_main_visualization.py -v
|
|
```
|
|
|
|
---
|
|
|
|
## Development Notes
|
|
|
|
### Performance Improvements
|
|
- **v2.0.0**: >70% memory reduction, batch processing, persistent storage
|
|
- **v1.0.0**: In-memory processing, real-time calculations
|
|
|
|
### Architecture Evolution
|
|
- **v2.0.0**: Streaming processing with metrics storage, separated visualization
|
|
- **v1.0.0**: Full snapshot retention, integrated visualization in strategies
|
|
|
|
### Testing Coverage
|
|
- **v2.0.0**: 27 tests across 6 files, integration and unit coverage
|
|
- **v1.0.0**: Basic unit tests for core components
|
|
|
|
---
|
|
|
|
*For detailed technical documentation, see [docs/](../docs/) directory.*
|