# Changelog

All notable changes to the Orderflow Backtest System are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [2.0.0] - 2024-Current

### Added
- **OBI Metrics Calculation**: Order Book Imbalance calculation with formula `(Vb - Va) / (Vb + Va)`
- **CVD Metrics Calculation**: Cumulative Volume Delta with incremental calculation and reset functionality
- **Persistent Metrics Storage**: SQLite-based storage for calculated metrics to avoid recalculation
- **Memory Optimization**: >70% reduction in peak memory usage through streaming processing
- **Enhanced Visualization**: Multi-subplot charts with OHLC, Volume, OBI, and CVD displays
- **Metrics Repository**: `SQLiteMetricsRepository` for write-enabled database operations
- **MetricCalculator Class**: Static methods for financial metrics computation
- **Batch Processing**: High-performance batch inserts (1000 records per operation)
- **Time-Range Queries**: Efficient metrics retrieval for specified time periods
- **Strategy Enhancement**: Metrics analysis capabilities in `DefaultStrategy`
- **Comprehensive Testing**: 27 tests across 6 test files with full integration coverage

### Changed
- **Storage Architecture**: Modified `Storage.build_booktick_from_db()` to integrate metrics calculation
- **Visualization Separation**: Moved visualization from strategy to main application for better separation of concerns
- **Strategy Interface**: Simplified `DefaultStrategy` constructor (removed `enable_visualization` parameter)
- **Main Application Flow**: Enhanced orchestration with per-database visualization updates
- **Database Schema**: Auto-creation of metrics table with proper indexes and foreign key constraints
- **Memory Management**: Stream processing instead of keeping full snapshot history

### Improved
- **Performance**: Batch database operations and optimized SQLite PRAGMAs
- **Scalability**: Support for months to years of high-frequency trading data
- **Code Quality**: All functions <50 lines, all files <250 lines
- **Documentation**: Comprehensive module and API documentation
- **Error Handling**: Graceful degradation and comprehensive logging
- **Type Safety**: Full type annotations throughout codebase

### Technical Details
- **New Tables**: `metrics` table with indexes on timestamp and snapshot_id
- **New Models**: `Metric` dataclass for calculated values
- **Processing Pipeline**: Snapshot → Calculate → Store → Discard workflow
- **Query Interface**: Time-range based metrics retrieval
- **Visualization Layout**: 4-subplot layout with shared time axis

## [1.0.0] - Previous Version

### Features
- **Orderbook Reconstruction**: Build complete orderbooks from SQLite database files
- **Data Models**: Core structures for `OrderbookLevel`, `Trade`, `BookSnapshot`, `Book`
- **SQLite Repository**: Read-only data access for orderbook and trades data
- **Orderbook Parser**: Text parsing with price caching optimization
- **Storage Orchestration**: High-level facade for book building
- **Basic Visualization**: OHLC candlestick charts with Qt5Agg backend
- **Strategy Framework**: Basic strategy pattern with `DefaultStrategy`
- **CLI Interface**: Command-line application for date range processing
- **Test Suite**: Unit and integration tests

### Architecture
- **Repository Pattern**: Clean separation of data access logic
- **Dataclass Models**: Lightweight, type-safe data structures
- **Parser Optimization**: Price caching for performance
- **Modular Design**: Clear separation between components

---

## Migration Guide

### Upgrading from v1.0.0 to v2.0.0

#### Code Changes Required

1. **Strategy Constructor**
   ```python
   # Before (v1.0.0)
   strategy = DefaultStrategy("BTC-USDT", enable_visualization=True)
   
   # After (v2.0.0)  
   strategy = DefaultStrategy("BTC-USDT")
   visualizer = Visualizer(window_seconds=60, max_bars=500)
   ```

2. **Main Application Flow**
   ```python
   # Before (v1.0.0)
   strategy = DefaultStrategy(instrument, enable_visualization=True)
   storage.build_booktick_from_db(db_path, db_date)
   strategy.on_booktick(storage.book)
   
   # After (v2.0.0)
   strategy = DefaultStrategy(instrument)
   visualizer = Visualizer(window_seconds=60, max_bars=500)
   
   strategy.set_db_path(db_path)
   visualizer.set_db_path(db_path)
   storage.build_booktick_from_db(db_path, db_date)
   strategy.on_booktick(storage.book)
   visualizer.update_from_book(storage.book)
   ```

#### Database Migration
- **Automatic**: Metrics table created automatically on first run
- **No Data Loss**: Existing orderbook and trades data unchanged
- **Schema Addition**: New `metrics` table with indexes added to existing databases

#### Benefits of Upgrading
- **Memory Efficiency**: >70% reduction in memory usage
- **Performance**: Faster processing through persistent metrics storage
- **Enhanced Analysis**: Access to OBI and CVD financial indicators
- **Better Visualization**: Multi-chart display with synchronized time axis
- **Improved Architecture**: Cleaner separation of concerns

#### Testing Migration
```bash
# Verify upgrade compatibility
uv run pytest tests/test_main_integration.py -v

# Test new metrics functionality  
uv run pytest tests/test_storage_metrics.py -v

# Validate visualization separation
uv run pytest tests/test_main_visualization.py -v
```

---

## Development Notes

### Performance Improvements
- **v2.0.0**: >70% memory reduction, batch processing, persistent storage
- **v1.0.0**: In-memory processing, real-time calculations

### Architecture Evolution
- **v2.0.0**: Streaming processing with metrics storage, separated visualization
- **v1.0.0**: Full snapshot retention, integrated visualization in strategies

### Testing Coverage
- **v2.0.0**: 27 tests across 6 files, integration and unit coverage
- **v1.0.0**: Basic unit tests for core components

---

*For detailed technical documentation, see [docs/](../docs/) directory.*