460 lines
11 KiB
Markdown
460 lines
11 KiB
Markdown
# Incremental Backtester
|
|
|
|
A high-performance backtesting system for incremental trading strategies with multiprocessing support for parameter optimization.
|
|
|
|
## Overview
|
|
|
|
The Incremental Backtester provides a complete solution for testing incremental trading strategies:
|
|
|
|
- **IncTrader**: Manages a single strategy during backtesting
|
|
- **IncBacktester**: Orchestrates multiple traders and parameter optimization
|
|
- **Multiprocessing Support**: Parallel execution across CPU cores
|
|
- **Memory Efficient**: Bounded memory usage regardless of data length
|
|
- **Real-time Compatible**: Same interface as live trading systems
|
|
|
|
## Quick Start
|
|
|
|
### 1. Basic Single Strategy Backtest
|
|
|
|
```python
|
|
from cycles.IncStrategies import (
|
|
IncBacktester, BacktestConfig, IncRandomStrategy
|
|
)
|
|
|
|
# Configure backtest
|
|
config = BacktestConfig(
|
|
data_file="btc_1min_2023.csv",
|
|
start_date="2023-01-01",
|
|
end_date="2023-12-31",
|
|
initial_usd=10000,
|
|
stop_loss_pct=0.02, # 2% stop loss
|
|
take_profit_pct=0.05 # 5% take profit
|
|
)
|
|
|
|
# Create strategy
|
|
strategy = IncRandomStrategy(params={
|
|
"timeframe": "15min",
|
|
"entry_probability": 0.1,
|
|
"exit_probability": 0.15
|
|
})
|
|
|
|
# Run backtest
|
|
backtester = IncBacktester(config)
|
|
results = backtester.run_single_strategy(strategy)
|
|
|
|
print(f"Profit: {results['profit_ratio']*100:.2f}%")
|
|
print(f"Trades: {results['n_trades']}")
|
|
print(f"Win Rate: {results['win_rate']*100:.1f}%")
|
|
```
|
|
|
|
### 2. Multiple Strategies
|
|
|
|
```python
|
|
strategies = [
|
|
IncRandomStrategy(params={"timeframe": "15min"}),
|
|
IncRandomStrategy(params={"timeframe": "30min"}),
|
|
IncMetaTrendStrategy(params={"timeframe": "15min"})
|
|
]
|
|
|
|
results = backtester.run_multiple_strategies(strategies)
|
|
|
|
for result in results:
|
|
print(f"{result['strategy_name']}: {result['profit_ratio']*100:.2f}%")
|
|
```
|
|
|
|
### 3. Parameter Optimization
|
|
|
|
```python
|
|
# Define parameter grids
|
|
strategy_param_grid = {
|
|
"timeframe": ["15min", "30min", "1h"],
|
|
"entry_probability": [0.05, 0.1, 0.15],
|
|
"exit_probability": [0.1, 0.15, 0.2]
|
|
}
|
|
|
|
trader_param_grid = {
|
|
"stop_loss_pct": [0.01, 0.02, 0.03],
|
|
"take_profit_pct": [0.03, 0.05, 0.07]
|
|
}
|
|
|
|
# Run optimization (uses all CPU cores)
|
|
results = backtester.optimize_parameters(
|
|
strategy_class=IncRandomStrategy,
|
|
param_grid=strategy_param_grid,
|
|
trader_param_grid=trader_param_grid,
|
|
max_workers=8 # Use 8 CPU cores
|
|
)
|
|
|
|
# Get summary statistics
|
|
summary = backtester.get_summary_statistics(results)
|
|
print(f"Best profit: {summary['profit_ratio']['max']*100:.2f}%")
|
|
|
|
# Save results
|
|
backtester.save_results(results, "optimization_results.csv")
|
|
```
|
|
|
|
## Architecture
|
|
|
|
### IncTrader Class
|
|
|
|
Manages a single strategy during backtesting:
|
|
|
|
```python
|
|
trader = IncTrader(
|
|
strategy=strategy,
|
|
initial_usd=10000,
|
|
params={
|
|
"stop_loss_pct": 0.02,
|
|
"take_profit_pct": 0.05
|
|
}
|
|
)
|
|
|
|
# Process data sequentially
|
|
for timestamp, ohlcv_data in data_stream:
|
|
trader.process_data_point(timestamp, ohlcv_data)
|
|
|
|
# Get results
|
|
results = trader.get_results()
|
|
```
|
|
|
|
**Key Features:**
|
|
- Position management (USD/coin balance)
|
|
- Trade execution based on strategy signals
|
|
- Stop loss and take profit handling
|
|
- Performance tracking and metrics
|
|
- Fee calculation using existing MarketFees
|
|
|
|
### IncBacktester Class
|
|
|
|
Orchestrates multiple traders and handles data loading:
|
|
|
|
```python
|
|
backtester = IncBacktester(config, storage)
|
|
|
|
# Single strategy
|
|
results = backtester.run_single_strategy(strategy)
|
|
|
|
# Multiple strategies
|
|
results = backtester.run_multiple_strategies(strategies)
|
|
|
|
# Parameter optimization
|
|
results = backtester.optimize_parameters(strategy_class, param_grid)
|
|
```
|
|
|
|
**Key Features:**
|
|
- Data loading using existing Storage class
|
|
- Multiprocessing for parameter optimization
|
|
- Result aggregation and analysis
|
|
- Summary statistics calculation
|
|
- CSV export functionality
|
|
|
|
### BacktestConfig Class
|
|
|
|
Configuration for backtesting runs:
|
|
|
|
```python
|
|
config = BacktestConfig(
|
|
data_file="btc_1min_2023.csv",
|
|
start_date="2023-01-01",
|
|
end_date="2023-12-31",
|
|
initial_usd=10000,
|
|
timeframe="1min",
|
|
|
|
# Trader parameters
|
|
stop_loss_pct=0.02,
|
|
take_profit_pct=0.05,
|
|
|
|
# Performance settings
|
|
max_workers=None, # Auto-detect CPU cores
|
|
chunk_size=1000
|
|
)
|
|
```
|
|
|
|
## Data Requirements
|
|
|
|
### Input Data Format
|
|
|
|
The backtester expects minute-level OHLCV data in CSV format:
|
|
|
|
```csv
|
|
timestamp,open,high,low,close,volume
|
|
1672531200,16625.1,16634.5,16620.0,16628.3,125.45
|
|
1672531260,16628.3,16635.2,16625.8,16631.7,98.32
|
|
...
|
|
```
|
|
|
|
**Requirements:**
|
|
- Timestamp column (Unix timestamp or datetime)
|
|
- OHLCV columns: open, high, low, close, volume
|
|
- Minute-level frequency (strategies handle timeframe aggregation)
|
|
- Sorted by timestamp (ascending)
|
|
|
|
### Data Loading
|
|
|
|
Uses the existing Storage class for data loading:
|
|
|
|
```python
|
|
from cycles.utils.storage import Storage
|
|
|
|
storage = Storage()
|
|
data = storage.load_data(
|
|
"btc_1min_2023.csv",
|
|
"2023-01-01",
|
|
"2023-12-31"
|
|
)
|
|
```
|
|
|
|
## Performance Features
|
|
|
|
### Multiprocessing Support
|
|
|
|
Parameter optimization automatically distributes work across CPU cores:
|
|
|
|
```python
|
|
# Automatic CPU detection
|
|
results = backtester.optimize_parameters(strategy_class, param_grid)
|
|
|
|
# Manual worker count
|
|
results = backtester.optimize_parameters(
|
|
strategy_class, param_grid, max_workers=4
|
|
)
|
|
|
|
# Single-threaded (for debugging)
|
|
results = backtester.optimize_parameters(
|
|
strategy_class, param_grid, max_workers=1
|
|
)
|
|
```
|
|
|
|
### Memory Efficiency
|
|
|
|
- **Bounded Memory**: Strategy buffers have fixed size limits
|
|
- **Incremental Processing**: No need to load entire datasets into memory
|
|
- **Efficient Data Structures**: Optimized for sequential processing
|
|
|
|
### Performance Monitoring
|
|
|
|
Built-in performance tracking:
|
|
|
|
```python
|
|
results = backtester.run_single_strategy(strategy)
|
|
|
|
print(f"Backtest duration: {results['backtest_duration_seconds']:.2f}s")
|
|
print(f"Data points processed: {results['data_points_processed']}")
|
|
print(f"Processing rate: {results['data_points']/results['backtest_duration_seconds']:.0f} points/sec")
|
|
```
|
|
|
|
## Result Analysis
|
|
|
|
### Individual Results
|
|
|
|
Each backtest returns comprehensive metrics:
|
|
|
|
```python
|
|
{
|
|
"strategy_name": "IncRandomStrategy",
|
|
"strategy_params": {"timeframe": "15min", ...},
|
|
"trader_params": {"stop_loss_pct": 0.02, ...},
|
|
"initial_usd": 10000.0,
|
|
"final_usd": 10250.0,
|
|
"profit_ratio": 0.025,
|
|
"n_trades": 15,
|
|
"win_rate": 0.6,
|
|
"max_drawdown": 0.08,
|
|
"avg_trade": 0.0167,
|
|
"total_fees_usd": 45.32,
|
|
"trades": [...], # Individual trade records
|
|
"backtest_duration_seconds": 2.45
|
|
}
|
|
```
|
|
|
|
### Summary Statistics
|
|
|
|
For parameter optimization runs:
|
|
|
|
```python
|
|
summary = backtester.get_summary_statistics(results)
|
|
|
|
{
|
|
"total_runs": 108,
|
|
"successful_runs": 105,
|
|
"failed_runs": 3,
|
|
"profit_ratio": {
|
|
"mean": 0.023,
|
|
"std": 0.045,
|
|
"min": -0.12,
|
|
"max": 0.18,
|
|
"median": 0.019
|
|
},
|
|
"best_run": {...},
|
|
"worst_run": {...}
|
|
}
|
|
```
|
|
|
|
### Export Results
|
|
|
|
Save results to CSV for further analysis:
|
|
|
|
```python
|
|
backtester.save_results(results, "backtest_results.csv")
|
|
```
|
|
|
|
Output includes:
|
|
- Strategy and trader parameters
|
|
- Performance metrics
|
|
- Trade statistics
|
|
- Execution timing
|
|
|
|
## Integration with Existing System
|
|
|
|
### Compatibility
|
|
|
|
The incremental backtester integrates seamlessly with existing components:
|
|
|
|
- **Storage Class**: Uses existing data loading infrastructure
|
|
- **MarketFees**: Uses existing fee calculation
|
|
- **Strategy Interface**: Compatible with incremental strategies
|
|
- **Result Format**: Similar to existing Backtest class
|
|
|
|
### Migration from Original Backtester
|
|
|
|
```python
|
|
# Original backtester
|
|
from cycles.backtest import Backtest
|
|
|
|
# Incremental backtester
|
|
from cycles.IncStrategies import IncBacktester, BacktestConfig
|
|
|
|
# Similar interface, enhanced capabilities
|
|
config = BacktestConfig(...)
|
|
backtester = IncBacktester(config)
|
|
results = backtester.run_single_strategy(strategy)
|
|
```
|
|
|
|
## Testing
|
|
|
|
### Synthetic Data Testing
|
|
|
|
Test with synthetic data before using real market data:
|
|
|
|
```python
|
|
from cycles.IncStrategies.test_inc_backtester import main
|
|
|
|
# Run all tests
|
|
main()
|
|
```
|
|
|
|
### Unit Tests
|
|
|
|
Individual component testing:
|
|
|
|
```python
|
|
# Test IncTrader
|
|
from cycles.IncStrategies.test_inc_backtester import test_inc_trader
|
|
test_inc_trader()
|
|
|
|
# Test IncBacktester
|
|
from cycles.IncStrategies.test_inc_backtester import test_inc_backtester
|
|
test_inc_backtester()
|
|
```
|
|
|
|
## Examples
|
|
|
|
See `example_backtest.py` for comprehensive usage examples:
|
|
|
|
```python
|
|
from cycles.IncStrategies.example_backtest import (
|
|
example_single_strategy_backtest,
|
|
example_parameter_optimization,
|
|
example_custom_analysis
|
|
)
|
|
|
|
# Run examples
|
|
example_single_strategy_backtest()
|
|
example_parameter_optimization()
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
### 1. Data Preparation
|
|
|
|
- Ensure data quality (no gaps, correct format)
|
|
- Use appropriate date ranges for testing
|
|
- Consider market conditions in test periods
|
|
|
|
### 2. Parameter Optimization
|
|
|
|
- Start with small parameter grids for testing
|
|
- Use representative time periods
|
|
- Consider overfitting risks
|
|
- Validate results on out-of-sample data
|
|
|
|
### 3. Performance Optimization
|
|
|
|
- Use multiprocessing for large parameter grids
|
|
- Monitor memory usage for long backtests
|
|
- Profile bottlenecks for optimization
|
|
|
|
### 4. Result Validation
|
|
|
|
- Compare with original backtester for validation
|
|
- Check trade logic manually for small samples
|
|
- Verify fee calculations and position management
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **Data Loading Errors**
|
|
- Check file path and format
|
|
- Verify date range availability
|
|
- Ensure required columns exist
|
|
|
|
2. **Strategy Errors**
|
|
- Check strategy initialization
|
|
- Verify parameter validity
|
|
- Monitor warmup period completion
|
|
|
|
3. **Performance Issues**
|
|
- Reduce parameter grid size
|
|
- Limit worker count for memory constraints
|
|
- Use shorter time periods for testing
|
|
|
|
### Debug Mode
|
|
|
|
Enable detailed logging:
|
|
|
|
```python
|
|
import logging
|
|
logging.basicConfig(level=logging.DEBUG)
|
|
|
|
# Run with detailed output
|
|
results = backtester.run_single_strategy(strategy)
|
|
```
|
|
|
|
### Memory Monitoring
|
|
|
|
Monitor memory usage during optimization:
|
|
|
|
```python
|
|
import psutil
|
|
import os
|
|
|
|
process = psutil.Process(os.getpid())
|
|
print(f"Memory usage: {process.memory_info().rss / 1024 / 1024:.1f} MB")
|
|
```
|
|
|
|
## Future Enhancements
|
|
|
|
- **Live Trading Integration**: Direct connection to trading systems
|
|
- **Advanced Analytics**: Risk metrics, Sharpe ratio, etc.
|
|
- **Visualization**: Built-in plotting and analysis tools
|
|
- **Database Support**: Direct database connectivity
|
|
- **Strategy Combinations**: Multi-strategy portfolio testing
|
|
|
|
## Support
|
|
|
|
For issues and questions:
|
|
1. Check the test scripts for working examples
|
|
2. Review the TODO.md for known limitations
|
|
3. Examine the base strategy implementations
|
|
4. Use debug logging for detailed troubleshooting |