bactester for strategies
This commit is contained in:
Ajasra
333
test/backtest/README.md
Normal file
333
test/backtest/README.md
Normal file
@@ -0,0 +1,333 @@
|
||||
# Strategy Backtest Runner
|
||||
|
||||
A comprehensive and efficient backtest runner for executing predefined trading strategies with advanced visualization and analysis capabilities.
|
||||
|
||||
## Overview
|
||||
|
||||
The Strategy Backtest Runner (`strategy_run.py`) executes specific trading strategies with predefined parameters defined in a JSON configuration file. Unlike the parameter optimization script, this runner focuses on testing and comparing specific strategy configurations with detailed market analysis and visualization.
|
||||
|
||||
## Features
|
||||
|
||||
- **JSON Configuration**: Define strategies and parameters in easy-to-edit JSON files
|
||||
- **Multiple Strategy Support**: Run multiple strategies in sequence with a single command
|
||||
- **All Strategy Types**: Support for MetaTrend, BBRS, and Random strategies
|
||||
- **Organized Results**: Automatic folder structure creation for each run
|
||||
- **Advanced Visualization**: Detailed plots showing portfolio performance and market context
|
||||
- **Full Market Data Integration**: Continuous price charts with buy/sell signals overlay
|
||||
- **Signal Export**: Complete buy/sell signal data exported to CSV files
|
||||
- **Real-time File Saving**: Individual strategy results saved immediately upon completion
|
||||
- **Comprehensive Analysis**: Multiple plot types for thorough performance analysis
|
||||
- **Detailed Results**: Comprehensive result reporting with CSV and JSON export
|
||||
- **Result Analysis**: Automatic summary generation and performance comparison
|
||||
- **Error Handling**: Robust error handling with detailed logging
|
||||
- **Flexible Configuration**: Support for different data files, date ranges, and trader parameters
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Run strategies from a configuration file
|
||||
python test/backtest/strategy_run.py --config configs/strategy/example_strategies.json
|
||||
|
||||
# Save results to a custom directory
|
||||
python test/backtest/strategy_run.py --config configs/strategy/my_strategies.json --results-dir my_results
|
||||
|
||||
# Enable verbose logging
|
||||
python test/backtest/strategy_run.py --config configs/strategy/example_strategies.json --verbose
|
||||
```
|
||||
|
||||
### Enhanced Analysis Features
|
||||
|
||||
Each run automatically generates:
|
||||
- **Organized folder structure** with timestamp for easy management
|
||||
- **Real-time file saving** - results saved immediately after each strategy completes
|
||||
- **Full market data visualization** - continuous price charts show complete market context
|
||||
- **Signal tracking** - all buy/sell decisions exported with precise timing and pricing
|
||||
- **Multi-layered analysis** - from individual trade details to portfolio-wide comparisons
|
||||
- **Professional plots** - high-resolution (300 DPI) charts suitable for reports and presentations
|
||||
|
||||
### Create Example Configuration
|
||||
|
||||
```bash
|
||||
# Create an example configuration file
|
||||
python test/backtest/strategy_run.py --create-example configs/example_strategies.json
|
||||
```
|
||||
|
||||
## Configuration File Format
|
||||
|
||||
The configuration file uses JSON format with two main sections:
|
||||
|
||||
### Backtest Settings
|
||||
|
||||
```json
|
||||
{
|
||||
"backtest_settings": {
|
||||
"data_file": "btcusd_1-min_data.csv",
|
||||
"data_dir": "data",
|
||||
"start_date": "2023-01-01",
|
||||
"end_date": "2023-01-31",
|
||||
"initial_usd": 10000
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Strategy Definitions
|
||||
|
||||
```json
|
||||
{
|
||||
"strategies": [
|
||||
{
|
||||
"name": "MetaTrend_Conservative",
|
||||
"type": "metatrend",
|
||||
"params": {
|
||||
"supertrend_periods": [12, 10, 11],
|
||||
"supertrend_multipliers": [3.0, 1.0, 2.0],
|
||||
"min_trend_agreement": 0.8,
|
||||
"timeframe": "15min"
|
||||
},
|
||||
"trader_params": {
|
||||
"stop_loss_pct": 0.02,
|
||||
"portfolio_percent_per_trade": 0.5
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Strategy Types
|
||||
|
||||
### MetaTrend Strategy
|
||||
|
||||
Parameters:
|
||||
- `supertrend_periods`: List of periods for multiple supertrend indicators
|
||||
- `supertrend_multipliers`: List of multipliers for supertrend indicators
|
||||
- `min_trend_agreement`: Minimum agreement threshold between indicators (0.0-1.0)
|
||||
- `timeframe`: Data aggregation timeframe ("1min", "5min", "15min", "30min", "1h")
|
||||
|
||||
### BBRS Strategy
|
||||
|
||||
Parameters:
|
||||
- `bb_length`: Bollinger Bands period
|
||||
- `bb_std`: Bollinger Bands standard deviation multiplier
|
||||
- `rsi_length`: RSI period
|
||||
- `rsi_overbought`: RSI overbought threshold
|
||||
- `rsi_oversold`: RSI oversold threshold
|
||||
- `timeframe`: Data aggregation timeframe
|
||||
|
||||
### Random Strategy
|
||||
|
||||
Parameters:
|
||||
- `signal_probability`: Probability of generating a signal (0.0-1.0)
|
||||
- `timeframe`: Data aggregation timeframe
|
||||
|
||||
## Trader Parameters
|
||||
|
||||
All strategies support these trader parameters:
|
||||
- `stop_loss_pct`: Stop loss percentage (e.g., 0.02 for 2%)
|
||||
- `portfolio_percent_per_trade`: Percentage of portfolio to use per trade (0.0-1.0)
|
||||
|
||||
## Results Organization
|
||||
|
||||
Each run creates an organized folder structure for easy navigation and analysis:
|
||||
|
||||
```
|
||||
results/
|
||||
└── [config_name]_[timestamp]/
|
||||
├── strategy_1_[strategy_name].json # Individual strategy data
|
||||
├── strategy_1_[strategy_name]_plot.png # 4-panel performance plot
|
||||
├── strategy_1_[strategy_name]_detailed_plot.png # 3-panel market analysis
|
||||
├── strategy_1_[strategy_name]_trades.csv # Trade details
|
||||
├── strategy_1_[strategy_name]_signals.csv # All buy/sell signals
|
||||
├── strategy_2_[strategy_name].* # Second strategy files
|
||||
├── ... # Additional strategies
|
||||
├── summary.csv # Strategy comparison table
|
||||
├── summary_plot.png # Multi-strategy comparison
|
||||
└── summary_*.json # Comprehensive results
|
||||
```
|
||||
|
||||
## Visualization Types
|
||||
|
||||
The runner generates three types of plots for comprehensive analysis:
|
||||
|
||||
### 1. Individual Strategy Plot (4-Panel)
|
||||
- **Equity Curve**: Portfolio value over time
|
||||
- **Trade P&L**: Individual trade profits/losses
|
||||
- **Drawdown**: Portfolio drawdown visualization
|
||||
- **Statistics**: Strategy performance summary
|
||||
|
||||
### 2. Detailed Market Analysis Plot (3-Panel)
|
||||
- **Portfolio Signals**: Portfolio value with buy/sell signal markers
|
||||
- **Market Price**: Full continuous market price with entry/exit points
|
||||
- **Combined View**: Dual-axis plot showing market vs portfolio performance
|
||||
|
||||
### 3. Summary Comparison Plot (4-Panel)
|
||||
- **Returns Comparison**: Total returns across all strategies
|
||||
- **Trade Counts**: Number of trades per strategy
|
||||
- **Risk vs Return**: Win rate vs maximum drawdown scatter plot
|
||||
- **Statistics Table**: Comprehensive performance metrics
|
||||
|
||||
## Output Files
|
||||
|
||||
The runner generates comprehensive output files organized in dedicated folders:
|
||||
|
||||
### Individual Strategy Files (per strategy)
|
||||
- `strategy_N_[name].json`: Complete strategy data and metadata
|
||||
- `strategy_N_[name]_plot.png`: 4-panel performance analysis plot
|
||||
- `strategy_N_[name]_detailed_plot.png`: 3-panel market context plot
|
||||
- `strategy_N_[name]_trades.csv`: Detailed trade information
|
||||
- `strategy_N_[name]_signals.csv`: All buy/sell signals with timestamps
|
||||
|
||||
### Summary Files (per run)
|
||||
- `summary.csv`: Strategy comparison table
|
||||
- `summary_plot.png`: Multi-strategy comparison visualization
|
||||
- `summary_*.json`: Comprehensive results and metadata
|
||||
|
||||
### Signal Data Format
|
||||
Each signal CSV contains:
|
||||
- `signal_id`: Unique signal identifier
|
||||
- `signal_type`: BUY or SELL
|
||||
- `time`: Signal timestamp
|
||||
- `price`: Execution price
|
||||
- `trade_id`: Associated trade number
|
||||
- `quantity`: Trade quantity
|
||||
- `value`: Trade value (quantity × price)
|
||||
- `strategy`: Strategy name
|
||||
|
||||
## Example Configurations
|
||||
|
||||
### Simple MetaTrend Test
|
||||
|
||||
```json
|
||||
{
|
||||
"backtest_settings": {
|
||||
"data_file": "btcusd_1-min_data.csv",
|
||||
"start_date": "2023-01-01",
|
||||
"end_date": "2023-01-07",
|
||||
"initial_usd": 10000
|
||||
},
|
||||
"strategies": [
|
||||
{
|
||||
"name": "MetaTrend_Test",
|
||||
"type": "metatrend",
|
||||
"params": {
|
||||
"supertrend_periods": [12, 10],
|
||||
"supertrend_multipliers": [3.0, 1.0],
|
||||
"min_trend_agreement": 0.5,
|
||||
"timeframe": "15min"
|
||||
},
|
||||
"trader_params": {
|
||||
"stop_loss_pct": 0.02,
|
||||
"portfolio_percent_per_trade": 0.5
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Multiple Strategy Comparison
|
||||
|
||||
```json
|
||||
{
|
||||
"backtest_settings": {
|
||||
"data_file": "btcusd_1-min_data.csv",
|
||||
"start_date": "2023-01-01",
|
||||
"end_date": "2023-01-31",
|
||||
"initial_usd": 10000
|
||||
},
|
||||
"strategies": [
|
||||
{
|
||||
"name": "Conservative_MetaTrend",
|
||||
"type": "metatrend",
|
||||
"params": {
|
||||
"supertrend_periods": [12, 10, 11],
|
||||
"supertrend_multipliers": [3.0, 1.0, 2.0],
|
||||
"min_trend_agreement": 0.8,
|
||||
"timeframe": "15min"
|
||||
},
|
||||
"trader_params": {
|
||||
"stop_loss_pct": 0.02,
|
||||
"portfolio_percent_per_trade": 0.5
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Aggressive_MetaTrend",
|
||||
"type": "metatrend",
|
||||
"params": {
|
||||
"supertrend_periods": [10, 8],
|
||||
"supertrend_multipliers": [2.0, 1.0],
|
||||
"min_trend_agreement": 0.5,
|
||||
"timeframe": "5min"
|
||||
},
|
||||
"trader_params": {
|
||||
"stop_loss_pct": 0.03,
|
||||
"portfolio_percent_per_trade": 0.8
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "BBRS_Baseline",
|
||||
"type": "bbrs",
|
||||
"params": {
|
||||
"bb_length": 20,
|
||||
"bb_std": 2.0,
|
||||
"rsi_length": 14,
|
||||
"rsi_overbought": 70,
|
||||
"rsi_oversold": 30,
|
||||
"timeframe": "15min"
|
||||
},
|
||||
"trader_params": {
|
||||
"stop_loss_pct": 0.025,
|
||||
"portfolio_percent_per_trade": 0.6
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Command Line Options
|
||||
|
||||
- `--config`: Path to JSON configuration file (required)
|
||||
- `--results-dir`: Directory for saving results (default: "results")
|
||||
- `--create-example`: Create example config file at specified path
|
||||
- `--verbose`: Enable verbose logging for debugging
|
||||
|
||||
## Error Handling
|
||||
|
||||
The runner includes comprehensive error handling:
|
||||
|
||||
- **Configuration Validation**: Validates JSON structure and required fields
|
||||
- **Data File Verification**: Checks if data files exist before running
|
||||
- **Strategy Creation**: Handles unknown strategy types gracefully
|
||||
- **Backtest Execution**: Captures and logs individual strategy failures
|
||||
- **Result Saving**: Ensures results are saved even if some strategies fail
|
||||
|
||||
## Integration
|
||||
|
||||
This runner integrates seamlessly with the existing IncrementalTrader framework:
|
||||
|
||||
- Uses the same `IncBacktester` and strategy classes
|
||||
- Compatible with all existing data formats
|
||||
- Leverages the same result saving utilities
|
||||
- Maintains consistency with optimization scripts
|
||||
|
||||
## Performance
|
||||
|
||||
- **Sequential Execution**: Strategies run one after another for clear logging
|
||||
- **Real-time Results**: Individual strategy files saved immediately upon completion
|
||||
- **Efficient Data Loading**: Market data loaded once per run for all visualizations
|
||||
- **Progress Tracking**: Clear progress indication for long-running backtests
|
||||
- **Detailed Timing**: Individual strategy execution times are tracked
|
||||
- **High-Quality Output**: Professional 300 DPI plots suitable for presentations
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Start Small**: Test with short date ranges first
|
||||
2. **Validate Data**: Ensure data files exist and cover the specified date range
|
||||
3. **Monitor Resources**: Watch memory usage for very long backtests
|
||||
4. **Save Configs**: Keep configuration files organized for reproducibility
|
||||
5. **Use Descriptive Names**: Give strategies clear, descriptive names
|
||||
6. **Test Incrementally**: Add strategies one by one when debugging
|
||||
7. **Leverage Visualizations**: Use detailed plots to understand market context and strategy behavior
|
||||
8. **Analyze Signals**: Review signal CSV files to understand strategy decision patterns
|
||||
9. **Compare Runs**: Use organized folder structure to compare different parameter sets
|
||||
10. **Monitor Execution**: Watch real-time progress as individual strategies complete
|
||||
1303
test/backtest/strategy_run.py
Normal file
1303
test/backtest/strategy_run.py
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user