parameter optimisation for strategies (can run a matrix of strategy with different parameters)
This commit is contained in:
parent
df19ef32db
commit
d8cc1a3192
333
test/strategy_optimisation/README.md
Normal file
333
test/strategy_optimisation/README.md
Normal file
@ -0,0 +1,333 @@
|
||||
# Strategy Parameter Optimization
|
||||
|
||||
This directory contains comprehensive tools for optimizing trading strategy parameters using the IncrementalTrader framework.
|
||||
|
||||
## Overview
|
||||
|
||||
The strategy optimization script provides:
|
||||
|
||||
- **Parallel Parameter Testing**: Uses multiple CPU cores for efficient optimization
|
||||
- **Configurable Supertrend Parameters**: Test different period and multiplier combinations
|
||||
- **Risk Management Optimization**: Optimize stop-loss and take-profit settings
|
||||
- **Multiple Timeframes**: Test strategies across different timeframes
|
||||
- **Comprehensive Results**: Detailed analysis and sensitivity reports
|
||||
- **Custom Parameter Ranges**: Support for custom parameter configurations
|
||||
|
||||
## Files
|
||||
|
||||
- `strategy_parameter_optimization.py` - Main optimization script
|
||||
- `custom_params_example.json` - Example custom parameter configuration
|
||||
- `README.md` - This documentation
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Basic Quick Test
|
||||
|
||||
Run a quick test with a smaller parameter space:
|
||||
|
||||
```bash
|
||||
python tasks/strategy_parameter_optimization.py --quick-test --create-sample-data
|
||||
```
|
||||
|
||||
This will:
|
||||
- Create sample data if it doesn't exist
|
||||
- Test a limited set of parameters for faster execution
|
||||
- Use the optimal number of CPU cores automatically
|
||||
|
||||
### 2. Full Optimization
|
||||
|
||||
Run comprehensive parameter optimization:
|
||||
|
||||
```bash
|
||||
python tasks/strategy_parameter_optimization.py \
|
||||
--data-file "your_data.csv" \
|
||||
--start-date "2024-01-01" \
|
||||
--end-date "2024-12-31" \
|
||||
--optimization-metric "sharpe_ratio"
|
||||
```
|
||||
|
||||
### 3. Custom Parameter Ranges
|
||||
|
||||
Create a custom parameter file and use it:
|
||||
|
||||
```bash
|
||||
python tasks/strategy_parameter_optimization.py \
|
||||
--custom-params "tasks/custom_params_example.json" \
|
||||
--max-workers 4
|
||||
```
|
||||
|
||||
## Parameter Configuration
|
||||
|
||||
### Strategy Parameters
|
||||
|
||||
The MetaTrend strategy now supports the following configurable parameters:
|
||||
|
||||
| Parameter | Type | Description | Example Values |
|
||||
|-----------|------|-------------|----------------|
|
||||
| `timeframe` | str | Analysis timeframe | `"5min"`, `"15min"`, `"30min"`, `"1h"` |
|
||||
| `supertrend_periods` | List[int] | Periods for Supertrend indicators | `[10, 12, 14]`, `[12, 15, 18]` |
|
||||
| `supertrend_multipliers` | List[float] | Multipliers for Supertrend indicators | `[2.0, 2.5, 3.0]`, `[1.5, 2.0, 2.5]` |
|
||||
| `min_trend_agreement` | float | Minimum agreement threshold (0.0-1.0) | `0.6`, `0.8`, `1.0` |
|
||||
|
||||
### Risk Management Parameters
|
||||
|
||||
| Parameter | Type | Description | Example Values |
|
||||
|-----------|------|-------------|----------------|
|
||||
| `stop_loss_pct` | float | Stop loss percentage | `0.02` (2%), `0.03` (3%) |
|
||||
| `take_profit_pct` | float | Take profit percentage | `0.04` (4%), `0.06` (6%) |
|
||||
|
||||
### Understanding min_trend_agreement
|
||||
|
||||
The `min_trend_agreement` parameter controls how many Supertrend indicators must agree:
|
||||
|
||||
- `1.0` - All indicators must agree (original behavior)
|
||||
- `0.8` - 80% of indicators must agree
|
||||
- `0.6` - 60% of indicators must agree
|
||||
- `0.5` - Simple majority must agree
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Example 1: Test Different Timeframes
|
||||
|
||||
```json
|
||||
{
|
||||
"timeframe": ["5min", "15min", "30min", "1h"],
|
||||
"min_trend_agreement": [1.0],
|
||||
"stop_loss_pct": [0.03],
|
||||
"take_profit_pct": [0.06]
|
||||
}
|
||||
```
|
||||
|
||||
### Example 2: Optimize Supertrend Parameters
|
||||
|
||||
```json
|
||||
{
|
||||
"timeframe": ["15min"],
|
||||
"supertrend_periods": [
|
||||
[8, 10, 12],
|
||||
[10, 12, 14],
|
||||
[12, 15, 18],
|
||||
[15, 20, 25]
|
||||
],
|
||||
"supertrend_multipliers": [
|
||||
[1.5, 2.0, 2.5],
|
||||
[2.0, 2.5, 3.0],
|
||||
[2.5, 3.0, 3.5]
|
||||
],
|
||||
"min_trend_agreement": [0.6, 0.8, 1.0]
|
||||
}
|
||||
```
|
||||
|
||||
### Example 3: Risk Management Focus
|
||||
|
||||
```json
|
||||
{
|
||||
"timeframe": ["15min"],
|
||||
"stop_loss_pct": [0.01, 0.015, 0.02, 0.025, 0.03, 0.04, 0.05],
|
||||
"take_profit_pct": [0.02, 0.03, 0.04, 0.05, 0.06, 0.08, 0.10]
|
||||
}
|
||||
```
|
||||
|
||||
## Command Line Options
|
||||
|
||||
```bash
|
||||
python tasks/strategy_parameter_optimization.py [OPTIONS]
|
||||
```
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `--data-file` | str | `sample_btc_1min.csv` | Data file for backtesting |
|
||||
| `--data-dir` | str | `data` | Directory containing data files |
|
||||
| `--results-dir` | str | `results` | Directory for saving results |
|
||||
| `--start-date` | str | `2024-01-01` | Start date (YYYY-MM-DD) |
|
||||
| `--end-date` | str | `2024-03-31` | End date (YYYY-MM-DD) |
|
||||
| `--initial-usd` | float | `10000` | Initial USD balance |
|
||||
| `--max-workers` | int | `auto` | Maximum parallel workers |
|
||||
| `--quick-test` | flag | `false` | Use smaller parameter space |
|
||||
| `--optimization-metric` | str | `sharpe_ratio` | Metric to optimize |
|
||||
| `--create-sample-data` | flag | `false` | Create sample data |
|
||||
| `--custom-params` | str | `none` | JSON file with custom ranges |
|
||||
|
||||
### Optimization Metrics
|
||||
|
||||
Available optimization metrics:
|
||||
|
||||
- `profit_ratio` - Total profit ratio
|
||||
- `sharpe_ratio` - Risk-adjusted return (recommended)
|
||||
- `sortino_ratio` - Downside risk-adjusted return
|
||||
- `calmar_ratio` - Return to max drawdown ratio
|
||||
|
||||
## Output Files
|
||||
|
||||
The script generates several output files in the results directory:
|
||||
|
||||
### 1. Summary Report
|
||||
`optimization_MetaTrendStrategy_sharpe_ratio_TIMESTAMP_summary.json`
|
||||
|
||||
Contains:
|
||||
- Best performing parameters
|
||||
- Summary statistics across all runs
|
||||
- Session information
|
||||
|
||||
### 2. Detailed Results
|
||||
`optimization_MetaTrendStrategy_sharpe_ratio_TIMESTAMP_detailed.csv`
|
||||
|
||||
Contains:
|
||||
- All parameter combinations tested
|
||||
- Performance metrics for each combination
|
||||
- Success/failure status
|
||||
|
||||
### 3. Individual Strategy Results
|
||||
`optimization_MetaTrendStrategy_sharpe_ratio_TIMESTAMP_strategy_N_metatrend.json`
|
||||
|
||||
Contains:
|
||||
- Detailed results for each parameter combination
|
||||
- Trade-by-trade breakdown
|
||||
- Strategy-specific metrics
|
||||
|
||||
### 4. Sensitivity Analysis
|
||||
`sensitivity_analysis_TIMESTAMP.json`
|
||||
|
||||
Contains:
|
||||
- Parameter correlation analysis
|
||||
- Performance impact of each parameter
|
||||
- Top performing configurations
|
||||
|
||||
### 5. Master Index
|
||||
`optimization_MetaTrendStrategy_sharpe_ratio_TIMESTAMP_index.json`
|
||||
|
||||
Contains:
|
||||
- File index for easy navigation
|
||||
- Quick statistics summary
|
||||
- Session metadata
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### System Resources
|
||||
|
||||
The script automatically detects your system capabilities and uses optimal worker counts:
|
||||
|
||||
- **CPU Cores**: Uses ~75% of available cores
|
||||
- **Memory**: Limits workers based on available RAM
|
||||
- **I/O**: Handles large result datasets efficiently
|
||||
|
||||
### Parameter Space Size
|
||||
|
||||
Be aware of exponential growth in parameter combinations:
|
||||
|
||||
- Quick test: ~48 combinations
|
||||
- Full test: ~5,000+ combinations
|
||||
- Custom ranges: Varies based on configuration
|
||||
|
||||
### Execution Time
|
||||
|
||||
Approximate execution times (varies by system and data size):
|
||||
|
||||
- Quick test: 2-10 minutes
|
||||
- Medium test: 30-60 minutes
|
||||
- Full test: 2-8 hours
|
||||
|
||||
## Data Requirements
|
||||
|
||||
### Data Format
|
||||
|
||||
The script expects CSV data with columns:
|
||||
- `timestamp` - Unix timestamp in milliseconds
|
||||
- `open` - Opening price
|
||||
- `high` - Highest price
|
||||
- `low` - Lowest price
|
||||
- `close` - Closing price
|
||||
- `volume` - Trading volume
|
||||
|
||||
### Sample Data
|
||||
|
||||
Use `--create-sample-data` to generate sample data for testing:
|
||||
|
||||
```bash
|
||||
python tasks/strategy_parameter_optimization.py --create-sample-data --quick-test
|
||||
```
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### 1. Distributed Optimization
|
||||
|
||||
For very large parameter spaces, consider running multiple instances:
|
||||
|
||||
```bash
|
||||
# Terminal 1 - Test timeframes 5min, 15min
|
||||
python tasks/strategy_parameter_optimization.py --custom-params timeframe_5_15.json
|
||||
|
||||
# Terminal 2 - Test timeframes 30min, 1h
|
||||
python tasks/strategy_parameter_optimization.py --custom-params timeframe_30_1h.json
|
||||
```
|
||||
|
||||
### 2. Walk-Forward Analysis
|
||||
|
||||
For more robust results, test across multiple time periods:
|
||||
|
||||
```bash
|
||||
# Q1 2024
|
||||
python tasks/strategy_parameter_optimization.py --start-date 2024-01-01 --end-date 2024-03-31
|
||||
|
||||
# Q2 2024
|
||||
python tasks/strategy_parameter_optimization.py --start-date 2024-04-01 --end-date 2024-06-30
|
||||
```
|
||||
|
||||
### 3. Custom Metrics
|
||||
|
||||
The script supports custom optimization metrics. See the documentation for implementation details.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Memory Errors**: Reduce `--max-workers` or use `--quick-test`
|
||||
2. **Data Not Found**: Use `--create-sample-data` or check file path
|
||||
3. **Import Errors**: Ensure IncrementalTrader is properly installed
|
||||
4. **Slow Performance**: Check system resources and reduce parameter space
|
||||
|
||||
### Logging
|
||||
|
||||
The script provides detailed logging. For debug information:
|
||||
|
||||
```python
|
||||
import logging
|
||||
logging.getLogger().setLevel(logging.DEBUG)
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Quick Start Example
|
||||
|
||||
```bash
|
||||
# Run quick optimization with sample data
|
||||
python tasks/strategy_parameter_optimization.py \
|
||||
--quick-test \
|
||||
--create-sample-data \
|
||||
--optimization-metric sharpe_ratio \
|
||||
--max-workers 4
|
||||
```
|
||||
|
||||
### Production Example
|
||||
|
||||
```bash
|
||||
# Run comprehensive optimization with real data
|
||||
python tasks/strategy_parameter_optimization.py \
|
||||
--data-file "BTCUSDT_1m_2024.csv" \
|
||||
--start-date "2024-01-01" \
|
||||
--end-date "2024-12-31" \
|
||||
--optimization-metric calmar_ratio \
|
||||
--custom-params "production_params.json"
|
||||
```
|
||||
|
||||
This comprehensive setup allows you to:
|
||||
|
||||
1. **Test the modified MetaTrend strategy** with configurable Supertrend parameters
|
||||
2. **Run parameter optimization in parallel** using system utilities from utils.py
|
||||
3. **Test multiple timeframes and risk management settings**
|
||||
4. **Get detailed analysis and sensitivity reports**
|
||||
5. **Use custom parameter ranges** for focused optimization
|
||||
|
||||
The script leverages the existing IncrementalTrader framework and integrates with the utilities you already have in place.
|
||||
18
test/strategy_optimisation/custom_params_example.json
Normal file
18
test/strategy_optimisation/custom_params_example.json
Normal file
@ -0,0 +1,18 @@
|
||||
{
|
||||
"timeframe": ["15min", "30min"],
|
||||
"supertrend_periods": [
|
||||
[8, 12, 16],
|
||||
[10, 15, 20],
|
||||
[12, 18, 24],
|
||||
[14, 21, 28]
|
||||
],
|
||||
"supertrend_multipliers": [
|
||||
[1.5, 2.0, 2.5],
|
||||
[2.0, 3.0, 4.0],
|
||||
[1.0, 2.0, 3.0],
|
||||
[1.0, 2.0, 3.0]
|
||||
],
|
||||
"min_trend_agreement": [0.6, 0.7, 0.8, 1.0, 1.0],
|
||||
"stop_loss_pct": [0.02, 0.03, 0.04, 0.05],
|
||||
"take_profit_pct": [0.00, 0.00, 0.00, 0.00]
|
||||
}
|
||||
466
test/strategy_optimisation/strategy_parameter_optimization.py
Normal file
466
test/strategy_optimisation/strategy_parameter_optimization.py
Normal file
@ -0,0 +1,466 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Strategy Parameter Optimization Script for IncrementalTrader
|
||||
|
||||
This script provides comprehensive parameter optimization for trading strategies,
|
||||
specifically designed for testing MetaTrend strategy with various configurations
|
||||
including supertrend parameters, timeframes, and risk management settings.
|
||||
|
||||
Features:
|
||||
- Parallel execution using multiple CPU cores
|
||||
- Configurable parameter grids for strategy and risk management
|
||||
- Comprehensive results analysis and reporting
|
||||
- Support for custom optimization metrics
|
||||
- Detailed logging and progress tracking
|
||||
- Individual strategy plotting and analysis
|
||||
|
||||
Usage:
|
||||
python tasks/strategy_parameter_optimization.py --help
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import logging
|
||||
import json
|
||||
import time
|
||||
import traceback
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Dict, List, Any, Optional, Tuple
|
||||
from concurrent.futures import ProcessPoolExecutor, as_completed
|
||||
from itertools import product
|
||||
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
from tqdm import tqdm
|
||||
|
||||
# Import plotting libraries for result visualization
|
||||
try:
|
||||
import matplotlib.pyplot as plt
|
||||
import seaborn as sns
|
||||
plt.style.use('default')
|
||||
PLOTTING_AVAILABLE = True
|
||||
except ImportError:
|
||||
PLOTTING_AVAILABLE = False
|
||||
|
||||
# Add project root to path
|
||||
project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, project_root)
|
||||
|
||||
# Import IncrementalTrader components
|
||||
from IncrementalTrader.backtester import IncBacktester, BacktestConfig
|
||||
from IncrementalTrader.backtester.utils import DataLoader, SystemUtils, ResultsSaver
|
||||
from IncrementalTrader.strategies import MetaTrendStrategy
|
||||
from IncrementalTrader.trader import IncTrader
|
||||
|
||||
# Set up logging
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
|
||||
handlers=[
|
||||
logging.StreamHandler(sys.stdout),
|
||||
logging.FileHandler('optimization.log')
|
||||
]
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Reduce verbosity for entry/exit logging
|
||||
logging.getLogger('IncrementalTrader.strategies').setLevel(logging.WARNING)
|
||||
logging.getLogger('IncrementalTrader.trader').setLevel(logging.WARNING)
|
||||
|
||||
|
||||
class StrategyOptimizer:
|
||||
"""
|
||||
Advanced parameter optimization for IncrementalTrader strategies.
|
||||
|
||||
This class provides comprehensive parameter optimization with parallel processing,
|
||||
sensitivity analysis, and detailed result reporting.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the StrategyOptimizer."""
|
||||
# Initialize utilities
|
||||
self.system_utils = SystemUtils()
|
||||
|
||||
# Session tracking
|
||||
self.session_start_time = datetime.now()
|
||||
self.optimization_results = []
|
||||
|
||||
logger.info(f"StrategyOptimizer initialized")
|
||||
logger.info(f"System info: {self.system_utils.get_system_info()}")
|
||||
|
||||
def generate_parameter_combinations(self, params_dict: Dict[str, List]) -> List[Dict[str, Dict]]:
|
||||
"""
|
||||
Generate all possible parameter combinations.
|
||||
|
||||
Args:
|
||||
params_dict: Dictionary with strategy_params and trader_params lists
|
||||
|
||||
Returns:
|
||||
List of parameter combinations
|
||||
"""
|
||||
strategy_params = params_dict.get('strategy_params', {})
|
||||
trader_params = params_dict.get('trader_params', {})
|
||||
|
||||
# Generate all combinations
|
||||
combinations = []
|
||||
|
||||
# Get all strategy parameter combinations
|
||||
strategy_keys = list(strategy_params.keys())
|
||||
strategy_values = list(strategy_params.values())
|
||||
|
||||
trader_keys = list(trader_params.keys())
|
||||
trader_values = list(trader_params.values())
|
||||
|
||||
for strategy_combo in product(*strategy_values):
|
||||
strategy_dict = dict(zip(strategy_keys, strategy_combo))
|
||||
|
||||
for trader_combo in product(*trader_values):
|
||||
trader_dict = dict(zip(trader_keys, trader_combo))
|
||||
|
||||
combinations.append({
|
||||
'strategy_params': strategy_dict,
|
||||
'trader_params': trader_dict
|
||||
})
|
||||
|
||||
return combinations
|
||||
|
||||
def get_quick_test_params(self) -> Dict[str, List]:
|
||||
"""
|
||||
Get parameters for quick testing (smaller parameter space for faster execution).
|
||||
|
||||
Returns:
|
||||
Dictionary with parameter ranges for quick testing
|
||||
"""
|
||||
return {
|
||||
"strategy_params": {
|
||||
"supertrend_periods": [[12, 10], [10, 8]], # Only 2 period combinations
|
||||
"supertrend_multipliers": [[3.0, 1.0], [2.0, 1.5]], # Only 2 multiplier combinations
|
||||
"min_trend_agreement": [0.5, 0.8], # Only 2 agreement levels
|
||||
"timeframe": ["5min", "15min"] # Only 2 timeframes
|
||||
},
|
||||
"trader_params": {
|
||||
"stop_loss_pct": [0.02, 0.05], # Only 2 stop loss levels
|
||||
"portfolio_percent_per_trade": [0.8, 0.9] # Only 2 position sizes
|
||||
}
|
||||
}
|
||||
|
||||
def get_comprehensive_params(self) -> Dict[str, List]:
|
||||
"""
|
||||
Get parameters for comprehensive optimization (larger parameter space).
|
||||
|
||||
Returns:
|
||||
Dictionary with parameter ranges for comprehensive optimization
|
||||
"""
|
||||
return {
|
||||
"strategy_params": {
|
||||
"supertrend_periods": [
|
||||
[12, 10, 11], [10, 8, 9], [14, 12, 13],
|
||||
[16, 14, 15], [20, 18, 19]
|
||||
],
|
||||
"supertrend_multipliers": [
|
||||
[3.0, 1.0, 2.0], [2.5, 1.5, 2.0], [3.5, 2.0, 2.5],
|
||||
[2.0, 1.0, 1.5], [4.0, 2.5, 3.0]
|
||||
],
|
||||
"min_trend_agreement": [0.33, 0.5, 0.67, 0.8, 1.0],
|
||||
"timeframe": ["1min", "5min", "15min", "30min", "1h"]
|
||||
},
|
||||
"trader_params": {
|
||||
"stop_loss_pct": [0.01, 0.015, 0.02, 0.025, 0.03, 0.04, 0.05],
|
||||
"portfolio_percent_per_trade": [0.1, 0.2, 0.3, 0.5, 0.8, 0.9, 1.0]
|
||||
}
|
||||
}
|
||||
|
||||
def run_single_backtest(self, params: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Run a single backtest with given parameters.
|
||||
|
||||
Args:
|
||||
params: Dictionary containing all parameters for the backtest
|
||||
|
||||
Returns:
|
||||
Dictionary with backtest results
|
||||
"""
|
||||
try:
|
||||
start_time = time.time()
|
||||
|
||||
# Extract parameters
|
||||
strategy_params = params['strategy_params']
|
||||
trader_params = params['trader_params']
|
||||
data_file = params['data_file']
|
||||
start_date = params['start_date']
|
||||
end_date = params['end_date']
|
||||
data_dir = params['data_dir']
|
||||
|
||||
# Create strategy name for identification
|
||||
strategy_name = f"MetaTrend_TF{strategy_params['timeframe']}_ST{len(strategy_params['supertrend_periods'])}_SL{trader_params['stop_loss_pct']}_POS{trader_params['portfolio_percent_per_trade']}"
|
||||
|
||||
# Create strategy
|
||||
strategy = MetaTrendStrategy(name="metatrend", params=strategy_params)
|
||||
|
||||
# Create backtest config (only with BacktestConfig-supported parameters)
|
||||
config = BacktestConfig(
|
||||
data_file=data_file,
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
initial_usd=10000,
|
||||
data_dir=data_dir,
|
||||
stop_loss_pct=trader_params.get('stop_loss_pct', 0.0)
|
||||
)
|
||||
|
||||
# Create backtester
|
||||
backtester = IncBacktester(config)
|
||||
|
||||
# Run backtest with trader-specific parameters
|
||||
results = backtester.run_single_strategy(strategy, trader_params)
|
||||
|
||||
# Calculate additional metrics
|
||||
end_time = time.time()
|
||||
backtest_duration = end_time - start_time
|
||||
|
||||
# Format results
|
||||
formatted_results = {
|
||||
"success": True,
|
||||
"strategy_name": strategy_name,
|
||||
"strategy_params": strategy_params,
|
||||
"trader_params": trader_params,
|
||||
"initial_usd": results["initial_usd"],
|
||||
"final_usd": results["final_usd"],
|
||||
"profit_ratio": results["profit_ratio"],
|
||||
"n_trades": results["n_trades"],
|
||||
"win_rate": results["win_rate"],
|
||||
"max_drawdown": results["max_drawdown"],
|
||||
"avg_trade": results["avg_trade"],
|
||||
"total_fees_usd": results["total_fees_usd"],
|
||||
"backtest_duration_seconds": backtest_duration,
|
||||
"data_points_processed": results.get("data_points", 0),
|
||||
"warmup_complete": results.get("warmup_complete", False),
|
||||
"trades": results.get("trades", [])
|
||||
}
|
||||
|
||||
return formatted_results
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error in backtest {params.get('strategy_params', {}).get('timeframe', 'unknown')}: {e}")
|
||||
return {
|
||||
"success": False,
|
||||
"error": str(e),
|
||||
"strategy_name": strategy_name if 'strategy_name' in locals() else "Unknown",
|
||||
"strategy_params": params.get('strategy_params', {}),
|
||||
"trader_params": params.get('trader_params', {}),
|
||||
"traceback": traceback.format_exc()
|
||||
}
|
||||
|
||||
def optimize_parallel(self, params_dict: Dict[str, List],
|
||||
data_file: str, start_date: str, end_date: str,
|
||||
data_dir: str = "data", max_workers: Optional[int] = None) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Run parameter optimization using parallel processing with progress tracking.
|
||||
|
||||
Args:
|
||||
params_dict: Dictionary with parameter ranges
|
||||
data_file: Data file for backtesting
|
||||
start_date: Start date for backtesting
|
||||
end_date: End date for backtesting
|
||||
data_dir: Directory containing data files
|
||||
max_workers: Maximum number of worker processes
|
||||
|
||||
Returns:
|
||||
List of backtest results
|
||||
"""
|
||||
# Generate parameter combinations
|
||||
param_combinations = self.generate_parameter_combinations(params_dict)
|
||||
total_combinations = len(param_combinations)
|
||||
|
||||
logger.info(f"Starting optimization with {total_combinations} parameter combinations")
|
||||
logger.info(f"Using {max_workers or self.system_utils.get_optimal_workers()} worker processes")
|
||||
|
||||
# Prepare jobs
|
||||
jobs = []
|
||||
for combo in param_combinations:
|
||||
job_params = {
|
||||
'strategy_params': combo['strategy_params'],
|
||||
'trader_params': combo['trader_params'],
|
||||
'data_file': data_file,
|
||||
'start_date': start_date,
|
||||
'end_date': end_date,
|
||||
'data_dir': data_dir
|
||||
}
|
||||
jobs.append(job_params)
|
||||
|
||||
# Run parallel optimization with progress bar
|
||||
results = []
|
||||
failed_jobs = []
|
||||
|
||||
max_workers = max_workers or self.system_utils.get_optimal_workers()
|
||||
|
||||
with ProcessPoolExecutor(max_workers=max_workers) as executor:
|
||||
# Submit all jobs
|
||||
future_to_params = {executor.submit(self.run_single_backtest, job): job for job in jobs}
|
||||
|
||||
# Process results with progress bar
|
||||
with tqdm(total=total_combinations, desc="Optimizing strategies", unit="strategy") as pbar:
|
||||
for future in as_completed(future_to_params):
|
||||
try:
|
||||
result = future.result(timeout=300) # 5 minute timeout per job
|
||||
results.append(result)
|
||||
|
||||
if result['success']:
|
||||
pbar.set_postfix({
|
||||
'Success': f"{len([r for r in results if r['success']])}/{len(results)}",
|
||||
'Best Profit': f"{max([r.get('profit_ratio', 0) for r in results if r['success']], default=0):.1%}"
|
||||
})
|
||||
else:
|
||||
failed_jobs.append(future_to_params[future])
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Job failed with exception: {e}")
|
||||
failed_jobs.append(future_to_params[future])
|
||||
results.append({
|
||||
"success": False,
|
||||
"error": f"Job exception: {e}",
|
||||
"strategy_name": "Failed",
|
||||
"strategy_params": future_to_params[future].get('strategy_params', {}),
|
||||
"trader_params": future_to_params[future].get('trader_params', {})
|
||||
})
|
||||
|
||||
pbar.update(1)
|
||||
|
||||
# Log summary
|
||||
successful_results = [r for r in results if r['success']]
|
||||
logger.info(f"Optimization completed: {len(successful_results)}/{total_combinations} successful")
|
||||
|
||||
if failed_jobs:
|
||||
logger.warning(f"{len(failed_jobs)} jobs failed")
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def main():
|
||||
"""Main function for running parameter optimization."""
|
||||
parser = argparse.ArgumentParser(description="Strategy Parameter Optimization")
|
||||
|
||||
parser.add_argument("--data-file", type=str, default="btcusd_1-min_data.csv",
|
||||
help="Data file for backtesting")
|
||||
parser.add_argument("--data-dir", type=str, default="data",
|
||||
help="Directory containing data files")
|
||||
parser.add_argument("--results-dir", type=str, default="results",
|
||||
help="Directory for saving results")
|
||||
parser.add_argument("--start-date", type=str, default="2023-01-01",
|
||||
help="Start date for backtesting (YYYY-MM-DD)")
|
||||
parser.add_argument("--end-date", type=str, default="2023-01-31",
|
||||
help="End date for backtesting (YYYY-MM-DD)")
|
||||
parser.add_argument("--max-workers", type=int, default=None,
|
||||
help="Maximum number of worker processes")
|
||||
parser.add_argument("--quick-test", action="store_true",
|
||||
help="Run quick test with smaller parameter space")
|
||||
parser.add_argument("--custom-params", type=str, default=None,
|
||||
help="Path to custom parameter configuration JSON file")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Adjust dates for quick test - use only 3 days for very fast testing
|
||||
if args.quick_test:
|
||||
args.start_date = "2023-01-01"
|
||||
args.end_date = "2023-01-03" # Only 3 days for quick test
|
||||
logger.info("Quick test mode: Using shortened time period (2023-01-01 to 2023-01-03)")
|
||||
|
||||
# Create optimizer
|
||||
optimizer = StrategyOptimizer()
|
||||
|
||||
# Determine parameter configuration
|
||||
if args.custom_params:
|
||||
# Load custom parameters from JSON file
|
||||
if not os.path.exists(args.custom_params):
|
||||
logger.error(f"Custom parameter file not found: {args.custom_params}")
|
||||
return
|
||||
|
||||
with open(args.custom_params, 'r') as f:
|
||||
params_dict = json.load(f)
|
||||
logger.info(f"Using custom parameters from: {args.custom_params}")
|
||||
elif args.quick_test:
|
||||
# Quick test parameters
|
||||
params_dict = optimizer.get_quick_test_params()
|
||||
logger.info("Using quick test parameter configuration")
|
||||
else:
|
||||
# Comprehensive optimization parameters
|
||||
params_dict = optimizer.get_comprehensive_params()
|
||||
logger.info("Using comprehensive optimization parameter configuration")
|
||||
|
||||
# Log optimization details
|
||||
total_combinations = len(optimizer.generate_parameter_combinations(params_dict))
|
||||
logger.info(f"Total parameter combinations: {total_combinations}")
|
||||
logger.info(f"Data file: {args.data_file}")
|
||||
logger.info(f"Date range: {args.start_date} to {args.end_date}")
|
||||
logger.info(f"Results directory: {args.results_dir}")
|
||||
|
||||
# Check if data file exists
|
||||
data_path = os.path.join(args.data_dir, args.data_file)
|
||||
if not os.path.exists(data_path):
|
||||
logger.error(f"Data file not found: {data_path}")
|
||||
return
|
||||
|
||||
# Create results directory
|
||||
os.makedirs(args.results_dir, exist_ok=True)
|
||||
|
||||
try:
|
||||
# Run optimization
|
||||
session_start_time = datetime.now()
|
||||
logger.info("Starting parameter optimization...")
|
||||
|
||||
results = optimizer.optimize_parallel(
|
||||
params_dict=params_dict,
|
||||
data_file=args.data_file,
|
||||
start_date=args.start_date,
|
||||
end_date=args.end_date,
|
||||
data_dir=args.data_dir,
|
||||
max_workers=args.max_workers
|
||||
)
|
||||
|
||||
# Save results
|
||||
saver = ResultsSaver(args.results_dir)
|
||||
|
||||
# Generate base filename
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
test_type = "quick_test" if args.quick_test else "comprehensive"
|
||||
base_filename = f"metatrend_optimization_{test_type}"
|
||||
|
||||
# Save comprehensive results
|
||||
saver.save_comprehensive_results(
|
||||
results=results,
|
||||
base_filename=base_filename,
|
||||
session_start_time=session_start_time
|
||||
)
|
||||
|
||||
# Calculate and display summary statistics
|
||||
successful_results = [r for r in results if r['success']]
|
||||
|
||||
if successful_results:
|
||||
# Sort by profit ratio
|
||||
sorted_results = sorted(successful_results, key=lambda x: x['profit_ratio'], reverse=True)
|
||||
|
||||
print(f"\nOptimization Summary:")
|
||||
print(f" Successful runs: {len(successful_results)}/{len(results)}")
|
||||
print(f" Total duration: {(datetime.now() - session_start_time).total_seconds():.1f} seconds")
|
||||
|
||||
print(f"\nTop 5 Strategies:")
|
||||
for i, result in enumerate(sorted_results[:5], 1):
|
||||
print(f" {i}. {result['strategy_name']}")
|
||||
print(f" Profit: {result['profit_ratio']:.1%} (${result['final_usd']:.2f})")
|
||||
print(f" Trades: {result['n_trades']} | Win Rate: {result['win_rate']:.1%}")
|
||||
print(f" Max DD: {result['max_drawdown']:.1%}")
|
||||
else:
|
||||
print(f"\nNo successful optimization runs completed")
|
||||
logger.error("All optimization runs failed")
|
||||
|
||||
print(f"\nFull results saved to: {args.results_dir}/")
|
||||
|
||||
except KeyboardInterrupt:
|
||||
logger.info("Optimization interrupted by user")
|
||||
except Exception as e:
|
||||
logger.error(f"Optimization failed: {e}")
|
||||
traceback.print_exc()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
x
Reference in New Issue
Block a user