Cycles/IncrementalTrader/docs/backtesting.md

# Backtesting Guide

This guide explains how to use the IncrementalTrader backtesting framework for comprehensive strategy testing and optimization.

## Overview

The IncrementalTrader backtesting framework provides:
- **Single Strategy Testing**: Test individual strategies with detailed metrics
- **Parameter Optimization**: Systematic parameter sweeps with parallel execution
- **Performance Analysis**: Comprehensive performance metrics and reporting
- **Data Management**: Flexible data loading and validation
- **Result Export**: Multiple output formats for analysis

## Quick Start

### Basic Backtesting

```python
from IncrementalTrader import IncBacktester, BacktestConfig, MetaTrendStrategy

# Configure backtest
config = BacktestConfig(
    initial_usd=10000,
    stop_loss_pct=0.03,
    take_profit_pct=0.06,
    start_date="2024-01-01",
    end_date="2024-12-31"
)

# Create backtester
backtester = IncBacktester()

# Run single strategy test
results = backtester.run_single_strategy(
    strategy_class=MetaTrendStrategy,
    strategy_params={"timeframe": "15min"},
    config=config,
    data_file="data/BTCUSDT_1m.csv"
)

# Print results
print(f"Total Return: {results['performance_metrics']['total_return_pct']:.2f}%")
print(f"Sharpe Ratio: {results['performance_metrics']['sharpe_ratio']:.2f}")
print(f"Max Drawdown: {results['performance_metrics']['max_drawdown_pct']:.2f}%")
```

## Configuration

### BacktestConfig

The main configuration class for backtesting parameters.

```python
from IncrementalTrader import BacktestConfig

config = BacktestConfig(
    # Portfolio settings
    initial_usd=10000,           # Starting capital
    
    # Risk management
    stop_loss_pct=0.03,          # 3% stop loss
    take_profit_pct=0.06,        # 6% take profit
    
    # Time range
    start_date="2024-01-01",     # Start date (YYYY-MM-DD)
    end_date="2024-12-31",       # End date (YYYY-MM-DD)
    
    # Trading settings
    fee_pct=0.001,               # 0.1% trading fee
    slippage_pct=0.0005,         # 0.05% slippage
    
    # Output settings
    output_dir="backtest_results",
    save_trades=True,
    save_portfolio_history=True,
    
    # Performance settings
    risk_free_rate=0.02          # 2% annual risk-free rate
)
```

**Parameters:**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `initial_usd` | float | 10000 | Starting capital in USD |
| `stop_loss_pct` | float | None | Stop loss percentage (0.03 = 3%) |
| `take_profit_pct` | float | None | Take profit percentage (0.06 = 6%) |
| `start_date` | str | None | Start date in YYYY-MM-DD format |
| `end_date` | str | None | End date in YYYY-MM-DD format |
| `fee_pct` | float | 0.001 | Trading fee percentage |
| `slippage_pct` | float | 0.0005 | Slippage percentage |
| `output_dir` | str | "backtest_results" | Output directory |
| `save_trades` | bool | True | Save individual trades |
| `save_portfolio_history` | bool | True | Save portfolio history |
| `risk_free_rate` | float | 0.02 | Annual risk-free rate for Sharpe ratio |

### OptimizationConfig

Configuration for parameter optimization.

```python
from IncrementalTrader import OptimizationConfig

# Define parameter ranges
param_ranges = {
    "supertrend_periods": [[10, 20, 30], [15, 25, 35], [20, 30, 40]],
    "supertrend_multipliers": [[2.0, 3.0, 4.0], [1.5, 2.5, 3.5]],
    "min_trend_agreement": [0.5, 0.6, 0.7, 0.8]
}

# Create optimization config
opt_config = OptimizationConfig(
    base_config=config,          # Base BacktestConfig
    param_ranges=param_ranges,   # Parameter combinations to test
    max_workers=4,               # Number of parallel workers
    optimization_metric="sharpe_ratio",  # Metric to optimize
    save_all_results=True        # Save all parameter combinations
)
```

## Single Strategy Testing

### Basic Usage

```python
# Test MetaTrend strategy
results = backtester.run_single_strategy(
    strategy_class=MetaTrendStrategy,
    strategy_params={
        "timeframe": "15min",
        "supertrend_periods": [10, 20, 30],
        "supertrend_multipliers": [2.0, 3.0, 4.0],
        "min_trend_agreement": 0.6
    },
    config=config,
    data_file="data/BTCUSDT_1m.csv"
)
```

### Results Structure

```python
# Access different result components
performance = results['performance_metrics']
trades = results['trades']
portfolio_history = results['portfolio_history']
config_used = results['config']

# Performance metrics
print(f"Total Trades: {performance['total_trades']}")
print(f"Win Rate: {performance['win_rate']:.2f}%")
print(f"Profit Factor: {performance['profit_factor']:.2f}")
print(f"Sharpe Ratio: {performance['sharpe_ratio']:.2f}")
print(f"Sortino Ratio: {performance['sortino_ratio']:.2f}")
print(f"Max Drawdown: {performance['max_drawdown_pct']:.2f}%")
print(f"Calmar Ratio: {performance['calmar_ratio']:.2f}")

# Trade analysis
winning_trades = [t for t in trades if t['pnl'] > 0]
losing_trades = [t for t in trades if t['pnl'] < 0]

print(f"Average Win: ${sum(t['pnl'] for t in winning_trades) / len(winning_trades):.2f}")
print(f"Average Loss: ${sum(t['pnl'] for t in losing_trades) / len(losing_trades):.2f}")
```

### Performance Metrics

The backtester calculates comprehensive performance metrics:

| Metric | Description | Formula |
|--------|-------------|---------|
| Total Return | Overall portfolio return | (Final Value - Initial Value) / Initial Value |
| Annualized Return | Yearly return rate | (Total Return + 1)^(365/days) - 1 |
| Volatility | Annualized standard deviation | std(daily_returns) × √365 |
| Sharpe Ratio | Risk-adjusted return | (Return - Risk Free Rate) / Volatility |
| Sortino Ratio | Downside risk-adjusted return | (Return - Risk Free Rate) / Downside Deviation |
| Max Drawdown | Maximum peak-to-trough decline | max((Peak - Trough) / Peak) |
| Calmar Ratio | Return to max drawdown ratio | Annualized Return / Max Drawdown |
| Win Rate | Percentage of profitable trades | Winning Trades / Total Trades |
| Profit Factor | Ratio of gross profit to loss | Gross Profit / Gross Loss |

## Parameter Optimization

### Basic Optimization

```python
# Define parameter ranges to test
param_ranges = {
    "timeframe": ["5min", "15min", "30min"],
    "supertrend_periods": [[10, 20, 30], [15, 25, 35]],
    "min_trend_agreement": [0.5, 0.6, 0.7]
}

# Create optimization config
opt_config = OptimizationConfig(
    base_config=config,
    param_ranges=param_ranges,
    max_workers=4,
    optimization_metric="sharpe_ratio"
)

# Run optimization
optimization_results = backtester.optimize_strategy(
    strategy_class=MetaTrendStrategy,
    optimization_config=opt_config,
    data_file="data/BTCUSDT_1m.csv"
)

# Get best parameters
best_params = optimization_results['best_params']
best_performance = optimization_results['best_performance']
all_results = optimization_results['all_results']

print(f"Best Parameters: {best_params}")
print(f"Best Sharpe Ratio: {best_performance['sharpe_ratio']:.2f}")
```

### Advanced Optimization

```python
# More complex parameter optimization
param_ranges = {
    # Strategy parameters
    "timeframe": ["5min", "15min", "30min"],
    "supertrend_periods": [
        [10, 20, 30], [15, 25, 35], [20, 30, 40],
        [10, 15, 20], [25, 35, 45]
    ],
    "supertrend_multipliers": [
        [2.0, 3.0, 4.0], [1.5, 2.5, 3.5], [2.5, 3.5, 4.5]
    ],
    "min_trend_agreement": [0.4, 0.5, 0.6, 0.7, 0.8],
    
    # Risk management (will override config values)
    "stop_loss_pct": [0.02, 0.03, 0.04, 0.05],
    "take_profit_pct": [0.04, 0.06, 0.08, 0.10]
}

# Optimization with custom metric
def custom_metric(performance):
    """Custom optimization metric combining return and drawdown."""
    return performance['total_return_pct'] / max(performance['max_drawdown_pct'], 1.0)

opt_config = OptimizationConfig(
    base_config=config,
    param_ranges=param_ranges,
    max_workers=8,
    optimization_metric=custom_metric,  # Custom function
    save_all_results=True
)

results = backtester.optimize_strategy(
    strategy_class=MetaTrendStrategy,
    optimization_config=opt_config,
    data_file="data/BTCUSDT_1m.csv"
)
```

### Optimization Metrics

You can optimize for different metrics:

```python
# Built-in metrics (string names)
optimization_metrics = [
    "total_return_pct",
    "sharpe_ratio",
    "sortino_ratio",
    "calmar_ratio",
    "profit_factor",
    "win_rate"
]

# Custom metric function
def risk_adjusted_return(performance):
    return (performance['total_return_pct'] / 
            max(performance['max_drawdown_pct'], 1.0))

opt_config = OptimizationConfig(
    base_config=config,
    param_ranges=param_ranges,
    optimization_metric=risk_adjusted_return  # Custom function
)
```

## Data Management

### Data Format

The backtester expects CSV data with the following columns:

```csv
timestamp,open,high,low,close,volume
1640995200000,46222.5,46850.0,46150.0,46800.0,1250.5
1640995260000,46800.0,47000.0,46750.0,46950.0,980.2
...
```

**Required Columns:**
- `timestamp`: Unix timestamp in milliseconds
- `open`: Opening price
- `high`: Highest price
- `low`: Lowest price
- `close`: Closing price
- `volume`: Trading volume

### Data Loading

```python
# The backtester automatically loads and validates data
results = backtester.run_single_strategy(
    strategy_class=MetaTrendStrategy,
    strategy_params={"timeframe": "15min"},
    config=config,
    data_file="data/BTCUSDT_1m.csv"  # Automatically loaded and validated
)

# Data is automatically filtered by start_date and end_date from config
```

### Data Validation

The backtester performs automatic data validation:

```python
# Validation checks performed:
# 1. Required columns present
# 2. No missing values
# 3. Timestamps in ascending order
# 4. Price consistency (high >= low, etc.)
# 5. Date range filtering
# 6. Data type validation
```

## Advanced Features

### Custom Strategy Testing

```python
# Test your custom strategy
class MyCustomStrategy(IncStrategyBase):
    def __init__(self, name: str, params: dict = None):
        super().__init__(name, params)
        # Your strategy implementation
    
    def _process_aggregated_data(self, timestamp: int, ohlcv: tuple):
        # Your strategy logic
        return IncStrategySignal.HOLD()

# Test custom strategy
results = backtester.run_single_strategy(
    strategy_class=MyCustomStrategy,
    strategy_params={"timeframe": "15min", "custom_param": 42},
    config=config,
    data_file="data/BTCUSDT_1m.csv"
)
```

### Multiple Strategy Comparison

```python
# Compare different strategies
strategies_to_test = [
    (MetaTrendStrategy, {"timeframe": "15min"}),
    (BBRSStrategy, {"timeframe": "15min"}),
    (RandomStrategy, {"timeframe": "15min"})
]

comparison_results = {}

for strategy_class, params in strategies_to_test:
    results = backtester.run_single_strategy(
        strategy_class=strategy_class,
        strategy_params=params,
        config=config,
        data_file="data/BTCUSDT_1m.csv"
    )
    
    strategy_name = strategy_class.__name__
    comparison_results[strategy_name] = results['performance_metrics']

# Compare results
for name, performance in comparison_results.items():
    print(f"{name}:")
    print(f"  Return: {performance['total_return_pct']:.2f}%")
    print(f"  Sharpe: {performance['sharpe_ratio']:.2f}")
    print(f"  Max DD: {performance['max_drawdown_pct']:.2f}%")
```

### Walk-Forward Analysis

```python
# Implement walk-forward analysis
import pandas as pd
from datetime import datetime, timedelta

def walk_forward_analysis(strategy_class, params, data_file, 
                         train_months=6, test_months=1):
    """Perform walk-forward analysis."""
    
    # Load full dataset to determine date range
    data = pd.read_csv(data_file)
    data['timestamp'] = pd.to_datetime(data['timestamp'], unit='ms')
    
    start_date = data['timestamp'].min()
    end_date = data['timestamp'].max()
    
    results = []
    current_date = start_date
    
    while current_date + timedelta(days=30*(train_months + test_months)) <= end_date:
        # Define train and test periods
        train_start = current_date
        train_end = current_date + timedelta(days=30*train_months)
        test_start = train_end
        test_end = test_start + timedelta(days=30*test_months)
        
        # Optimize on training data
        train_config = BacktestConfig(
            initial_usd=10000,
            start_date=train_start.strftime("%Y-%m-%d"),
            end_date=train_end.strftime("%Y-%m-%d")
        )
        
        # Simple parameter optimization (you can expand this)
        best_params = params  # In practice, optimize here
        
        # Test on out-of-sample data
        test_config = BacktestConfig(
            initial_usd=10000,
            start_date=test_start.strftime("%Y-%m-%d"),
            end_date=test_end.strftime("%Y-%m-%d")
        )
        
        test_results = backtester.run_single_strategy(
            strategy_class=strategy_class,
            strategy_params=best_params,
            config=test_config,
            data_file=data_file
        )
        
        results.append({
            'test_start': test_start,
            'test_end': test_end,
            'performance': test_results['performance_metrics']
        })
        
        # Move to next period
        current_date = test_start
    
    return results

# Run walk-forward analysis
wf_results = walk_forward_analysis(
    MetaTrendStrategy,
    {"timeframe": "15min"},
    "data/BTCUSDT_1m.csv"
)

# Analyze walk-forward results
total_returns = [r['performance']['total_return_pct'] for r in wf_results]
avg_return = sum(total_returns) / len(total_returns)
print(f"Average out-of-sample return: {avg_return:.2f}%")
```

## Result Analysis

### Detailed Performance Analysis

```python
# Comprehensive result analysis
def analyze_results(results):
    """Analyze backtest results in detail."""
    
    performance = results['performance_metrics']
    trades = results['trades']
    portfolio_history = results['portfolio_history']
    
    print("=== PERFORMANCE SUMMARY ===")
    print(f"Total Return: {performance['total_return_pct']:.2f}%")
    print(f"Annualized Return: {performance['annualized_return_pct']:.2f}%")
    print(f"Volatility: {performance['volatility_pct']:.2f}%")
    print(f"Sharpe Ratio: {performance['sharpe_ratio']:.2f}")
    print(f"Sortino Ratio: {performance['sortino_ratio']:.2f}")
    print(f"Max Drawdown: {performance['max_drawdown_pct']:.2f}%")
    print(f"Calmar Ratio: {performance['calmar_ratio']:.2f}")
    
    print("\n=== TRADING STATISTICS ===")
    print(f"Total Trades: {performance['total_trades']}")
    print(f"Win Rate: {performance['win_rate']:.2f}%")
    print(f"Profit Factor: {performance['profit_factor']:.2f}")
    
    # Trade analysis
    if trades:
        winning_trades = [t for t in trades if t['pnl'] > 0]
        losing_trades = [t for t in trades if t['pnl'] < 0]
        
        if winning_trades:
            avg_win = sum(t['pnl'] for t in winning_trades) / len(winning_trades)
            max_win = max(t['pnl'] for t in winning_trades)
            print(f"Average Win: ${avg_win:.2f}")
            print(f"Largest Win: ${max_win:.2f}")
        
        if losing_trades:
            avg_loss = sum(t['pnl'] for t in losing_trades) / len(losing_trades)
            max_loss = min(t['pnl'] for t in losing_trades)
            print(f"Average Loss: ${avg_loss:.2f}")
            print(f"Largest Loss: ${max_loss:.2f}")
    
    print("\n=== RISK METRICS ===")
    print(f"Value at Risk (95%): {performance.get('var_95', 'N/A')}")
    print(f"Expected Shortfall (95%): {performance.get('es_95', 'N/A')}")
    
    return performance

# Analyze results
performance = analyze_results(results)
```

### Export Results

```python
# Export results to different formats
def export_results(results, output_dir="backtest_results"):
    """Export backtest results to files."""
    
    import os
    import json
    import pandas as pd
    
    os.makedirs(output_dir, exist_ok=True)
    
    # Export performance metrics
    with open(f"{output_dir}/performance_metrics.json", 'w') as f:
        json.dump(results['performance_metrics'], f, indent=2)
    
    # Export trades
    if results['trades']:
        trades_df = pd.DataFrame(results['trades'])
        trades_df.to_csv(f"{output_dir}/trades.csv", index=False)
    
    # Export portfolio history
    if results['portfolio_history']:
        portfolio_df = pd.DataFrame(results['portfolio_history'])
        portfolio_df.to_csv(f"{output_dir}/portfolio_history.csv", index=False)
    
    # Export configuration
    config_dict = {
        'initial_usd': results['config'].initial_usd,
        'stop_loss_pct': results['config'].stop_loss_pct,
        'take_profit_pct': results['config'].take_profit_pct,
        'start_date': results['config'].start_date,
        'end_date': results['config'].end_date,
        'fee_pct': results['config'].fee_pct,
        'slippage_pct': results['config'].slippage_pct
    }
    
    with open(f"{output_dir}/config.json", 'w') as f:
        json.dump(config_dict, f, indent=2)
    
    print(f"Results exported to {output_dir}/")

# Export results
export_results(results)
```

## Best Practices

### 1. Data Quality

```python
# Ensure high-quality data
# - Use clean, validated OHLCV data
# - Check for gaps and inconsistencies
# - Use appropriate timeframes for your strategy
# - Include sufficient history for indicator warmup
```

### 2. Realistic Parameters

```python
# Use realistic trading parameters
config = BacktestConfig(
    initial_usd=10000,
    fee_pct=0.001,        # Realistic trading fees
    slippage_pct=0.0005,  # Account for slippage
    stop_loss_pct=0.03,   # Reasonable stop loss
    take_profit_pct=0.06  # Reasonable take profit
)
```

### 3. Overfitting Prevention

```python
# Prevent overfitting
# - Use out-of-sample testing
# - Implement walk-forward analysis
# - Limit parameter optimization ranges
# - Use cross-validation techniques
# - Test on multiple time periods and market conditions
```

### 4. Performance Validation

```python
# Validate performance metrics
# - Check for statistical significance
# - Analyze trade distribution
# - Examine drawdown periods
# - Verify risk-adjusted returns
# - Compare to benchmarks
```

### 5. Strategy Robustness

```python
# Test strategy robustness
# - Test on different market conditions
# - Vary parameter ranges
# - Check sensitivity to transaction costs
# - Analyze performance across different timeframes
# - Test with different data sources
```

This comprehensive backtesting guide provides everything you need to thoroughly test and optimize your trading strategies using the IncrementalTrader framework. Remember that backtesting is just one part of strategy development - always validate results with forward testing before live trading.