Cycles/IncrementalTrader/docs/backtesting.md

Backtesting Guide

This guide explains how to use the IncrementalTrader backtesting framework for comprehensive strategy testing and optimization.

Overview

The IncrementalTrader backtesting framework provides:

• Single Strategy Testing: Test individual strategies with detailed metrics
• Parameter Optimization: Systematic parameter sweeps with parallel execution
• Performance Analysis: Comprehensive performance metrics and reporting
• Data Management: Flexible data loading and validation
• Result Export: Multiple output formats for analysis

Quick Start

Basic Backtesting

from IncrementalTrader import IncBacktester, BacktestConfig, MetaTrendStrategy

# Configure backtest
config = BacktestConfig(
    initial_usd=10000,
    stop_loss_pct=0.03,
    take_profit_pct=0.06,
    start_date="2024-01-01",
    end_date="2024-12-31"
)

# Create backtester
backtester = IncBacktester()

# Run single strategy test
results = backtester.run_single_strategy(
    strategy_class=MetaTrendStrategy,
    strategy_params={"timeframe": "15min"},
    config=config,
    data_file="data/BTCUSDT_1m.csv"
)

# Print results
print(f"Total Return: {results['performance_metrics']['total_return_pct']:.2f}%")
print(f"Sharpe Ratio: {results['performance_metrics']['sharpe_ratio']:.2f}")
print(f"Max Drawdown: {results['performance_metrics']['max_drawdown_pct']:.2f}%")

Configuration

BacktestConfig

The main configuration class for backtesting parameters.

from IncrementalTrader import BacktestConfig

config = BacktestConfig(
    # Portfolio settings
    initial_usd=10000,           # Starting capital
    
    # Risk management
    stop_loss_pct=0.03,          # 3% stop loss
    take_profit_pct=0.06,        # 6% take profit
    
    # Time range
    start_date="2024-01-01",     # Start date (YYYY-MM-DD)
    end_date="2024-12-31",       # End date (YYYY-MM-DD)
    
    # Trading settings
    fee_pct=0.001,               # 0.1% trading fee
    slippage_pct=0.0005,         # 0.05% slippage
    
    # Output settings
    output_dir="backtest_results",
    save_trades=True,
    save_portfolio_history=True,
    
    # Performance settings
    risk_free_rate=0.02          # 2% annual risk-free rate
)

Parameters:

Parameter	Type	Default	Description
initial_usd	float	10000	Starting capital in USD
stop_loss_pct	float	None	Stop loss percentage (0.03 = 3%)
take_profit_pct	float	None	Take profit percentage (0.06 = 6%)
start_date	str	None	Start date in YYYY-MM-DD format
end_date	str	None	End date in YYYY-MM-DD format
fee_pct	float	0.001	Trading fee percentage
slippage_pct	float	0.0005	Slippage percentage
output_dir	str	"backtest_results"	Output directory
save_trades	bool	True	Save individual trades
save_portfolio_history	bool	True	Save portfolio history
risk_free_rate	float	0.02	Annual risk-free rate for Sharpe ratio

OptimizationConfig

Configuration for parameter optimization.

from IncrementalTrader import OptimizationConfig

# Define parameter ranges
param_ranges = {
    "supertrend_periods": [[10, 20, 30], [15, 25, 35], [20, 30, 40]],
    "supertrend_multipliers": [[2.0, 3.0, 4.0], [1.5, 2.5, 3.5]],
    "min_trend_agreement": [0.5, 0.6, 0.7, 0.8]
}

# Create optimization config
opt_config = OptimizationConfig(
    base_config=config,          # Base BacktestConfig
    param_ranges=param_ranges,   # Parameter combinations to test
    max_workers=4,               # Number of parallel workers
    optimization_metric="sharpe_ratio",  # Metric to optimize
    save_all_results=True        # Save all parameter combinations
)

Single Strategy Testing

Basic Usage

# Test MetaTrend strategy
results = backtester.run_single_strategy(
    strategy_class=MetaTrendStrategy,
    strategy_params={
        "timeframe": "15min",
        "supertrend_periods": [10, 20, 30],
        "supertrend_multipliers": [2.0, 3.0, 4.0],
        "min_trend_agreement": 0.6
    },
    config=config,
    data_file="data/BTCUSDT_1m.csv"
)

Results Structure

# Access different result components
performance = results['performance_metrics']
trades = results['trades']
portfolio_history = results['portfolio_history']
config_used = results['config']

# Performance metrics
print(f"Total Trades: {performance['total_trades']}")
print(f"Win Rate: {performance['win_rate']:.2f}%")
print(f"Profit Factor: {performance['profit_factor']:.2f}")
print(f"Sharpe Ratio: {performance['sharpe_ratio']:.2f}")
print(f"Sortino Ratio: {performance['sortino_ratio']:.2f}")
print(f"Max Drawdown: {performance['max_drawdown_pct']:.2f}%")
print(f"Calmar Ratio: {performance['calmar_ratio']:.2f}")

# Trade analysis
winning_trades = [t for t in trades if t['pnl'] > 0]
losing_trades = [t for t in trades if t['pnl'] < 0]

print(f"Average Win: ${sum(t['pnl'] for t in winning_trades) / len(winning_trades):.2f}")
print(f"Average Loss: ${sum(t['pnl'] for t in losing_trades) / len(losing_trades):.2f}")

Performance Metrics

The backtester calculates comprehensive performance metrics:

Metric	Description	Formula
Total Return	Overall portfolio return	(Final Value - Initial Value) / Initial Value
Annualized Return	Yearly return rate	(Total Return + 1)^(365/days) - 1
Volatility	Annualized standard deviation	std(daily_returns) × √365
Sharpe Ratio	Risk-adjusted return	(Return - Risk Free Rate) / Volatility
Sortino Ratio	Downside risk-adjusted return	(Return - Risk Free Rate) / Downside Deviation
Max Drawdown	Maximum peak-to-trough decline	max((Peak - Trough) / Peak)
Calmar Ratio	Return to max drawdown ratio	Annualized Return / Max Drawdown
Win Rate	Percentage of profitable trades	Winning Trades / Total Trades
Profit Factor	Ratio of gross profit to loss	Gross Profit / Gross Loss

Parameter Optimization

Basic Optimization

# Define parameter ranges to test
param_ranges = {
    "timeframe": ["5min", "15min", "30min"],
    "supertrend_periods": [[10, 20, 30], [15, 25, 35]],
    "min_trend_agreement": [0.5, 0.6, 0.7]
}

# Create optimization config
opt_config = OptimizationConfig(
    base_config=config,
    param_ranges=param_ranges,
    max_workers=4,
    optimization_metric="sharpe_ratio"
)

# Run optimization
optimization_results = backtester.optimize_strategy(
    strategy_class=MetaTrendStrategy,
    optimization_config=opt_config,
    data_file="data/BTCUSDT_1m.csv"
)

# Get best parameters
best_params = optimization_results['best_params']
best_performance = optimization_results['best_performance']
all_results = optimization_results['all_results']

print(f"Best Parameters: {best_params}")
print(f"Best Sharpe Ratio: {best_performance['sharpe_ratio']:.2f}")

Advanced Optimization

# More complex parameter optimization
param_ranges = {
    # Strategy parameters
    "timeframe": ["5min", "15min", "30min"],
    "supertrend_periods": [
        [10, 20, 30], [15, 25, 35], [20, 30, 40],
        [10, 15, 20], [25, 35, 45]
    ],
    "supertrend_multipliers": [
        [2.0, 3.0, 4.0], [1.5, 2.5, 3.5], [2.5, 3.5, 4.5]
    ],
    "min_trend_agreement": [0.4, 0.5, 0.6, 0.7, 0.8],
    
    # Risk management (will override config values)
    "stop_loss_pct": [0.02, 0.03, 0.04, 0.05],
    "take_profit_pct": [0.04, 0.06, 0.08, 0.10]
}

# Optimization with custom metric
def custom_metric(performance):
    """Custom optimization metric combining return and drawdown."""
    return performance['total_return_pct'] / max(performance['max_drawdown_pct'], 1.0)

opt_config = OptimizationConfig(
    base_config=config,
    param_ranges=param_ranges,
    max_workers=8,
    optimization_metric=custom_metric,  # Custom function
    save_all_results=True
)

results = backtester.optimize_strategy(
    strategy_class=MetaTrendStrategy,
    optimization_config=opt_config,
    data_file="data/BTCUSDT_1m.csv"
)

Optimization Metrics

You can optimize for different metrics:

# Built-in metrics (string names)
optimization_metrics = [
    "total_return_pct",
    "sharpe_ratio",
    "sortino_ratio",
    "calmar_ratio",
    "profit_factor",
    "win_rate"
]

# Custom metric function
def risk_adjusted_return(performance):
    return (performance['total_return_pct'] / 
            max(performance['max_drawdown_pct'], 1.0))

opt_config = OptimizationConfig(
    base_config=config,
    param_ranges=param_ranges,
    optimization_metric=risk_adjusted_return  # Custom function
)

Data Management

Data Format

The backtester expects CSV data with the following columns:

timestamp,open,high,low,close,volume
1640995200000,46222.5,46850.0,46150.0,46800.0,1250.5
1640995260000,46800.0,47000.0,46750.0,46950.0,980.2
...

Required Columns:

• timestamp: Unix timestamp in milliseconds
• open: Opening price
• high: Highest price
• low: Lowest price
• close: Closing price
• volume: Trading volume

Data Loading

# The backtester automatically loads and validates data
results = backtester.run_single_strategy(
    strategy_class=MetaTrendStrategy,
    strategy_params={"timeframe": "15min"},
    config=config,
    data_file="data/BTCUSDT_1m.csv"  # Automatically loaded and validated
)

# Data is automatically filtered by start_date and end_date from config

Data Validation

The backtester performs automatic data validation:

# Validation checks performed:
# 1. Required columns present
# 2. No missing values
# 3. Timestamps in ascending order
# 4. Price consistency (high >= low, etc.)
# 5. Date range filtering
# 6. Data type validation

Advanced Features

Custom Strategy Testing

# Test your custom strategy
class MyCustomStrategy(IncStrategyBase):
    def __init__(self, name: str, params: dict = None):
        super().__init__(name, params)
        # Your strategy implementation
    
    def _process_aggregated_data(self, timestamp: int, ohlcv: tuple):
        # Your strategy logic
        return IncStrategySignal.HOLD()

# Test custom strategy
results = backtester.run_single_strategy(
    strategy_class=MyCustomStrategy,
    strategy_params={"timeframe": "15min", "custom_param": 42},
    config=config,
    data_file="data/BTCUSDT_1m.csv"
)

Multiple Strategy Comparison

# Compare different strategies
strategies_to_test = [
    (MetaTrendStrategy, {"timeframe": "15min"}),
    (BBRSStrategy, {"timeframe": "15min"}),
    (RandomStrategy, {"timeframe": "15min"})
]

comparison_results = {}

for strategy_class, params in strategies_to_test:
    results = backtester.run_single_strategy(
        strategy_class=strategy_class,
        strategy_params=params,
        config=config,
        data_file="data/BTCUSDT_1m.csv"
    )
    
    strategy_name = strategy_class.__name__
    comparison_results[strategy_name] = results['performance_metrics']

# Compare results
for name, performance in comparison_results.items():
    print(f"{name}:")
    print(f"  Return: {performance['total_return_pct']:.2f}%")
    print(f"  Sharpe: {performance['sharpe_ratio']:.2f}")
    print(f"  Max DD: {performance['max_drawdown_pct']:.2f}%")

Walk-Forward Analysis

# Implement walk-forward analysis
import pandas as pd
from datetime import datetime, timedelta

def walk_forward_analysis(strategy_class, params, data_file, 
                         train_months=6, test_months=1):
    """Perform walk-forward analysis."""
    
    # Load full dataset to determine date range
    data = pd.read_csv(data_file)
    data['timestamp'] = pd.to_datetime(data['timestamp'], unit='ms')
    
    start_date = data['timestamp'].min()
    end_date = data['timestamp'].max()
    
    results = []
    current_date = start_date
    
    while current_date + timedelta(days=30*(train_months + test_months)) <= end_date:
        # Define train and test periods
        train_start = current_date
        train_end = current_date + timedelta(days=30*train_months)
        test_start = train_end
        test_end = test_start + timedelta(days=30*test_months)
        
        # Optimize on training data
        train_config = BacktestConfig(
            initial_usd=10000,
            start_date=train_start.strftime("%Y-%m-%d"),
            end_date=train_end.strftime("%Y-%m-%d")
        )
        
        # Simple parameter optimization (you can expand this)
        best_params = params  # In practice, optimize here
        
        # Test on out-of-sample data
        test_config = BacktestConfig(
            initial_usd=10000,
            start_date=test_start.strftime("%Y-%m-%d"),
            end_date=test_end.strftime("%Y-%m-%d")
        )
        
        test_results = backtester.run_single_strategy(
            strategy_class=strategy_class,
            strategy_params=best_params,
            config=test_config,
            data_file=data_file
        )
        
        results.append({
            'test_start': test_start,
            'test_end': test_end,
            'performance': test_results['performance_metrics']
        })
        
        # Move to next period
        current_date = test_start
    
    return results

# Run walk-forward analysis
wf_results = walk_forward_analysis(
    MetaTrendStrategy,
    {"timeframe": "15min"},
    "data/BTCUSDT_1m.csv"
)

# Analyze walk-forward results
total_returns = [r['performance']['total_return_pct'] for r in wf_results]
avg_return = sum(total_returns) / len(total_returns)
print(f"Average out-of-sample return: {avg_return:.2f}%")

Result Analysis

Detailed Performance Analysis

# Comprehensive result analysis
def analyze_results(results):
    """Analyze backtest results in detail."""
    
    performance = results['performance_metrics']
    trades = results['trades']
    portfolio_history = results['portfolio_history']
    
    print("=== PERFORMANCE SUMMARY ===")
    print(f"Total Return: {performance['total_return_pct']:.2f}%")
    print(f"Annualized Return: {performance['annualized_return_pct']:.2f}%")
    print(f"Volatility: {performance['volatility_pct']:.2f}%")
    print(f"Sharpe Ratio: {performance['sharpe_ratio']:.2f}")
    print(f"Sortino Ratio: {performance['sortino_ratio']:.2f}")
    print(f"Max Drawdown: {performance['max_drawdown_pct']:.2f}%")
    print(f"Calmar Ratio: {performance['calmar_ratio']:.2f}")
    
    print("\n=== TRADING STATISTICS ===")
    print(f"Total Trades: {performance['total_trades']}")
    print(f"Win Rate: {performance['win_rate']:.2f}%")
    print(f"Profit Factor: {performance['profit_factor']:.2f}")
    
    # Trade analysis
    if trades:
        winning_trades = [t for t in trades if t['pnl'] > 0]
        losing_trades = [t for t in trades if t['pnl'] < 0]
        
        if winning_trades:
            avg_win = sum(t['pnl'] for t in winning_trades) / len(winning_trades)
            max_win = max(t['pnl'] for t in winning_trades)
            print(f"Average Win: ${avg_win:.2f}")
            print(f"Largest Win: ${max_win:.2f}")
        
        if losing_trades:
            avg_loss = sum(t['pnl'] for t in losing_trades) / len(losing_trades)
            max_loss = min(t['pnl'] for t in losing_trades)
            print(f"Average Loss: ${avg_loss:.2f}")
            print(f"Largest Loss: ${max_loss:.2f}")
    
    print("\n=== RISK METRICS ===")
    print(f"Value at Risk (95%): {performance.get('var_95', 'N/A')}")
    print(f"Expected Shortfall (95%): {performance.get('es_95', 'N/A')}")
    
    return performance

# Analyze results
performance = analyze_results(results)

Export Results

# Export results to different formats
def export_results(results, output_dir="backtest_results"):
    """Export backtest results to files."""
    
    import os
    import json
    import pandas as pd
    
    os.makedirs(output_dir, exist_ok=True)
    
    # Export performance metrics
    with open(f"{output_dir}/performance_metrics.json", 'w') as f:
        json.dump(results['performance_metrics'], f, indent=2)
    
    # Export trades
    if results['trades']:
        trades_df = pd.DataFrame(results['trades'])
        trades_df.to_csv(f"{output_dir}/trades.csv", index=False)
    
    # Export portfolio history
    if results['portfolio_history']:
        portfolio_df = pd.DataFrame(results['portfolio_history'])
        portfolio_df.to_csv(f"{output_dir}/portfolio_history.csv", index=False)
    
    # Export configuration
    config_dict = {
        'initial_usd': results['config'].initial_usd,
        'stop_loss_pct': results['config'].stop_loss_pct,
        'take_profit_pct': results['config'].take_profit_pct,
        'start_date': results['config'].start_date,
        'end_date': results['config'].end_date,
        'fee_pct': results['config'].fee_pct,
        'slippage_pct': results['config'].slippage_pct
    }
    
    with open(f"{output_dir}/config.json", 'w') as f:
        json.dump(config_dict, f, indent=2)
    
    print(f"Results exported to {output_dir}/")

# Export results
export_results(results)

Best Practices

1. Data Quality

# Ensure high-quality data
# - Use clean, validated OHLCV data
# - Check for gaps and inconsistencies
# - Use appropriate timeframes for your strategy
# - Include sufficient history for indicator warmup

2. Realistic Parameters

# Use realistic trading parameters
config = BacktestConfig(
    initial_usd=10000,
    fee_pct=0.001,        # Realistic trading fees
    slippage_pct=0.0005,  # Account for slippage
    stop_loss_pct=0.03,   # Reasonable stop loss
    take_profit_pct=0.06  # Reasonable take profit
)

3. Overfitting Prevention

# Prevent overfitting
# - Use out-of-sample testing
# - Implement walk-forward analysis
# - Limit parameter optimization ranges
# - Use cross-validation techniques
# - Test on multiple time periods and market conditions

4. Performance Validation

# Validate performance metrics
# - Check for statistical significance
# - Analyze trade distribution
# - Examine drawdown periods
# - Verify risk-adjusted returns
# - Compare to benchmarks

5. Strategy Robustness

# Test strategy robustness
# - Test on different market conditions
# - Vary parameter ranges
# - Check sensitivity to transaction costs
# - Analyze performance across different timeframes
# - Test with different data sources

This comprehensive backtesting guide provides everything you need to thoroughly test and optimize your trading strategies using the IncrementalTrader framework. Remember that backtesting is just one part of strategy development - always validate results with forward testing before live trading.