Cycles/test/backtest/README.md

# Strategy Backtest Runner

A comprehensive and efficient backtest runner for executing predefined trading strategies with advanced visualization and analysis capabilities.

## Overview

The Strategy Backtest Runner (`strategy_run.py`) executes specific trading strategies with predefined parameters defined in a JSON configuration file. Unlike the parameter optimization script, this runner focuses on testing and comparing specific strategy configurations with detailed market analysis and visualization.

## Features

- **JSON Configuration**: Define strategies and parameters in easy-to-edit JSON files
- **Multiple Strategy Support**: Run multiple strategies in sequence with a single command
- **All Strategy Types**: Support for MetaTrend, BBRS, and Random strategies
- **Organized Results**: Automatic folder structure creation for each run
- **Advanced Visualization**: Detailed plots showing portfolio performance and market context
- **Full Market Data Integration**: Continuous price charts with buy/sell signals overlay
- **Signal Export**: Complete buy/sell signal data exported to CSV files
- **Real-time File Saving**: Individual strategy results saved immediately upon completion
- **Comprehensive Analysis**: Multiple plot types for thorough performance analysis
- **Detailed Results**: Comprehensive result reporting with CSV and JSON export
- **Result Analysis**: Automatic summary generation and performance comparison
- **Error Handling**: Robust error handling with detailed logging
- **Flexible Configuration**: Support for different data files, date ranges, and trader parameters

## Usage

### Basic Usage

```bash
# Run strategies from a configuration file
python test/backtest/strategy_run.py --config configs/strategy/example_strategies.json

# Save results to a custom directory
python test/backtest/strategy_run.py --config configs/strategy/my_strategies.json --results-dir my_results

# Enable verbose logging
python test/backtest/strategy_run.py --config configs/strategy/example_strategies.json --verbose
```

### Enhanced Analysis Features

Each run automatically generates:
- **Organized folder structure** with timestamp for easy management
- **Real-time file saving** - results saved immediately after each strategy completes
- **Full market data visualization** - continuous price charts show complete market context
- **Signal tracking** - all buy/sell decisions exported with precise timing and pricing
- **Multi-layered analysis** - from individual trade details to portfolio-wide comparisons
- **Professional plots** - high-resolution (300 DPI) charts suitable for reports and presentations

### Create Example Configuration

```bash
# Create an example configuration file
python test/backtest/strategy_run.py --create-example configs/example_strategies.json
```

## Configuration File Format

The configuration file uses JSON format with two main sections:

### Backtest Settings

```json
{
  "backtest_settings": {
    "data_file": "btcusd_1-min_data.csv",
    "data_dir": "data",
    "start_date": "2023-01-01",
    "end_date": "2023-01-31",
    "initial_usd": 10000
  }
}
```

### Strategy Definitions

```json
{
  "strategies": [
    {
      "name": "MetaTrend_Conservative",
      "type": "metatrend",
      "params": {
        "supertrend_periods": [12, 10, 11],
        "supertrend_multipliers": [3.0, 1.0, 2.0],
        "min_trend_agreement": 0.8,
        "timeframe": "15min"
      },
      "trader_params": {
        "stop_loss_pct": 0.02,
        "portfolio_percent_per_trade": 0.5
      }
    }
  ]
}
```

## Strategy Types

### MetaTrend Strategy

Parameters:
- `supertrend_periods`: List of periods for multiple supertrend indicators
- `supertrend_multipliers`: List of multipliers for supertrend indicators
- `min_trend_agreement`: Minimum agreement threshold between indicators (0.0-1.0)
- `timeframe`: Data aggregation timeframe ("1min", "5min", "15min", "30min", "1h")

### BBRS Strategy

Parameters:
- `bb_length`: Bollinger Bands period
- `bb_std`: Bollinger Bands standard deviation multiplier
- `rsi_length`: RSI period
- `rsi_overbought`: RSI overbought threshold
- `rsi_oversold`: RSI oversold threshold
- `timeframe`: Data aggregation timeframe

### Random Strategy

Parameters:
- `signal_probability`: Probability of generating a signal (0.0-1.0)
- `timeframe`: Data aggregation timeframe

## Trader Parameters

All strategies support these trader parameters:
- `stop_loss_pct`: Stop loss percentage (e.g., 0.02 for 2%)
- `portfolio_percent_per_trade`: Percentage of portfolio to use per trade (0.0-1.0)

## Results Organization

Each run creates an organized folder structure for easy navigation and analysis:

```
results/
└── [config_name]_[timestamp]/
    ├── strategy_1_[strategy_name].json                    # Individual strategy data
    ├── strategy_1_[strategy_name]_plot.png               # 4-panel performance plot
    ├── strategy_1_[strategy_name]_detailed_plot.png      # 3-panel market analysis
    ├── strategy_1_[strategy_name]_trades.csv             # Trade details
    ├── strategy_1_[strategy_name]_signals.csv            # All buy/sell signals
    ├── strategy_2_[strategy_name].*                      # Second strategy files
    ├── ...                                               # Additional strategies
    ├── summary.csv                                       # Strategy comparison table
    ├── summary_plot.png                                  # Multi-strategy comparison
    └── summary_*.json                                    # Comprehensive results
```

## Visualization Types

The runner generates three types of plots for comprehensive analysis:

### 1. Individual Strategy Plot (4-Panel)
- **Equity Curve**: Portfolio value over time
- **Trade P&L**: Individual trade profits/losses
- **Drawdown**: Portfolio drawdown visualization
- **Statistics**: Strategy performance summary

### 2. Detailed Market Analysis Plot (3-Panel)
- **Portfolio Signals**: Portfolio value with buy/sell signal markers
- **Market Price**: Full continuous market price with entry/exit points
- **Combined View**: Dual-axis plot showing market vs portfolio performance

### 3. Summary Comparison Plot (4-Panel)
- **Returns Comparison**: Total returns across all strategies
- **Trade Counts**: Number of trades per strategy
- **Risk vs Return**: Win rate vs maximum drawdown scatter plot
- **Statistics Table**: Comprehensive performance metrics

## Output Files

The runner generates comprehensive output files organized in dedicated folders:

### Individual Strategy Files (per strategy)
- `strategy_N_[name].json`: Complete strategy data and metadata
- `strategy_N_[name]_plot.png`: 4-panel performance analysis plot
- `strategy_N_[name]_detailed_plot.png`: 3-panel market context plot
- `strategy_N_[name]_trades.csv`: Detailed trade information
- `strategy_N_[name]_signals.csv`: All buy/sell signals with timestamps

### Summary Files (per run)
- `summary.csv`: Strategy comparison table
- `summary_plot.png`: Multi-strategy comparison visualization
- `summary_*.json`: Comprehensive results and metadata

### Signal Data Format
Each signal CSV contains:
- `signal_id`: Unique signal identifier
- `signal_type`: BUY or SELL
- `time`: Signal timestamp
- `price`: Execution price
- `trade_id`: Associated trade number
- `quantity`: Trade quantity
- `value`: Trade value (quantity × price)
- `strategy`: Strategy name

## Example Configurations

### Simple MetaTrend Test

```json
{
  "backtest_settings": {
    "data_file": "btcusd_1-min_data.csv",
    "start_date": "2023-01-01",
    "end_date": "2023-01-07",
    "initial_usd": 10000
  },
  "strategies": [
    {
      "name": "MetaTrend_Test",
      "type": "metatrend",
      "params": {
        "supertrend_periods": [12, 10],
        "supertrend_multipliers": [3.0, 1.0],
        "min_trend_agreement": 0.5,
        "timeframe": "15min"
      },
      "trader_params": {
        "stop_loss_pct": 0.02,
        "portfolio_percent_per_trade": 0.5
      }
    }
  ]
}
```

### Multiple Strategy Comparison

```json
{
  "backtest_settings": {
    "data_file": "btcusd_1-min_data.csv",
    "start_date": "2023-01-01",
    "end_date": "2023-01-31",
    "initial_usd": 10000
  },
  "strategies": [
    {
      "name": "Conservative_MetaTrend",
      "type": "metatrend",
      "params": {
        "supertrend_periods": [12, 10, 11],
        "supertrend_multipliers": [3.0, 1.0, 2.0],
        "min_trend_agreement": 0.8,
        "timeframe": "15min"
      },
      "trader_params": {
        "stop_loss_pct": 0.02,
        "portfolio_percent_per_trade": 0.5
      }
    },
    {
      "name": "Aggressive_MetaTrend",
      "type": "metatrend",
      "params": {
        "supertrend_periods": [10, 8],
        "supertrend_multipliers": [2.0, 1.0],
        "min_trend_agreement": 0.5,
        "timeframe": "5min"
      },
      "trader_params": {
        "stop_loss_pct": 0.03,
        "portfolio_percent_per_trade": 0.8
      }
    },
    {
      "name": "BBRS_Baseline",
      "type": "bbrs",
      "params": {
        "bb_length": 20,
        "bb_std": 2.0,
        "rsi_length": 14,
        "rsi_overbought": 70,
        "rsi_oversold": 30,
        "timeframe": "15min"
      },
      "trader_params": {
        "stop_loss_pct": 0.025,
        "portfolio_percent_per_trade": 0.6
      }
    }
  ]
}
```

## Command Line Options

- `--config`: Path to JSON configuration file (required)
- `--results-dir`: Directory for saving results (default: "results")
- `--create-example`: Create example config file at specified path
- `--verbose`: Enable verbose logging for debugging

## Error Handling

The runner includes comprehensive error handling:

- **Configuration Validation**: Validates JSON structure and required fields
- **Data File Verification**: Checks if data files exist before running
- **Strategy Creation**: Handles unknown strategy types gracefully
- **Backtest Execution**: Captures and logs individual strategy failures
- **Result Saving**: Ensures results are saved even if some strategies fail

## Integration

This runner integrates seamlessly with the existing IncrementalTrader framework:

- Uses the same `IncBacktester` and strategy classes
- Compatible with all existing data formats
- Leverages the same result saving utilities
- Maintains consistency with optimization scripts

## Performance

- **Sequential Execution**: Strategies run one after another for clear logging
- **Real-time Results**: Individual strategy files saved immediately upon completion
- **Efficient Data Loading**: Market data loaded once per run for all visualizations
- **Progress Tracking**: Clear progress indication for long-running backtests
- **Detailed Timing**: Individual strategy execution times are tracked
- **High-Quality Output**: Professional 300 DPI plots suitable for presentations

## Best Practices

1. **Start Small**: Test with short date ranges first
2. **Validate Data**: Ensure data files exist and cover the specified date range
3. **Monitor Resources**: Watch memory usage for very long backtests
4. **Save Configs**: Keep configuration files organized for reproducibility
5. **Use Descriptive Names**: Give strategies clear, descriptive names
6. **Test Incrementally**: Add strategies one by one when debugging
7. **Leverage Visualizations**: Use detailed plots to understand market context and strategy behavior
8. **Analyze Signals**: Review signal CSV files to understand strategy decision patterns
9. **Compare Runs**: Use organized folder structure to compare different parameter sets
10. **Monitor Execution**: Watch real-time progress as individual strategies complete