Cycles/README.md

# Cycles - Cryptocurrency Trading Strategy Backtesting Framework

A comprehensive Python framework for backtesting cryptocurrency trading strategies using technical indicators, with advanced features like machine learning price prediction to eliminate lookahead bias.

## Table of Contents

- [Overview](#overview)
- [Features](#features)
- [Quick Start](#quick-start)
- [Project Structure](#project-structure)
- [Core Modules](#core-modules)
- [Configuration](#configuration)
- [Usage Examples](#usage-examples)
- [API Documentation](#api-documentation)
- [Testing](#testing)
- [Contributing](#contributing)
- [License](#license)

## Overview

Cycles is a sophisticated backtesting framework designed specifically for cryptocurrency trading strategies. It provides robust tools for:

- **Strategy Backtesting**: Test trading strategies across multiple timeframes with comprehensive metrics
- **Technical Analysis**: Built-in indicators including SuperTrend, RSI, Bollinger Bands, and more
- **Machine Learning Integration**: Eliminate lookahead bias using XGBoost price prediction
- **Multi-timeframe Analysis**: Support for various timeframes from 1-minute to daily data
- **Performance Analytics**: Detailed reporting with profit ratios, drawdowns, win rates, and fee calculations

### Key Goals

1. **Realistic Trading Simulation**: Eliminate common backtesting pitfalls like lookahead bias
2. **Modular Architecture**: Easy to extend with new indicators and strategies
3. **Performance Optimization**: Parallel processing for efficient large-scale backtesting
4. **Comprehensive Analysis**: Rich reporting and visualization capabilities

## Features

### 🚀 Core Features

- **Multi-Strategy Backtesting**: Test multiple trading strategies simultaneously
- **Advanced Stop Loss Management**: Precise stop-loss execution using 1-minute data
- **Fee Integration**: Realistic trading fee calculations (OKX exchange fees)
- **Parallel Processing**: Efficient multi-core backtesting execution
- **Rich Analytics**: Comprehensive performance metrics and reporting

### 📊 Technical Indicators

- **SuperTrend**: Multi-parameter SuperTrend indicator with meta-trend analysis
- **RSI**: Relative Strength Index with customizable periods
- **Bollinger Bands**: Configurable period and standard deviation multipliers
- **Extensible Framework**: Easy to add new technical indicators

### 🤖 Machine Learning

- **Price Prediction**: XGBoost-based closing price prediction
- **Lookahead Bias Elimination**: Realistic trading simulations
- **Feature Engineering**: Advanced technical feature extraction
- **Model Persistence**: Save and load trained models

### 📈 Data Management

- **Multiple Data Sources**: Support for various cryptocurrency exchanges
- **Flexible Timeframes**: 1-minute to daily data aggregation
- **Efficient Storage**: Optimized data loading and caching
- **Google Sheets Integration**: External data source connectivity

## Quick Start

### Prerequisites

- Python 3.10 or higher
- UV package manager (recommended)
- Git

### Installation

1. **Clone the repository**:
   ```bash
   git clone <repository-url>
   cd Cycles
   ```

2. **Install dependencies**:
   ```bash
   uv sync
   ```

3. **Activate virtual environment**:
   ```bash
   source .venv/bin/activate  # Linux/Mac
   # or
   .venv\Scripts\activate     # Windows
   ```

### Basic Usage

1. **Prepare your configuration file** (`config.json`):
   ```json
   {
     "start_date": "2023-01-01",
     "stop_date": "2023-12-31",
     "initial_usd": 10000,
     "timeframes": ["5T", "15T", "1H", "4H"],
     "stop_loss_pcts": [0.02, 0.05, 0.10]
   }
   ```

2. **Run a backtest**:
   ```bash
   uv run python main.py --config config.json
   ```

3. **View results**:
   Results will be saved in timestamped CSV files with comprehensive metrics.

## Project Structure

```
Cycles/
├── cycles/                    # Core library modules
│   ├── Analysis/             # Technical analysis indicators
│   │   ├── boillinger_band.py
│   │   ├── rsi.py
│   │   └── __init__.py
│   ├── utils/                # Utility modules
│   │   ├── storage.py        # Data storage and management
│   │   ├── system.py         # System utilities
│   │   ├── data_utils.py     # Data processing utilities
│   │   └── gsheets.py        # Google Sheets integration
│   ├── backtest.py           # Core backtesting engine
│   ├── supertrend.py         # SuperTrend indicator implementation
│   ├── charts.py             # Visualization utilities
│   ├── market_fees.py        # Trading fee calculations
│   └── __init__.py
├── docs/                     # Documentation
│   ├── analysis.md           # Analysis module documentation
│   ├── utils_storage.md      # Storage utilities documentation
│   └── utils_system.md       # System utilities documentation
├── data/                     # Data directory (not in repo)
├── results/                  # Backtest results (not in repo)
├── xgboost/                  # Machine learning components
├── OHLCVPredictor/           # Price prediction module
├── main.py                   # Main execution script
├── test_bbrsi.py            # Example strategy test
├── pyproject.toml           # Project configuration
├── requirements.txt         # Dependencies
├── uv.lock                  # UV lock file
└── README.md                # This file
```

## Core Modules

### Backtest Engine (`cycles/backtest.py`)

The heart of the framework, providing comprehensive backtesting capabilities:

```python
from cycles.backtest import Backtest

results = Backtest.run(
    min1_df=minute_data,
    df=timeframe_data,
    initial_usd=10000,
    stop_loss_pct=0.05,
    debug=False
)
```

**Key Features**:
- Meta-SuperTrend strategy implementation
- Precise stop-loss execution using 1-minute data
- Comprehensive trade logging and statistics
- Fee-aware profit calculations

### Technical Analysis (`cycles/Analysis/`)

Modular technical indicator implementations:

#### RSI (Relative Strength Index)
```python
from cycles.Analysis.rsi import RSI

rsi_calculator = RSI(period=14)
data_with_rsi = rsi_calculator.calculate(df, price_column='close')
```

#### Bollinger Bands
```python
from cycles.Analysis.boillinger_band import BollingerBands

bb = BollingerBands(period=20, std_dev_multiplier=2.0)
data_with_bb = bb.calculate(df)
```

### Data Management (`cycles/utils/storage.py`)

Efficient data loading, processing, and result storage:

```python
from cycles.utils.storage import Storage

storage = Storage(data_dir='./data', logging=logging)
data = storage.load_data('btcusd_1-min_data.csv', '2023-01-01', '2023-12-31')
```

## Configuration

### Backtest Configuration

Create a `config.json` file with the following structure:

```json
{
  "start_date": "2023-01-01",
  "stop_date": "2023-12-31",
  "initial_usd": 10000,
  "timeframes": [
    "1T",    // 1 minute
    "5T",    // 5 minutes
    "15T",   // 15 minutes
    "1H",    // 1 hour
    "4H",    // 4 hours
    "1D"     // 1 day
  ],
  "stop_loss_pcts": [0.02, 0.05, 0.10, 0.15]
}
```

### Environment Variables

Set the following environment variables for enhanced functionality:

```bash
# Google Sheets integration (optional)
export GOOGLE_SHEETS_CREDENTIALS_PATH="/path/to/credentials.json"

# Data directory (optional, defaults to ./data)
export DATA_DIR="/path/to/data"

# Results directory (optional, defaults to ./results)
export RESULTS_DIR="/path/to/results"
```

## Usage Examples

### Basic Backtest

```python
import json
from cycles.utils.storage import Storage
from cycles.backtest import Backtest

# Load configuration
with open('config.json', 'r') as f:
    config = json.load(f)

# Initialize storage
storage = Storage(data_dir='./data')

# Load data
data_1min = storage.load_data(
    'btcusd_1-min_data.csv',
    config['start_date'],
    config['stop_date']
)

# Run backtest
results = Backtest.run(
    min1_df=data_1min,
    df=data_1min,  # Same data for 1-minute strategy
    initial_usd=config['initial_usd'],
    stop_loss_pct=0.05,
    debug=True
)

print(f"Final USD: {results['final_usd']:.2f}")
print(f"Number of trades: {results['n_trades']}")
print(f"Win rate: {results['win_rate']:.2%}")
```

### Multi-Timeframe Analysis

```python
from main import process

# Define timeframes to test
timeframes = ['5T', '15T', '1H', '4H']
stop_loss_pcts = [0.02, 0.05, 0.10]

# Create tasks for parallel processing
tasks = [
    (timeframe, data_1min, stop_loss_pct, 10000)
    for timeframe in timeframes
    for stop_loss_pct in stop_loss_pcts
]

# Process each task
for task in tasks:
    results, trades = process(task, debug=False)
    print(f"Timeframe: {task[0]}, Stop Loss: {task[2]:.1%}")
    for result in results:
        print(f"  Final USD: {result['final_usd']:.2f}")
```

### Custom Strategy Development

```python
from cycles.Analysis.rsi import RSI
from cycles.Analysis.boillinger_band import BollingerBands

def custom_strategy(df):
    """Example custom trading strategy using RSI and Bollinger Bands"""
    
    # Calculate indicators
    rsi = RSI(period=14)
    bb = BollingerBands(period=20, std_dev_multiplier=2.0)
    
    df_with_rsi = rsi.calculate(df.copy())
    df_with_bb = bb.calculate(df_with_rsi)
    
    # Define signals
    buy_signals = (
        (df_with_bb['close'] < df_with_bb['LowerBand']) & 
        (df_with_bb['RSI'] < 30)
    )
    
    sell_signals = (
        (df_with_bb['close'] > df_with_bb['UpperBand']) & 
        (df_with_bb['RSI'] > 70)
    )
    
    return buy_signals, sell_signals
```

## API Documentation

### Core Classes

#### `Backtest`
Main backtesting engine with static methods for strategy execution.

**Methods**:
- `run(min1_df, df, initial_usd, stop_loss_pct, debug=False)`: Execute backtest
- `check_stop_loss(...)`: Check stop-loss conditions using 1-minute data
- `handle_entry(...)`: Process trade entry logic
- `handle_exit(...)`: Process trade exit logic

#### `Storage`
Data management and persistence utilities.

**Methods**:
- `load_data(filename, start_date, stop_date)`: Load and filter historical data
- `save_data(df, filename)`: Save processed data
- `write_backtest_results(...)`: Save backtest results to CSV

#### `SystemUtils`
System optimization and resource management.

**Methods**:
- `get_optimal_workers()`: Determine optimal number of parallel workers
- `get_memory_usage()`: Monitor memory consumption

### Configuration Parameters

| Parameter | Type | Description | Default |
|-----------|------|-------------|---------|
| `start_date` | string | Backtest start date (YYYY-MM-DD) | Required |
| `stop_date` | string | Backtest end date (YYYY-MM-DD) | Required |
| `initial_usd` | float | Starting capital in USD | Required |
| `timeframes` | array | List of timeframes to test | Required |
| `stop_loss_pcts` | array | Stop-loss percentages to test | Required |

## Testing

### Running Tests

```bash
# Run all tests
uv run pytest

# Run specific test file
uv run pytest test_bbrsi.py

# Run with verbose output
uv run pytest -v

# Run with coverage
uv run pytest --cov=cycles
```

### Test Structure

- `test_bbrsi.py`: Example strategy testing with RSI and Bollinger Bands
- Unit tests for individual modules (add as needed)
- Integration tests for complete workflows

### Example Test

```python
# test_bbrsi.py demonstrates strategy testing
from cycles.Analysis.rsi import RSI
from cycles.Analysis.boillinger_band import BollingerBands

def test_strategy_signals():
    # Load test data
    storage = Storage()
    data = storage.load_data('test_data.csv', '2023-01-01', '2023-02-01')
    
    # Calculate indicators
    rsi = RSI(period=14)
    bb = BollingerBands(period=20)
    
    data_with_indicators = bb.calculate(rsi.calculate(data))
    
    # Test signal generation
    assert 'RSI' in data_with_indicators.columns
    assert 'UpperBand' in data_with_indicators.columns
    assert 'LowerBand' in data_with_indicators.columns
```

## Contributing

### Development Setup

1. Fork the repository
2. Create a feature branch: `git checkout -b feature/new-indicator`
3. Install development dependencies: `uv sync --dev`
4. Make your changes following the coding standards
5. Add tests for new functionality
6. Run tests: `uv run pytest`
7. Submit a pull request

### Coding Standards

- **Maximum file size**: 250 lines
- **Maximum function size**: 50 lines
- **Documentation**: All public functions must have docstrings
- **Type hints**: Use type hints for all function parameters and returns
- **Error handling**: Include proper error handling and meaningful error messages
- **No emoji**: Avoid emoji in code and comments

### Adding New Indicators

1. Create a new file in `cycles/Analysis/`
2. Follow the existing pattern (see `rsi.py` or `boillinger_band.py`)
3. Include comprehensive docstrings and type hints
4. Add tests for the new indicator
5. Update documentation

## Performance Considerations

### Optimization Tips

1. **Parallel Processing**: Use the built-in parallel processing for multiple timeframes
2. **Data Caching**: Cache frequently used calculations
3. **Memory Management**: Monitor memory usage for large datasets
4. **Efficient Data Types**: Use appropriate pandas data types

### Benchmarks

Typical performance on modern hardware:
- **1-minute data**: ~1M candles processed in 2-3 minutes
- **Multiple timeframes**: 4 timeframes × 4 stop-loss values in 5-10 minutes
- **Memory usage**: ~2-4GB for 1 year of 1-minute BTC data

## Troubleshooting

### Common Issues

1. **Memory errors with large datasets**:
   - Reduce date range or use data chunking
   - Increase system RAM or use swap space

2. **Slow performance**:
   - Enable parallel processing
   - Reduce number of timeframes/stop-loss values
   - Use SSD storage for data files

3. **Missing data errors**:
   - Verify data file format and column names
   - Check date range availability in data
   - Ensure proper data cleaning

### Debug Mode

Enable debug mode for detailed logging:

```python
# Set debug=True for detailed output
results = Backtest.run(..., debug=True)
```

## License

This project is licensed under the MIT License. See the LICENSE file for details.

## Changelog

### Version 0.1.0 (Current)
- Initial release
- Core backtesting framework
- SuperTrend strategy implementation
- Technical indicators (RSI, Bollinger Bands)
- Multi-timeframe analysis
- Machine learning price prediction
- Parallel processing support

---

For more detailed documentation, see the `docs/` directory or visit our [documentation website](link-to-docs).

**Support**: For questions or issues, please create an issue on GitHub or contact the development team.