Simon/lowkey_backtest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Remove deprecated modules and files related to the backtesting framework, including backtest.py, cli.py, config.py, data.py, intrabar.py, logging_utils.py, market_costs.py, metrics.py, trade.py, and supertrend indicators. Introduce a new structure for the backtesting engine with improved organization and functionality, including a CLI handler, data manager, and reporting capabilities. Update dependencies in pyproject.toml to support the new architecture.

Browse Source
This commit is contained in:
Simon Moisy
2026-01-12 21:11:39 +08:00
parent c4aa965a98
commit 44fac1ed25
37 changed files with 5253 additions and 393 deletions
Download Patch File Download Diff File Expand all files Collapse all files

295
tasks/prd-market-type-selection.md Normal file
View File

@@ -0,0 +1,295 @@
# PRD: Market Type Selection for Backtesting
## Introduction/Overview
Currently, the backtesting system operates with a single, implicit market type assumption. This PRD defines the implementation of **market type selection** (Spot vs. USDT-M Perpetual Futures) to enable realistic simulation of different trading conditions.
**Problem Statement:**
- Strategies cannot be backtested against different market mechanics (leverage, funding, short-selling)
- Fee structures are uniform regardless of market type
- No support for short-selling strategies
- Data fetching doesn't distinguish between spot and futures markets
**Goal:**
Enable users to backtest strategies against specific market types (Spot or USDT-M Perpetual) with realistic trading conditions matching OKX's live environment.
---
## Goals
1. **Support two market types:** Spot and USDT-M Perpetual Futures
2. **Realistic fee simulation:** Match OKX's fee structure per market type
3. **Leverage support:** Per-strategy configurable leverage (perpetuals only)
4. **Funding rate simulation:** Simplified funding rate model for perpetuals
5. **Short-selling support:** Enable strategies to generate short signals
6. **Liquidation awareness:** Warn when positions would be liquidated (no full simulation)
7. **Separate data storage:** Download and store data per market type
8. **Grid search integration:** Allow leverage optimization in parameter grids
---
## User Stories
1. **As a trader**, I want to backtest my strategy on perpetual futures so that I can simulate leveraged trading with funding costs.
2. **As a trader**, I want to backtest on spot markets so that I can compare performance without leverage or funding overhead.
3. **As a strategy developer**, I want to define a default market type for my strategy so that it runs with appropriate settings by default.
4. **As a trader**, I want to test different leverage levels so that I can find the optimal risk/reward balance.
5. **As a trader**, I want to see warnings when my position would have been liquidated so that I can adjust my risk parameters.
6. **As a strategy developer**, I want to create strategies that can go short so that I can profit from downward price movements.
---
## Functional Requirements
### FR1: Market Type Enum and Configuration
1.1. Create a `MarketType` enum with values: `SPOT`, `PERPETUAL`
1.2. Each strategy class must have a `default_market_type` class attribute
1.3. Market type can be overridden via CLI (optional, for testing)
### FR2: Data Management
2.1. Modify `DataManager` to support market type in data paths:
- Spot: `data/ccxt/{exchange}/spot/{symbol}/{timeframe}.csv`
- Perpetual: `data/ccxt/{exchange}/perpetual/{symbol}/{timeframe}.csv`
2.2. Update `download` command to accept `--market` flag:
```bash
uv run python main.py download --pair BTC/USDT --market perpetual
```
2.3. Use CCXT's market type parameter when fetching data:
- Spot: `exchange.fetch_ohlcv(symbol, timeframe, ...)`
- Perpetual: `exchange.fetch_ohlcv(symbol + ':USDT', timeframe, ...)`
### FR3: Fee Structure
3.1. Define default fees per market type (matching OKX):
| Market Type | Maker Fee | Taker Fee | Notes |
|-------------|-----------|-----------|-------|
| Spot | 0.08% | 0.10% | No funding |
| Perpetual | 0.02% | 0.05% | + funding |
3.2. Allow fee override via CLI (existing `--fees` flag)
### FR4: Leverage Support (Perpetual Only)
4.1. Add `default_leverage` class attribute to strategies (default: 1 for spot, configurable for perpetual)
4.2. Add `--leverage` CLI flag for backtest command
4.3. Leverage affects:
- Position sizing (notional = cash * leverage)
- PnL calculation (multiplied by leverage)
- Liquidation threshold calculation
4.4. Support leverage in grid search parameter grids
### FR5: Funding Rate Simulation (Perpetual Only)
5.1. Implement simplified funding rate model:
- Default rate: 0.01% per 8 hours (configurable)
- Applied every 8 hours to open positions
- Positive rate: Longs pay shorts
- Negative rate: Shorts pay longs
5.2. Add `--funding-rate` CLI flag to override default
5.3. Track cumulative funding paid/received in backtest stats
### FR6: Short-Selling Support
6.1. Modify `BaseStrategy.run()` signature to return 4 signal arrays:
```python
def run(self, close, **kwargs) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]:
"""
Returns:
long_entries: Boolean signals to open long positions
long_exits: Boolean signals to close long positions
short_entries: Boolean signals to open short positions
short_exits: Boolean signals to close short positions
"""
```
6.2. Update `Backtester` to use VectorBT's `direction` parameter or dual portfolio simulation
6.3. For spot market: Ignore short signals (log warning if present)
6.4. For perpetual market: Process both long and short signals
### FR7: Liquidation Warning
7.1. Calculate liquidation price based on:
- Entry price
- Leverage
- Maintenance margin rate (OKX: ~0.4% for BTC)
7.2. During backtest, check if price crosses liquidation threshold
7.3. Log warning with details:
```
WARNING: Position would be liquidated at bar 1234 (price: $45,000, liq_price: $44,820)
```
7.4. Include liquidation event count in backtest summary stats
### FR8: Backtester Integration
8.1. Modify `Backtester.run_strategy()` to accept market type from strategy
8.2. Apply market-specific simulation parameters:
- Fees (if not overridden)
- Leverage
- Funding rate calculation
- Short-selling capability
8.3. Update portfolio simulation to handle leveraged positions
### FR9: Reporting Updates
9.1. Add market type to backtest summary output
9.2. Add new stats for perpetual backtests:
- Total funding paid/received
- Number of liquidation warnings
- Effective leverage used
9.3. Update CSV exports to include market-specific columns
---
## Non-Goals (Out of Scope)
- **Coin-M (Inverse) Perpetuals:** Not included in v1
- **Spot Margin Trading:** Not included in v1
- **Expiry Futures:** Not included in v1
- **Full Liquidation Simulation:** Only warnings, no automatic position closure
- **Real Funding Rate Data:** Use simplified model; historical funding API integration is future work
- **Cross-Margin Mode:** Assume isolated margin for simplicity
- **Partial Liquidation:** Assume full liquidation threshold only
---
## Design Considerations
### Data Directory Structure (New)
```
data/ccxt/
okx/
spot/
BTC-USDT/
1m.csv
1d.csv
perpetual/
BTC-USDT/
1m.csv
1d.csv
```
### Strategy Class Example
```python
class MetaSupertrendStrategy(BaseStrategy):
default_market_type = MarketType.PERPETUAL
default_leverage = 5
default_sl_stop = 0.02
def run(self, close, **kwargs):
# ... indicator logic ...
return long_entries, long_exits, short_entries, short_exits
```
### CLI Usage Examples
```bash
# Download perpetual data
uv run python main.py download --pair BTC/USDT --market perpetual
# Backtest with strategy defaults (uses strategy's default_market_type)
uv run python main.py backtest --strategy meta_st --pair BTC/USDT
# Override leverage
uv run python main.py backtest --strategy meta_st --pair BTC/USDT --leverage 10
# Grid search including leverage
uv run python main.py backtest --strategy meta_st --pair BTC/USDT --grid
# (leverage can be part of param grid in strategy factory)
```
---
## Technical Considerations
1. **VectorBT Compatibility:**
- VectorBT's `Portfolio.from_signals()` supports `direction` parameter for long/short
- Alternatively, run two portfolios (long-only, short-only) and combine
- Leverage can be simulated via `size` parameter or post-processing returns
2. **CCXT Market Type Handling:**
- OKX perpetual symbols use format: `BTC/USDT:USDT`
- Need to handle symbol conversion in DataManager
3. **Funding Rate Timing:**
- OKX funding at 00:00, 08:00, 16:00 UTC
- Need to identify these timestamps in the data and apply funding
4. **Backward Compatibility:**
- Existing strategies should work with minimal changes
- Default to `MarketType.SPOT` if not specified
- Existing 2-tuple return from `run()` should be interpreted as long-only
---
## Success Metrics
1. **Functional:** All existing backtests produce same results when run with `MarketType.SPOT`
2. **Functional:** Perpetual backtests correctly apply funding every 8 hours
3. **Functional:** Leverage multiplies both gains and losses correctly
4. **Functional:** Short signals are processed for perpetual, ignored for spot
5. **Usability:** Users can switch market types with minimal configuration
6. **Accuracy:** Fee structures match OKX's published rates
---
## Open Questions
1. **Position Sizing with Leverage:**
- Should leverage affect `init_cash` interpretation (notional value) or position size directly?
- Recommendation: Affect position size; `init_cash` remains the actual margin deposited.
2. **Multiple Positions:**
- Can strategies hold both long and short simultaneously (hedging)?
- Recommendation: No for v1; only one direction at a time.
3. **Funding Rate Sign:**
- When funding is positive, longs pay shorts. Should we assume the user is always the "taker" of funding?
- Recommendation: Yes, apply funding based on position direction.
4. **Migration Path:**
- Should we migrate existing data to new directory structure?
- Recommendation: No auto-migration; users re-download with `--market` flag.
---
## Implementation Priority
| Priority | Component | Complexity |
|----------|-----------|------------|
| 1 | MarketType enum + strategy defaults | Low |
| 2 | DataManager market type support | Medium |
| 3 | Fee structure per market type | Low |
| 4 | Short-selling signal support | Medium |
| 5 | Leverage simulation | Medium |
| 6 | Funding rate simulation | Medium |
| 7 | Liquidation warnings | Low |
| 8 | Reporting updates | Low |
| 9 | Grid search leverage support | Low |

76
tasks/prd-vectorbt-migration.md Normal file
View File

@@ -0,0 +1,76 @@
# PRD: VectorBT Migration & CCXT Integration
## 1. Introduction
The goal of this project is to refactor the current backtesting infrastructure to a professional-grade stack using **VectorBT** for high-performance backtesting and **CCXT** for robust historical data acquisition. The system will support rapid prototyping of "many simple strategies," parameter optimization (Grid Search), and stability testing (Walk-Forward Analysis).
## 2. Goals
- **Replace Custom Backtester:** Retire the existing loop-based backtesting logic in favor of vectorized operations using `vectorbt`.
- **Automate Data Collection:** Implement a `ccxt` based downloader to fetch and cache OHLCV data from OKX (and other exchanges) automatically.
- **Enable Optimization:** Built-in support for Grid Search to find optimal strategy parameters.
- **Validation:** Implement Walk-Forward Analysis (WFA) to validate strategy robustness and prevent overfitting.
- **Standardized Reporting:** Generate consistent outputs: Console summaries, CSV logs, and VectorBT interactive plots.
## 3. User Stories
- **Data Acquisition:** "As a user, I want to run a command `download_data --pair BTC/USDT --exchange okx` and have the system fetch historical 1-minute candles and save them to `data/ccxt/okx/BTC-USDT/1m.csv`."
- **Strategy Dev:** "As a researcher, I want to define a new strategy by simply writing a class/function that defines entry/exit signals, without worrying about the backtesting loop."
- **Optimization:** "As a researcher, I want to say 'Optimize RSI period between 10 and 20' and get a heatmap of results."
- **Validation:** "As a researcher, I want to verify if my 'best' parameters work on unseen data using Walk-Forward Analysis."
- **Analysis:** "As a user, I want to see an equity curve and key metrics (Sharpe, Drawdown) immediately after a test run."
## 4. Functional Requirements
### 4.1 Data Module (`data_manager`)
- **Exchange Interface:** Use `ccxt` to connect to exchanges (initially OKX).
- **Fetching Logic:** Fetch OHLCV data in chunks to handle rate limits and long histories.
- **Storage:** Save data to standardized paths: `data/ccxt/{exchange}/{pair}_{timeframe}.csv`.
- **Loading:** Utility to load saved CSVs into a Pandas DataFrame compatible with `vectorbt`.
### 4.2 Strategy Interface (`strategies/`)
- **Base Protocol:** Define a standard structure for strategies. A strategy should return/define:
- Indicator calculations (Vectorized).
- Entry signals (Boolean Series).
- Exit signals (Boolean Series).
- **Parameterization:** Strategies must accept dynamic parameters to support Grid Search.
### 4.3 Backtest Engine (`engine.py`)
- **Simulation:** Use `vectorbt.Portfolio.from_signals` (or similar) for fast simulation.
- **Cost Model:** Support configurable fees (maker/taker) and slippage estimates.
- **Grid Search:** Utilize `vectorbt`'s parameter broadcasting to run many variations simultaneously.
- **Walk-Forward Analysis:**
- Implement a splitting mechanism (e.g., `vectorbt.Splitter`) to divide data into In-Sample (Train) and Out-of-Sample (Test) sets.
- Execute optimization on Train, validate on Test.
### 4.4 Reporting (`reporting.py`)
- **Console:** Print key metrics: Total Return, Sharpe Ratio, Max Drawdown, Win Rate, Count of Trades.
- **Files:** Save detailed trade logs and metrics summaries to `backtest_logs/`.
- **Visuals:** Generate and save/show `vectorbt` plots (Equity curve, Drawdowns).
## 5. Non-Goals
- Real-time live trading execution (this is strictly for research/backtesting).
- Complex Machine Learning models (initially focusing on indicator-based logic).
- High-frequency tick-level backtesting (1-minute granularity is the target).
## 6. Technical Architecture Proposal
```text
project_root/
├── data/
│ └── ccxt/ # New data storage structure
├── strategies/ # Strategy definitions
│ ├── __init__.py
│ ├── base.py # Abstract Base Class
│ └── ma_cross.py # Example strategy
├── engine/
│ ├── data_loader.py # CCXT wrapper
│ ├── backtester.py # VBT runner
│ └── optimizer.py # Grid Search & WFA logic
├── main.py # CLI entry point
└── pyproject.toml
```
## 7. Success Metrics
- Can download 1 year of 1m BTC/USDT data from OKX in < 2 minutes.
- Can run a 100-parameter grid search on 1 year of 1m data in < 10 seconds.
- Walk-forward analysis produces a clear "Robustness Score" or visual comparison of Train vs Test performance.
## 8. Open Questions
- Do we need to handle funding rates for perp futures in the PnL calculation immediately? (Assumed NO for V1, stick to spot/simple futures price action).