Compare commits
11 Commits
main
...
regime-imb
| Author | SHA1 | Date | |
|---|---|---|---|
| 582a43cd4a | |||
| b5550f4ff4 | |||
| 35992ee374 | |||
| 62c470b3de | |||
| 7e4a6874a2 | |||
| c4ecb29d4c | |||
| 0c82c4f366 | |||
| 1e4cb87da3 | |||
| 10bb371054 | |||
| e6d69ed04d | |||
| 44fac1ed25 |
11
.gitignore
vendored
11
.gitignore
vendored
@@ -169,4 +169,13 @@ cython_debug/
|
||||
#.idea/
|
||||
|
||||
./logs/
|
||||
*.csv
|
||||
*.csv
|
||||
research/regime_results.html
|
||||
data/backtest_runs.db
|
||||
.gitignore
|
||||
live_trading/regime_model.pkl
|
||||
live_trading/positions.json
|
||||
|
||||
|
||||
*.pkl
|
||||
*.db
|
||||
@@ -1 +0,0 @@
|
||||
../OHLCVPredictor
|
||||
304
README.md
304
README.md
@@ -1,82 +1,262 @@
|
||||
### lowkey_backtest — Supertrend Backtester
|
||||
# Lowkey Backtest
|
||||
|
||||
### Overview
|
||||
Backtest a simple, long-only strategy driven by a meta Supertrend signal on aggregated OHLCV data. The script:
|
||||
- Loads 1-minute BTC/USD data from `../data/btcusd_1-min_data.csv`
|
||||
- Aggregates to multiple timeframes (e.g., `5min`, `15min`, `30min`, `1h`, `4h`, `1d`)
|
||||
- Computes three Supertrend variants and creates a meta signal when all agree
|
||||
- Executes entries/exits at the aggregated bar open price
|
||||
- Applies OKX spot fee assumptions (taker by default)
|
||||
- Evaluates stop-loss using intra-bar 1-minute data
|
||||
- Writes detailed trade logs and a summary CSV
|
||||
A backtesting framework supporting multiple market types (spot, perpetual) with realistic trading simulation including leverage, funding, and shorts.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirements
|
||||
- Python 3.12+
|
||||
- Dependencies: `pandas`, `numpy`, `ta`
|
||||
- Package management: `uv`
|
||||
- Package manager: `uv`
|
||||
|
||||
Install dependencies with uv:
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
# If a dependency is missing, add it explicitly and sync
|
||||
uv add pandas numpy ta
|
||||
uv sync
|
||||
```
|
||||
|
||||
### Data
|
||||
- Expected CSV location: `../data/btcusd_1-min_data.csv` (relative to the repo root)
|
||||
- Required columns: `Timestamp`, `Open`, `High`, `Low`, `Close`, `Volume`
|
||||
- `Timestamp` should be UNIX seconds; zero-volume rows are ignored
|
||||
## Quick Reference
|
||||
|
||||
### Quickstart
|
||||
Run the backtest with defaults:
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `uv run python main.py download -p BTC-USDT` | Download data |
|
||||
| `uv run python main.py backtest -s meta_st -p BTC-USDT` | Run backtest |
|
||||
| `uv run python main.py wfa -s regime -p BTC-USDT` | Walk-forward analysis |
|
||||
| `uv run python train_model.py --download` | Train/retrain ML model |
|
||||
| `uv run python research/regime_detection.py` | Run research script |
|
||||
|
||||
---
|
||||
|
||||
## Backtest CLI
|
||||
|
||||
The main entry point is `main.py` which provides three commands: `download`, `backtest`, and `wfa`.
|
||||
|
||||
### Download Data
|
||||
|
||||
Download historical OHLCV data from exchanges.
|
||||
|
||||
```bash
|
||||
uv run python main.py
|
||||
uv run python main.py download -p BTC-USDT -t 1h
|
||||
```
|
||||
|
||||
Outputs:
|
||||
- Per-run trade logs in `backtest_logs/` named like `trade_log_<TIMEFRAME>_sl<STOPLOSS>.csv`
|
||||
- Run-level summary in `backtest_summary.csv`
|
||||
**Options:**
|
||||
- `-p, --pair` (required): Trading pair (e.g., `BTC-USDT`, `ETH-USDT`)
|
||||
- `-t, --timeframe`: Timeframe (default: `1m`)
|
||||
- `-e, --exchange`: Exchange (default: `okx`)
|
||||
- `-m, --market`: Market type: `spot` or `perpetual` (default: `spot`)
|
||||
- `--start`: Start date in `YYYY-MM-DD` format
|
||||
|
||||
### Configuring a Run
|
||||
Adjust parameters directly in `main.py`:
|
||||
- Date range (in `load_data`): `load_data('2021-11-01', '2024-10-16')`
|
||||
- Timeframes to test (any subset of `"5min", "15min", "30min", "1h", "4h", "1d"`):
|
||||
- `timeframes = ["5min", "15min", "30min", "1h", "4h", "1d"]`
|
||||
- Stop-loss percentages:
|
||||
- `stoplosses = [0.03, 0.05, 0.1]`
|
||||
- Supertrend settings (in `add_supertrend_indicators`): `(period, multiplier)` pairs `(12, 3.0)`, `(10, 1.0)`, `(11, 2.0)`
|
||||
- Fee model (in `calculate_okx_taker_maker_fee`): taker `0.0010`, maker `0.0008`
|
||||
**Examples:**
|
||||
```bash
|
||||
# Download 1-hour spot data
|
||||
uv run python main.py download -p ETH-USDT -t 1h
|
||||
|
||||
### What the Backtester Does
|
||||
- Aggregation: Resamples 1-minute data to your selected timeframe using OHLCV rules
|
||||
- Supertrend signals: Computes three Supertrends and derives a meta trend of `+1` (bullish) or `-1` (bearish) when all agree; otherwise `0`
|
||||
- Trade logic (long-only):
|
||||
- Entry when the meta trend changes to bullish; uses aggregated bar open price
|
||||
- Exit when the meta trend changes to bearish; uses aggregated bar open price
|
||||
- Stop-loss: For each aggregated bar, scans corresponding 1-minute closes to detect stop-loss and exits using a realistic fill (threshold or next 1-minute open if gapped)
|
||||
- Performance metrics: total return, max drawdown, Sharpe (daily, factor 252), win rate, number of trades, final/initial equity, and total fees
|
||||
|
||||
### Important: Lookahead Bias Note
|
||||
The current implementation uses the meta Supertrend signal of the same bar for entries/exits, which introduces lookahead bias. To avoid this, lag the signal by one bar inside `backtest()` in `main.py`:
|
||||
|
||||
```python
|
||||
# Replace the current line
|
||||
meta_trend_signal = meta_trend
|
||||
|
||||
# With a one-bar lag to remove lookahead
|
||||
# meta_trend_signal = np.roll(meta_trend, 1)
|
||||
# meta_trend_signal[0] = 0
|
||||
# Download perpetual data from a specific date
|
||||
uv run python main.py download -p BTC-USDT -m perpetual --start 2024-01-01
|
||||
```
|
||||
|
||||
### Outputs
|
||||
- `backtest_logs/trade_log_<TIMEFRAME>_sl<STOPLOSS>.csv`: trade-by-trade records including type (`buy`, `sell`, `stop_loss`, `forced_close`), timestamps, prices, balances, PnL, and fees
|
||||
- `backtest_summary.csv`: one row per (timeframe, stop-loss) combination with `timeframe`, `stop_loss`, `total_return`, `max_drawdown`, `sharpe_ratio`, `win_rate`, `num_trades`, `final_equity`, `initial_equity`, `num_stop_losses`, `total_fees`
|
||||
### Run Backtest
|
||||
|
||||
### Troubleshooting
|
||||
- CSV not found: Ensure the dataset is located at `../data/btcusd_1-min_data.csv`
|
||||
- Missing packages: Run `uv add pandas numpy ta` then `uv sync`
|
||||
- Memory/performance: Large date ranges on 1-minute data can be heavy; narrow the date span or test fewer timeframes
|
||||
Run a backtest with a specific strategy.
|
||||
|
||||
```bash
|
||||
uv run python main.py backtest -s <strategy> -p <pair> [options]
|
||||
```
|
||||
|
||||
**Available Strategies:**
|
||||
- `meta_st` - Meta Supertrend (triple supertrend consensus)
|
||||
- `regime` - Regime Reversion (ML-based spread trading)
|
||||
- `rsi` - RSI overbought/oversold
|
||||
- `macross` - Moving Average Crossover
|
||||
|
||||
**Options:**
|
||||
- `-s, --strategy` (required): Strategy name
|
||||
- `-p, --pair` (required): Trading pair
|
||||
- `-t, --timeframe`: Timeframe (default: `1m`)
|
||||
- `--start`: Start date
|
||||
- `--end`: End date
|
||||
- `-g, --grid`: Run grid search optimization
|
||||
- `--plot`: Show equity curve plot
|
||||
- `--sl`: Stop loss percentage
|
||||
- `--tp`: Take profit percentage
|
||||
- `--trail`: Enable trailing stop
|
||||
- `--fees`: Override fee rate
|
||||
- `--slippage`: Slippage (default: `0.001`)
|
||||
- `-l, --leverage`: Leverage multiplier
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Basic backtest with Meta Supertrend
|
||||
uv run python main.py backtest -s meta_st -p BTC-USDT -t 1h
|
||||
|
||||
# Backtest with date range and plot
|
||||
uv run python main.py backtest -s meta_st -p BTC-USDT --start 2024-01-01 --end 2024-12-31 --plot
|
||||
|
||||
# Grid search optimization
|
||||
uv run python main.py backtest -s meta_st -p BTC-USDT -t 4h -g
|
||||
|
||||
# Backtest with risk parameters
|
||||
uv run python main.py backtest -s meta_st -p BTC-USDT --sl 0.05 --tp 0.10 --trail
|
||||
|
||||
# Regime strategy on ETH/BTC spread
|
||||
uv run python main.py backtest -s regime -p ETH-USDT -t 1h
|
||||
```
|
||||
|
||||
### Walk-Forward Analysis (WFA)
|
||||
|
||||
Run walk-forward optimization to avoid overfitting.
|
||||
|
||||
```bash
|
||||
uv run python main.py wfa -s <strategy> -p <pair> [options]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `-s, --strategy` (required): Strategy name
|
||||
- `-p, --pair` (required): Trading pair
|
||||
- `-t, --timeframe`: Timeframe (default: `1d`)
|
||||
- `-w, --windows`: Number of walk-forward windows (default: `10`)
|
||||
- `--plot`: Show WFA results plot
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Walk-forward analysis with 10 windows
|
||||
uv run python main.py wfa -s meta_st -p BTC-USDT -t 1d -w 10
|
||||
|
||||
# WFA with plot output
|
||||
uv run python main.py wfa -s regime -p ETH-USDT --plot
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Research Scripts
|
||||
|
||||
Research scripts are located in the `research/` directory for experimental analysis.
|
||||
|
||||
### Regime Detection Research
|
||||
|
||||
Tests multiple holding horizons for the regime reversion strategy using walk-forward training.
|
||||
|
||||
```bash
|
||||
uv run python research/regime_detection.py
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--days DAYS`: Number of days of historical data (default: 90)
|
||||
- `--start DATE`: Start date (YYYY-MM-DD), overrides `--days`
|
||||
- `--end DATE`: End date (YYYY-MM-DD), defaults to now
|
||||
- `--output PATH`: Output CSV path
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Use last 90 days (default)
|
||||
uv run python research/regime_detection.py
|
||||
|
||||
# Use last 180 days
|
||||
uv run python research/regime_detection.py --days 180
|
||||
|
||||
# Specific date range
|
||||
uv run python research/regime_detection.py --start 2025-07-01 --end 2025-12-31
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Loads BTC and ETH hourly data
|
||||
- Calculates spread features (Z-score, RSI, volume ratios)
|
||||
- Trains RandomForest classifier with walk-forward methodology
|
||||
- Tests horizons from 6h to 150h
|
||||
- Outputs best parameters by F1 score, Net PnL, and MAE
|
||||
|
||||
**Output:**
|
||||
- Console: Summary of results for each horizon
|
||||
- File: `research/horizon_optimization_results.csv`
|
||||
|
||||
---
|
||||
|
||||
## ML Model Training
|
||||
|
||||
The `regime` strategy uses a RandomForest classifier that can be trained with new data.
|
||||
|
||||
### Train Model
|
||||
|
||||
Train or retrain the ML model with latest data:
|
||||
|
||||
```bash
|
||||
uv run python train_model.py [options]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--days DAYS`: Days of historical data (default: 90)
|
||||
- `--pair PAIR`: Base pair for context (default: BTC-USDT)
|
||||
- `--spread-pair PAIR`: Trading pair (default: ETH-USDT)
|
||||
- `--timeframe TF`: Timeframe (default: 1h)
|
||||
- `--market TYPE`: Market type: `spot` or `perpetual` (default: perpetual)
|
||||
- `--output PATH`: Model output path (default: `data/regime_model.pkl`)
|
||||
- `--train-ratio R`: Train/test split ratio (default: 0.7)
|
||||
- `--horizon H`: Prediction horizon in bars (default: 102)
|
||||
- `--download`: Download latest data before training
|
||||
- `--dry-run`: Run without saving model
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Train with last 90 days of data
|
||||
uv run python train_model.py
|
||||
|
||||
# Download fresh data and train
|
||||
uv run python train_model.py --download
|
||||
|
||||
# Train with 180 days of data
|
||||
uv run python train_model.py --days 180
|
||||
|
||||
# Train on spot market data
|
||||
uv run python train_model.py --market spot
|
||||
|
||||
# Dry run to see metrics without saving
|
||||
uv run python train_model.py --dry-run
|
||||
```
|
||||
|
||||
### Daily Retraining (Cron)
|
||||
|
||||
To automate daily model retraining, add a cron job:
|
||||
|
||||
```bash
|
||||
# Edit crontab
|
||||
crontab -e
|
||||
|
||||
# Add entry to retrain daily at 00:30 UTC
|
||||
30 0 * * * cd /path/to/lowkey_backtest_live && uv run python train_model.py --download >> logs/training.log 2>&1
|
||||
```
|
||||
|
||||
### Model Files
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `data/regime_model.pkl` | Current production model |
|
||||
| `data/regime_model_YYYYMMDD_HHMMSS.pkl` | Versioned model snapshots |
|
||||
|
||||
The model file contains:
|
||||
- Trained RandomForest classifier
|
||||
- Feature column names
|
||||
- Training metrics (F1 score, sample counts)
|
||||
- Training timestamp
|
||||
|
||||
---
|
||||
|
||||
## Output Files
|
||||
|
||||
| Location | Description |
|
||||
|----------|-------------|
|
||||
| `backtest_logs/` | Trade logs and WFA results |
|
||||
| `research/` | Research output files |
|
||||
| `data/` | Downloaded OHLCV data and ML models |
|
||||
| `data/regime_model.pkl` | Trained ML model for regime strategy |
|
||||
|
||||
---
|
||||
|
||||
## Running Tests
|
||||
|
||||
```bash
|
||||
uv run pytest tests/
|
||||
```
|
||||
|
||||
Run a specific test file:
|
||||
|
||||
```bash
|
||||
uv run pytest tests/test_data_manager.py
|
||||
```
|
||||
|
||||
3
api/__init__.py
Normal file
3
api/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
"""
|
||||
FastAPI backend for Lowkey Backtest UI.
|
||||
"""
|
||||
47
api/main.py
Normal file
47
api/main.py
Normal file
@@ -0,0 +1,47 @@
|
||||
"""
|
||||
FastAPI application entry point for Lowkey Backtest UI.
|
||||
|
||||
Run with: uvicorn api.main:app --reload
|
||||
"""
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from api.models.database import init_db
|
||||
from api.routers import backtest, data, strategies
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Initialize database on startup."""
|
||||
init_db()
|
||||
yield
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
title="Lowkey Backtest API",
|
||||
description="API for running and analyzing trading strategy backtests",
|
||||
version="0.1.0",
|
||||
lifespan=lifespan,
|
||||
)
|
||||
|
||||
# CORS configuration for local development
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["http://localhost:5173", "http://127.0.0.1:5173"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
# Register routers
|
||||
app.include_router(strategies.router, prefix="/api", tags=["strategies"])
|
||||
app.include_router(data.router, prefix="/api", tags=["data"])
|
||||
app.include_router(backtest.router, prefix="/api", tags=["backtest"])
|
||||
|
||||
|
||||
@app.get("/api/health")
|
||||
async def health_check():
|
||||
"""Health check endpoint."""
|
||||
return {"status": "ok", "service": "lowkey-backtest-api"}
|
||||
3
api/models/__init__.py
Normal file
3
api/models/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
"""
|
||||
Pydantic schemas and database models.
|
||||
"""
|
||||
99
api/models/database.py
Normal file
99
api/models/database.py
Normal file
@@ -0,0 +1,99 @@
|
||||
"""
|
||||
SQLAlchemy database models and session management for backtest run persistence.
|
||||
"""
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy import JSON, Column, DateTime, Float, Integer, String, Text, create_engine
|
||||
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
|
||||
|
||||
# Database file location
|
||||
DB_PATH = Path(__file__).parent.parent.parent / "data" / "backtest_runs.db"
|
||||
DATABASE_URL = f"sqlite:///{DB_PATH}"
|
||||
|
||||
engine = create_engine(DATABASE_URL, connect_args={"check_same_thread": False})
|
||||
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
||||
|
||||
|
||||
class Base(DeclarativeBase):
|
||||
"""Base class for SQLAlchemy models."""
|
||||
pass
|
||||
|
||||
|
||||
class BacktestRun(Base):
|
||||
"""
|
||||
Persisted backtest run record.
|
||||
|
||||
Stores all information needed to display and compare runs.
|
||||
"""
|
||||
__tablename__ = "backtest_runs"
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
run_id = Column(String(36), unique=True, nullable=False, index=True)
|
||||
|
||||
# Configuration
|
||||
strategy = Column(String(50), nullable=False, index=True)
|
||||
symbol = Column(String(20), nullable=False, index=True)
|
||||
exchange = Column(String(20), nullable=False, default="okx")
|
||||
market_type = Column(String(20), nullable=False)
|
||||
timeframe = Column(String(10), nullable=False)
|
||||
leverage = Column(Integer, nullable=False, default=1)
|
||||
params = Column(JSON, nullable=False, default=dict)
|
||||
|
||||
# Date range
|
||||
start_date = Column(String(20), nullable=True)
|
||||
end_date = Column(String(20), nullable=True)
|
||||
|
||||
# Metrics (denormalized for quick listing)
|
||||
total_return = Column(Float, nullable=False)
|
||||
benchmark_return = Column(Float, nullable=False, default=0.0)
|
||||
alpha = Column(Float, nullable=False, default=0.0)
|
||||
sharpe_ratio = Column(Float, nullable=False)
|
||||
max_drawdown = Column(Float, nullable=False)
|
||||
win_rate = Column(Float, nullable=False)
|
||||
total_trades = Column(Integer, nullable=False)
|
||||
profit_factor = Column(Float, nullable=True)
|
||||
total_fees = Column(Float, nullable=False, default=0.0)
|
||||
total_funding = Column(Float, nullable=False, default=0.0)
|
||||
liquidation_count = Column(Integer, nullable=False, default=0)
|
||||
liquidation_loss = Column(Float, nullable=False, default=0.0)
|
||||
adjusted_return = Column(Float, nullable=True)
|
||||
|
||||
# Full data (JSON serialized)
|
||||
equity_curve = Column(Text, nullable=False) # JSON array
|
||||
trades = Column(Text, nullable=False) # JSON array
|
||||
|
||||
# Metadata
|
||||
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
|
||||
|
||||
def set_equity_curve(self, data: list[dict]):
|
||||
"""Serialize equity curve to JSON string."""
|
||||
self.equity_curve = json.dumps(data)
|
||||
|
||||
def get_equity_curve(self) -> list[dict]:
|
||||
"""Deserialize equity curve from JSON string."""
|
||||
return json.loads(self.equity_curve) if self.equity_curve else []
|
||||
|
||||
def set_trades(self, data: list[dict]):
|
||||
"""Serialize trades to JSON string."""
|
||||
self.trades = json.dumps(data)
|
||||
|
||||
def get_trades(self) -> list[dict]:
|
||||
"""Deserialize trades from JSON string."""
|
||||
return json.loads(self.trades) if self.trades else []
|
||||
|
||||
|
||||
def init_db():
|
||||
"""Create database tables if they don't exist."""
|
||||
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
Base.metadata.create_all(bind=engine)
|
||||
|
||||
|
||||
def get_db() -> Session:
|
||||
"""Get database session (dependency injection)."""
|
||||
db = SessionLocal()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
162
api/models/schemas.py
Normal file
162
api/models/schemas.py
Normal file
@@ -0,0 +1,162 @@
|
||||
"""
|
||||
Pydantic schemas for API request/response models.
|
||||
"""
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# --- Strategy Schemas ---
|
||||
|
||||
class StrategyParam(BaseModel):
|
||||
"""Single strategy parameter definition."""
|
||||
name: str
|
||||
value: Any
|
||||
param_type: str = Field(description="Type: int, float, bool, list")
|
||||
min_value: float | None = None
|
||||
max_value: float | None = None
|
||||
description: str | None = None
|
||||
|
||||
|
||||
class StrategyInfo(BaseModel):
|
||||
"""Strategy information with parameters."""
|
||||
name: str
|
||||
display_name: str
|
||||
market_type: str
|
||||
default_leverage: int
|
||||
default_params: dict[str, Any]
|
||||
grid_params: dict[str, Any]
|
||||
|
||||
|
||||
class StrategiesResponse(BaseModel):
|
||||
"""Response for GET /api/strategies."""
|
||||
strategies: list[StrategyInfo]
|
||||
|
||||
|
||||
# --- Symbol/Data Schemas ---
|
||||
|
||||
class SymbolInfo(BaseModel):
|
||||
"""Available symbol information."""
|
||||
symbol: str
|
||||
exchange: str
|
||||
market_type: str
|
||||
timeframes: list[str]
|
||||
start_date: str | None = None
|
||||
end_date: str | None = None
|
||||
row_count: int = 0
|
||||
|
||||
|
||||
class DataStatusResponse(BaseModel):
|
||||
"""Response for GET /api/data/status."""
|
||||
symbols: list[SymbolInfo]
|
||||
|
||||
|
||||
# --- Backtest Schemas ---
|
||||
|
||||
class BacktestRequest(BaseModel):
|
||||
"""Request body for POST /api/backtest."""
|
||||
strategy: str
|
||||
symbol: str
|
||||
exchange: str = "okx"
|
||||
timeframe: str = "1h"
|
||||
market_type: str = "perpetual"
|
||||
start_date: str | None = None
|
||||
end_date: str | None = None
|
||||
init_cash: float = 10000.0
|
||||
leverage: int | None = None
|
||||
fees: float | None = None
|
||||
slippage: float = 0.001
|
||||
sl_stop: float | None = None
|
||||
tp_stop: float | None = None
|
||||
sl_trail: bool = False
|
||||
params: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class TradeRecord(BaseModel):
|
||||
"""Single trade record."""
|
||||
entry_time: str
|
||||
exit_time: str | None = None
|
||||
entry_price: float
|
||||
exit_price: float | None = None
|
||||
size: float
|
||||
direction: str
|
||||
pnl: float | None = None
|
||||
return_pct: float | None = None
|
||||
status: str = "closed"
|
||||
|
||||
|
||||
class EquityPoint(BaseModel):
|
||||
"""Single point on equity curve."""
|
||||
timestamp: str
|
||||
value: float
|
||||
drawdown: float = 0.0
|
||||
|
||||
|
||||
class BacktestMetrics(BaseModel):
|
||||
"""Backtest performance metrics."""
|
||||
total_return: float
|
||||
benchmark_return: float = 0.0
|
||||
alpha: float = 0.0
|
||||
sharpe_ratio: float
|
||||
max_drawdown: float
|
||||
win_rate: float
|
||||
total_trades: int
|
||||
profit_factor: float | None = None
|
||||
avg_trade_return: float | None = None
|
||||
total_fees: float = 0.0
|
||||
total_funding: float = 0.0
|
||||
liquidation_count: int = 0
|
||||
liquidation_loss: float = 0.0
|
||||
adjusted_return: float | None = None
|
||||
|
||||
|
||||
class BacktestResult(BaseModel):
|
||||
"""Complete backtest result."""
|
||||
run_id: str
|
||||
strategy: str
|
||||
symbol: str
|
||||
market_type: str
|
||||
timeframe: str
|
||||
start_date: str
|
||||
end_date: str
|
||||
leverage: int
|
||||
params: dict[str, Any]
|
||||
metrics: BacktestMetrics
|
||||
equity_curve: list[EquityPoint]
|
||||
trades: list[TradeRecord]
|
||||
created_at: str
|
||||
|
||||
|
||||
class BacktestSummary(BaseModel):
|
||||
"""Summary for backtest list view."""
|
||||
run_id: str
|
||||
strategy: str
|
||||
symbol: str
|
||||
market_type: str
|
||||
timeframe: str
|
||||
total_return: float
|
||||
sharpe_ratio: float
|
||||
max_drawdown: float
|
||||
total_trades: int
|
||||
created_at: str
|
||||
params: dict[str, Any]
|
||||
|
||||
|
||||
class BacktestListResponse(BaseModel):
|
||||
"""Response for GET /api/backtests."""
|
||||
runs: list[BacktestSummary]
|
||||
total: int
|
||||
|
||||
|
||||
# --- Comparison Schemas ---
|
||||
|
||||
class CompareRequest(BaseModel):
|
||||
"""Request body for POST /api/compare."""
|
||||
run_ids: list[str] = Field(min_length=2, max_length=5)
|
||||
|
||||
|
||||
class CompareResult(BaseModel):
|
||||
"""Comparison of multiple backtest runs."""
|
||||
runs: list[BacktestResult]
|
||||
param_diff: dict[str, list[Any]]
|
||||
3
api/routers/__init__.py
Normal file
3
api/routers/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
"""
|
||||
API routers for backtest, strategies, and data endpoints.
|
||||
"""
|
||||
193
api/routers/backtest.py
Normal file
193
api/routers/backtest.py
Normal file
@@ -0,0 +1,193 @@
|
||||
"""
|
||||
Backtest execution and history endpoints.
|
||||
"""
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from api.models.database import get_db
|
||||
from api.models.schemas import (
|
||||
BacktestListResponse,
|
||||
BacktestRequest,
|
||||
BacktestResult,
|
||||
CompareRequest,
|
||||
CompareResult,
|
||||
)
|
||||
from api.services.runner import get_runner
|
||||
from api.services.storage import get_storage
|
||||
from engine.logging_config import get_logger
|
||||
|
||||
router = APIRouter()
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@router.post("/backtest", response_model=BacktestResult)
|
||||
async def run_backtest(
|
||||
request: BacktestRequest,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Execute a backtest with the specified configuration.
|
||||
|
||||
Runs the strategy on historical data and returns metrics,
|
||||
equity curve, and trade records. Results are automatically saved.
|
||||
"""
|
||||
runner = get_runner()
|
||||
storage = get_storage()
|
||||
|
||||
try:
|
||||
# Execute backtest
|
||||
result = runner.run(request)
|
||||
|
||||
# Save to database
|
||||
storage.save_run(db, result)
|
||||
|
||||
logger.info(
|
||||
"Backtest completed and saved: %s (return=%.2f%%, sharpe=%.2f)",
|
||||
result.run_id,
|
||||
result.metrics.total_return,
|
||||
result.metrics.sharpe_ratio,
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
except KeyError as e:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid strategy: {e}")
|
||||
except FileNotFoundError as e:
|
||||
raise HTTPException(status_code=404, detail=f"Data not found: {e}")
|
||||
except Exception as e:
|
||||
logger.error("Backtest failed: %s", e, exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=str(e))
|
||||
|
||||
|
||||
@router.get("/backtests", response_model=BacktestListResponse)
|
||||
async def list_backtests(
|
||||
limit: int = Query(50, ge=1, le=200),
|
||||
offset: int = Query(0, ge=0),
|
||||
strategy: str | None = None,
|
||||
symbol: str | None = None,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
List saved backtest runs with optional filtering.
|
||||
|
||||
Returns summaries for quick display in the history sidebar.
|
||||
"""
|
||||
storage = get_storage()
|
||||
|
||||
runs, total = storage.list_runs(
|
||||
db,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
strategy=strategy,
|
||||
symbol=symbol,
|
||||
)
|
||||
|
||||
return BacktestListResponse(runs=runs, total=total)
|
||||
|
||||
|
||||
@router.get("/backtest/{run_id}", response_model=BacktestResult)
|
||||
async def get_backtest(
|
||||
run_id: str,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Retrieve a specific backtest run by ID.
|
||||
|
||||
Returns full result including equity curve and trades.
|
||||
"""
|
||||
storage = get_storage()
|
||||
|
||||
result = storage.get_run(db, run_id)
|
||||
|
||||
if not result:
|
||||
raise HTTPException(status_code=404, detail=f"Run not found: {run_id}")
|
||||
|
||||
return result
|
||||
|
||||
|
||||
@router.delete("/backtest/{run_id}")
|
||||
async def delete_backtest(
|
||||
run_id: str,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Delete a backtest run.
|
||||
"""
|
||||
storage = get_storage()
|
||||
|
||||
deleted = storage.delete_run(db, run_id)
|
||||
|
||||
if not deleted:
|
||||
raise HTTPException(status_code=404, detail=f"Run not found: {run_id}")
|
||||
|
||||
return {"status": "deleted", "run_id": run_id}
|
||||
|
||||
|
||||
@router.post("/compare", response_model=CompareResult)
|
||||
async def compare_runs(
|
||||
request: CompareRequest,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Compare multiple backtest runs (2-5 runs).
|
||||
|
||||
Returns full results for each run plus parameter differences.
|
||||
"""
|
||||
storage = get_storage()
|
||||
|
||||
runs = storage.get_runs_by_ids(db, request.run_ids)
|
||||
|
||||
if len(runs) != len(request.run_ids):
|
||||
found_ids = {r.run_id for r in runs}
|
||||
missing = [rid for rid in request.run_ids if rid not in found_ids]
|
||||
raise HTTPException(
|
||||
status_code=404,
|
||||
detail=f"Runs not found: {missing}"
|
||||
)
|
||||
|
||||
# Calculate parameter differences
|
||||
param_diff = _calculate_param_diff(runs)
|
||||
|
||||
return CompareResult(runs=runs, param_diff=param_diff)
|
||||
|
||||
|
||||
def _calculate_param_diff(runs: list[BacktestResult]) -> dict[str, list[Any]]:
|
||||
"""
|
||||
Find parameters that differ between runs.
|
||||
|
||||
Returns dict mapping param name to list of values (one per run).
|
||||
"""
|
||||
if not runs:
|
||||
return {}
|
||||
|
||||
# Collect all param keys
|
||||
all_keys: set[str] = set()
|
||||
for run in runs:
|
||||
all_keys.update(run.params.keys())
|
||||
|
||||
# Also include strategy and key config
|
||||
all_keys.update(['strategy', 'symbol', 'leverage', 'timeframe'])
|
||||
|
||||
diff: dict[str, list[Any]] = {}
|
||||
|
||||
for key in sorted(all_keys):
|
||||
values = []
|
||||
for run in runs:
|
||||
if key == 'strategy':
|
||||
values.append(run.strategy)
|
||||
elif key == 'symbol':
|
||||
values.append(run.symbol)
|
||||
elif key == 'leverage':
|
||||
values.append(run.leverage)
|
||||
elif key == 'timeframe':
|
||||
values.append(run.timeframe)
|
||||
else:
|
||||
values.append(run.params.get(key))
|
||||
|
||||
# Only include if values differ
|
||||
if len(set(str(v) for v in values)) > 1:
|
||||
diff[key] = values
|
||||
|
||||
return diff
|
||||
97
api/routers/data.py
Normal file
97
api/routers/data.py
Normal file
@@ -0,0 +1,97 @@
|
||||
"""
|
||||
Data status and symbol information endpoints.
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
from fastapi import APIRouter
|
||||
|
||||
from api.models.schemas import DataStatusResponse, SymbolInfo
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# Base path for CCXT data
|
||||
DATA_BASE = Path(__file__).parent.parent.parent / "data" / "ccxt"
|
||||
|
||||
|
||||
def _scan_available_data() -> list[SymbolInfo]:
|
||||
"""
|
||||
Scan the data directory for available symbols and timeframes.
|
||||
|
||||
Returns list of SymbolInfo with date ranges and row counts.
|
||||
"""
|
||||
symbols = []
|
||||
|
||||
if not DATA_BASE.exists():
|
||||
return symbols
|
||||
|
||||
# Structure: data/ccxt/{exchange}/{market_type}/{symbol}/{timeframe}.csv
|
||||
for exchange_dir in DATA_BASE.iterdir():
|
||||
if not exchange_dir.is_dir():
|
||||
continue
|
||||
exchange = exchange_dir.name
|
||||
|
||||
for market_dir in exchange_dir.iterdir():
|
||||
if not market_dir.is_dir():
|
||||
continue
|
||||
market_type = market_dir.name
|
||||
|
||||
for symbol_dir in market_dir.iterdir():
|
||||
if not symbol_dir.is_dir():
|
||||
continue
|
||||
symbol = symbol_dir.name
|
||||
|
||||
# Find all timeframes
|
||||
timeframes = []
|
||||
start_date = None
|
||||
end_date = None
|
||||
row_count = 0
|
||||
|
||||
for csv_file in symbol_dir.glob("*.csv"):
|
||||
tf = csv_file.stem
|
||||
timeframes.append(tf)
|
||||
|
||||
# Read first and last rows for date range
|
||||
try:
|
||||
df = pd.read_csv(csv_file, parse_dates=['timestamp'])
|
||||
if not df.empty:
|
||||
row_count = len(df)
|
||||
start_date = df['timestamp'].min().strftime("%Y-%m-%d")
|
||||
end_date = df['timestamp'].max().strftime("%Y-%m-%d")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
if timeframes:
|
||||
symbols.append(SymbolInfo(
|
||||
symbol=symbol,
|
||||
exchange=exchange,
|
||||
market_type=market_type,
|
||||
timeframes=sorted(timeframes),
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
row_count=row_count,
|
||||
))
|
||||
|
||||
return symbols
|
||||
|
||||
|
||||
@router.get("/symbols", response_model=DataStatusResponse)
|
||||
async def get_symbols():
|
||||
"""
|
||||
Get list of available symbols with their data ranges.
|
||||
|
||||
Scans the local data directory for downloaded OHLCV data.
|
||||
"""
|
||||
symbols = _scan_available_data()
|
||||
return DataStatusResponse(symbols=symbols)
|
||||
|
||||
|
||||
@router.get("/data/status", response_model=DataStatusResponse)
|
||||
async def get_data_status():
|
||||
"""
|
||||
Get detailed data inventory status.
|
||||
|
||||
Alias for /symbols with additional metadata.
|
||||
"""
|
||||
symbols = _scan_available_data()
|
||||
return DataStatusResponse(symbols=symbols)
|
||||
67
api/routers/strategies.py
Normal file
67
api/routers/strategies.py
Normal file
@@ -0,0 +1,67 @@
|
||||
"""
|
||||
Strategy information endpoints.
|
||||
"""
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
from fastapi import APIRouter
|
||||
|
||||
from api.models.schemas import StrategiesResponse, StrategyInfo
|
||||
from strategies.factory import get_registry
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _serialize_param_value(value: Any) -> Any:
|
||||
"""Convert numpy arrays and other types to JSON-serializable format."""
|
||||
if isinstance(value, np.ndarray):
|
||||
return value.tolist()
|
||||
if isinstance(value, (np.integer, np.floating)):
|
||||
return value.item()
|
||||
return value
|
||||
|
||||
|
||||
def _get_display_name(name: str) -> str:
|
||||
"""Convert strategy key to display name."""
|
||||
display_names = {
|
||||
"rsi": "RSI Strategy",
|
||||
"macross": "MA Crossover",
|
||||
"meta_st": "Meta Supertrend",
|
||||
"regime": "Regime Reversion (ML)",
|
||||
}
|
||||
return display_names.get(name, name.replace("_", " ").title())
|
||||
|
||||
|
||||
@router.get("/strategies", response_model=StrategiesResponse)
|
||||
async def get_strategies():
|
||||
"""
|
||||
Get list of available strategies with their parameters.
|
||||
|
||||
Returns strategy names, default parameters, and grid search ranges.
|
||||
"""
|
||||
registry = get_registry()
|
||||
strategies = []
|
||||
|
||||
for name, config in registry.items():
|
||||
strategy_instance = config.strategy_class()
|
||||
|
||||
# Serialize parameters (convert numpy arrays to lists)
|
||||
default_params = {
|
||||
k: _serialize_param_value(v)
|
||||
for k, v in config.default_params.items()
|
||||
}
|
||||
grid_params = {
|
||||
k: _serialize_param_value(v)
|
||||
for k, v in config.grid_params.items()
|
||||
}
|
||||
|
||||
strategies.append(StrategyInfo(
|
||||
name=name,
|
||||
display_name=_get_display_name(name),
|
||||
market_type=strategy_instance.default_market_type.value,
|
||||
default_leverage=strategy_instance.default_leverage,
|
||||
default_params=default_params,
|
||||
grid_params=grid_params,
|
||||
))
|
||||
|
||||
return StrategiesResponse(strategies=strategies)
|
||||
3
api/services/__init__.py
Normal file
3
api/services/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
"""
|
||||
Business logic services for backtest execution and storage.
|
||||
"""
|
||||
300
api/services/runner.py
Normal file
300
api/services/runner.py
Normal file
@@ -0,0 +1,300 @@
|
||||
"""
|
||||
Backtest runner service that wraps the existing Backtester engine.
|
||||
"""
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from api.models.schemas import (
|
||||
BacktestMetrics,
|
||||
BacktestRequest,
|
||||
BacktestResult,
|
||||
EquityPoint,
|
||||
TradeRecord,
|
||||
)
|
||||
from engine.backtester import Backtester
|
||||
from engine.data_manager import DataManager
|
||||
from engine.logging_config import get_logger
|
||||
from engine.market import MarketType
|
||||
from strategies.factory import get_strategy
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class BacktestRunner:
|
||||
"""
|
||||
Service for executing backtests via the API.
|
||||
|
||||
Wraps the existing Backtester engine and converts results
|
||||
to API response format.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.dm = DataManager()
|
||||
self.bt = Backtester(self.dm)
|
||||
|
||||
def run(self, request: BacktestRequest) -> BacktestResult:
|
||||
"""
|
||||
Execute a backtest and return structured results.
|
||||
|
||||
Args:
|
||||
request: BacktestRequest with strategy, symbol, and parameters
|
||||
|
||||
Returns:
|
||||
BacktestResult with metrics, equity curve, and trades
|
||||
"""
|
||||
# Get strategy instance
|
||||
strategy, default_params = get_strategy(request.strategy, is_grid=False)
|
||||
|
||||
# Merge default params with request params
|
||||
params = {**default_params, **request.params}
|
||||
|
||||
# Convert market type string to enum
|
||||
market_type = MarketType(request.market_type)
|
||||
|
||||
# Override strategy market type if specified
|
||||
strategy.default_market_type = market_type
|
||||
|
||||
logger.info(
|
||||
"Running backtest: %s on %s (%s), params=%s",
|
||||
request.strategy, request.symbol, request.timeframe, params
|
||||
)
|
||||
|
||||
# Execute backtest
|
||||
result = self.bt.run_strategy(
|
||||
strategy=strategy,
|
||||
exchange_id=request.exchange,
|
||||
symbol=request.symbol,
|
||||
timeframe=request.timeframe,
|
||||
start_date=request.start_date,
|
||||
end_date=request.end_date,
|
||||
init_cash=request.init_cash,
|
||||
fees=request.fees,
|
||||
slippage=request.slippage,
|
||||
sl_stop=request.sl_stop,
|
||||
tp_stop=request.tp_stop,
|
||||
sl_trail=request.sl_trail,
|
||||
leverage=request.leverage,
|
||||
**params
|
||||
)
|
||||
|
||||
# Extract data from portfolio
|
||||
portfolio = result.portfolio
|
||||
|
||||
# Build trade records
|
||||
trades = self._build_trade_records(portfolio)
|
||||
|
||||
# Build equity curve (trimmed to trading period)
|
||||
equity_curve = self._build_equity_curve(portfolio)
|
||||
|
||||
# Build metrics
|
||||
metrics = self._build_metrics(result, portfolio)
|
||||
|
||||
# Get date range from actual trading period (first trade to end)
|
||||
idx = portfolio.wrapper.index
|
||||
end_date = idx[-1].strftime("%Y-%m-%d %H:%M")
|
||||
|
||||
# Use first trade time as start if trades exist
|
||||
trades_df = portfolio.trades.records_readable
|
||||
if not trades_df.empty:
|
||||
first_entry_col = 'Entry Timestamp' if 'Entry Timestamp' in trades_df.columns else 'Entry Time'
|
||||
if first_entry_col in trades_df.columns:
|
||||
first_trade_time = pd.to_datetime(trades_df[first_entry_col].iloc[0])
|
||||
start_date = first_trade_time.strftime("%Y-%m-%d %H:%M")
|
||||
else:
|
||||
start_date = idx[0].strftime("%Y-%m-%d %H:%M")
|
||||
else:
|
||||
start_date = idx[0].strftime("%Y-%m-%d %H:%M")
|
||||
|
||||
return BacktestResult(
|
||||
run_id=str(uuid.uuid4()),
|
||||
strategy=request.strategy,
|
||||
symbol=request.symbol,
|
||||
market_type=result.market_type.value,
|
||||
timeframe=request.timeframe,
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
leverage=result.leverage,
|
||||
params=params,
|
||||
metrics=metrics,
|
||||
equity_curve=equity_curve,
|
||||
trades=trades,
|
||||
created_at=datetime.now(timezone.utc).isoformat(),
|
||||
)
|
||||
|
||||
def _build_equity_curve(self, portfolio) -> list[EquityPoint]:
|
||||
"""Extract equity curve with drawdown from portfolio, starting from first trade."""
|
||||
value_series = portfolio.value()
|
||||
drawdown_series = portfolio.drawdown()
|
||||
|
||||
# Handle multi-column case (from grid search)
|
||||
if hasattr(value_series, 'columns') and len(value_series.columns) > 1:
|
||||
value_series = value_series.iloc[:, 0]
|
||||
drawdown_series = drawdown_series.iloc[:, 0]
|
||||
elif hasattr(value_series, 'columns'):
|
||||
value_series = value_series.iloc[:, 0]
|
||||
drawdown_series = drawdown_series.iloc[:, 0]
|
||||
|
||||
# Find first trade time to trim equity curve
|
||||
first_trade_idx = 0
|
||||
trades_df = portfolio.trades.records_readable
|
||||
if not trades_df.empty:
|
||||
first_entry_col = 'Entry Timestamp' if 'Entry Timestamp' in trades_df.columns else 'Entry Time'
|
||||
if first_entry_col in trades_df.columns:
|
||||
first_trade_time = pd.to_datetime(trades_df[first_entry_col].iloc[0])
|
||||
# Find index in value_series closest to first trade
|
||||
if hasattr(value_series.index, 'get_indexer'):
|
||||
first_trade_idx = value_series.index.get_indexer([first_trade_time], method='nearest')[0]
|
||||
# Start a few bars before first trade for context
|
||||
first_trade_idx = max(0, first_trade_idx - 5)
|
||||
|
||||
# Slice from first trade onwards
|
||||
value_series = value_series.iloc[first_trade_idx:]
|
||||
drawdown_series = drawdown_series.iloc[first_trade_idx:]
|
||||
|
||||
points = []
|
||||
for i, (ts, val) in enumerate(value_series.items()):
|
||||
dd = drawdown_series.iloc[i] if i < len(drawdown_series) else 0.0
|
||||
points.append(EquityPoint(
|
||||
timestamp=ts.isoformat(),
|
||||
value=float(val),
|
||||
drawdown=float(dd) * 100, # Convert to percentage
|
||||
))
|
||||
|
||||
return points
|
||||
|
||||
def _build_trade_records(self, portfolio) -> list[TradeRecord]:
|
||||
"""Extract trade records from portfolio."""
|
||||
trades_df = portfolio.trades.records_readable
|
||||
|
||||
if trades_df.empty:
|
||||
return []
|
||||
|
||||
records = []
|
||||
for _, row in trades_df.iterrows():
|
||||
# Handle different column names in vectorbt
|
||||
entry_time = row.get('Entry Timestamp', row.get('Entry Time', ''))
|
||||
exit_time = row.get('Exit Timestamp', row.get('Exit Time', ''))
|
||||
|
||||
records.append(TradeRecord(
|
||||
entry_time=str(entry_time) if pd.notna(entry_time) else "",
|
||||
exit_time=str(exit_time) if pd.notna(exit_time) else None,
|
||||
entry_price=float(row.get('Avg Entry Price', row.get('Entry Price', 0))),
|
||||
exit_price=float(row.get('Avg Exit Price', row.get('Exit Price', 0)))
|
||||
if pd.notna(row.get('Avg Exit Price', row.get('Exit Price'))) else None,
|
||||
size=float(row.get('Size', 0)),
|
||||
direction=str(row.get('Direction', 'Long')),
|
||||
pnl=float(row.get('PnL', 0)) if pd.notna(row.get('PnL')) else None,
|
||||
return_pct=float(row.get('Return', 0)) * 100
|
||||
if pd.notna(row.get('Return')) else None,
|
||||
status="closed" if pd.notna(exit_time) else "open",
|
||||
))
|
||||
|
||||
return records
|
||||
|
||||
def _build_metrics(self, result, portfolio) -> BacktestMetrics:
|
||||
"""Build metrics from backtest result."""
|
||||
stats = portfolio.stats()
|
||||
|
||||
# Extract values, handling potential multi-column results
|
||||
def get_stat(key: str, default: float = 0.0) -> float:
|
||||
val = stats.get(key, default)
|
||||
if hasattr(val, 'mean'):
|
||||
return float(val.mean())
|
||||
return float(val) if pd.notna(val) else default
|
||||
|
||||
total_return = portfolio.total_return()
|
||||
if hasattr(total_return, 'mean'):
|
||||
total_return = total_return.mean()
|
||||
|
||||
# Calculate benchmark return from first trade to end (not full period)
|
||||
# This gives accurate comparison when strategy has training period
|
||||
close = portfolio.close
|
||||
benchmark_return = 0.0
|
||||
|
||||
if hasattr(close, 'iloc'):
|
||||
# Find first trade entry time
|
||||
trades_df = portfolio.trades.records_readable
|
||||
if not trades_df.empty:
|
||||
# Get the first trade entry timestamp
|
||||
first_entry_col = 'Entry Timestamp' if 'Entry Timestamp' in trades_df.columns else 'Entry Time'
|
||||
if first_entry_col in trades_df.columns:
|
||||
first_trade_time = pd.to_datetime(trades_df[first_entry_col].iloc[0])
|
||||
|
||||
# Find the price at first trade
|
||||
if hasattr(close.index, 'get_indexer'):
|
||||
# Find closest index to first trade time
|
||||
idx = close.index.get_indexer([first_trade_time], method='nearest')[0]
|
||||
start_price = close.iloc[idx]
|
||||
else:
|
||||
start_price = close.iloc[0]
|
||||
|
||||
end_price = close.iloc[-1]
|
||||
|
||||
if hasattr(start_price, 'mean'):
|
||||
start_price = start_price.mean()
|
||||
if hasattr(end_price, 'mean'):
|
||||
end_price = end_price.mean()
|
||||
|
||||
benchmark_return = ((end_price - start_price) / start_price)
|
||||
else:
|
||||
# No trades - use full period
|
||||
start_price = close.iloc[0]
|
||||
end_price = close.iloc[-1]
|
||||
if hasattr(start_price, 'mean'):
|
||||
start_price = start_price.mean()
|
||||
if hasattr(end_price, 'mean'):
|
||||
end_price = end_price.mean()
|
||||
benchmark_return = ((end_price - start_price) / start_price)
|
||||
|
||||
# Alpha = strategy return - benchmark return
|
||||
alpha = float(total_return) - float(benchmark_return)
|
||||
|
||||
sharpe = portfolio.sharpe_ratio()
|
||||
if hasattr(sharpe, 'mean'):
|
||||
sharpe = sharpe.mean()
|
||||
|
||||
max_dd = portfolio.max_drawdown()
|
||||
if hasattr(max_dd, 'mean'):
|
||||
max_dd = max_dd.mean()
|
||||
|
||||
win_rate = portfolio.trades.win_rate()
|
||||
if hasattr(win_rate, 'mean'):
|
||||
win_rate = win_rate.mean()
|
||||
|
||||
trade_count = portfolio.trades.count()
|
||||
if hasattr(trade_count, 'mean'):
|
||||
trade_count = int(trade_count.mean())
|
||||
else:
|
||||
trade_count = int(trade_count)
|
||||
|
||||
return BacktestMetrics(
|
||||
total_return=float(total_return) * 100,
|
||||
benchmark_return=float(benchmark_return) * 100,
|
||||
alpha=float(alpha) * 100,
|
||||
sharpe_ratio=float(sharpe) if pd.notna(sharpe) else 0.0,
|
||||
max_drawdown=float(max_dd) * 100,
|
||||
win_rate=float(win_rate) * 100 if pd.notna(win_rate) else 0.0,
|
||||
total_trades=trade_count,
|
||||
profit_factor=get_stat('Profit Factor'),
|
||||
avg_trade_return=get_stat('Avg Winning Trade [%]'),
|
||||
total_fees=get_stat('Total Fees Paid'),
|
||||
total_funding=result.total_funding_paid,
|
||||
liquidation_count=result.liquidation_count,
|
||||
liquidation_loss=result.total_liquidation_loss,
|
||||
adjusted_return=result.adjusted_return,
|
||||
)
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_runner: BacktestRunner | None = None
|
||||
|
||||
|
||||
def get_runner() -> BacktestRunner:
|
||||
"""Get or create the backtest runner instance."""
|
||||
global _runner
|
||||
if _runner is None:
|
||||
_runner = BacktestRunner()
|
||||
return _runner
|
||||
225
api/services/storage.py
Normal file
225
api/services/storage.py
Normal file
@@ -0,0 +1,225 @@
|
||||
"""
|
||||
Storage service for persisting and retrieving backtest runs.
|
||||
"""
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from api.models.database import BacktestRun
|
||||
from api.models.schemas import (
|
||||
BacktestResult,
|
||||
BacktestSummary,
|
||||
EquityPoint,
|
||||
BacktestMetrics,
|
||||
TradeRecord,
|
||||
)
|
||||
|
||||
|
||||
class StorageService:
|
||||
"""
|
||||
Service for saving and loading backtest runs from SQLite.
|
||||
"""
|
||||
|
||||
def save_run(self, db: Session, result: BacktestResult) -> BacktestRun:
|
||||
"""
|
||||
Persist a backtest result to the database.
|
||||
|
||||
Args:
|
||||
db: Database session
|
||||
result: BacktestResult to save
|
||||
|
||||
Returns:
|
||||
Created BacktestRun record
|
||||
"""
|
||||
run = BacktestRun(
|
||||
run_id=result.run_id,
|
||||
strategy=result.strategy,
|
||||
symbol=result.symbol,
|
||||
market_type=result.market_type,
|
||||
timeframe=result.timeframe,
|
||||
leverage=result.leverage,
|
||||
params=result.params,
|
||||
start_date=result.start_date,
|
||||
end_date=result.end_date,
|
||||
total_return=result.metrics.total_return,
|
||||
benchmark_return=result.metrics.benchmark_return,
|
||||
alpha=result.metrics.alpha,
|
||||
sharpe_ratio=result.metrics.sharpe_ratio,
|
||||
max_drawdown=result.metrics.max_drawdown,
|
||||
win_rate=result.metrics.win_rate,
|
||||
total_trades=result.metrics.total_trades,
|
||||
profit_factor=result.metrics.profit_factor,
|
||||
total_fees=result.metrics.total_fees,
|
||||
total_funding=result.metrics.total_funding,
|
||||
liquidation_count=result.metrics.liquidation_count,
|
||||
liquidation_loss=result.metrics.liquidation_loss,
|
||||
adjusted_return=result.metrics.adjusted_return,
|
||||
)
|
||||
|
||||
# Serialize complex data
|
||||
run.set_equity_curve([p.model_dump() for p in result.equity_curve])
|
||||
run.set_trades([t.model_dump() for t in result.trades])
|
||||
|
||||
db.add(run)
|
||||
db.commit()
|
||||
db.refresh(run)
|
||||
|
||||
return run
|
||||
|
||||
def get_run(self, db: Session, run_id: str) -> BacktestResult | None:
|
||||
"""
|
||||
Retrieve a backtest run by ID.
|
||||
|
||||
Args:
|
||||
db: Database session
|
||||
run_id: UUID of the run
|
||||
|
||||
Returns:
|
||||
BacktestResult or None if not found
|
||||
"""
|
||||
run = db.query(BacktestRun).filter(BacktestRun.run_id == run_id).first()
|
||||
|
||||
if not run:
|
||||
return None
|
||||
|
||||
return self._to_result(run)
|
||||
|
||||
def list_runs(
|
||||
self,
|
||||
db: Session,
|
||||
limit: int = 50,
|
||||
offset: int = 0,
|
||||
strategy: str | None = None,
|
||||
symbol: str | None = None,
|
||||
) -> tuple[list[BacktestSummary], int]:
|
||||
"""
|
||||
List backtest runs with optional filtering.
|
||||
|
||||
Args:
|
||||
db: Database session
|
||||
limit: Maximum number of runs to return
|
||||
offset: Offset for pagination
|
||||
strategy: Filter by strategy name
|
||||
symbol: Filter by symbol
|
||||
|
||||
Returns:
|
||||
Tuple of (list of summaries, total count)
|
||||
"""
|
||||
query = db.query(BacktestRun)
|
||||
|
||||
if strategy:
|
||||
query = query.filter(BacktestRun.strategy == strategy)
|
||||
if symbol:
|
||||
query = query.filter(BacktestRun.symbol == symbol)
|
||||
|
||||
total = query.count()
|
||||
|
||||
runs = query.order_by(BacktestRun.created_at.desc()).offset(offset).limit(limit).all()
|
||||
|
||||
summaries = [self._to_summary(run) for run in runs]
|
||||
|
||||
return summaries, total
|
||||
|
||||
def get_runs_by_ids(self, db: Session, run_ids: list[str]) -> list[BacktestResult]:
|
||||
"""
|
||||
Retrieve multiple runs by their IDs.
|
||||
|
||||
Args:
|
||||
db: Database session
|
||||
run_ids: List of run UUIDs
|
||||
|
||||
Returns:
|
||||
List of BacktestResults (preserves order)
|
||||
"""
|
||||
runs = db.query(BacktestRun).filter(BacktestRun.run_id.in_(run_ids)).all()
|
||||
|
||||
# Create lookup and preserve order
|
||||
run_map = {run.run_id: run for run in runs}
|
||||
results = []
|
||||
|
||||
for run_id in run_ids:
|
||||
if run_id in run_map:
|
||||
results.append(self._to_result(run_map[run_id]))
|
||||
|
||||
return results
|
||||
|
||||
def delete_run(self, db: Session, run_id: str) -> bool:
|
||||
"""
|
||||
Delete a backtest run.
|
||||
|
||||
Args:
|
||||
db: Database session
|
||||
run_id: UUID of the run to delete
|
||||
|
||||
Returns:
|
||||
True if deleted, False if not found
|
||||
"""
|
||||
run = db.query(BacktestRun).filter(BacktestRun.run_id == run_id).first()
|
||||
|
||||
if not run:
|
||||
return False
|
||||
|
||||
db.delete(run)
|
||||
db.commit()
|
||||
return True
|
||||
|
||||
def _to_result(self, run: BacktestRun) -> BacktestResult:
|
||||
"""Convert database record to BacktestResult schema."""
|
||||
equity_data = run.get_equity_curve()
|
||||
trades_data = run.get_trades()
|
||||
|
||||
return BacktestResult(
|
||||
run_id=run.run_id,
|
||||
strategy=run.strategy,
|
||||
symbol=run.symbol,
|
||||
market_type=run.market_type,
|
||||
timeframe=run.timeframe,
|
||||
start_date=run.start_date or "",
|
||||
end_date=run.end_date or "",
|
||||
leverage=run.leverage,
|
||||
params=run.params or {},
|
||||
metrics=BacktestMetrics(
|
||||
total_return=run.total_return,
|
||||
benchmark_return=run.benchmark_return or 0.0,
|
||||
alpha=run.alpha or 0.0,
|
||||
sharpe_ratio=run.sharpe_ratio,
|
||||
max_drawdown=run.max_drawdown,
|
||||
win_rate=run.win_rate,
|
||||
total_trades=run.total_trades,
|
||||
profit_factor=run.profit_factor,
|
||||
total_fees=run.total_fees,
|
||||
total_funding=run.total_funding,
|
||||
liquidation_count=run.liquidation_count,
|
||||
liquidation_loss=run.liquidation_loss,
|
||||
adjusted_return=run.adjusted_return,
|
||||
),
|
||||
equity_curve=[EquityPoint(**p) for p in equity_data],
|
||||
trades=[TradeRecord(**t) for t in trades_data],
|
||||
created_at=run.created_at.isoformat() if run.created_at else "",
|
||||
)
|
||||
|
||||
def _to_summary(self, run: BacktestRun) -> BacktestSummary:
|
||||
"""Convert database record to BacktestSummary schema."""
|
||||
return BacktestSummary(
|
||||
run_id=run.run_id,
|
||||
strategy=run.strategy,
|
||||
symbol=run.symbol,
|
||||
market_type=run.market_type,
|
||||
timeframe=run.timeframe,
|
||||
total_return=run.total_return,
|
||||
sharpe_ratio=run.sharpe_ratio,
|
||||
max_drawdown=run.max_drawdown,
|
||||
total_trades=run.total_trades,
|
||||
created_at=run.created_at.isoformat() if run.created_at else "",
|
||||
params=run.params or {},
|
||||
)
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_storage: StorageService | None = None
|
||||
|
||||
|
||||
def get_storage() -> StorageService:
|
||||
"""Get or create the storage service instance."""
|
||||
global _storage
|
||||
if _storage is None:
|
||||
_storage = StorageService()
|
||||
return _storage
|
||||
70
backtest.py
70
backtest.py
@@ -1,70 +0,0 @@
|
||||
from __future__ import annotations
|
||||
import pandas as pd
|
||||
from pathlib import Path
|
||||
from trade import TradeState, enter_long, exit_long, maybe_trailing_stop
|
||||
from indicators import add_supertrends, compute_meta_trend
|
||||
from metrics import compute_metrics
|
||||
from logging_utils import write_trade_log
|
||||
|
||||
DEFAULT_ST_SETTINGS = [(12, 3.0), (10, 1.0), (11, 2.0)]
|
||||
|
||||
def backtest(
|
||||
df: pd.DataFrame,
|
||||
df_1min: pd.DataFrame,
|
||||
timeframe_minutes: int,
|
||||
stop_loss: float,
|
||||
exit_on_bearish_flip: bool,
|
||||
fee_bps: float,
|
||||
slippage_bps: float,
|
||||
log_path: Path | None = None,
|
||||
):
|
||||
df = add_supertrends(df, DEFAULT_ST_SETTINGS)
|
||||
df["meta_bull"] = compute_meta_trend(df, DEFAULT_ST_SETTINGS)
|
||||
|
||||
state = TradeState(stop_loss_frac=stop_loss, fee_bps=fee_bps, slippage_bps=slippage_bps)
|
||||
equity, trades = [], []
|
||||
|
||||
for i, row in df.iterrows():
|
||||
price = float(row["Close"])
|
||||
ts = pd.Timestamp(row["Timestamp"])
|
||||
|
||||
if state.qty <= 0 and row["meta_bull"] == 1:
|
||||
evt = enter_long(state, price)
|
||||
if evt:
|
||||
evt.update({"t": ts.isoformat(), "reason": "bull_flip"})
|
||||
trades.append(evt)
|
||||
|
||||
start = ts
|
||||
end = df["Timestamp"].iat[i + 1] if i + 1 < len(df) else ts + pd.Timedelta(minutes=timeframe_minutes)
|
||||
|
||||
if state.qty > 0:
|
||||
win = df_1min[(df_1min["Timestamp"] >= start) & (df_1min["Timestamp"] < end)]
|
||||
for _, m in win.iterrows():
|
||||
hi = float(m["High"])
|
||||
lo = float(m["Low"])
|
||||
state.max_px = max(state.max_px or hi, hi)
|
||||
trail = state.max_px * (1.0 - state.stop_loss_frac)
|
||||
if lo <= trail:
|
||||
evt = exit_long(state, trail)
|
||||
if evt:
|
||||
prev = trades[-1]
|
||||
pnl = (evt["price"] - (prev.get("price") or evt["price"])) * (prev.get("qty") or 0.0)
|
||||
evt.update({"t": pd.Timestamp(m["Timestamp"]).isoformat(), "reason": "stop", "pnl": pnl})
|
||||
trades.append(evt)
|
||||
break
|
||||
|
||||
if state.qty > 0 and exit_on_bearish_flip and row["meta_bull"] == 0:
|
||||
evt = exit_long(state, price)
|
||||
if evt:
|
||||
prev = trades[-1]
|
||||
pnl = (evt["price"] - (prev.get("price") or evt["price"])) * (prev.get("qty") or 0.0)
|
||||
evt.update({"t": ts.isoformat(), "reason": "bearish_flip", "pnl": pnl})
|
||||
trades.append(evt)
|
||||
|
||||
equity.append(state.cash + state.qty * price)
|
||||
|
||||
equity_curve = pd.Series(equity, index=df["Timestamp"])
|
||||
if log_path:
|
||||
write_trade_log(trades, log_path)
|
||||
perf = compute_metrics(equity_curve, trades)
|
||||
return perf, equity_curve, trades
|
||||
28
check_symbols.py
Normal file
28
check_symbols.py
Normal file
@@ -0,0 +1,28 @@
|
||||
import ccxt
|
||||
import sys
|
||||
|
||||
def main():
|
||||
try:
|
||||
exchange = ccxt.okx()
|
||||
print("Loading markets...")
|
||||
markets = exchange.load_markets()
|
||||
|
||||
# Filter for ETH perpetuals
|
||||
eth_perps = [
|
||||
symbol for symbol, market in markets.items()
|
||||
if 'ETH' in symbol and 'USDT' in symbol and market.get('swap') and market.get('linear')
|
||||
]
|
||||
|
||||
print(f"\nFound {len(eth_perps)} ETH Linear Perps:")
|
||||
for symbol in eth_perps:
|
||||
market = markets[symbol]
|
||||
print(f" CCXT Symbol: {symbol}")
|
||||
print(f" Exchange ID: {market['id']}")
|
||||
print(f" Type: {market['type']}")
|
||||
print("-" * 30)
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error: {e}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
80
cli.py
80
cli.py
@@ -1,80 +0,0 @@
|
||||
from __future__ import annotations
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import pandas as pd
|
||||
|
||||
from config import CLIConfig
|
||||
from data import load_data
|
||||
from backtest import backtest
|
||||
|
||||
def parse_args() -> CLIConfig:
|
||||
p = argparse.ArgumentParser(prog="bt", description="Simple supertrend backtester")
|
||||
p.add_argument("start")
|
||||
p.add_argument("end")
|
||||
p.add_argument("--timeframe-minutes", type=int, default=15) # single TF
|
||||
p.add_argument("--timeframes-minutes", nargs="+", type=int) # multi TF: e.g. 5 15 60 240
|
||||
p.add_argument("--stop-loss", dest="stop_losses", type=float, nargs="+", default=[0.02, 0.05])
|
||||
p.add_argument("--exit-on-bearish-flip", action="store_true")
|
||||
p.add_argument("--csv", dest="csv_path", type=Path, required=True)
|
||||
p.add_argument("--out-csv", type=Path, default=Path("summary.csv"))
|
||||
p.add_argument("--log-dir", type=Path, default=Path("./logs"))
|
||||
p.add_argument("--fee-bps", type=float, default=10.0)
|
||||
p.add_argument("--slippage-bps", type=float, default=2.0)
|
||||
a = p.parse_args()
|
||||
|
||||
return CLIConfig(
|
||||
start=a.start,
|
||||
end=a.end,
|
||||
timeframe_minutes=a.timeframe_minutes,
|
||||
timeframes_minutes=a.timeframes_minutes,
|
||||
stop_losses=a.stop_losses,
|
||||
exit_on_bearish_flip=a.exit_on_bearish_flip,
|
||||
csv_path=a.csv_path,
|
||||
out_csv=a.out_csv,
|
||||
log_dir=a.log_dir,
|
||||
fee_bps=a.fee_bps,
|
||||
slippage_bps=a.slippage_bps,
|
||||
)
|
||||
|
||||
def main():
|
||||
cfg = parse_args()
|
||||
frames = cfg.timeframes_minutes or [cfg.timeframe_minutes]
|
||||
|
||||
rows: list[dict] = []
|
||||
for tfm in frames:
|
||||
df_1min, df = load_data(cfg.start, cfg.end, tfm, cfg.csv_path)
|
||||
for sl in cfg.stop_losses:
|
||||
log_path = cfg.log_dir / f"{tfm}m_sl{sl:.2%}.csv"
|
||||
perf, equity, _ = backtest(
|
||||
df=df,
|
||||
df_1min=df_1min,
|
||||
timeframe_minutes=tfm,
|
||||
stop_loss=sl,
|
||||
exit_on_bearish_flip=cfg.exit_on_bearish_flip,
|
||||
fee_bps=cfg.fee_bps,
|
||||
slippage_bps=cfg.slippage_bps,
|
||||
log_path=log_path,
|
||||
)
|
||||
rows.append({
|
||||
"timeframe": f"{tfm}min",
|
||||
"stop_loss": sl,
|
||||
"exit_on_bearish_flip": cfg.exit_on_bearish_flip,
|
||||
"total_return": f"{perf.total_return:.2%}",
|
||||
"max_drawdown": f"{perf.max_drawdown:.2%}",
|
||||
"sharpe_ratio": f"{perf.sharpe_ratio:.2f}",
|
||||
"win_rate": f"{perf.win_rate:.2%}",
|
||||
"num_trades": perf.num_trades,
|
||||
"final_equity": f"${perf.final_equity:.2f}",
|
||||
"initial_equity": f"${perf.initial_equity:.2f}",
|
||||
"num_stop_losses": perf.num_stop_losses,
|
||||
"total_fees": perf.total_fees,
|
||||
"total_slippage_usd": perf.total_slippage_usd,
|
||||
"avg_slippage_bps": perf.avg_slippage_bps,
|
||||
})
|
||||
|
||||
out = pd.DataFrame(rows)
|
||||
out.to_csv(cfg.out_csv, index=False)
|
||||
print(out.to_string(index=False))
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
18
config.py
18
config.py
@@ -1,18 +0,0 @@
|
||||
from __future__ import annotations
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Sequence
|
||||
|
||||
@dataclass
|
||||
class CLIConfig:
|
||||
start: str
|
||||
end: str
|
||||
timeframe_minutes: int
|
||||
timeframes_minutes: list[int] | None
|
||||
stop_losses: Sequence[float]
|
||||
exit_on_bearish_flip: bool
|
||||
csv_path: Path | None
|
||||
out_csv: Path
|
||||
log_dir: Path
|
||||
fee_bps: float
|
||||
slippage_bps: float
|
||||
24
data.py
24
data.py
@@ -1,24 +0,0 @@
|
||||
from __future__ import annotations
|
||||
import pandas as pd
|
||||
from pathlib import Path
|
||||
|
||||
def load_data(start: str, end: str, timeframe_minutes: int, csv_path: Path) -> tuple[pd.DataFrame, pd.DataFrame]:
|
||||
df_1min = pd.read_csv(csv_path)
|
||||
df_1min["Timestamp"] = pd.to_datetime(df_1min["Timestamp"], unit="s", utc=True)
|
||||
df_1min = df_1min[(df_1min["Timestamp"] >= pd.Timestamp(start, tz="UTC")) &
|
||||
(df_1min["Timestamp"] <= pd.Timestamp(end, tz="UTC"))] \
|
||||
.sort_values("Timestamp").reset_index(drop=True)
|
||||
|
||||
if timeframe_minutes != 1:
|
||||
g = df_1min.set_index("Timestamp").resample(f"{timeframe_minutes}min")
|
||||
df = pd.DataFrame({
|
||||
"Open": g["Open"].first(),
|
||||
"High": g["High"].max(),
|
||||
"Low": g["Low"].min(),
|
||||
"Close": g["Close"].last(),
|
||||
"Volume": g["Volume"].sum(),
|
||||
}).dropna().reset_index()
|
||||
else:
|
||||
df = df_1min.copy()
|
||||
|
||||
return df_1min, df
|
||||
377
engine/backtester.py
Normal file
377
engine/backtester.py
Normal file
@@ -0,0 +1,377 @@
|
||||
"""
|
||||
Core backtesting engine for running strategy simulations.
|
||||
|
||||
Supports multiple market types with realistic trading conditions.
|
||||
"""
|
||||
from dataclasses import dataclass
|
||||
|
||||
import pandas as pd
|
||||
import vectorbt as vbt
|
||||
|
||||
from engine.data_manager import DataManager
|
||||
from engine.logging_config import get_logger
|
||||
from engine.market import MarketType, get_market_config
|
||||
from engine.optimizer import WalkForwardOptimizer
|
||||
from engine.portfolio import run_long_only_portfolio, run_long_short_portfolio
|
||||
from engine.risk import (
|
||||
LiquidationEvent,
|
||||
calculate_funding,
|
||||
calculate_liquidation_adjustment,
|
||||
inject_liquidation_exits,
|
||||
)
|
||||
from strategies.base import BaseStrategy
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class BacktestResult:
|
||||
"""
|
||||
Container for backtest results with market-specific metrics.
|
||||
|
||||
Attributes:
|
||||
portfolio: VectorBT Portfolio object
|
||||
market_type: Market type used for the backtest
|
||||
leverage: Effective leverage used
|
||||
total_funding_paid: Total funding fees paid (perpetuals only)
|
||||
liquidation_count: Number of positions that were liquidated
|
||||
liquidation_events: Detailed list of liquidation events
|
||||
total_liquidation_loss: Total margin lost from liquidations
|
||||
adjusted_return: Return adjusted for liquidation losses (percentage)
|
||||
"""
|
||||
portfolio: vbt.Portfolio
|
||||
market_type: MarketType
|
||||
leverage: int
|
||||
total_funding_paid: float = 0.0
|
||||
liquidation_count: int = 0
|
||||
liquidation_events: list[LiquidationEvent] | None = None
|
||||
total_liquidation_loss: float = 0.0
|
||||
adjusted_return: float | None = None
|
||||
|
||||
|
||||
class Backtester:
|
||||
"""
|
||||
Backtester supporting multiple market types with realistic simulation.
|
||||
|
||||
Features:
|
||||
- Spot and Perpetual market support
|
||||
- Long and short position handling
|
||||
- Leverage simulation
|
||||
- Funding rate calculation (perpetuals)
|
||||
- Liquidation warnings
|
||||
"""
|
||||
|
||||
def __init__(self, data_manager: DataManager):
|
||||
self.dm = data_manager
|
||||
|
||||
def run_strategy(
|
||||
self,
|
||||
strategy: BaseStrategy,
|
||||
exchange_id: str,
|
||||
symbol: str,
|
||||
timeframe: str = '1m',
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
init_cash: float = 10000,
|
||||
fees: float | None = None,
|
||||
slippage: float = 0.001,
|
||||
sl_stop: float | None = None,
|
||||
tp_stop: float | None = None,
|
||||
sl_trail: bool = False,
|
||||
leverage: int | None = None,
|
||||
**strategy_params
|
||||
) -> BacktestResult:
|
||||
"""
|
||||
Run a backtest with market-type-aware simulation.
|
||||
|
||||
Args:
|
||||
strategy: Strategy instance to backtest
|
||||
exchange_id: Exchange identifier (e.g., 'okx')
|
||||
symbol: Trading pair (e.g., 'BTC/USDT')
|
||||
timeframe: Data timeframe (e.g., '1m', '1h', '1d')
|
||||
start_date: Start date filter (YYYY-MM-DD)
|
||||
end_date: End date filter (YYYY-MM-DD)
|
||||
init_cash: Initial capital (margin for leveraged)
|
||||
fees: Transaction fee override (uses market default if None)
|
||||
slippage: Slippage percentage
|
||||
sl_stop: Stop loss percentage
|
||||
tp_stop: Take profit percentage
|
||||
sl_trail: Enable trailing stop loss
|
||||
leverage: Leverage override (uses strategy default if None)
|
||||
**strategy_params: Additional strategy parameters
|
||||
|
||||
Returns:
|
||||
BacktestResult with portfolio and market-specific metrics
|
||||
"""
|
||||
# Get market configuration from strategy
|
||||
market_type = strategy.default_market_type
|
||||
market_config = get_market_config(market_type)
|
||||
|
||||
# Resolve leverage and fees
|
||||
effective_leverage = self._resolve_leverage(leverage, strategy, market_type)
|
||||
effective_fees = fees if fees is not None else market_config.taker_fee
|
||||
|
||||
# Load and filter data
|
||||
df = self._load_data(
|
||||
exchange_id, symbol, timeframe, market_type, start_date, end_date
|
||||
)
|
||||
|
||||
close_price = df['close']
|
||||
high_price = df['high']
|
||||
low_price = df['low']
|
||||
open_price = df['open']
|
||||
volume = df['volume']
|
||||
|
||||
# Run strategy logic
|
||||
signals = strategy.run(
|
||||
close_price,
|
||||
high=high_price,
|
||||
low=low_price,
|
||||
open=open_price,
|
||||
volume=volume,
|
||||
**strategy_params
|
||||
)
|
||||
|
||||
# Normalize signals to 5-tuple format
|
||||
signals = self._normalize_signals(signals, close_price, market_config)
|
||||
long_entries, long_exits, short_entries, short_exits, size = signals
|
||||
|
||||
# Default size if None
|
||||
if size is None:
|
||||
size = 1.0
|
||||
|
||||
# Convert leverage multiplier to raw value (USD) for vbt
|
||||
# This works around "SizeType.Percent reversal" error
|
||||
# Effectively "Fixed Fractional" sizing based on Initial Capital
|
||||
# (Does not compound, but safe for backtesting)
|
||||
if isinstance(size, pd.Series):
|
||||
size = size * init_cash
|
||||
else:
|
||||
size = size * init_cash
|
||||
|
||||
# Process liquidations - inject forced exits at liquidation points
|
||||
liquidation_events: list[LiquidationEvent] = []
|
||||
if effective_leverage > 1:
|
||||
long_exits, short_exits, liquidation_events = inject_liquidation_exits(
|
||||
close_price, high_price, low_price,
|
||||
long_entries, long_exits,
|
||||
short_entries, short_exits,
|
||||
effective_leverage,
|
||||
market_config.maintenance_margin_rate
|
||||
)
|
||||
|
||||
# Calculate perpetual-specific metrics (after liquidation processing)
|
||||
total_funding = 0.0
|
||||
if market_type == MarketType.PERPETUAL:
|
||||
total_funding = calculate_funding(
|
||||
close_price,
|
||||
long_entries, long_exits,
|
||||
short_entries, short_exits,
|
||||
market_config,
|
||||
effective_leverage
|
||||
)
|
||||
|
||||
# Run portfolio simulation with liquidation-aware exits
|
||||
portfolio = self._run_portfolio(
|
||||
close_price, market_config,
|
||||
long_entries, long_exits,
|
||||
short_entries, short_exits,
|
||||
init_cash, effective_fees, slippage, timeframe,
|
||||
sl_stop, tp_stop, sl_trail, effective_leverage,
|
||||
size=size
|
||||
)
|
||||
|
||||
# Calculate adjusted returns accounting for liquidation losses
|
||||
total_liq_loss, liq_adjustment = calculate_liquidation_adjustment(
|
||||
liquidation_events, init_cash, effective_leverage
|
||||
)
|
||||
|
||||
raw_return = portfolio.total_return().mean() * 100
|
||||
adjusted_return = raw_return - liq_adjustment
|
||||
|
||||
if liquidation_events:
|
||||
logger.info(
|
||||
"Liquidation impact: %d events, $%.2f margin lost, %.2f%% adjustment",
|
||||
len(liquidation_events), total_liq_loss, liq_adjustment
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"Backtest completed: %s market, %dx leverage, fees=%.4f%%",
|
||||
market_type.value, effective_leverage, effective_fees * 100
|
||||
)
|
||||
|
||||
return BacktestResult(
|
||||
portfolio=portfolio,
|
||||
market_type=market_type,
|
||||
leverage=effective_leverage,
|
||||
total_funding_paid=total_funding,
|
||||
liquidation_count=len(liquidation_events),
|
||||
liquidation_events=liquidation_events,
|
||||
total_liquidation_loss=total_liq_loss,
|
||||
adjusted_return=adjusted_return
|
||||
)
|
||||
|
||||
def _resolve_leverage(
|
||||
self,
|
||||
leverage: int | None,
|
||||
strategy: BaseStrategy,
|
||||
market_type: MarketType
|
||||
) -> int:
|
||||
"""Resolve effective leverage from CLI, strategy default, or market type."""
|
||||
effective = leverage or strategy.default_leverage
|
||||
if market_type == MarketType.SPOT:
|
||||
return 1 # Spot cannot have leverage
|
||||
return effective
|
||||
|
||||
def _load_data(
|
||||
self,
|
||||
exchange_id: str,
|
||||
symbol: str,
|
||||
timeframe: str,
|
||||
market_type: MarketType,
|
||||
start_date: str | None,
|
||||
end_date: str | None
|
||||
) -> pd.DataFrame:
|
||||
"""Load and filter OHLCV data."""
|
||||
try:
|
||||
df = self.dm.load_data(exchange_id, symbol, timeframe, market_type)
|
||||
except FileNotFoundError:
|
||||
logger.warning("Data not found locally. Attempting download...")
|
||||
df = self.dm.download_data(
|
||||
exchange_id, symbol, timeframe,
|
||||
start_date, end_date, market_type
|
||||
)
|
||||
|
||||
if start_date:
|
||||
df = df[df.index >= pd.Timestamp(start_date, tz="UTC")]
|
||||
if end_date:
|
||||
df = df[df.index <= pd.Timestamp(end_date, tz="UTC")]
|
||||
|
||||
return df
|
||||
|
||||
def _normalize_signals(
|
||||
self,
|
||||
signals: tuple,
|
||||
close: pd.Series,
|
||||
market_config
|
||||
) -> tuple:
|
||||
"""
|
||||
Normalize strategy signals to 5-tuple format.
|
||||
|
||||
Returns:
|
||||
(long_entries, long_exits, short_entries, short_exits, size)
|
||||
"""
|
||||
# Default size is None (will be treated as 1.0 or default later)
|
||||
size = None
|
||||
|
||||
if len(signals) == 2:
|
||||
long_entries, long_exits = signals
|
||||
short_entries = BaseStrategy.create_empty_signals(long_entries)
|
||||
short_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
return long_entries, long_exits, short_entries, short_exits, size
|
||||
|
||||
if len(signals) == 4:
|
||||
long_entries, long_exits, short_entries, short_exits = signals
|
||||
elif len(signals) == 5:
|
||||
long_entries, long_exits, short_entries, short_exits, size = signals
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Strategy must return 2, 4, or 5 signal arrays, got {len(signals)}"
|
||||
)
|
||||
|
||||
# Warn and clear short signals on spot markets
|
||||
if not market_config.supports_short:
|
||||
has_shorts = (
|
||||
short_entries.any().any()
|
||||
if hasattr(short_entries, 'any')
|
||||
else short_entries.any()
|
||||
)
|
||||
if has_shorts:
|
||||
logger.warning(
|
||||
"Short signals detected but market type is SPOT. "
|
||||
"Short signals will be ignored."
|
||||
)
|
||||
short_entries = BaseStrategy.create_empty_signals(long_entries)
|
||||
short_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
|
||||
return long_entries, long_exits, short_entries, short_exits, size
|
||||
|
||||
def _run_portfolio(
|
||||
self,
|
||||
close: pd.Series,
|
||||
market_config,
|
||||
long_entries, long_exits,
|
||||
short_entries, short_exits,
|
||||
init_cash: float,
|
||||
fees: float,
|
||||
slippage: float,
|
||||
freq: str,
|
||||
sl_stop: float | None,
|
||||
tp_stop: float | None,
|
||||
sl_trail: bool,
|
||||
leverage: int,
|
||||
size: pd.Series | float = 1.0
|
||||
) -> vbt.Portfolio:
|
||||
"""Select and run appropriate portfolio simulation."""
|
||||
has_shorts = (
|
||||
short_entries.any().any()
|
||||
if hasattr(short_entries, 'any')
|
||||
else short_entries.any()
|
||||
)
|
||||
|
||||
if market_config.supports_short and has_shorts:
|
||||
return run_long_short_portfolio(
|
||||
close,
|
||||
long_entries, long_exits,
|
||||
short_entries, short_exits,
|
||||
init_cash, fees, slippage, freq,
|
||||
sl_stop, tp_stop, sl_trail, leverage,
|
||||
size=size
|
||||
)
|
||||
|
||||
return run_long_only_portfolio(
|
||||
close,
|
||||
long_entries, long_exits,
|
||||
init_cash, fees, slippage, freq,
|
||||
sl_stop, tp_stop, sl_trail, leverage,
|
||||
# Long-only doesn't support variable size in current implementation
|
||||
# without modification, but we can add it if needed.
|
||||
# For now, only regime strategy uses it, which is Long/Short.
|
||||
)
|
||||
|
||||
def run_wfa(
|
||||
self,
|
||||
strategy: BaseStrategy,
|
||||
exchange_id: str,
|
||||
symbol: str,
|
||||
param_grid: dict,
|
||||
n_windows: int = 10,
|
||||
timeframe: str = '1m'
|
||||
):
|
||||
"""
|
||||
Execute Walk-Forward Analysis.
|
||||
|
||||
Args:
|
||||
strategy: Strategy instance to optimize
|
||||
exchange_id: Exchange identifier
|
||||
symbol: Trading pair symbol
|
||||
param_grid: Parameter grid for optimization
|
||||
n_windows: Number of walk-forward windows
|
||||
timeframe: Data timeframe to load
|
||||
|
||||
Returns:
|
||||
Tuple of (results DataFrame, stitched equity curve)
|
||||
"""
|
||||
market_type = strategy.default_market_type
|
||||
df = self.dm.load_data(exchange_id, symbol, timeframe, market_type)
|
||||
|
||||
wfa = WalkForwardOptimizer(self, strategy, param_grid)
|
||||
|
||||
results, stitched_curve = wfa.run(
|
||||
df['close'],
|
||||
high=df['high'],
|
||||
low=df['low'],
|
||||
n_windows=n_windows
|
||||
)
|
||||
|
||||
return results, stitched_curve
|
||||
243
engine/cli.py
Normal file
243
engine/cli.py
Normal file
@@ -0,0 +1,243 @@
|
||||
"""
|
||||
CLI handler for Lowkey Backtest.
|
||||
"""
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
from engine.backtester import Backtester
|
||||
from engine.data_manager import DataManager
|
||||
from engine.logging_config import get_logger, setup_logging
|
||||
from engine.market import MarketType
|
||||
from engine.reporting import Reporter
|
||||
from strategies.factory import get_strategy, get_strategy_names
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def create_parser() -> argparse.ArgumentParser:
|
||||
"""Create and configure the argument parser."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Lowkey Backtest CLI (VectorBT Edition)"
|
||||
)
|
||||
subparsers = parser.add_subparsers(dest="command", help="Command to run")
|
||||
|
||||
_add_download_parser(subparsers)
|
||||
_add_backtest_parser(subparsers)
|
||||
_add_wfa_parser(subparsers)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def _add_download_parser(subparsers) -> None:
|
||||
"""Add download command parser."""
|
||||
dl_parser = subparsers.add_parser("download", help="Download historical data")
|
||||
dl_parser.add_argument("--exchange", "-e", type=str, default="okx")
|
||||
dl_parser.add_argument("--pair", "-p", type=str, required=True)
|
||||
dl_parser.add_argument("--timeframe", "-t", type=str, default="1m")
|
||||
dl_parser.add_argument("--start", type=str, help="Start Date (YYYY-MM-DD)")
|
||||
dl_parser.add_argument(
|
||||
"--market", "-m",
|
||||
type=str,
|
||||
choices=["spot", "perpetual"],
|
||||
default="spot"
|
||||
)
|
||||
|
||||
|
||||
def _add_backtest_parser(subparsers) -> None:
|
||||
"""Add backtest command parser."""
|
||||
strategy_choices = get_strategy_names()
|
||||
|
||||
bt_parser = subparsers.add_parser("backtest", help="Run a backtest")
|
||||
bt_parser.add_argument(
|
||||
"--strategy", "-s",
|
||||
type=str,
|
||||
choices=strategy_choices,
|
||||
required=True
|
||||
)
|
||||
bt_parser.add_argument("--exchange", "-e", type=str, default="okx")
|
||||
bt_parser.add_argument("--pair", "-p", type=str, required=True)
|
||||
bt_parser.add_argument("--timeframe", "-t", type=str, default="1m")
|
||||
bt_parser.add_argument("--start", type=str)
|
||||
bt_parser.add_argument("--end", type=str)
|
||||
bt_parser.add_argument("--grid", "-g", action="store_true")
|
||||
bt_parser.add_argument("--plot", action="store_true")
|
||||
|
||||
# Risk parameters
|
||||
bt_parser.add_argument("--sl", type=float, help="Stop Loss %%")
|
||||
bt_parser.add_argument("--tp", type=float, help="Take Profit %%")
|
||||
bt_parser.add_argument("--trail", action="store_true")
|
||||
bt_parser.add_argument("--no-bear-exit", action="store_true")
|
||||
|
||||
# Cost parameters
|
||||
bt_parser.add_argument("--fees", type=float, default=None)
|
||||
bt_parser.add_argument("--slippage", type=float, default=0.001)
|
||||
bt_parser.add_argument("--leverage", "-l", type=int, default=None)
|
||||
|
||||
|
||||
def _add_wfa_parser(subparsers) -> None:
|
||||
"""Add walk-forward analysis command parser."""
|
||||
strategy_choices = get_strategy_names()
|
||||
|
||||
wfa_parser = subparsers.add_parser("wfa", help="Run Walk-Forward Analysis")
|
||||
wfa_parser.add_argument(
|
||||
"--strategy", "-s",
|
||||
type=str,
|
||||
choices=strategy_choices,
|
||||
required=True
|
||||
)
|
||||
wfa_parser.add_argument("--pair", "-p", type=str, required=True)
|
||||
wfa_parser.add_argument("--timeframe", "-t", type=str, default="1d")
|
||||
wfa_parser.add_argument("--windows", "-w", type=int, default=10)
|
||||
wfa_parser.add_argument("--plot", action="store_true")
|
||||
|
||||
|
||||
def run_download(args) -> None:
|
||||
"""Execute download command."""
|
||||
dm = DataManager()
|
||||
market_type = MarketType(args.market)
|
||||
dm.download_data(
|
||||
args.exchange,
|
||||
args.pair,
|
||||
args.timeframe,
|
||||
start_date=args.start,
|
||||
market_type=market_type
|
||||
)
|
||||
|
||||
|
||||
def run_backtest(args) -> None:
|
||||
"""Execute backtest command."""
|
||||
dm = DataManager()
|
||||
bt = Backtester(dm)
|
||||
reporter = Reporter()
|
||||
|
||||
strategy, params = get_strategy(args.strategy, args.grid)
|
||||
|
||||
# Apply CLI overrides for meta_st strategy
|
||||
params = _apply_strategy_overrides(args, strategy, params)
|
||||
|
||||
if args.grid and args.strategy == "meta_st":
|
||||
logger.info("Running Grid Search for Meta Supertrend...")
|
||||
|
||||
try:
|
||||
result = bt.run_strategy(
|
||||
strategy,
|
||||
args.exchange,
|
||||
args.pair,
|
||||
timeframe=args.timeframe,
|
||||
start_date=args.start,
|
||||
end_date=args.end,
|
||||
fees=args.fees,
|
||||
slippage=args.slippage,
|
||||
sl_stop=args.sl,
|
||||
tp_stop=args.tp,
|
||||
sl_trail=args.trail,
|
||||
leverage=args.leverage,
|
||||
**params
|
||||
)
|
||||
|
||||
reporter.print_summary(result)
|
||||
reporter.save_reports(result, f"{args.strategy}_{args.pair.replace('/','-')}")
|
||||
|
||||
if args.plot and not args.grid:
|
||||
reporter.plot(result.portfolio)
|
||||
elif args.plot and args.grid:
|
||||
logger.info("Plotting skipped for Grid Search. Check CSV results.")
|
||||
|
||||
except Exception as e:
|
||||
logger.error("Backtest failed: %s", e, exc_info=True)
|
||||
|
||||
|
||||
def run_wfa(args) -> None:
|
||||
"""Execute walk-forward analysis command."""
|
||||
dm = DataManager()
|
||||
bt = Backtester(dm)
|
||||
reporter = Reporter()
|
||||
|
||||
strategy, params = get_strategy(args.strategy, is_grid=True)
|
||||
|
||||
logger.info(
|
||||
"Running WFA on %s for %s (%s) with %d windows...",
|
||||
args.strategy, args.pair, args.timeframe, args.windows
|
||||
)
|
||||
|
||||
try:
|
||||
results, stitched_curve = bt.run_wfa(
|
||||
strategy,
|
||||
"okx",
|
||||
args.pair,
|
||||
params,
|
||||
n_windows=args.windows,
|
||||
timeframe=args.timeframe
|
||||
)
|
||||
|
||||
_log_wfa_results(results)
|
||||
_save_wfa_results(args, results, stitched_curve, reporter)
|
||||
|
||||
except Exception as e:
|
||||
logger.error("WFA failed: %s", e, exc_info=True)
|
||||
|
||||
|
||||
def _apply_strategy_overrides(args, strategy, params: dict) -> dict:
|
||||
"""Apply CLI argument overrides to strategy parameters."""
|
||||
if args.strategy != "meta_st":
|
||||
return params
|
||||
|
||||
if args.no_bear_exit:
|
||||
params['exit_on_bearish_flip'] = False
|
||||
|
||||
if args.sl is None:
|
||||
args.sl = strategy.default_sl_stop
|
||||
|
||||
if not args.trail:
|
||||
args.trail = strategy.default_sl_trail
|
||||
|
||||
return params
|
||||
|
||||
|
||||
def _log_wfa_results(results) -> None:
|
||||
"""Log WFA results summary."""
|
||||
logger.info("Walk-Forward Analysis Results:")
|
||||
|
||||
if results.empty or 'window' not in results.columns:
|
||||
logger.warning("No valid WFA results. All windows may have failed.")
|
||||
return
|
||||
|
||||
columns = ['window', 'train_score', 'test_score', 'test_return']
|
||||
logger.info("\n%s", results[columns].to_string(index=False))
|
||||
|
||||
avg_test_sharpe = results['test_score'].mean()
|
||||
avg_test_return = results['test_return'].mean()
|
||||
logger.info("Average Test Sharpe: %.2f", avg_test_sharpe)
|
||||
logger.info("Average Test Return: %.2f%%", avg_test_return * 100)
|
||||
|
||||
|
||||
def _save_wfa_results(args, results, stitched_curve, reporter) -> None:
|
||||
"""Save WFA results to file and optionally plot."""
|
||||
if results.empty:
|
||||
return
|
||||
|
||||
output_path = f"backtest_logs/wfa_{args.strategy}_{args.pair.replace('/','-')}.csv"
|
||||
results.to_csv(output_path)
|
||||
logger.info("Saved full results to %s", output_path)
|
||||
|
||||
if args.plot:
|
||||
reporter.plot_wfa(results, stitched_curve)
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point."""
|
||||
setup_logging()
|
||||
|
||||
parser = create_parser()
|
||||
args = parser.parse_args()
|
||||
|
||||
commands = {
|
||||
"download": run_download,
|
||||
"backtest": run_backtest,
|
||||
"wfa": run_wfa,
|
||||
}
|
||||
|
||||
if args.command in commands:
|
||||
commands[args.command](args)
|
||||
else:
|
||||
parser.print_help()
|
||||
201
engine/cryptoquant.py
Normal file
201
engine/cryptoquant.py
Normal file
@@ -0,0 +1,201 @@
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import requests
|
||||
import pandas as pd
|
||||
from datetime import datetime, timedelta
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load env vars from .env file
|
||||
load_dotenv()
|
||||
|
||||
# Fix path for direct execution
|
||||
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from engine.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
class CryptoQuantClient:
|
||||
"""
|
||||
Client for fetching data from CryptoQuant API.
|
||||
"""
|
||||
BASE_URL = "https://api.cryptoquant.com/v1"
|
||||
|
||||
def __init__(self, api_key: str | None = None):
|
||||
self.api_key = api_key or os.getenv("CRYPTOQUANT_API_KEY")
|
||||
if not self.api_key:
|
||||
raise ValueError("CryptoQuant API Key not found. Set CRYPTOQUANT_API_KEY env var.")
|
||||
|
||||
self.headers = {
|
||||
"Authorization": f"Bearer {self.api_key}"
|
||||
}
|
||||
|
||||
def fetch_metric(
|
||||
self,
|
||||
metric_path: str,
|
||||
symbol: str,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
exchange: str | None = "all_exchange",
|
||||
window: str = "day"
|
||||
) -> pd.DataFrame:
|
||||
"""
|
||||
Fetch a specific metric from CryptoQuant.
|
||||
"""
|
||||
url = f"{self.BASE_URL}/{metric_path}"
|
||||
|
||||
params = {
|
||||
"window": window,
|
||||
"from": start_date,
|
||||
"to": end_date,
|
||||
"limit": 100000
|
||||
}
|
||||
|
||||
if exchange:
|
||||
params["exchange"] = exchange
|
||||
|
||||
logger.info(f"Fetching {metric_path} for {symbol} ({start_date}-{end_date})...")
|
||||
|
||||
try:
|
||||
response = requests.get(url, headers=self.headers, params=params)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
if 'result' in data and 'data' in data['result']:
|
||||
df = pd.DataFrame(data['result']['data'])
|
||||
if not df.empty:
|
||||
if 'date' in df.columns:
|
||||
df['timestamp'] = pd.to_datetime(df['date'])
|
||||
df.set_index('timestamp', inplace=True)
|
||||
df.sort_index(inplace=True)
|
||||
return df
|
||||
|
||||
return pd.DataFrame()
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error fetching CQ data {metric_path}: {e}")
|
||||
if 'response' in locals() and hasattr(response, 'text'):
|
||||
logger.error(f"Response: {response.text}")
|
||||
return pd.DataFrame()
|
||||
|
||||
def fetch_multi_metrics(self, symbols: list[str], metrics: dict, start_date: str, end_date: str):
|
||||
"""
|
||||
Fetch multiple metrics for multiple symbols and combine them.
|
||||
"""
|
||||
combined_df = pd.DataFrame()
|
||||
|
||||
for symbol in symbols:
|
||||
asset = symbol.lower()
|
||||
|
||||
for metric_name, api_path in metrics.items():
|
||||
full_path = f"{asset}/{api_path}"
|
||||
|
||||
# Some metrics (like funding rates) might need specific exchange vs all_exchange
|
||||
# Defaulting to all_exchange is usually safe for flows, but check specific logic if needed
|
||||
exchange_param = "all_exchange"
|
||||
if "funding-rates" in api_path:
|
||||
# For funding rates, 'all_exchange' might not be valid or might be aggregated
|
||||
# Let's try 'binance' as a proxy for market sentiment if all fails,
|
||||
# or keep 'all_exchange' if supported.
|
||||
# Based on testing, 'all_exchange' is standard for flows.
|
||||
pass
|
||||
|
||||
df = self.fetch_metric(full_path, asset, start_date, end_date, exchange=exchange_param)
|
||||
|
||||
if not df.empty:
|
||||
target_col = None
|
||||
# Heuristic to find the value column
|
||||
candidates = ['funding_rate', 'reserve', 'inflow_total', 'outflow_total', 'open_interest', 'ratio', 'value']
|
||||
|
||||
for col in df.columns:
|
||||
if col in candidates:
|
||||
target_col = col
|
||||
break
|
||||
|
||||
if not target_col:
|
||||
# Fallback: take first numeric col that isn't date
|
||||
for col in df.columns:
|
||||
if col not in ['date', 'datetime', 'timestamp_str', 'block_height']:
|
||||
target_col = col
|
||||
break
|
||||
|
||||
if target_col:
|
||||
col_name = f"{asset}_{metric_name}"
|
||||
subset = df[[target_col]].rename(columns={target_col: col_name})
|
||||
|
||||
if combined_df.empty:
|
||||
combined_df = subset
|
||||
else:
|
||||
combined_df = combined_df.join(subset, how='outer')
|
||||
|
||||
time.sleep(0.2)
|
||||
|
||||
return combined_df
|
||||
|
||||
def fetch_history_chunked(
|
||||
self,
|
||||
symbols: list[str],
|
||||
metrics: dict,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
chunk_months: int = 3
|
||||
) -> pd.DataFrame:
|
||||
"""
|
||||
Fetch historical data in chunks to avoid API limits.
|
||||
"""
|
||||
start_dt = datetime.strptime(start_date, "%Y%m%d")
|
||||
end_dt = datetime.strptime(end_date, "%Y%m%d")
|
||||
|
||||
all_data = []
|
||||
|
||||
current = start_dt
|
||||
while current < end_dt:
|
||||
next_chunk = current + timedelta(days=chunk_months * 30)
|
||||
if next_chunk > end_dt:
|
||||
next_chunk = end_dt
|
||||
|
||||
s_str = current.strftime("%Y%m%d")
|
||||
e_str = next_chunk.strftime("%Y%m%d")
|
||||
|
||||
logger.info(f"Processing chunk: {s_str} to {e_str}")
|
||||
chunk_df = self.fetch_multi_metrics(symbols, metrics, s_str, e_str)
|
||||
|
||||
if not chunk_df.empty:
|
||||
all_data.append(chunk_df)
|
||||
|
||||
current = next_chunk + timedelta(days=1)
|
||||
time.sleep(1) # Be nice to API
|
||||
|
||||
if not all_data:
|
||||
return pd.DataFrame()
|
||||
|
||||
# Combine all chunks
|
||||
full_df = pd.concat(all_data)
|
||||
# Remove duplicates if any overlap
|
||||
full_df = full_df[~full_df.index.duplicated(keep='first')]
|
||||
full_df.sort_index(inplace=True)
|
||||
|
||||
return full_df
|
||||
|
||||
if __name__ == "__main__":
|
||||
cq = CryptoQuantClient()
|
||||
|
||||
# 12 Months Data (Jan 1 2025 - Jan 14 2026)
|
||||
start = "20250101"
|
||||
end = "20260114"
|
||||
|
||||
metrics = {
|
||||
"reserves": "exchange-flows/exchange-reserve",
|
||||
"inflow": "exchange-flows/inflow",
|
||||
"funding": "market-data/funding-rates"
|
||||
}
|
||||
|
||||
print(f"Fetching training data from {start} to {end}...")
|
||||
df = cq.fetch_history_chunked(["btc", "eth"], metrics, start, end)
|
||||
|
||||
output_file = "data/cq_training_data.csv"
|
||||
os.makedirs("data", exist_ok=True)
|
||||
df.to_csv(output_file)
|
||||
print(f"\nSaved {len(df)} rows to {output_file}")
|
||||
print(df.head())
|
||||
209
engine/data_manager.py
Normal file
209
engine/data_manager.py
Normal file
@@ -0,0 +1,209 @@
|
||||
"""
|
||||
Data management for OHLCV data download and storage.
|
||||
|
||||
Handles data retrieval from exchanges and local file management.
|
||||
"""
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import ccxt
|
||||
import pandas as pd
|
||||
|
||||
from engine.logging_config import get_logger
|
||||
from engine.market import MarketType, get_ccxt_symbol
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class DataManager:
|
||||
"""
|
||||
Manages OHLCV data download and storage for different market types.
|
||||
|
||||
Data is stored in: data/ccxt/{exchange}/{market_type}/{symbol}/{timeframe}.csv
|
||||
"""
|
||||
|
||||
def __init__(self, data_dir: str = "data/ccxt"):
|
||||
self.data_dir = Path(data_dir)
|
||||
self.data_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.exchanges: dict[str, ccxt.Exchange] = {}
|
||||
|
||||
def get_exchange(self, exchange_id: str) -> ccxt.Exchange:
|
||||
"""Get or create a CCXT exchange instance."""
|
||||
if exchange_id not in self.exchanges:
|
||||
exchange_class = getattr(ccxt, exchange_id)
|
||||
self.exchanges[exchange_id] = exchange_class({
|
||||
'enableRateLimit': True,
|
||||
})
|
||||
return self.exchanges[exchange_id]
|
||||
|
||||
def _get_data_path(
|
||||
self,
|
||||
exchange_id: str,
|
||||
symbol: str,
|
||||
timeframe: str,
|
||||
market_type: MarketType
|
||||
) -> Path:
|
||||
"""
|
||||
Get the file path for storing/loading data.
|
||||
|
||||
Args:
|
||||
exchange_id: Exchange name (e.g., 'okx')
|
||||
symbol: Trading pair (e.g., 'BTC/USDT')
|
||||
timeframe: Candle timeframe (e.g., '1m')
|
||||
market_type: Market type (spot or perpetual)
|
||||
|
||||
Returns:
|
||||
Path to the CSV file
|
||||
"""
|
||||
safe_symbol = symbol.replace('/', '-')
|
||||
return (
|
||||
self.data_dir
|
||||
/ exchange_id
|
||||
/ market_type.value
|
||||
/ safe_symbol
|
||||
/ f"{timeframe}.csv"
|
||||
)
|
||||
|
||||
def download_data(
|
||||
self,
|
||||
exchange_id: str,
|
||||
symbol: str,
|
||||
timeframe: str = '1m',
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
market_type: MarketType = MarketType.SPOT
|
||||
) -> pd.DataFrame | None:
|
||||
"""
|
||||
Download OHLCV data from exchange and save to CSV.
|
||||
|
||||
Args:
|
||||
exchange_id: Exchange name (e.g., 'okx')
|
||||
symbol: Trading pair (e.g., 'BTC/USDT')
|
||||
timeframe: Candle timeframe (e.g., '1m')
|
||||
start_date: Start date string (YYYY-MM-DD)
|
||||
end_date: End date string (YYYY-MM-DD)
|
||||
market_type: Market type (spot or perpetual)
|
||||
|
||||
Returns:
|
||||
DataFrame with OHLCV data, or None if download failed
|
||||
"""
|
||||
exchange = self.get_exchange(exchange_id)
|
||||
|
||||
file_path = self._get_data_path(exchange_id, symbol, timeframe, market_type)
|
||||
file_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ccxt_symbol = get_ccxt_symbol(symbol, market_type)
|
||||
|
||||
since, until = self._parse_date_range(exchange, start_date, end_date)
|
||||
|
||||
logger.info(
|
||||
"Downloading %s (%s) from %s...",
|
||||
symbol, market_type.value, exchange_id
|
||||
)
|
||||
|
||||
all_ohlcv = self._fetch_all_candles(exchange, ccxt_symbol, timeframe, since, until)
|
||||
|
||||
if not all_ohlcv:
|
||||
logger.warning("No data downloaded.")
|
||||
return None
|
||||
|
||||
df = self._convert_to_dataframe(all_ohlcv)
|
||||
df.to_csv(file_path)
|
||||
logger.info("Saved %d candles to %s", len(df), file_path)
|
||||
return df
|
||||
|
||||
def load_data(
|
||||
self,
|
||||
exchange_id: str,
|
||||
symbol: str,
|
||||
timeframe: str = '1m',
|
||||
market_type: MarketType = MarketType.SPOT
|
||||
) -> pd.DataFrame:
|
||||
"""
|
||||
Load saved OHLCV data for vectorbt.
|
||||
|
||||
Args:
|
||||
exchange_id: Exchange name (e.g., 'okx')
|
||||
symbol: Trading pair (e.g., 'BTC/USDT')
|
||||
timeframe: Candle timeframe (e.g., '1m')
|
||||
market_type: Market type (spot or perpetual)
|
||||
|
||||
Returns:
|
||||
DataFrame with OHLCV data indexed by timestamp
|
||||
|
||||
Raises:
|
||||
FileNotFoundError: If data file does not exist
|
||||
"""
|
||||
file_path = self._get_data_path(exchange_id, symbol, timeframe, market_type)
|
||||
|
||||
if not file_path.exists():
|
||||
raise FileNotFoundError(
|
||||
f"Data not found at {file_path}. "
|
||||
f"Run: uv run python main.py download --pair {symbol} "
|
||||
f"--market {market_type.value}"
|
||||
)
|
||||
|
||||
return pd.read_csv(file_path, index_col='timestamp', parse_dates=True)
|
||||
|
||||
def _parse_date_range(
|
||||
self,
|
||||
exchange: ccxt.Exchange,
|
||||
start_date: str | None,
|
||||
end_date: str | None
|
||||
) -> tuple[int, int]:
|
||||
"""Parse date strings into millisecond timestamps."""
|
||||
if start_date:
|
||||
since = exchange.parse8601(f"{start_date}T00:00:00Z")
|
||||
else:
|
||||
since = exchange.milliseconds() - 365 * 24 * 60 * 60 * 1000
|
||||
|
||||
if end_date:
|
||||
until = exchange.parse8601(f"{end_date}T23:59:59Z")
|
||||
else:
|
||||
until = exchange.milliseconds()
|
||||
|
||||
return since, until
|
||||
|
||||
def _fetch_all_candles(
|
||||
self,
|
||||
exchange: ccxt.Exchange,
|
||||
symbol: str,
|
||||
timeframe: str,
|
||||
since: int,
|
||||
until: int
|
||||
) -> list:
|
||||
"""Fetch all candles in the date range."""
|
||||
all_ohlcv = []
|
||||
|
||||
while since < until:
|
||||
try:
|
||||
ohlcv = exchange.fetch_ohlcv(symbol, timeframe, since, limit=100)
|
||||
if not ohlcv:
|
||||
break
|
||||
|
||||
all_ohlcv.extend(ohlcv)
|
||||
since = ohlcv[-1][0] + 1
|
||||
|
||||
current_date = datetime.fromtimestamp(
|
||||
since/1000, tz=timezone.utc
|
||||
).strftime('%Y-%m-%d')
|
||||
logger.debug("Fetched up to %s", current_date)
|
||||
|
||||
time.sleep(exchange.rateLimit / 1000)
|
||||
|
||||
except Exception as e:
|
||||
logger.error("Error fetching data: %s", e)
|
||||
break
|
||||
|
||||
return all_ohlcv
|
||||
|
||||
def _convert_to_dataframe(self, ohlcv: list) -> pd.DataFrame:
|
||||
"""Convert OHLCV list to DataFrame."""
|
||||
df = pd.DataFrame(
|
||||
ohlcv,
|
||||
columns=['timestamp', 'open', 'high', 'low', 'close', 'volume']
|
||||
)
|
||||
df['timestamp'] = pd.to_datetime(df['timestamp'], unit='ms', utc=True)
|
||||
df.set_index('timestamp', inplace=True)
|
||||
return df
|
||||
124
engine/logging_config.py
Normal file
124
engine/logging_config.py
Normal file
@@ -0,0 +1,124 @@
|
||||
"""
|
||||
Centralized logging configuration for the backtest engine.
|
||||
|
||||
Provides colored console output and rotating file logs.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from logging.handlers import RotatingFileHandler
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
# ANSI color codes for terminal output
|
||||
class Colors:
|
||||
"""ANSI escape codes for colored terminal output."""
|
||||
|
||||
RESET = "\033[0m"
|
||||
BOLD = "\033[1m"
|
||||
|
||||
# Log level colors
|
||||
DEBUG = "\033[36m" # Cyan
|
||||
INFO = "\033[32m" # Green
|
||||
WARNING = "\033[33m" # Yellow
|
||||
ERROR = "\033[31m" # Red
|
||||
CRITICAL = "\033[35m" # Magenta
|
||||
|
||||
|
||||
class ColoredFormatter(logging.Formatter):
|
||||
"""
|
||||
Custom formatter that adds colors to log level names in terminal output.
|
||||
"""
|
||||
|
||||
LEVEL_COLORS = {
|
||||
logging.DEBUG: Colors.DEBUG,
|
||||
logging.INFO: Colors.INFO,
|
||||
logging.WARNING: Colors.WARNING,
|
||||
logging.ERROR: Colors.ERROR,
|
||||
logging.CRITICAL: Colors.CRITICAL,
|
||||
}
|
||||
|
||||
def __init__(self, fmt: str = None, datefmt: str = None):
|
||||
super().__init__(fmt, datefmt)
|
||||
|
||||
def format(self, record: logging.LogRecord) -> str:
|
||||
# Save original levelname
|
||||
original_levelname = record.levelname
|
||||
|
||||
# Add color to levelname
|
||||
color = self.LEVEL_COLORS.get(record.levelno, Colors.RESET)
|
||||
record.levelname = f"{color}{record.levelname}{Colors.RESET}"
|
||||
|
||||
# Format the message
|
||||
result = super().format(record)
|
||||
|
||||
# Restore original levelname
|
||||
record.levelname = original_levelname
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def setup_logging(
|
||||
log_dir: str = "logs",
|
||||
log_level: int = logging.INFO,
|
||||
console_level: int = logging.INFO,
|
||||
max_bytes: int = 5 * 1024 * 1024, # 5MB
|
||||
backup_count: int = 3
|
||||
) -> None:
|
||||
"""
|
||||
Configure logging for the application.
|
||||
|
||||
Args:
|
||||
log_dir: Directory for log files
|
||||
log_level: File logging level
|
||||
console_level: Console logging level
|
||||
max_bytes: Max size per log file before rotation
|
||||
backup_count: Number of backup files to keep
|
||||
"""
|
||||
log_path = Path(log_dir)
|
||||
log_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Get root logger
|
||||
root_logger = logging.getLogger()
|
||||
root_logger.setLevel(logging.DEBUG) # Capture all, handlers filter
|
||||
|
||||
# Clear existing handlers
|
||||
root_logger.handlers.clear()
|
||||
|
||||
# Console handler with colors
|
||||
console_handler = logging.StreamHandler(sys.stdout)
|
||||
console_handler.setLevel(console_level)
|
||||
console_fmt = ColoredFormatter(
|
||||
fmt="[%(asctime)s] %(levelname)s - %(message)s",
|
||||
datefmt="%H:%M:%S"
|
||||
)
|
||||
console_handler.setFormatter(console_fmt)
|
||||
root_logger.addHandler(console_handler)
|
||||
|
||||
# File handler with rotation
|
||||
file_handler = RotatingFileHandler(
|
||||
log_path / "backtest.log",
|
||||
maxBytes=max_bytes,
|
||||
backupCount=backup_count,
|
||||
encoding="utf-8"
|
||||
)
|
||||
file_handler.setLevel(log_level)
|
||||
file_fmt = logging.Formatter(
|
||||
fmt="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S"
|
||||
)
|
||||
file_handler.setFormatter(file_fmt)
|
||||
root_logger.addHandler(file_handler)
|
||||
|
||||
|
||||
def get_logger(name: str) -> logging.Logger:
|
||||
"""
|
||||
Get a logger instance for the given module name.
|
||||
|
||||
Args:
|
||||
name: Module name (typically __name__)
|
||||
|
||||
Returns:
|
||||
Configured logger instance
|
||||
"""
|
||||
return logging.getLogger(name)
|
||||
168
engine/market.py
Normal file
168
engine/market.py
Normal file
@@ -0,0 +1,168 @@
|
||||
"""
|
||||
Market type definitions and configuration for backtesting.
|
||||
|
||||
Supports different market types with their specific trading conditions:
|
||||
- SPOT: No leverage, no funding, long-only
|
||||
- PERPETUAL: Leverage, funding rates, long/short
|
||||
"""
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
|
||||
|
||||
class MarketType(Enum):
|
||||
"""Supported market types for backtesting."""
|
||||
SPOT = "spot"
|
||||
PERPETUAL = "perpetual"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MarketConfig:
|
||||
"""
|
||||
Configuration for a specific market type.
|
||||
|
||||
Attributes:
|
||||
market_type: The market type enum value
|
||||
maker_fee: Maker fee as decimal (e.g., 0.0008 = 0.08%)
|
||||
taker_fee: Taker fee as decimal (e.g., 0.001 = 0.1%)
|
||||
max_leverage: Maximum allowed leverage
|
||||
funding_rate: Funding rate per 8 hours as decimal (perpetuals only)
|
||||
funding_interval_hours: Hours between funding payments
|
||||
maintenance_margin_rate: Rate for liquidation calculation
|
||||
supports_short: Whether short-selling is supported
|
||||
"""
|
||||
market_type: MarketType
|
||||
maker_fee: float
|
||||
taker_fee: float
|
||||
max_leverage: int
|
||||
funding_rate: float
|
||||
funding_interval_hours: int
|
||||
maintenance_margin_rate: float
|
||||
supports_short: bool
|
||||
|
||||
|
||||
# OKX-based default configurations
|
||||
SPOT_CONFIG = MarketConfig(
|
||||
market_type=MarketType.SPOT,
|
||||
maker_fee=0.0008, # 0.08%
|
||||
taker_fee=0.0010, # 0.10%
|
||||
max_leverage=1,
|
||||
funding_rate=0.0,
|
||||
funding_interval_hours=0,
|
||||
maintenance_margin_rate=0.0,
|
||||
supports_short=False,
|
||||
)
|
||||
|
||||
PERPETUAL_CONFIG = MarketConfig(
|
||||
market_type=MarketType.PERPETUAL,
|
||||
maker_fee=0.0002, # 0.02%
|
||||
taker_fee=0.0005, # 0.05%
|
||||
max_leverage=125,
|
||||
funding_rate=0.0001, # 0.01% per 8 hours (simplified average)
|
||||
funding_interval_hours=8,
|
||||
maintenance_margin_rate=0.004, # 0.4% for BTC on OKX
|
||||
supports_short=True,
|
||||
)
|
||||
|
||||
|
||||
def get_market_config(market_type: MarketType) -> MarketConfig:
|
||||
"""
|
||||
Get the configuration for a specific market type.
|
||||
|
||||
Args:
|
||||
market_type: The market type to get configuration for
|
||||
|
||||
Returns:
|
||||
MarketConfig with default values for that market type
|
||||
"""
|
||||
configs = {
|
||||
MarketType.SPOT: SPOT_CONFIG,
|
||||
MarketType.PERPETUAL: PERPETUAL_CONFIG,
|
||||
}
|
||||
return configs[market_type]
|
||||
|
||||
|
||||
def get_ccxt_symbol(symbol: str, market_type: MarketType) -> str:
|
||||
"""
|
||||
Convert a standard symbol to CCXT format for the given market type.
|
||||
|
||||
Args:
|
||||
symbol: Standard symbol (e.g., 'BTC/USDT')
|
||||
market_type: The market type
|
||||
|
||||
Returns:
|
||||
CCXT-formatted symbol (e.g., 'BTC/USDT:USDT' for perpetuals)
|
||||
"""
|
||||
if market_type == MarketType.PERPETUAL:
|
||||
# OKX perpetual format: BTC/USDT:USDT
|
||||
if '/' in symbol:
|
||||
base, quote = symbol.split('/')
|
||||
return f"{symbol}:{quote}"
|
||||
elif '-' in symbol:
|
||||
base, quote = symbol.split('-')
|
||||
return f"{base}/{quote}:{quote}"
|
||||
else:
|
||||
# Assume base is symbol, quote is USDT default
|
||||
return f"{symbol}/USDT:USDT"
|
||||
|
||||
# For spot, normalize dash to slash for CCXT
|
||||
if '-' in symbol:
|
||||
return symbol.replace('-', '/')
|
||||
|
||||
return symbol
|
||||
|
||||
|
||||
def calculate_leverage_stop_loss(
|
||||
leverage: int,
|
||||
maintenance_margin_rate: float = 0.004
|
||||
) -> float:
|
||||
"""
|
||||
Calculate the implicit stop-loss percentage from leverage.
|
||||
|
||||
At a given leverage, liquidation occurs when the position loses
|
||||
approximately (1/leverage - maintenance_margin_rate) of its value.
|
||||
|
||||
Args:
|
||||
leverage: Position leverage multiplier
|
||||
maintenance_margin_rate: Maintenance margin rate (default OKX BTC: 0.4%)
|
||||
|
||||
Returns:
|
||||
Stop-loss percentage as decimal (e.g., 0.196 for 19.6%)
|
||||
"""
|
||||
if leverage <= 1:
|
||||
return 1.0 # No forced stop for spot
|
||||
|
||||
return (1 / leverage) - maintenance_margin_rate
|
||||
|
||||
|
||||
def calculate_liquidation_price(
|
||||
entry_price: float,
|
||||
leverage: float,
|
||||
is_long: bool,
|
||||
maintenance_margin_rate: float = 0.004
|
||||
) -> float:
|
||||
"""
|
||||
Calculate the liquidation price for a leveraged position.
|
||||
|
||||
Args:
|
||||
entry_price: Position entry price
|
||||
leverage: Position leverage
|
||||
is_long: True for long positions, False for short
|
||||
maintenance_margin_rate: Maintenance margin rate (default OKX BTC: 0.4%)
|
||||
|
||||
Returns:
|
||||
Liquidation price
|
||||
"""
|
||||
if leverage <= 1:
|
||||
return 0.0 if is_long else float('inf')
|
||||
|
||||
# Simplified liquidation formula
|
||||
# Long: liq_price = entry * (1 - 1/leverage + maintenance_margin_rate)
|
||||
# Short: liq_price = entry * (1 + 1/leverage - maintenance_margin_rate)
|
||||
margin_ratio = 1 / leverage
|
||||
|
||||
if is_long:
|
||||
liq_price = entry_price * (1 - margin_ratio + maintenance_margin_rate)
|
||||
else:
|
||||
liq_price = entry_price * (1 + margin_ratio - maintenance_margin_rate)
|
||||
|
||||
return liq_price
|
||||
245
engine/optimizer.py
Normal file
245
engine/optimizer.py
Normal file
@@ -0,0 +1,245 @@
|
||||
"""
|
||||
Walk-Forward Analysis optimizer for strategy parameter optimization.
|
||||
|
||||
Implements expanding window walk-forward analysis with train/test splits.
|
||||
"""
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import vectorbt as vbt
|
||||
|
||||
from engine.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def create_rolling_windows(
|
||||
index: pd.Index,
|
||||
n_windows: int,
|
||||
train_split: float = 0.7
|
||||
):
|
||||
"""
|
||||
Create rolling train/test split indices using expanding window approach.
|
||||
|
||||
Args:
|
||||
index: DataFrame index to split
|
||||
n_windows: Number of walk-forward windows
|
||||
train_split: Unused, kept for API compatibility
|
||||
|
||||
Yields:
|
||||
Tuples of (train_idx, test_idx) numpy arrays
|
||||
"""
|
||||
chunks = np.array_split(index, n_windows + 1)
|
||||
|
||||
for i in range(n_windows):
|
||||
train_idx = np.concatenate([c for c in chunks[:i+1]])
|
||||
test_idx = chunks[i+1]
|
||||
yield train_idx, test_idx
|
||||
|
||||
|
||||
class WalkForwardOptimizer:
|
||||
"""
|
||||
Walk-Forward Analysis optimizer for strategy backtesting.
|
||||
|
||||
Optimizes strategy parameters on training windows and validates
|
||||
on out-of-sample test windows.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
backtester,
|
||||
strategy,
|
||||
param_grid: dict,
|
||||
metric: str = 'Sharpe Ratio',
|
||||
fees: float = 0.001,
|
||||
freq: str = '1m'
|
||||
):
|
||||
"""
|
||||
Initialize the optimizer.
|
||||
|
||||
Args:
|
||||
backtester: Backtester instance
|
||||
strategy: Strategy instance to optimize
|
||||
param_grid: Parameter grid for optimization
|
||||
metric: Performance metric to optimize
|
||||
fees: Transaction fees for simulation
|
||||
freq: Data frequency for portfolio simulation
|
||||
"""
|
||||
self.bt = backtester
|
||||
self.strategy = strategy
|
||||
self.param_grid = param_grid
|
||||
self.metric = metric
|
||||
self.fees = fees
|
||||
self.freq = freq
|
||||
|
||||
# Separate grid params (lists) from fixed params (scalars)
|
||||
self.grid_keys = []
|
||||
self.fixed_params = {}
|
||||
for k, v in param_grid.items():
|
||||
if isinstance(v, (list, np.ndarray)):
|
||||
self.grid_keys.append(k)
|
||||
else:
|
||||
self.fixed_params[k] = v
|
||||
|
||||
def run(
|
||||
self,
|
||||
close_price: pd.Series,
|
||||
high: pd.Series | None = None,
|
||||
low: pd.Series | None = None,
|
||||
n_windows: int = 10
|
||||
) -> tuple[pd.DataFrame, pd.Series | None]:
|
||||
"""
|
||||
Execute walk-forward analysis.
|
||||
|
||||
Args:
|
||||
close_price: Close price series
|
||||
high: High price series (optional)
|
||||
low: Low price series (optional)
|
||||
n_windows: Number of walk-forward windows
|
||||
|
||||
Returns:
|
||||
Tuple of (results DataFrame, stitched equity curve)
|
||||
"""
|
||||
results = []
|
||||
equity_curves = []
|
||||
|
||||
logger.info(
|
||||
"Starting Walk-Forward Analysis with %d windows (Expanding Train)...",
|
||||
n_windows
|
||||
)
|
||||
|
||||
splitter = create_rolling_windows(close_price.index, n_windows)
|
||||
|
||||
for i, (train_idx, test_idx) in enumerate(splitter):
|
||||
logger.info("Processing Window %d/%d...", i + 1, n_windows)
|
||||
|
||||
window_result = self._process_window(
|
||||
i, train_idx, test_idx, close_price, high, low
|
||||
)
|
||||
|
||||
if window_result is not None:
|
||||
result_dict, eq_curve = window_result
|
||||
results.append(result_dict)
|
||||
equity_curves.append(eq_curve)
|
||||
|
||||
stitched_series = self._stitch_equity_curves(equity_curves)
|
||||
return pd.DataFrame(results), stitched_series
|
||||
|
||||
def _process_window(
|
||||
self,
|
||||
window_idx: int,
|
||||
train_idx: np.ndarray,
|
||||
test_idx: np.ndarray,
|
||||
close_price: pd.Series,
|
||||
high: pd.Series | None,
|
||||
low: pd.Series | None
|
||||
) -> tuple[dict, pd.Series] | None:
|
||||
"""Process a single WFA window."""
|
||||
try:
|
||||
# Slice data for train/test
|
||||
train_close = close_price.loc[train_idx]
|
||||
train_high = high.loc[train_idx] if high is not None else None
|
||||
train_low = low.loc[train_idx] if low is not None else None
|
||||
|
||||
# Train phase: find best parameters
|
||||
best_params, best_score = self._optimize_train(
|
||||
train_close, train_high, train_low
|
||||
)
|
||||
|
||||
# Test phase: validate with best params
|
||||
test_close = close_price.loc[test_idx]
|
||||
test_high = high.loc[test_idx] if high is not None else None
|
||||
test_low = low.loc[test_idx] if low is not None else None
|
||||
|
||||
test_params = {**self.fixed_params, **best_params}
|
||||
test_score, test_return, eq_curve = self._run_test(
|
||||
test_close, test_high, test_low, test_params
|
||||
)
|
||||
|
||||
return {
|
||||
'window': window_idx + 1,
|
||||
'train_start': train_idx[0],
|
||||
'train_end': train_idx[-1],
|
||||
'test_start': test_idx[0],
|
||||
'test_end': test_idx[-1],
|
||||
'best_params': best_params,
|
||||
'train_score': best_score,
|
||||
'test_score': test_score,
|
||||
'test_return': test_return
|
||||
}, eq_curve
|
||||
|
||||
except Exception as e:
|
||||
logger.error("Error in window %d: %s", window_idx + 1, e, exc_info=True)
|
||||
return None
|
||||
|
||||
def _optimize_train(
|
||||
self,
|
||||
close: pd.Series,
|
||||
high: pd.Series | None,
|
||||
low: pd.Series | None
|
||||
) -> tuple[dict, float]:
|
||||
"""Run grid search on training data to find best parameters."""
|
||||
entries, exits = self.strategy.run(
|
||||
close, high=high, low=low, **self.param_grid
|
||||
)
|
||||
|
||||
pf_train = vbt.Portfolio.from_signals(
|
||||
close, entries, exits,
|
||||
fees=self.fees,
|
||||
freq=self.freq
|
||||
)
|
||||
|
||||
perf_stats = pf_train.sharpe_ratio()
|
||||
perf_stats = perf_stats.fillna(-999)
|
||||
|
||||
best_idx = perf_stats.idxmax()
|
||||
best_score = perf_stats.max()
|
||||
|
||||
# Extract best params from grid search
|
||||
if len(self.grid_keys) == 1:
|
||||
best_params = {self.grid_keys[0]: best_idx}
|
||||
elif len(self.grid_keys) > 1:
|
||||
best_params = dict(zip(self.grid_keys, best_idx))
|
||||
else:
|
||||
best_params = {}
|
||||
|
||||
return best_params, best_score
|
||||
|
||||
def _run_test(
|
||||
self,
|
||||
close: pd.Series,
|
||||
high: pd.Series | None,
|
||||
low: pd.Series | None,
|
||||
params: dict
|
||||
) -> tuple[float, float, pd.Series]:
|
||||
"""Run test phase with given parameters."""
|
||||
entries, exits = self.strategy.run(
|
||||
close, high=high, low=low, **params
|
||||
)
|
||||
|
||||
pf_test = vbt.Portfolio.from_signals(
|
||||
close, entries, exits,
|
||||
fees=self.fees,
|
||||
freq=self.freq
|
||||
)
|
||||
|
||||
return pf_test.sharpe_ratio(), pf_test.total_return(), pf_test.value()
|
||||
|
||||
def _stitch_equity_curves(
|
||||
self,
|
||||
equity_curves: list[pd.Series]
|
||||
) -> pd.Series | None:
|
||||
"""Stitch multiple equity curves into a continuous series."""
|
||||
if not equity_curves:
|
||||
return None
|
||||
|
||||
stitched = [equity_curves[0]]
|
||||
for j in range(1, len(equity_curves)):
|
||||
prev_end_val = stitched[-1].iloc[-1]
|
||||
curr_curve = equity_curves[j]
|
||||
init_cash = curr_curve.iloc[0]
|
||||
|
||||
# Scale curve to continue from previous end value
|
||||
scaled_curve = (curr_curve / init_cash) * prev_end_val
|
||||
stitched.append(scaled_curve)
|
||||
|
||||
return pd.concat(stitched)
|
||||
108
engine/portfolio.py
Normal file
108
engine/portfolio.py
Normal file
@@ -0,0 +1,108 @@
|
||||
"""
|
||||
Portfolio simulation utilities for backtesting.
|
||||
|
||||
Handles long-only and long/short portfolio creation using VectorBT.
|
||||
"""
|
||||
import pandas as pd
|
||||
import vectorbt as vbt
|
||||
|
||||
from engine.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def run_long_only_portfolio(
|
||||
close: pd.Series,
|
||||
entries: pd.DataFrame,
|
||||
exits: pd.DataFrame,
|
||||
init_cash: float,
|
||||
fees: float,
|
||||
slippage: float,
|
||||
freq: str,
|
||||
sl_stop: float | None,
|
||||
tp_stop: float | None,
|
||||
sl_trail: bool,
|
||||
leverage: int
|
||||
) -> vbt.Portfolio:
|
||||
"""
|
||||
Run a long-only portfolio simulation.
|
||||
|
||||
Args:
|
||||
close: Close price series
|
||||
entries: Entry signals
|
||||
exits: Exit signals
|
||||
init_cash: Initial capital
|
||||
fees: Transaction fee percentage
|
||||
slippage: Slippage percentage
|
||||
freq: Data frequency string
|
||||
sl_stop: Stop loss percentage
|
||||
tp_stop: Take profit percentage
|
||||
sl_trail: Enable trailing stop loss
|
||||
leverage: Leverage multiplier
|
||||
|
||||
Returns:
|
||||
VectorBT Portfolio object
|
||||
"""
|
||||
effective_cash = init_cash * leverage
|
||||
|
||||
return vbt.Portfolio.from_signals(
|
||||
close=close,
|
||||
entries=entries,
|
||||
exits=exits,
|
||||
init_cash=effective_cash,
|
||||
fees=fees,
|
||||
slippage=slippage,
|
||||
freq=freq,
|
||||
sl_stop=sl_stop,
|
||||
tp_stop=tp_stop,
|
||||
sl_trail=sl_trail,
|
||||
size=1.0,
|
||||
size_type='percent',
|
||||
)
|
||||
|
||||
|
||||
def run_long_short_portfolio(
|
||||
close: pd.Series,
|
||||
long_entries: pd.DataFrame,
|
||||
long_exits: pd.DataFrame,
|
||||
short_entries: pd.DataFrame,
|
||||
short_exits: pd.DataFrame,
|
||||
init_cash: float,
|
||||
fees: float,
|
||||
slippage: float,
|
||||
freq: str,
|
||||
sl_stop: float | None,
|
||||
tp_stop: float | None,
|
||||
sl_trail: bool,
|
||||
leverage: int,
|
||||
size: pd.Series | float = 1.0,
|
||||
size_type: str = 'value' # Changed to 'value' to support reversals/sizing
|
||||
) -> vbt.Portfolio:
|
||||
"""
|
||||
Run a portfolio supporting both long and short positions.
|
||||
|
||||
Uses VectorBT's native support for short_entries/short_exits
|
||||
to simulate a single unified portfolio.
|
||||
"""
|
||||
effective_cash = init_cash * leverage
|
||||
|
||||
# If size is passed as value (USD), we don't scale it by leverage here
|
||||
# The backtester has already scaled it by init_cash.
|
||||
# If using 'value', vbt treats it as "Amount of CASH to use for the trade"
|
||||
|
||||
return vbt.Portfolio.from_signals(
|
||||
close=close,
|
||||
entries=long_entries,
|
||||
exits=long_exits,
|
||||
short_entries=short_entries,
|
||||
short_exits=short_exits,
|
||||
init_cash=effective_cash,
|
||||
fees=fees,
|
||||
slippage=slippage,
|
||||
freq=freq,
|
||||
sl_stop=sl_stop,
|
||||
tp_stop=tp_stop,
|
||||
sl_trail=sl_trail,
|
||||
size=size,
|
||||
size_type=size_type,
|
||||
)
|
||||
228
engine/reporting.py
Normal file
228
engine/reporting.py
Normal file
@@ -0,0 +1,228 @@
|
||||
"""
|
||||
Reporting module for backtest results.
|
||||
|
||||
Handles summary printing, CSV exports, and plotting.
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
import pandas as pd
|
||||
import plotly.graph_objects as go
|
||||
import vectorbt as vbt
|
||||
from plotly.subplots import make_subplots
|
||||
|
||||
from engine.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class Reporter:
|
||||
"""Reporter for backtest results with market-specific metrics."""
|
||||
|
||||
def __init__(self, output_dir: str = "backtest_logs"):
|
||||
self.output_dir = Path(output_dir)
|
||||
self.output_dir.mkdir(exist_ok=True)
|
||||
|
||||
def print_summary(self, result) -> None:
|
||||
"""
|
||||
Print backtest summary to console via logger.
|
||||
|
||||
Args:
|
||||
result: BacktestResult or vbt.Portfolio object
|
||||
"""
|
||||
(portfolio, market_type, leverage, funding_paid,
|
||||
liq_count, liq_loss, adjusted_return) = self._extract_result_data(result)
|
||||
|
||||
# Extract period info
|
||||
idx = portfolio.wrapper.index
|
||||
start_date = idx[0].strftime("%Y-%m-%d")
|
||||
end_date = idx[-1].strftime("%Y-%m-%d")
|
||||
|
||||
# Extract price info
|
||||
close = portfolio.close
|
||||
start_price = close.iloc[0].mean() if hasattr(close.iloc[0], 'mean') else close.iloc[0]
|
||||
end_price = close.iloc[-1].mean() if hasattr(close.iloc[-1], 'mean') else close.iloc[-1]
|
||||
price_change = ((end_price - start_price) / start_price) * 100
|
||||
|
||||
# Extract fees
|
||||
stats = portfolio.stats()
|
||||
total_fees = stats.get('Total Fees Paid', 0)
|
||||
|
||||
raw_return = portfolio.total_return().mean() * 100
|
||||
|
||||
# Build summary
|
||||
summary_lines = [
|
||||
"",
|
||||
"=" * 50,
|
||||
"BACKTEST RESULTS",
|
||||
"=" * 50,
|
||||
f"Market Type: [{market_type.upper()}]",
|
||||
f"Leverage: [{leverage}x]",
|
||||
f"Period: [{start_date} to {end_date}]",
|
||||
f"Price: [{start_price:,.2f} -> {end_price:,.2f} ({price_change:+.2f}%)]",
|
||||
]
|
||||
|
||||
# Show adjusted return if liquidations occurred
|
||||
if liq_count > 0 and adjusted_return is not None:
|
||||
summary_lines.append(f"Raw Return: [%{raw_return:.2f}] (before liq adjustment)")
|
||||
summary_lines.append(f"Adj Return: [%{adjusted_return:.2f}] (after liq losses)")
|
||||
else:
|
||||
summary_lines.append(f"Total Return: [%{raw_return:.2f}]")
|
||||
|
||||
summary_lines.extend([
|
||||
f"Sharpe Ratio: [{portfolio.sharpe_ratio().mean():.2f}]",
|
||||
f"Max Drawdown: [%{portfolio.max_drawdown().mean() * 100:.2f}]",
|
||||
f"Total Trades: [{portfolio.trades.count().mean():.0f}]",
|
||||
f"Win Rate: [%{portfolio.trades.win_rate().mean() * 100:.2f}]",
|
||||
f"Total Fees: [{total_fees:,.2f}]",
|
||||
])
|
||||
|
||||
if funding_paid != 0:
|
||||
summary_lines.append(f"Funding Paid: [{funding_paid:,.2f}]")
|
||||
if liq_count > 0:
|
||||
summary_lines.append(f"Liquidations: [{liq_count}] (${liq_loss:,.2f} margin lost)")
|
||||
|
||||
summary_lines.append("=" * 50)
|
||||
logger.info("\n".join(summary_lines))
|
||||
|
||||
def save_reports(self, result, filename_prefix: str) -> None:
|
||||
"""
|
||||
Save trade log, stats, and liquidation events to CSV files.
|
||||
|
||||
Args:
|
||||
result: BacktestResult or vbt.Portfolio object
|
||||
filename_prefix: Prefix for output filenames
|
||||
"""
|
||||
(portfolio, market_type, leverage, funding_paid,
|
||||
liq_count, liq_loss, adjusted_return) = self._extract_result_data(result)
|
||||
|
||||
# Save trades
|
||||
self._save_csv(
|
||||
data=portfolio.trades.records_readable,
|
||||
path=self.output_dir / f"{filename_prefix}_trades.csv",
|
||||
description="trade log"
|
||||
)
|
||||
|
||||
# Save stats with market-specific additions
|
||||
stats = portfolio.stats()
|
||||
stats['Market Type'] = market_type
|
||||
stats['Leverage'] = leverage
|
||||
stats['Total Funding Paid'] = funding_paid
|
||||
stats['Liquidations'] = liq_count
|
||||
stats['Liquidation Loss'] = liq_loss
|
||||
if adjusted_return is not None:
|
||||
stats['Adjusted Return'] = adjusted_return
|
||||
|
||||
self._save_csv(
|
||||
data=stats,
|
||||
path=self.output_dir / f"{filename_prefix}_stats.csv",
|
||||
description="stats"
|
||||
)
|
||||
|
||||
# Save liquidation events if any
|
||||
if hasattr(result, 'liquidation_events') and result.liquidation_events:
|
||||
liq_df = pd.DataFrame([
|
||||
{
|
||||
'entry_time': e.entry_time,
|
||||
'entry_price': e.entry_price,
|
||||
'liquidation_time': e.liquidation_time,
|
||||
'liquidation_price': e.liquidation_price,
|
||||
'actual_price': e.actual_price,
|
||||
'direction': e.direction,
|
||||
'margin_lost_pct': e.margin_lost_pct
|
||||
}
|
||||
for e in result.liquidation_events
|
||||
])
|
||||
self._save_csv(
|
||||
data=liq_df,
|
||||
path=self.output_dir / f"{filename_prefix}_liquidations.csv",
|
||||
description="liquidation events"
|
||||
)
|
||||
|
||||
def plot(self, portfolio: vbt.Portfolio, show: bool = True) -> None:
|
||||
"""Display portfolio plot."""
|
||||
if show:
|
||||
portfolio.plot().show()
|
||||
|
||||
def plot_wfa(
|
||||
self,
|
||||
wfa_results: pd.DataFrame,
|
||||
stitched_curve: pd.Series | None = None,
|
||||
show: bool = True
|
||||
) -> None:
|
||||
"""
|
||||
Plot Walk-Forward Analysis results.
|
||||
|
||||
Args:
|
||||
wfa_results: DataFrame with WFA window results
|
||||
stitched_curve: Stitched out-of-sample equity curve
|
||||
show: Whether to display the plot
|
||||
"""
|
||||
fig = make_subplots(
|
||||
rows=2, cols=1,
|
||||
shared_xaxes=False,
|
||||
vertical_spacing=0.1,
|
||||
subplot_titles=(
|
||||
"Walk-Forward Test Scores (Sharpe)",
|
||||
"Stitched Out-of-Sample Equity"
|
||||
)
|
||||
)
|
||||
|
||||
fig.add_trace(
|
||||
go.Bar(
|
||||
x=wfa_results['window'],
|
||||
y=wfa_results['test_score'],
|
||||
name="Test Sharpe"
|
||||
),
|
||||
row=1, col=1
|
||||
)
|
||||
|
||||
if stitched_curve is not None:
|
||||
fig.add_trace(
|
||||
go.Scatter(
|
||||
x=stitched_curve.index,
|
||||
y=stitched_curve.values,
|
||||
name="Equity",
|
||||
mode='lines'
|
||||
),
|
||||
row=2, col=1
|
||||
)
|
||||
|
||||
fig.update_layout(height=800, title_text="Walk-Forward Analysis Report")
|
||||
|
||||
if show:
|
||||
fig.show()
|
||||
|
||||
def _extract_result_data(self, result) -> tuple:
|
||||
"""
|
||||
Extract data from BacktestResult or raw Portfolio.
|
||||
|
||||
Returns:
|
||||
Tuple of (portfolio, market_type, leverage, funding_paid, liq_count,
|
||||
liq_loss, adjusted_return)
|
||||
"""
|
||||
if hasattr(result, 'portfolio'):
|
||||
return (
|
||||
result.portfolio,
|
||||
result.market_type.value,
|
||||
result.leverage,
|
||||
result.total_funding_paid,
|
||||
result.liquidation_count,
|
||||
getattr(result, 'total_liquidation_loss', 0.0),
|
||||
getattr(result, 'adjusted_return', None)
|
||||
)
|
||||
return (result, "unknown", 1, 0.0, 0, 0.0, None)
|
||||
|
||||
def _save_csv(self, data, path: Path, description: str) -> None:
|
||||
"""
|
||||
Save data to CSV with consistent error handling.
|
||||
|
||||
Args:
|
||||
data: DataFrame or Series to save
|
||||
path: Output file path
|
||||
description: Human-readable description for logging
|
||||
"""
|
||||
try:
|
||||
data.to_csv(path)
|
||||
logger.info("Saved %s to %s", description, path)
|
||||
except Exception as e:
|
||||
logger.error("Could not save %s: %s", description, e)
|
||||
395
engine/risk.py
Normal file
395
engine/risk.py
Normal file
@@ -0,0 +1,395 @@
|
||||
"""
|
||||
Risk management utilities for backtesting.
|
||||
|
||||
Handles funding rate calculations and liquidation detection.
|
||||
"""
|
||||
from dataclasses import dataclass
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from engine.logging_config import get_logger
|
||||
from engine.market import MarketConfig, calculate_liquidation_price
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class LiquidationEvent:
|
||||
"""
|
||||
Record of a liquidation event during backtesting.
|
||||
|
||||
Attributes:
|
||||
entry_time: Timestamp when position was opened
|
||||
entry_price: Price at position entry
|
||||
liquidation_time: Timestamp when liquidation occurred
|
||||
liquidation_price: Calculated liquidation price
|
||||
actual_price: Actual price that triggered liquidation (high/low)
|
||||
direction: 'long' or 'short'
|
||||
margin_lost_pct: Percentage of margin lost (typically 100%)
|
||||
"""
|
||||
entry_time: pd.Timestamp
|
||||
entry_price: float
|
||||
liquidation_time: pd.Timestamp
|
||||
liquidation_price: float
|
||||
actual_price: float
|
||||
direction: str
|
||||
margin_lost_pct: float = 1.0
|
||||
|
||||
|
||||
def calculate_funding(
|
||||
close: pd.Series,
|
||||
long_entries: pd.DataFrame,
|
||||
long_exits: pd.DataFrame,
|
||||
short_entries: pd.DataFrame,
|
||||
short_exits: pd.DataFrame,
|
||||
market_config: MarketConfig,
|
||||
leverage: int
|
||||
) -> float:
|
||||
"""
|
||||
Calculate total funding paid/received for perpetual positions.
|
||||
|
||||
Simplified model: applies funding rate every 8 hours to open positions.
|
||||
Positive rate means longs pay shorts.
|
||||
|
||||
Args:
|
||||
close: Price series
|
||||
long_entries: Long entry signals
|
||||
long_exits: Long exit signals
|
||||
short_entries: Short entry signals
|
||||
short_exits: Short exit signals
|
||||
market_config: Market configuration with funding parameters
|
||||
leverage: Position leverage
|
||||
|
||||
Returns:
|
||||
Total funding paid (positive) or received (negative)
|
||||
"""
|
||||
if market_config.funding_interval_hours == 0:
|
||||
return 0.0
|
||||
|
||||
funding_rate = market_config.funding_rate
|
||||
interval_hours = market_config.funding_interval_hours
|
||||
|
||||
# Determine position state at each bar
|
||||
long_position = long_entries.cumsum() - long_exits.cumsum()
|
||||
short_position = short_entries.cumsum() - short_exits.cumsum()
|
||||
|
||||
# Clamp to 0/1 (either in position or not)
|
||||
long_position = (long_position > 0).astype(int)
|
||||
short_position = (short_position > 0).astype(int)
|
||||
|
||||
# Find funding timestamps (every 8 hours: 00:00, 08:00, 16:00 UTC)
|
||||
funding_times = close.index[close.index.hour % interval_hours == 0]
|
||||
|
||||
total_funding = 0.0
|
||||
for ts in funding_times:
|
||||
if ts not in close.index:
|
||||
continue
|
||||
price = close.loc[ts]
|
||||
|
||||
# Long pays funding, short receives (when rate > 0)
|
||||
if isinstance(long_position, pd.DataFrame):
|
||||
long_open = long_position.loc[ts].any()
|
||||
short_open = short_position.loc[ts].any()
|
||||
else:
|
||||
long_open = long_position.loc[ts] > 0
|
||||
short_open = short_position.loc[ts] > 0
|
||||
|
||||
position_value = price * leverage
|
||||
if long_open:
|
||||
total_funding += position_value * funding_rate
|
||||
if short_open:
|
||||
total_funding -= position_value * funding_rate
|
||||
|
||||
return total_funding
|
||||
|
||||
|
||||
def inject_liquidation_exits(
|
||||
close: pd.Series,
|
||||
high: pd.Series,
|
||||
low: pd.Series,
|
||||
long_entries: pd.DataFrame | pd.Series,
|
||||
long_exits: pd.DataFrame | pd.Series,
|
||||
short_entries: pd.DataFrame | pd.Series,
|
||||
short_exits: pd.DataFrame | pd.Series,
|
||||
leverage: int,
|
||||
maintenance_margin_rate: float
|
||||
) -> tuple[pd.DataFrame | pd.Series, pd.DataFrame | pd.Series, list[LiquidationEvent]]:
|
||||
"""
|
||||
Modify exit signals to force position closure at liquidation points.
|
||||
|
||||
This function simulates realistic liquidation behavior by:
|
||||
1. Finding positions that would be liquidated before their normal exit
|
||||
2. Injecting forced exit signals at the liquidation bar
|
||||
3. Recording all liquidation events
|
||||
|
||||
Args:
|
||||
close: Close price series
|
||||
high: High price series
|
||||
low: Low price series
|
||||
long_entries: Long entry signals
|
||||
long_exits: Long exit signals
|
||||
short_entries: Short entry signals
|
||||
short_exits: Short exit signals
|
||||
leverage: Position leverage
|
||||
maintenance_margin_rate: Maintenance margin rate for liquidation
|
||||
|
||||
Returns:
|
||||
Tuple of (modified_long_exits, modified_short_exits, liquidation_events)
|
||||
"""
|
||||
if leverage <= 1:
|
||||
return long_exits, short_exits, []
|
||||
|
||||
liquidation_events: list[LiquidationEvent] = []
|
||||
|
||||
# Convert to DataFrame if Series for consistent handling
|
||||
is_series = isinstance(long_entries, pd.Series)
|
||||
if is_series:
|
||||
long_entries_df = long_entries.to_frame()
|
||||
long_exits_df = long_exits.to_frame()
|
||||
short_entries_df = short_entries.to_frame()
|
||||
short_exits_df = short_exits.to_frame()
|
||||
else:
|
||||
long_entries_df = long_entries
|
||||
long_exits_df = long_exits.copy()
|
||||
short_entries_df = short_entries
|
||||
short_exits_df = short_exits.copy()
|
||||
|
||||
modified_long_exits = long_exits_df.copy()
|
||||
modified_short_exits = short_exits_df.copy()
|
||||
|
||||
# Process long positions
|
||||
long_mask = long_entries_df.any(axis=1)
|
||||
for entry_idx in close.index[long_mask]:
|
||||
entry_price = close.loc[entry_idx]
|
||||
liq_price = calculate_liquidation_price(
|
||||
entry_price, leverage, is_long=True,
|
||||
maintenance_margin_rate=maintenance_margin_rate
|
||||
)
|
||||
|
||||
# Find the normal exit for this entry
|
||||
subsequent_exits = long_exits_df.loc[entry_idx:].any(axis=1)
|
||||
exit_indices = subsequent_exits[subsequent_exits].index
|
||||
normal_exit_idx = exit_indices[0] if len(exit_indices) > 0 else close.index[-1]
|
||||
|
||||
# Check if liquidation occurs before normal exit
|
||||
price_range = low.loc[entry_idx:normal_exit_idx]
|
||||
if (price_range < liq_price).any():
|
||||
liq_bar = price_range[price_range < liq_price].index[0]
|
||||
|
||||
# Inject forced exit at liquidation bar
|
||||
for col in modified_long_exits.columns:
|
||||
modified_long_exits.loc[liq_bar, col] = True
|
||||
|
||||
# Record the liquidation event
|
||||
liquidation_events.append(LiquidationEvent(
|
||||
entry_time=entry_idx,
|
||||
entry_price=entry_price,
|
||||
liquidation_time=liq_bar,
|
||||
liquidation_price=liq_price,
|
||||
actual_price=low.loc[liq_bar],
|
||||
direction='long',
|
||||
margin_lost_pct=1.0
|
||||
))
|
||||
|
||||
logger.warning(
|
||||
"LIQUIDATION (Long): Entry %s ($%.2f) -> Liquidated %s "
|
||||
"(liq=$%.2f, low=$%.2f)",
|
||||
entry_idx.strftime('%Y-%m-%d'), entry_price,
|
||||
liq_bar.strftime('%Y-%m-%d'), liq_price, low.loc[liq_bar]
|
||||
)
|
||||
|
||||
# Process short positions
|
||||
short_mask = short_entries_df.any(axis=1)
|
||||
for entry_idx in close.index[short_mask]:
|
||||
entry_price = close.loc[entry_idx]
|
||||
liq_price = calculate_liquidation_price(
|
||||
entry_price, leverage, is_long=False,
|
||||
maintenance_margin_rate=maintenance_margin_rate
|
||||
)
|
||||
|
||||
# Find the normal exit for this entry
|
||||
subsequent_exits = short_exits_df.loc[entry_idx:].any(axis=1)
|
||||
exit_indices = subsequent_exits[subsequent_exits].index
|
||||
normal_exit_idx = exit_indices[0] if len(exit_indices) > 0 else close.index[-1]
|
||||
|
||||
# Check if liquidation occurs before normal exit
|
||||
price_range = high.loc[entry_idx:normal_exit_idx]
|
||||
if (price_range > liq_price).any():
|
||||
liq_bar = price_range[price_range > liq_price].index[0]
|
||||
|
||||
# Inject forced exit at liquidation bar
|
||||
for col in modified_short_exits.columns:
|
||||
modified_short_exits.loc[liq_bar, col] = True
|
||||
|
||||
# Record the liquidation event
|
||||
liquidation_events.append(LiquidationEvent(
|
||||
entry_time=entry_idx,
|
||||
entry_price=entry_price,
|
||||
liquidation_time=liq_bar,
|
||||
liquidation_price=liq_price,
|
||||
actual_price=high.loc[liq_bar],
|
||||
direction='short',
|
||||
margin_lost_pct=1.0
|
||||
))
|
||||
|
||||
logger.warning(
|
||||
"LIQUIDATION (Short): Entry %s ($%.2f) -> Liquidated %s "
|
||||
"(liq=$%.2f, high=$%.2f)",
|
||||
entry_idx.strftime('%Y-%m-%d'), entry_price,
|
||||
liq_bar.strftime('%Y-%m-%d'), liq_price, high.loc[liq_bar]
|
||||
)
|
||||
|
||||
# Convert back to Series if input was Series
|
||||
if is_series:
|
||||
modified_long_exits = modified_long_exits.iloc[:, 0]
|
||||
modified_short_exits = modified_short_exits.iloc[:, 0]
|
||||
|
||||
return modified_long_exits, modified_short_exits, liquidation_events
|
||||
|
||||
|
||||
def calculate_liquidation_adjustment(
|
||||
liquidation_events: list[LiquidationEvent],
|
||||
init_cash: float,
|
||||
leverage: int
|
||||
) -> tuple[float, float]:
|
||||
"""
|
||||
Calculate the return adjustment for liquidated positions.
|
||||
|
||||
VectorBT calculates trade P&L using close price at exit bar.
|
||||
For liquidations, the actual loss is 100% of the position margin.
|
||||
This function calculates the difference between what VectorBT
|
||||
recorded and what actually would have happened.
|
||||
|
||||
In our portfolio setup:
|
||||
- Long/short each get half the capital (init_cash * leverage / 2)
|
||||
- Each trade uses 100% of that allocation (size=1.0, percent)
|
||||
- On liquidation, the margin for that trade is lost entirely
|
||||
|
||||
The adjustment is the DIFFERENCE between:
|
||||
- VectorBT's calculated P&L (exit at close price)
|
||||
- Actual liquidation P&L (100% margin loss)
|
||||
|
||||
Args:
|
||||
liquidation_events: List of liquidation events
|
||||
init_cash: Initial portfolio cash (before leverage)
|
||||
leverage: Position leverage used
|
||||
|
||||
Returns:
|
||||
Tuple of (total_margin_lost, adjustment_pct)
|
||||
- total_margin_lost: Estimated total margin lost from liquidations
|
||||
- adjustment_pct: Percentage adjustment to apply to returns
|
||||
"""
|
||||
if not liquidation_events:
|
||||
return 0.0, 0.0
|
||||
|
||||
# In our setup, each side (long/short) gets half the capital
|
||||
# Margin per side = init_cash / 2
|
||||
margin_per_side = init_cash / 2
|
||||
|
||||
# For each liquidation, VectorBT recorded some P&L based on close price
|
||||
# The actual P&L should be -100% of the margin used for that trade
|
||||
#
|
||||
# We estimate the adjustment as:
|
||||
# - Each liquidation should have resulted in ~-20% loss (at 5x leverage)
|
||||
# - VectorBT may have recorded a different value
|
||||
# - The margin loss is (1/leverage) per trade that gets liquidated
|
||||
|
||||
# Calculate liquidation loss rate based on leverage
|
||||
# At 5x leverage, liquidation = ~19.6% adverse move = 100% margin loss
|
||||
liq_loss_rate = 1.0 / leverage # Approximate loss per trade as % of position
|
||||
|
||||
# Count liquidations
|
||||
n_liquidations = len(liquidation_events)
|
||||
|
||||
# Estimate total margin lost:
|
||||
# Each liquidation on average loses the margin for that trade
|
||||
# Since VectorBT uses half capital per side, and we trade 100% size,
|
||||
# each liquidation loses approximately margin_per_side
|
||||
# But we cap at available capital
|
||||
total_margin_lost = min(n_liquidations * margin_per_side * liq_loss_rate, init_cash)
|
||||
|
||||
# Calculate as percentage of initial capital
|
||||
adjustment_pct = (total_margin_lost / init_cash) * 100
|
||||
|
||||
return total_margin_lost, adjustment_pct
|
||||
|
||||
|
||||
def check_liquidations(
|
||||
close: pd.Series,
|
||||
high: pd.Series,
|
||||
low: pd.Series,
|
||||
long_entries: pd.DataFrame,
|
||||
long_exits: pd.DataFrame,
|
||||
short_entries: pd.DataFrame,
|
||||
short_exits: pd.DataFrame,
|
||||
leverage: int,
|
||||
maintenance_margin_rate: float
|
||||
) -> int:
|
||||
"""
|
||||
Check for liquidation events and log warnings.
|
||||
|
||||
Args:
|
||||
close: Close price series
|
||||
high: High price series
|
||||
low: Low price series
|
||||
long_entries: Long entry signals
|
||||
long_exits: Long exit signals
|
||||
short_entries: Short entry signals
|
||||
short_exits: Short exit signals
|
||||
leverage: Position leverage
|
||||
maintenance_margin_rate: Maintenance margin rate for liquidation
|
||||
|
||||
Returns:
|
||||
Count of liquidation warnings
|
||||
"""
|
||||
warnings = 0
|
||||
|
||||
# For long positions
|
||||
long_mask = (
|
||||
long_entries.any(axis=1)
|
||||
if isinstance(long_entries, pd.DataFrame)
|
||||
else long_entries
|
||||
)
|
||||
|
||||
for entry_idx in close.index[long_mask]:
|
||||
entry_price = close.loc[entry_idx]
|
||||
liq_price = calculate_liquidation_price(
|
||||
entry_price, leverage, is_long=True,
|
||||
maintenance_margin_rate=maintenance_margin_rate
|
||||
)
|
||||
|
||||
subsequent = low.loc[entry_idx:]
|
||||
if (subsequent < liq_price).any():
|
||||
liq_bar = subsequent[subsequent < liq_price].index[0]
|
||||
logger.warning(
|
||||
"LIQUIDATION WARNING (Long): Entry at %s ($%.2f), "
|
||||
"would liquidate at %s (liq_price=$%.2f, low=$%.2f)",
|
||||
entry_idx, entry_price, liq_bar, liq_price, low.loc[liq_bar]
|
||||
)
|
||||
warnings += 1
|
||||
|
||||
# For short positions
|
||||
short_mask = (
|
||||
short_entries.any(axis=1)
|
||||
if isinstance(short_entries, pd.DataFrame)
|
||||
else short_entries
|
||||
)
|
||||
|
||||
for entry_idx in close.index[short_mask]:
|
||||
entry_price = close.loc[entry_idx]
|
||||
liq_price = calculate_liquidation_price(
|
||||
entry_price, leverage, is_long=False,
|
||||
maintenance_margin_rate=maintenance_margin_rate
|
||||
)
|
||||
|
||||
subsequent = high.loc[entry_idx:]
|
||||
if (subsequent > liq_price).any():
|
||||
liq_bar = subsequent[subsequent > liq_price].index[0]
|
||||
logger.warning(
|
||||
"LIQUIDATION WARNING (Short): Entry at %s ($%.2f), "
|
||||
"would liquidate at %s (liq_price=$%.2f, high=$%.2f)",
|
||||
entry_idx, entry_price, liq_bar, liq_price, high.loc[liq_bar]
|
||||
)
|
||||
warnings += 1
|
||||
|
||||
return warnings
|
||||
24
frontend/.gitignore
vendored
Normal file
24
frontend/.gitignore
vendored
Normal file
@@ -0,0 +1,24 @@
|
||||
# Logs
|
||||
logs
|
||||
*.log
|
||||
npm-debug.log*
|
||||
yarn-debug.log*
|
||||
yarn-error.log*
|
||||
pnpm-debug.log*
|
||||
lerna-debug.log*
|
||||
|
||||
node_modules
|
||||
dist
|
||||
dist-ssr
|
||||
*.local
|
||||
|
||||
# Editor directories and files
|
||||
.vscode/*
|
||||
!.vscode/extensions.json
|
||||
.idea
|
||||
.DS_Store
|
||||
*.suo
|
||||
*.ntvs*
|
||||
*.njsproj
|
||||
*.sln
|
||||
*.sw?
|
||||
3
frontend/.vscode/extensions.json
vendored
Normal file
3
frontend/.vscode/extensions.json
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"recommendations": ["Vue.volar"]
|
||||
}
|
||||
5
frontend/README.md
Normal file
5
frontend/README.md
Normal file
@@ -0,0 +1,5 @@
|
||||
# Vue 3 + TypeScript + Vite
|
||||
|
||||
This template should help get you started developing with Vue 3 and TypeScript in Vite. The template uses Vue 3 `<script setup>` SFCs, check out the [script setup docs](https://v3.vuejs.org/api/sfc-script-setup.html#sfc-script-setup) to learn more.
|
||||
|
||||
Learn more about the recommended Project Setup and IDE Support in the [Vue Docs TypeScript Guide](https://vuejs.org/guide/typescript/overview.html#project-setup).
|
||||
24
frontend/index.html
Normal file
24
frontend/index.html
Normal file
@@ -0,0 +1,24 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Lowkey Backtest</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500;600;700&family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
body {
|
||||
font-family: 'Inter', -apple-system, BlinkMacSystemFont, sans-serif;
|
||||
}
|
||||
code, pre, .font-mono, input, select {
|
||||
font-family: 'JetBrains Mono', 'Fira Code', monospace;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="app"></div>
|
||||
<script type="module" src="/src/main.ts"></script>
|
||||
</body>
|
||||
</html>
|
||||
2427
frontend/package-lock.json
generated
Normal file
2427
frontend/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
27
frontend/package.json
Normal file
27
frontend/package.json
Normal file
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"name": "frontend",
|
||||
"private": true,
|
||||
"version": "0.0.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
"build": "vue-tsc -b && vite build",
|
||||
"preview": "vite preview"
|
||||
},
|
||||
"dependencies": {
|
||||
"axios": "^1.13.2",
|
||||
"plotly.js-dist-min": "^3.3.1",
|
||||
"vue": "^3.5.24",
|
||||
"vue-router": "^4.6.4"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@tailwindcss/vite": "^4.1.18",
|
||||
"@types/node": "^24.10.1",
|
||||
"@vitejs/plugin-vue": "^6.0.1",
|
||||
"@vue/tsconfig": "^0.8.1",
|
||||
"tailwindcss": "^4.1.18",
|
||||
"typescript": "~5.9.3",
|
||||
"vite": "^7.2.4",
|
||||
"vue-tsc": "^3.1.4"
|
||||
}
|
||||
}
|
||||
1
frontend/public/vite.svg
Normal file
1
frontend/public/vite.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="31.88" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 257"><defs><linearGradient id="IconifyId1813088fe1fbc01fb466" x1="-.828%" x2="57.636%" y1="7.652%" y2="78.411%"><stop offset="0%" stop-color="#41D1FF"></stop><stop offset="100%" stop-color="#BD34FE"></stop></linearGradient><linearGradient id="IconifyId1813088fe1fbc01fb467" x1="43.376%" x2="50.316%" y1="2.242%" y2="89.03%"><stop offset="0%" stop-color="#FFEA83"></stop><stop offset="8.333%" stop-color="#FFDD35"></stop><stop offset="100%" stop-color="#FFA800"></stop></linearGradient></defs><path fill="url(#IconifyId1813088fe1fbc01fb466)" d="M255.153 37.938L134.897 252.976c-2.483 4.44-8.862 4.466-11.382.048L.875 37.958c-2.746-4.814 1.371-10.646 6.827-9.67l120.385 21.517a6.537 6.537 0 0 0 2.322-.004l117.867-21.483c5.438-.991 9.574 4.796 6.877 9.62Z"></path><path fill="url(#IconifyId1813088fe1fbc01fb467)" d="M185.432.063L96.44 17.501a3.268 3.268 0 0 0-2.634 3.014l-5.474 92.456a3.268 3.268 0 0 0 3.997 3.378l24.777-5.718c2.318-.535 4.413 1.507 3.936 3.838l-7.361 36.047c-.495 2.426 1.782 4.5 4.151 3.78l15.304-4.649c2.372-.72 4.652 1.36 4.15 3.788l-11.698 56.621c-.732 3.542 3.979 5.473 5.943 2.437l1.313-2.028l72.516-144.72c1.215-2.423-.88-5.186-3.54-4.672l-25.505 4.922c-2.396.462-4.435-1.77-3.759-4.114l16.646-57.705c.677-2.35-1.37-4.583-3.769-4.113Z"></path></svg>
|
||||
|
After Width: | Height: | Size: 1.5 KiB |
72
frontend/src/App.vue
Normal file
72
frontend/src/App.vue
Normal file
@@ -0,0 +1,72 @@
|
||||
<script setup lang="ts">
|
||||
import { ref } from 'vue'
|
||||
import { RouterLink, RouterView } from 'vue-router'
|
||||
import RunHistory from '@/components/RunHistory.vue'
|
||||
|
||||
const historyOpen = ref(true)
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="flex h-screen overflow-hidden">
|
||||
<!-- Sidebar Navigation -->
|
||||
<aside class="w-16 bg-bg-secondary border-r border-border flex flex-col items-center py-4 gap-4">
|
||||
<!-- Logo -->
|
||||
<div class="w-10 h-10 rounded-lg bg-accent-blue flex items-center justify-center text-black font-bold text-lg">
|
||||
LB
|
||||
</div>
|
||||
|
||||
<!-- Nav Links -->
|
||||
<nav class="flex flex-col gap-2 mt-4">
|
||||
<RouterLink
|
||||
to="/"
|
||||
class="w-10 h-10 rounded-lg flex items-center justify-center hover:bg-bg-hover transition-colors"
|
||||
:class="{ 'bg-bg-tertiary': $route.path === '/' }"
|
||||
title="Dashboard"
|
||||
>
|
||||
<svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 19v-6a2 2 0 00-2-2H5a2 2 0 00-2 2v6a2 2 0 002 2h2a2 2 0 002-2zm0 0V9a2 2 0 012-2h2a2 2 0 012 2v10m-6 0a2 2 0 002 2h2a2 2 0 002-2m0 0V5a2 2 0 012-2h2a2 2 0 012 2v14a2 2 0 01-2 2h-2a2 2 0 01-2-2z" />
|
||||
</svg>
|
||||
</RouterLink>
|
||||
|
||||
<RouterLink
|
||||
to="/compare"
|
||||
class="w-10 h-10 rounded-lg flex items-center justify-center hover:bg-bg-hover transition-colors"
|
||||
:class="{ 'bg-bg-tertiary': $route.path === '/compare' }"
|
||||
title="Compare Runs"
|
||||
>
|
||||
<svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 17V7m0 10a2 2 0 01-2 2H5a2 2 0 01-2-2V7a2 2 0 012-2h2a2 2 0 012 2m0 10a2 2 0 002 2h2a2 2 0 002-2M9 7a2 2 0 012-2h2a2 2 0 012 2m0 10V7m0 10a2 2 0 002 2h2a2 2 0 002-2V7a2 2 0 00-2-2h-2a2 2 0 00-2 2" />
|
||||
</svg>
|
||||
</RouterLink>
|
||||
</nav>
|
||||
|
||||
<!-- Spacer -->
|
||||
<div class="flex-1"></div>
|
||||
|
||||
<!-- Toggle History -->
|
||||
<button
|
||||
@click="historyOpen = !historyOpen"
|
||||
class="w-10 h-10 rounded-lg flex items-center justify-center hover:bg-bg-hover transition-colors"
|
||||
:class="{ 'bg-bg-tertiary': historyOpen }"
|
||||
title="Toggle Run History"
|
||||
>
|
||||
<svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 8v4l3 3m6-3a9 9 0 11-18 0 9 9 0 0118 0z" />
|
||||
</svg>
|
||||
</button>
|
||||
</aside>
|
||||
|
||||
<!-- Main Content -->
|
||||
<main class="flex-1 overflow-auto">
|
||||
<RouterView />
|
||||
</main>
|
||||
|
||||
<!-- Run History Sidebar -->
|
||||
<aside
|
||||
v-if="historyOpen"
|
||||
class="w-72 bg-bg-secondary border-l border-border overflow-hidden flex flex-col"
|
||||
>
|
||||
<RunHistory />
|
||||
</aside>
|
||||
</div>
|
||||
</template>
|
||||
81
frontend/src/api/client.ts
Normal file
81
frontend/src/api/client.ts
Normal file
@@ -0,0 +1,81 @@
|
||||
/**
|
||||
* API client for Lowkey Backtest backend.
|
||||
*/
|
||||
import axios from 'axios'
|
||||
import type {
|
||||
StrategiesResponse,
|
||||
DataStatusResponse,
|
||||
BacktestRequest,
|
||||
BacktestResult,
|
||||
BacktestListResponse,
|
||||
CompareResult,
|
||||
} from './types'
|
||||
|
||||
const api = axios.create({
|
||||
baseURL: '/api',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
})
|
||||
|
||||
/**
|
||||
* Get list of available strategies with parameters.
|
||||
*/
|
||||
export async function getStrategies(): Promise<StrategiesResponse> {
|
||||
const response = await api.get<StrategiesResponse>('/strategies')
|
||||
return response.data
|
||||
}
|
||||
|
||||
/**
|
||||
* Get list of available symbols with data status.
|
||||
*/
|
||||
export async function getSymbols(): Promise<DataStatusResponse> {
|
||||
const response = await api.get<DataStatusResponse>('/symbols')
|
||||
return response.data
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a backtest with the given configuration.
|
||||
*/
|
||||
export async function runBacktest(request: BacktestRequest): Promise<BacktestResult> {
|
||||
const response = await api.post<BacktestResult>('/backtest', request)
|
||||
return response.data
|
||||
}
|
||||
|
||||
/**
|
||||
* Get list of saved backtest runs.
|
||||
*/
|
||||
export async function getBacktests(params?: {
|
||||
limit?: number
|
||||
offset?: number
|
||||
strategy?: string
|
||||
symbol?: string
|
||||
}): Promise<BacktestListResponse> {
|
||||
const response = await api.get<BacktestListResponse>('/backtests', { params })
|
||||
return response.data
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a specific backtest run by ID.
|
||||
*/
|
||||
export async function getBacktest(runId: string): Promise<BacktestResult> {
|
||||
const response = await api.get<BacktestResult>(`/backtest/${runId}`)
|
||||
return response.data
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a backtest run.
|
||||
*/
|
||||
export async function deleteBacktest(runId: string): Promise<void> {
|
||||
await api.delete(`/backtest/${runId}`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare multiple backtest runs.
|
||||
*/
|
||||
export async function compareRuns(runIds: string[]): Promise<CompareResult> {
|
||||
const response = await api.post<CompareResult>('/compare', { run_ids: runIds })
|
||||
return response.data
|
||||
}
|
||||
|
||||
export default api
|
||||
131
frontend/src/api/types.ts
Normal file
131
frontend/src/api/types.ts
Normal file
@@ -0,0 +1,131 @@
|
||||
/**
|
||||
* TypeScript types matching the FastAPI Pydantic schemas.
|
||||
*/
|
||||
|
||||
// Strategy types
|
||||
export interface StrategyInfo {
|
||||
name: string
|
||||
display_name: string
|
||||
market_type: string
|
||||
default_leverage: number
|
||||
default_params: Record<string, unknown>
|
||||
grid_params: Record<string, unknown>
|
||||
}
|
||||
|
||||
export interface StrategiesResponse {
|
||||
strategies: StrategyInfo[]
|
||||
}
|
||||
|
||||
// Symbol/Data types
|
||||
export interface SymbolInfo {
|
||||
symbol: string
|
||||
exchange: string
|
||||
market_type: string
|
||||
timeframes: string[]
|
||||
start_date: string | null
|
||||
end_date: string | null
|
||||
row_count: number
|
||||
}
|
||||
|
||||
export interface DataStatusResponse {
|
||||
symbols: SymbolInfo[]
|
||||
}
|
||||
|
||||
// Backtest types
|
||||
export interface BacktestRequest {
|
||||
strategy: string
|
||||
symbol: string
|
||||
exchange?: string
|
||||
timeframe?: string
|
||||
market_type?: string
|
||||
start_date?: string | null
|
||||
end_date?: string | null
|
||||
init_cash?: number
|
||||
leverage?: number | null
|
||||
fees?: number | null
|
||||
slippage?: number
|
||||
sl_stop?: number | null
|
||||
tp_stop?: number | null
|
||||
sl_trail?: boolean
|
||||
params?: Record<string, unknown>
|
||||
}
|
||||
|
||||
export interface TradeRecord {
|
||||
entry_time: string
|
||||
exit_time: string | null
|
||||
entry_price: number
|
||||
exit_price: number | null
|
||||
size: number
|
||||
direction: string
|
||||
pnl: number | null
|
||||
return_pct: number | null
|
||||
status: string
|
||||
}
|
||||
|
||||
export interface EquityPoint {
|
||||
timestamp: string
|
||||
value: number
|
||||
drawdown: number
|
||||
}
|
||||
|
||||
export interface BacktestMetrics {
|
||||
total_return: number
|
||||
benchmark_return: number
|
||||
alpha: number
|
||||
sharpe_ratio: number
|
||||
max_drawdown: number
|
||||
win_rate: number
|
||||
total_trades: number
|
||||
profit_factor: number | null
|
||||
avg_trade_return: number | null
|
||||
total_fees: number
|
||||
total_funding: number
|
||||
liquidation_count: number
|
||||
liquidation_loss: number
|
||||
adjusted_return: number | null
|
||||
}
|
||||
|
||||
export interface BacktestResult {
|
||||
run_id: string
|
||||
strategy: string
|
||||
symbol: string
|
||||
market_type: string
|
||||
timeframe: string
|
||||
start_date: string
|
||||
end_date: string
|
||||
leverage: number
|
||||
params: Record<string, unknown>
|
||||
metrics: BacktestMetrics
|
||||
equity_curve: EquityPoint[]
|
||||
trades: TradeRecord[]
|
||||
created_at: string
|
||||
}
|
||||
|
||||
export interface BacktestSummary {
|
||||
run_id: string
|
||||
strategy: string
|
||||
symbol: string
|
||||
market_type: string
|
||||
timeframe: string
|
||||
total_return: number
|
||||
sharpe_ratio: number
|
||||
max_drawdown: number
|
||||
total_trades: number
|
||||
created_at: string
|
||||
params: Record<string, unknown>
|
||||
}
|
||||
|
||||
export interface BacktestListResponse {
|
||||
runs: BacktestSummary[]
|
||||
total: number
|
||||
}
|
||||
|
||||
// Comparison types
|
||||
export interface CompareRequest {
|
||||
run_ids: string[]
|
||||
}
|
||||
|
||||
export interface CompareResult {
|
||||
runs: BacktestResult[]
|
||||
param_diff: Record<string, unknown[]>
|
||||
}
|
||||
1
frontend/src/assets/vue.svg
Normal file
1
frontend/src/assets/vue.svg
Normal file
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="37.07" height="36" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 198"><path fill="#41B883" d="M204.8 0H256L128 220.8L0 0h97.92L128 51.2L157.44 0h47.36Z"></path><path fill="#41B883" d="m0 0l128 220.8L256 0h-51.2L128 132.48L50.56 0H0Z"></path><path fill="#35495E" d="M50.56 0L128 133.12L204.8 0h-47.36L128 51.2L97.92 0H50.56Z"></path></svg>
|
||||
|
After Width: | Height: | Size: 496 B |
186
frontend/src/components/BacktestConfig.vue
Normal file
186
frontend/src/components/BacktestConfig.vue
Normal file
@@ -0,0 +1,186 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, computed, watch, onMounted } from 'vue'
|
||||
import { useBacktest } from '@/composables/useBacktest'
|
||||
import type { BacktestRequest } from '@/api/types'
|
||||
|
||||
const { strategies, symbols, loading, init, executeBacktest } = useBacktest()
|
||||
|
||||
// Form state
|
||||
const selectedStrategy = ref('')
|
||||
const selectedSymbol = ref('')
|
||||
const selectedMarket = ref('perpetual')
|
||||
const timeframe = ref('1h')
|
||||
const initCash = ref(10000)
|
||||
const leverage = ref<number | null>(null)
|
||||
const slStop = ref<number | null>(null)
|
||||
const tpStop = ref<number | null>(null)
|
||||
const params = ref<Record<string, number | boolean>>({})
|
||||
|
||||
// Initialize
|
||||
onMounted(async () => {
|
||||
await init()
|
||||
if (strategies.value.length > 0 && strategies.value[0]) {
|
||||
selectedStrategy.value = strategies.value[0].name
|
||||
}
|
||||
})
|
||||
|
||||
// Get current strategy config
|
||||
const currentStrategy = computed(() =>
|
||||
strategies.value.find(s => s.name === selectedStrategy.value)
|
||||
)
|
||||
|
||||
// Filter symbols by market type
|
||||
const filteredSymbols = computed(() =>
|
||||
symbols.value.filter(s => s.market_type === selectedMarket.value)
|
||||
)
|
||||
|
||||
// Update params when strategy changes
|
||||
watch(selectedStrategy, (name) => {
|
||||
const strategy = strategies.value.find(s => s.name === name)
|
||||
if (strategy) {
|
||||
params.value = { ...strategy.default_params } as Record<string, number | boolean>
|
||||
selectedMarket.value = strategy.market_type
|
||||
leverage.value = strategy.default_leverage > 1 ? strategy.default_leverage : null
|
||||
}
|
||||
})
|
||||
|
||||
// Update symbol when market changes
|
||||
watch([filteredSymbols, selectedMarket], () => {
|
||||
const firstSymbol = filteredSymbols.value[0]
|
||||
if (filteredSymbols.value.length > 0 && firstSymbol && !filteredSymbols.value.find(s => s.symbol === selectedSymbol.value)) {
|
||||
selectedSymbol.value = firstSymbol.symbol
|
||||
}
|
||||
})
|
||||
|
||||
async function handleSubmit() {
|
||||
if (!selectedStrategy.value || !selectedSymbol.value) return
|
||||
|
||||
const request: BacktestRequest = {
|
||||
strategy: selectedStrategy.value,
|
||||
symbol: selectedSymbol.value,
|
||||
market_type: selectedMarket.value,
|
||||
timeframe: timeframe.value,
|
||||
init_cash: initCash.value,
|
||||
leverage: leverage.value,
|
||||
sl_stop: slStop.value,
|
||||
tp_stop: tpStop.value,
|
||||
params: params.value,
|
||||
}
|
||||
|
||||
await executeBacktest(request)
|
||||
}
|
||||
|
||||
function getParamType(value: unknown): 'number' | 'boolean' | 'unknown' {
|
||||
if (typeof value === 'boolean') return 'boolean'
|
||||
if (typeof value === 'number') return 'number'
|
||||
return 'unknown'
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="card">
|
||||
<h2 class="text-lg font-semibold mb-4">Backtest Configuration</h2>
|
||||
|
||||
<form @submit.prevent="handleSubmit" class="space-y-4">
|
||||
<!-- Strategy -->
|
||||
<div>
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">Strategy</label>
|
||||
<select v-model="selectedStrategy" class="w-full">
|
||||
<option v-for="s in strategies" :key="s.name" :value="s.name">
|
||||
{{ s.display_name }}
|
||||
</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<!-- Market Type & Symbol -->
|
||||
<div class="grid grid-cols-2 gap-3">
|
||||
<div>
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">Market</label>
|
||||
<select v-model="selectedMarket" class="w-full">
|
||||
<option value="spot">Spot</option>
|
||||
<option value="perpetual">Perpetual</option>
|
||||
</select>
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">Symbol</label>
|
||||
<select v-model="selectedSymbol" class="w-full">
|
||||
<option v-for="s in filteredSymbols" :key="s.symbol" :value="s.symbol">
|
||||
{{ s.symbol }}
|
||||
</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Timeframe & Cash -->
|
||||
<div class="grid grid-cols-2 gap-3">
|
||||
<div>
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">Timeframe</label>
|
||||
<select v-model="timeframe" class="w-full">
|
||||
<option value="1h">1 Hour</option>
|
||||
<option value="4h">4 Hours</option>
|
||||
<option value="1d">1 Day</option>
|
||||
</select>
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">Initial Cash</label>
|
||||
<input type="number" v-model.number="initCash" class="w-full" min="100" step="100" />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Leverage (perpetual only) -->
|
||||
<div v-if="selectedMarket === 'perpetual'" class="grid grid-cols-3 gap-3">
|
||||
<div>
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">Leverage</label>
|
||||
<input type="number" v-model.number="leverage" class="w-full" min="1" max="100" placeholder="1" />
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">Stop Loss %</label>
|
||||
<input type="number" v-model.number="slStop" class="w-full" min="0" max="100" step="0.1" placeholder="None" />
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">Take Profit %</label>
|
||||
<input type="number" v-model.number="tpStop" class="w-full" min="0" max="100" step="0.1" placeholder="None" />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Strategy Parameters -->
|
||||
<div v-if="currentStrategy && Object.keys(params).length > 0">
|
||||
<h3 class="text-sm font-medium text-text-secondary mb-2">Strategy Parameters</h3>
|
||||
<div class="grid grid-cols-2 gap-3">
|
||||
<div v-for="(value, key) in params" :key="key">
|
||||
<label class="block text-xs text-text-secondary uppercase mb-1">
|
||||
{{ String(key).replace(/_/g, ' ') }}
|
||||
</label>
|
||||
<template v-if="getParamType(value) === 'boolean'">
|
||||
<input
|
||||
type="checkbox"
|
||||
:checked="Boolean(value)"
|
||||
@change="params[key] = ($event.target as HTMLInputElement).checked"
|
||||
class="w-5 h-5"
|
||||
/>
|
||||
</template>
|
||||
<template v-else>
|
||||
<input
|
||||
type="number"
|
||||
:value="value"
|
||||
@input="params[key] = parseFloat(($event.target as HTMLInputElement).value)"
|
||||
class="w-full"
|
||||
step="any"
|
||||
/>
|
||||
</template>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Submit -->
|
||||
<button
|
||||
type="submit"
|
||||
class="btn btn-primary w-full"
|
||||
:disabled="loading || !selectedStrategy || !selectedSymbol"
|
||||
>
|
||||
<span v-if="loading" class="spinner"></span>
|
||||
<span v-else>Run Backtest</span>
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
</template>
|
||||
88
frontend/src/components/EquityCurve.vue
Normal file
88
frontend/src/components/EquityCurve.vue
Normal file
@@ -0,0 +1,88 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, watch, onMounted, onUnmounted } from 'vue'
|
||||
import Plotly from 'plotly.js-dist-min'
|
||||
import type { EquityPoint } from '@/api/types'
|
||||
|
||||
const props = defineProps<{
|
||||
data: EquityPoint[]
|
||||
title?: string
|
||||
}>()
|
||||
|
||||
const chartRef = ref<HTMLDivElement | null>(null)
|
||||
|
||||
const CHART_COLORS = {
|
||||
equity: '#58a6ff',
|
||||
grid: '#30363d',
|
||||
text: '#8b949e',
|
||||
}
|
||||
|
||||
function renderChart() {
|
||||
if (!chartRef.value || props.data.length === 0) return
|
||||
|
||||
const timestamps = props.data.map(p => p.timestamp)
|
||||
const values = props.data.map(p => p.value)
|
||||
|
||||
const traces: Plotly.Data[] = [
|
||||
{
|
||||
x: timestamps,
|
||||
y: values,
|
||||
type: 'scatter',
|
||||
mode: 'lines',
|
||||
name: 'Portfolio Value',
|
||||
line: { color: CHART_COLORS.equity, width: 2 },
|
||||
hovertemplate: '%{x}<br>Value: $%{y:,.2f}<extra></extra>',
|
||||
},
|
||||
]
|
||||
|
||||
const layout: Partial<Plotly.Layout> = {
|
||||
title: props.title ? {
|
||||
text: props.title,
|
||||
font: { color: CHART_COLORS.text, size: 14 },
|
||||
} : undefined,
|
||||
paper_bgcolor: 'transparent',
|
||||
plot_bgcolor: 'transparent',
|
||||
margin: { l: 60, r: 20, t: props.title ? 40 : 20, b: 40 },
|
||||
xaxis: {
|
||||
showgrid: true,
|
||||
gridcolor: CHART_COLORS.grid,
|
||||
tickfont: { color: CHART_COLORS.text, size: 10 },
|
||||
linecolor: CHART_COLORS.grid,
|
||||
},
|
||||
yaxis: {
|
||||
showgrid: true,
|
||||
gridcolor: CHART_COLORS.grid,
|
||||
tickfont: { color: CHART_COLORS.text, size: 10 },
|
||||
linecolor: CHART_COLORS.grid,
|
||||
tickprefix: '$',
|
||||
hoverformat: ',.2f',
|
||||
},
|
||||
showlegend: false,
|
||||
hovermode: 'x unified',
|
||||
}
|
||||
|
||||
const config: Partial<Plotly.Config> = {
|
||||
responsive: true,
|
||||
displayModeBar: false,
|
||||
}
|
||||
|
||||
Plotly.react(chartRef.value, traces, layout, config)
|
||||
}
|
||||
|
||||
watch(() => props.data, renderChart, { deep: true })
|
||||
|
||||
onMounted(() => {
|
||||
renderChart()
|
||||
window.addEventListener('resize', renderChart)
|
||||
})
|
||||
|
||||
onUnmounted(() => {
|
||||
window.removeEventListener('resize', renderChart)
|
||||
if (chartRef.value) {
|
||||
Plotly.purge(chartRef.value)
|
||||
}
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div ref="chartRef" class="w-full h-full min-h-[300px]"></div>
|
||||
</template>
|
||||
41
frontend/src/components/HelloWorld.vue
Normal file
41
frontend/src/components/HelloWorld.vue
Normal file
@@ -0,0 +1,41 @@
|
||||
<script setup lang="ts">
|
||||
import { ref } from 'vue'
|
||||
|
||||
defineProps<{ msg: string }>()
|
||||
|
||||
const count = ref(0)
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<h1>{{ msg }}</h1>
|
||||
|
||||
<div class="card">
|
||||
<button type="button" @click="count++">count is {{ count }}</button>
|
||||
<p>
|
||||
Edit
|
||||
<code>components/HelloWorld.vue</code> to test HMR
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<p>
|
||||
Check out
|
||||
<a href="https://vuejs.org/guide/quick-start.html#local" target="_blank"
|
||||
>create-vue</a
|
||||
>, the official Vue + Vite starter
|
||||
</p>
|
||||
<p>
|
||||
Learn more about IDE Support for Vue in the
|
||||
<a
|
||||
href="https://vuejs.org/guide/scaling-up/tooling.html#ide-support"
|
||||
target="_blank"
|
||||
>Vue Docs Scaling up Guide</a
|
||||
>.
|
||||
</p>
|
||||
<p class="read-the-docs">Click on the Vite and Vue logos to learn more</p>
|
||||
</template>
|
||||
|
||||
<style scoped>
|
||||
.read-the-docs {
|
||||
color: #888;
|
||||
}
|
||||
</style>
|
||||
144
frontend/src/components/MetricsPanel.vue
Normal file
144
frontend/src/components/MetricsPanel.vue
Normal file
@@ -0,0 +1,144 @@
|
||||
<script setup lang="ts">
|
||||
import type { BacktestMetrics } from '@/api/types'
|
||||
|
||||
const props = defineProps<{
|
||||
metrics: BacktestMetrics
|
||||
leverage?: number
|
||||
marketType?: string
|
||||
}>()
|
||||
|
||||
function formatPercent(val: number): string {
|
||||
return (val >= 0 ? '+' : '') + val.toFixed(2) + '%'
|
||||
}
|
||||
|
||||
function formatNumber(val: number | null | undefined, decimals = 2): string {
|
||||
if (val === null || val === undefined) return '-'
|
||||
return val.toFixed(decimals)
|
||||
}
|
||||
|
||||
function formatCurrency(val: number): string {
|
||||
return '$' + val.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 2 })
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="grid grid-cols-2 md:grid-cols-4 gap-4">
|
||||
<!-- Total Return -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Strategy Return</div>
|
||||
<div
|
||||
class="metric-value"
|
||||
:class="metrics.total_return >= 0 ? 'profit' : 'loss'"
|
||||
>
|
||||
{{ formatPercent(metrics.total_return) }}
|
||||
</div>
|
||||
<div v-if="metrics.adjusted_return !== null && metrics.adjusted_return !== metrics.total_return" class="text-xs text-text-muted mt-1">
|
||||
Adj: {{ formatPercent(metrics.adjusted_return) }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Benchmark Return -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Benchmark (B&H)</div>
|
||||
<div
|
||||
class="metric-value"
|
||||
:class="metrics.benchmark_return >= 0 ? 'profit' : 'loss'"
|
||||
>
|
||||
{{ formatPercent(metrics.benchmark_return) }}
|
||||
</div>
|
||||
<div class="text-xs text-text-muted mt-1">
|
||||
Market change
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Alpha -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Alpha</div>
|
||||
<div
|
||||
class="metric-value"
|
||||
:class="metrics.alpha >= 0 ? 'profit' : 'loss'"
|
||||
>
|
||||
{{ formatPercent(metrics.alpha) }}
|
||||
</div>
|
||||
<div class="text-xs text-text-muted mt-1">
|
||||
vs Buy & Hold
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Sharpe Ratio -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Sharpe Ratio</div>
|
||||
<div
|
||||
class="metric-value"
|
||||
:class="metrics.sharpe_ratio >= 1 ? 'profit' : metrics.sharpe_ratio < 0 ? 'loss' : ''"
|
||||
>
|
||||
{{ formatNumber(metrics.sharpe_ratio) }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Max Drawdown -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Max Drawdown</div>
|
||||
<div class="metric-value loss">
|
||||
{{ formatPercent(metrics.max_drawdown) }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Win Rate -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Win Rate</div>
|
||||
<div
|
||||
class="metric-value"
|
||||
:class="metrics.win_rate >= 50 ? 'profit' : 'loss'"
|
||||
>
|
||||
{{ formatNumber(metrics.win_rate, 1) }}%
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Total Trades -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Total Trades</div>
|
||||
<div class="metric-value">
|
||||
{{ metrics.total_trades }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Profit Factor -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Profit Factor</div>
|
||||
<div
|
||||
class="metric-value"
|
||||
:class="(metrics.profit_factor || 0) >= 1 ? 'profit' : 'loss'"
|
||||
>
|
||||
{{ formatNumber(metrics.profit_factor) }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Total Fees -->
|
||||
<div class="card">
|
||||
<div class="metric-label">Total Fees</div>
|
||||
<div class="metric-value text-warning">
|
||||
{{ formatCurrency(metrics.total_fees) }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Funding (perpetual only) -->
|
||||
<div v-if="marketType === 'perpetual'" class="card">
|
||||
<div class="metric-label">Funding Paid</div>
|
||||
<div class="metric-value text-warning">
|
||||
{{ formatCurrency(metrics.total_funding) }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Liquidations (if any) -->
|
||||
<div v-if="metrics.liquidation_count > 0" class="card">
|
||||
<div class="metric-label">Liquidations</div>
|
||||
<div class="metric-value loss">
|
||||
{{ metrics.liquidation_count }}
|
||||
</div>
|
||||
<div class="text-xs text-text-muted mt-1">
|
||||
Lost: {{ formatCurrency(metrics.liquidation_loss) }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
141
frontend/src/components/RunHistory.vue
Normal file
141
frontend/src/components/RunHistory.vue
Normal file
@@ -0,0 +1,141 @@
|
||||
<script setup lang="ts">
|
||||
import { onMounted } from 'vue'
|
||||
import { useBacktest } from '@/composables/useBacktest'
|
||||
import { useRouter } from 'vue-router'
|
||||
|
||||
const router = useRouter()
|
||||
const {
|
||||
runs,
|
||||
currentResult,
|
||||
selectedRuns,
|
||||
refreshRuns,
|
||||
loadRun,
|
||||
removeRun,
|
||||
toggleRunSelection
|
||||
} = useBacktest()
|
||||
|
||||
onMounted(() => {
|
||||
refreshRuns()
|
||||
})
|
||||
|
||||
function formatDate(iso: string): string {
|
||||
const d = new Date(iso)
|
||||
return d.toLocaleDateString('en-US', {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit'
|
||||
})
|
||||
}
|
||||
|
||||
function formatReturn(val: number): string {
|
||||
return (val >= 0 ? '+' : '') + val.toFixed(2) + '%'
|
||||
}
|
||||
|
||||
async function handleClick(runId: string) {
|
||||
await loadRun(runId)
|
||||
router.push('/')
|
||||
}
|
||||
|
||||
function handleCheckbox(e: Event, runId: string) {
|
||||
e.stopPropagation()
|
||||
toggleRunSelection(runId)
|
||||
}
|
||||
|
||||
function handleDelete(e: Event, runId: string) {
|
||||
e.stopPropagation()
|
||||
if (confirm('Delete this run?')) {
|
||||
removeRun(runId)
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="flex flex-col h-full">
|
||||
<!-- Header -->
|
||||
<div class="p-4 border-b border-border">
|
||||
<h2 class="text-sm font-semibold text-text-secondary uppercase tracking-wide">
|
||||
Run History
|
||||
</h2>
|
||||
<p class="text-xs text-text-muted mt-1">
|
||||
{{ runs.length }} runs | {{ selectedRuns.length }} selected
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<!-- Run List -->
|
||||
<div class="flex-1 overflow-y-auto">
|
||||
<div
|
||||
v-for="run in runs"
|
||||
:key="run.run_id"
|
||||
@click="handleClick(run.run_id)"
|
||||
class="p-3 border-b border-border-muted cursor-pointer hover:bg-bg-hover transition-colors"
|
||||
:class="{ 'bg-bg-tertiary': currentResult?.run_id === run.run_id }"
|
||||
>
|
||||
<div class="flex items-start gap-2">
|
||||
<!-- Checkbox for comparison -->
|
||||
<input
|
||||
type="checkbox"
|
||||
:checked="selectedRuns.includes(run.run_id)"
|
||||
@click="handleCheckbox($event, run.run_id)"
|
||||
class="mt-1 w-4 h-4 rounded border-border bg-bg-tertiary"
|
||||
/>
|
||||
|
||||
<div class="flex-1 min-w-0">
|
||||
<!-- Strategy & Symbol -->
|
||||
<div class="flex items-center gap-2">
|
||||
<span class="font-medium text-sm truncate">{{ run.strategy }}</span>
|
||||
<span class="text-xs px-1.5 py-0.5 rounded bg-bg-tertiary text-text-secondary">
|
||||
{{ run.symbol }}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<!-- Metrics -->
|
||||
<div class="flex items-center gap-3 mt-1">
|
||||
<span
|
||||
class="text-sm font-mono"
|
||||
:class="run.total_return >= 0 ? 'profit' : 'loss'"
|
||||
>
|
||||
{{ formatReturn(run.total_return) }}
|
||||
</span>
|
||||
<span class="text-xs text-text-muted">
|
||||
SR {{ run.sharpe_ratio.toFixed(2) }}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<!-- Date -->
|
||||
<div class="text-xs text-text-muted mt-1">
|
||||
{{ formatDate(run.created_at) }}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Delete button -->
|
||||
<button
|
||||
@click="handleDelete($event, run.run_id)"
|
||||
class="p-1 rounded hover:bg-loss/20 text-text-muted hover:text-loss transition-colors"
|
||||
title="Delete run"
|
||||
>
|
||||
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 7l-.867 12.142A2 2 0 0116.138 21H7.862a2 2 0 01-1.995-1.858L5 7m5 4v6m4-6v6m1-10V4a1 1 0 00-1-1h-4a1 1 0 00-1 1v3M4 7h16" />
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Empty state -->
|
||||
<div v-if="runs.length === 0" class="p-8 text-center text-text-muted">
|
||||
<p>No runs yet.</p>
|
||||
<p class="text-xs mt-1">Run a backtest to see results here.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Compare Button -->
|
||||
<div v-if="selectedRuns.length >= 2" class="p-4 border-t border-border">
|
||||
<router-link
|
||||
to="/compare"
|
||||
class="btn btn-primary w-full"
|
||||
>
|
||||
Compare {{ selectedRuns.length }} Runs
|
||||
</router-link>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
157
frontend/src/components/TradeLog.vue
Normal file
157
frontend/src/components/TradeLog.vue
Normal file
@@ -0,0 +1,157 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, computed } from 'vue'
|
||||
import type { TradeRecord } from '@/api/types'
|
||||
|
||||
const props = defineProps<{
|
||||
trades: TradeRecord[]
|
||||
}>()
|
||||
|
||||
type SortKey = 'entry_time' | 'pnl' | 'return_pct' | 'size'
|
||||
const sortKey = ref<SortKey>('entry_time')
|
||||
const sortDesc = ref(true)
|
||||
|
||||
const sortedTrades = computed(() => {
|
||||
return [...props.trades].sort((a, b) => {
|
||||
let aVal: number | string = 0
|
||||
let bVal: number | string = 0
|
||||
|
||||
switch (sortKey.value) {
|
||||
case 'entry_time':
|
||||
aVal = a.entry_time
|
||||
bVal = b.entry_time
|
||||
break
|
||||
case 'pnl':
|
||||
aVal = a.pnl ?? 0
|
||||
bVal = b.pnl ?? 0
|
||||
break
|
||||
case 'return_pct':
|
||||
aVal = a.return_pct ?? 0
|
||||
bVal = b.return_pct ?? 0
|
||||
break
|
||||
case 'size':
|
||||
aVal = a.size
|
||||
bVal = b.size
|
||||
break
|
||||
}
|
||||
|
||||
if (aVal < bVal) return sortDesc.value ? 1 : -1
|
||||
if (aVal > bVal) return sortDesc.value ? -1 : 1
|
||||
return 0
|
||||
})
|
||||
})
|
||||
|
||||
function toggleSort(key: SortKey) {
|
||||
if (sortKey.value === key) {
|
||||
sortDesc.value = !sortDesc.value
|
||||
} else {
|
||||
sortKey.value = key
|
||||
sortDesc.value = true
|
||||
}
|
||||
}
|
||||
|
||||
function formatDate(iso: string): string {
|
||||
if (!iso) return '-'
|
||||
const d = new Date(iso)
|
||||
return d.toLocaleDateString('en-US', {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
})
|
||||
}
|
||||
|
||||
function formatPrice(val: number | null): string {
|
||||
if (val === null) return '-'
|
||||
return val.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 2 })
|
||||
}
|
||||
|
||||
function formatPnL(val: number | null): string {
|
||||
if (val === null) return '-'
|
||||
const sign = val >= 0 ? '+' : ''
|
||||
return sign + val.toLocaleString('en-US', { minimumFractionDigits: 2, maximumFractionDigits: 2 })
|
||||
}
|
||||
|
||||
function formatReturn(val: number | null): string {
|
||||
if (val === null) return '-'
|
||||
return (val >= 0 ? '+' : '') + val.toFixed(2) + '%'
|
||||
}
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="card overflow-hidden">
|
||||
<div class="flex items-center justify-between mb-4">
|
||||
<h3 class="text-sm font-semibold text-text-secondary uppercase tracking-wide">
|
||||
Trade Log
|
||||
</h3>
|
||||
<span class="text-xs text-text-muted">{{ trades.length }} trades</span>
|
||||
</div>
|
||||
|
||||
<div class="overflow-x-auto max-h-[400px] overflow-y-auto">
|
||||
<table class="min-w-full">
|
||||
<thead class="sticky top-0 bg-bg-card">
|
||||
<tr>
|
||||
<th
|
||||
@click="toggleSort('entry_time')"
|
||||
class="cursor-pointer hover:text-text-primary"
|
||||
>
|
||||
Entry Time
|
||||
<span v-if="sortKey === 'entry_time'">{{ sortDesc ? ' v' : ' ^' }}</span>
|
||||
</th>
|
||||
<th>Exit Time</th>
|
||||
<th>Direction</th>
|
||||
<th>Entry</th>
|
||||
<th>Exit</th>
|
||||
<th
|
||||
@click="toggleSort('size')"
|
||||
class="cursor-pointer hover:text-text-primary"
|
||||
>
|
||||
Size
|
||||
<span v-if="sortKey === 'size'">{{ sortDesc ? ' v' : ' ^' }}</span>
|
||||
</th>
|
||||
<th
|
||||
@click="toggleSort('pnl')"
|
||||
class="cursor-pointer hover:text-text-primary"
|
||||
>
|
||||
PnL
|
||||
<span v-if="sortKey === 'pnl'">{{ sortDesc ? ' v' : ' ^' }}</span>
|
||||
</th>
|
||||
<th
|
||||
@click="toggleSort('return_pct')"
|
||||
class="cursor-pointer hover:text-text-primary"
|
||||
>
|
||||
Return
|
||||
<span v-if="sortKey === 'return_pct'">{{ sortDesc ? ' v' : ' ^' }}</span>
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr v-for="(trade, idx) in sortedTrades" :key="idx">
|
||||
<td class="text-text-secondary">{{ formatDate(trade.entry_time) }}</td>
|
||||
<td class="text-text-secondary">{{ formatDate(trade.exit_time || '') }}</td>
|
||||
<td>
|
||||
<span
|
||||
class="px-2 py-0.5 rounded text-xs font-medium"
|
||||
:class="trade.direction === 'Long' ? 'bg-profit/20 text-profit' : 'bg-loss/20 text-loss'"
|
||||
>
|
||||
{{ trade.direction }}
|
||||
</span>
|
||||
</td>
|
||||
<td>${{ formatPrice(trade.entry_price) }}</td>
|
||||
<td>${{ formatPrice(trade.exit_price) }}</td>
|
||||
<td>{{ trade.size.toFixed(4) }}</td>
|
||||
<td :class="(trade.pnl ?? 0) >= 0 ? 'profit' : 'loss'">
|
||||
${{ formatPnL(trade.pnl) }}
|
||||
</td>
|
||||
<td :class="(trade.return_pct ?? 0) >= 0 ? 'profit' : 'loss'">
|
||||
{{ formatReturn(trade.return_pct) }}
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<div v-if="trades.length === 0" class="p-8 text-center text-text-muted">
|
||||
No trades executed.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
150
frontend/src/composables/useBacktest.ts
Normal file
150
frontend/src/composables/useBacktest.ts
Normal file
@@ -0,0 +1,150 @@
|
||||
/**
|
||||
* Composable for managing backtest state across components.
|
||||
*/
|
||||
import { ref, computed } from 'vue'
|
||||
import type { BacktestResult, BacktestSummary, StrategyInfo, SymbolInfo } from '@/api/types'
|
||||
import { getStrategies, getSymbols, getBacktests, getBacktest, runBacktest, deleteBacktest } from '@/api/client'
|
||||
import type { BacktestRequest } from '@/api/types'
|
||||
|
||||
// Shared state
|
||||
const strategies = ref<StrategyInfo[]>([])
|
||||
const symbols = ref<SymbolInfo[]>([])
|
||||
const runs = ref<BacktestSummary[]>([])
|
||||
const currentResult = ref<BacktestResult | null>(null)
|
||||
const selectedRuns = ref<string[]>([])
|
||||
const loading = ref(false)
|
||||
const error = ref<string | null>(null)
|
||||
|
||||
// Computed
|
||||
const symbolsByMarket = computed(() => {
|
||||
const grouped: Record<string, SymbolInfo[]> = {}
|
||||
for (const s of symbols.value) {
|
||||
const key = `${s.market_type}`
|
||||
if (!grouped[key]) grouped[key] = []
|
||||
grouped[key].push(s)
|
||||
}
|
||||
return grouped
|
||||
})
|
||||
|
||||
export function useBacktest() {
|
||||
/**
|
||||
* Load strategies and symbols on app init.
|
||||
*/
|
||||
async function init() {
|
||||
try {
|
||||
const [stratRes, symRes] = await Promise.all([
|
||||
getStrategies(),
|
||||
getSymbols(),
|
||||
])
|
||||
strategies.value = stratRes.strategies
|
||||
symbols.value = symRes.symbols
|
||||
} catch (e) {
|
||||
error.value = `Failed to load initial data: ${e}`
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Refresh the run history list.
|
||||
*/
|
||||
async function refreshRuns() {
|
||||
try {
|
||||
const res = await getBacktests({ limit: 100 })
|
||||
runs.value = res.runs
|
||||
} catch (e) {
|
||||
error.value = `Failed to load runs: ${e}`
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a new backtest.
|
||||
*/
|
||||
async function executeBacktest(request: BacktestRequest) {
|
||||
loading.value = true
|
||||
error.value = null
|
||||
try {
|
||||
const result = await runBacktest(request)
|
||||
currentResult.value = result
|
||||
await refreshRuns()
|
||||
return result
|
||||
} catch (e: unknown) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
error.value = `Backtest failed: ${msg}`
|
||||
throw e
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Load a specific run by ID.
|
||||
*/
|
||||
async function loadRun(runId: string) {
|
||||
loading.value = true
|
||||
error.value = null
|
||||
try {
|
||||
const result = await getBacktest(runId)
|
||||
currentResult.value = result
|
||||
return result
|
||||
} catch (e) {
|
||||
error.value = `Failed to load run: ${e}`
|
||||
throw e
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a run.
|
||||
*/
|
||||
async function removeRun(runId: string) {
|
||||
try {
|
||||
await deleteBacktest(runId)
|
||||
await refreshRuns()
|
||||
if (currentResult.value?.run_id === runId) {
|
||||
currentResult.value = null
|
||||
}
|
||||
selectedRuns.value = selectedRuns.value.filter(id => id !== runId)
|
||||
} catch (e) {
|
||||
error.value = `Failed to delete run: ${e}`
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Toggle run selection for comparison.
|
||||
*/
|
||||
function toggleRunSelection(runId: string) {
|
||||
const idx = selectedRuns.value.indexOf(runId)
|
||||
if (idx >= 0) {
|
||||
selectedRuns.value.splice(idx, 1)
|
||||
} else if (selectedRuns.value.length < 5) {
|
||||
selectedRuns.value.push(runId)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear all selections.
|
||||
*/
|
||||
function clearSelections() {
|
||||
selectedRuns.value = []
|
||||
}
|
||||
|
||||
return {
|
||||
// State
|
||||
strategies,
|
||||
symbols,
|
||||
symbolsByMarket,
|
||||
runs,
|
||||
currentResult,
|
||||
selectedRuns,
|
||||
loading,
|
||||
error,
|
||||
// Actions
|
||||
init,
|
||||
refreshRuns,
|
||||
executeBacktest,
|
||||
loadRun,
|
||||
removeRun,
|
||||
toggleRunSelection,
|
||||
clearSelections,
|
||||
}
|
||||
}
|
||||
8
frontend/src/main.ts
Normal file
8
frontend/src/main.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
import { createApp } from 'vue'
|
||||
import App from './App.vue'
|
||||
import router from './router'
|
||||
import './style.css'
|
||||
|
||||
const app = createApp(App)
|
||||
app.use(router)
|
||||
app.mount('#app')
|
||||
21
frontend/src/router/index.ts
Normal file
21
frontend/src/router/index.ts
Normal file
@@ -0,0 +1,21 @@
|
||||
import { createRouter, createWebHistory } from 'vue-router'
|
||||
import DashboardView from '@/views/DashboardView.vue'
|
||||
import CompareView from '@/views/CompareView.vue'
|
||||
|
||||
const router = createRouter({
|
||||
history: createWebHistory(import.meta.env.BASE_URL),
|
||||
routes: [
|
||||
{
|
||||
path: '/',
|
||||
name: 'dashboard',
|
||||
component: DashboardView,
|
||||
},
|
||||
{
|
||||
path: '/compare',
|
||||
name: 'compare',
|
||||
component: CompareView,
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
export default router
|
||||
198
frontend/src/style.css
Normal file
198
frontend/src/style.css
Normal file
@@ -0,0 +1,198 @@
|
||||
@import "tailwindcss";
|
||||
|
||||
/* QuantConnect-inspired dark theme */
|
||||
@theme {
|
||||
/* Background colors */
|
||||
--color-bg-primary: #0d1117;
|
||||
--color-bg-secondary: #161b22;
|
||||
--color-bg-tertiary: #21262d;
|
||||
--color-bg-card: #1c2128;
|
||||
--color-bg-hover: #30363d;
|
||||
|
||||
/* Text colors */
|
||||
--color-text-primary: #e6edf3;
|
||||
--color-text-secondary: #8b949e;
|
||||
--color-text-muted: #6e7681;
|
||||
|
||||
/* Accent colors */
|
||||
--color-accent-blue: #58a6ff;
|
||||
--color-accent-purple: #a371f7;
|
||||
--color-accent-cyan: #39d4e8;
|
||||
|
||||
/* Status colors */
|
||||
--color-profit: #3fb950;
|
||||
--color-loss: #f85149;
|
||||
--color-warning: #d29922;
|
||||
|
||||
/* Border colors */
|
||||
--color-border: #30363d;
|
||||
--color-border-muted: #21262d;
|
||||
|
||||
/* Chart colors for comparison */
|
||||
--color-chart-1: #58a6ff;
|
||||
--color-chart-2: #a371f7;
|
||||
--color-chart-3: #39d4e8;
|
||||
--color-chart-4: #f0883e;
|
||||
--color-chart-5: #db61a2;
|
||||
}
|
||||
|
||||
/* Base styles */
|
||||
body {
|
||||
background-color: var(--color-bg-primary);
|
||||
color: var(--color-text-primary);
|
||||
font-family: 'JetBrains Mono', 'Fira Code', 'SF Mono', Consolas, monospace;
|
||||
}
|
||||
|
||||
/* Scrollbar styling */
|
||||
::-webkit-scrollbar {
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-track {
|
||||
background: var(--color-bg-secondary);
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-thumb {
|
||||
background: var(--color-bg-hover);
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
::-webkit-scrollbar-thumb:hover {
|
||||
background: var(--color-border);
|
||||
}
|
||||
|
||||
/* Input styling */
|
||||
input[type="number"],
|
||||
input[type="text"],
|
||||
select {
|
||||
background-color: var(--color-bg-tertiary);
|
||||
border: 1px solid var(--color-border);
|
||||
color: var(--color-text-primary);
|
||||
border-radius: 6px;
|
||||
padding: 0.5rem 0.75rem;
|
||||
font-size: 0.875rem;
|
||||
transition: border-color 0.2s, box-shadow 0.2s;
|
||||
}
|
||||
|
||||
input[type="number"]:focus,
|
||||
input[type="text"]:focus,
|
||||
select:focus {
|
||||
outline: none;
|
||||
border-color: var(--color-accent-blue);
|
||||
box-shadow: 0 0 0 3px rgba(88, 166, 255, 0.2);
|
||||
}
|
||||
|
||||
/* Button base */
|
||||
.btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 0.5rem;
|
||||
padding: 0.5rem 1rem;
|
||||
font-size: 0.875rem;
|
||||
font-weight: 500;
|
||||
border-radius: 6px;
|
||||
transition: all 0.2s;
|
||||
cursor: pointer;
|
||||
border: 1px solid transparent;
|
||||
}
|
||||
|
||||
.btn-primary {
|
||||
background-color: var(--color-accent-blue);
|
||||
color: #000;
|
||||
}
|
||||
|
||||
.btn-primary:hover {
|
||||
background-color: #79b8ff;
|
||||
}
|
||||
|
||||
.btn-primary:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.btn-secondary {
|
||||
background-color: var(--color-bg-tertiary);
|
||||
border-color: var(--color-border);
|
||||
color: var(--color-text-primary);
|
||||
}
|
||||
|
||||
.btn-secondary:hover {
|
||||
background-color: var(--color-bg-hover);
|
||||
}
|
||||
|
||||
/* Card styling */
|
||||
.card {
|
||||
background-color: var(--color-bg-card);
|
||||
border: 1px solid var(--color-border-muted);
|
||||
border-radius: 8px;
|
||||
padding: 1rem;
|
||||
}
|
||||
|
||||
/* Profit/Loss coloring */
|
||||
.profit {
|
||||
color: var(--color-profit);
|
||||
}
|
||||
|
||||
.loss {
|
||||
color: var(--color-loss);
|
||||
}
|
||||
|
||||
/* Metric value styling */
|
||||
.metric-value {
|
||||
font-size: 1.5rem;
|
||||
font-weight: 600;
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.metric-label {
|
||||
font-size: 0.75rem;
|
||||
color: var(--color-text-secondary);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.05em;
|
||||
}
|
||||
|
||||
/* Table styling */
|
||||
table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
}
|
||||
|
||||
th {
|
||||
text-align: left;
|
||||
padding: 0.75rem;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 500;
|
||||
color: var(--color-text-secondary);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.05em;
|
||||
border-bottom: 1px solid var(--color-border);
|
||||
}
|
||||
|
||||
td {
|
||||
padding: 0.75rem;
|
||||
font-size: 0.875rem;
|
||||
border-bottom: 1px solid var(--color-border-muted);
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
tr:hover td {
|
||||
background-color: var(--color-bg-hover);
|
||||
}
|
||||
|
||||
/* Loading spinner */
|
||||
.spinner {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border: 2px solid var(--color-border);
|
||||
border-top-color: var(--color-accent-blue);
|
||||
border-radius: 50%;
|
||||
animation: spin 0.8s linear infinite;
|
||||
}
|
||||
|
||||
@keyframes spin {
|
||||
to {
|
||||
transform: rotate(360deg);
|
||||
}
|
||||
}
|
||||
4
frontend/src/types/plotly.d.ts
vendored
Normal file
4
frontend/src/types/plotly.d.ts
vendored
Normal file
@@ -0,0 +1,4 @@
|
||||
declare module 'plotly.js-dist-min' {
|
||||
import Plotly from 'plotly.js'
|
||||
export default Plotly
|
||||
}
|
||||
362
frontend/src/views/CompareView.vue
Normal file
362
frontend/src/views/CompareView.vue
Normal file
@@ -0,0 +1,362 @@
|
||||
<script setup lang="ts">
|
||||
import { ref, watch, onMounted, onUnmounted } from 'vue'
|
||||
import { useRouter } from 'vue-router'
|
||||
import Plotly from 'plotly.js-dist-min'
|
||||
import { useBacktest } from '@/composables/useBacktest'
|
||||
import { compareRuns } from '@/api/client'
|
||||
import type { BacktestResult, CompareResult } from '@/api/types'
|
||||
|
||||
const router = useRouter()
|
||||
const { selectedRuns, clearSelections } = useBacktest()
|
||||
|
||||
const chartRef = ref<HTMLDivElement | null>(null)
|
||||
const loading = ref(false)
|
||||
const error = ref<string | null>(null)
|
||||
const compareResult = ref<CompareResult | null>(null)
|
||||
|
||||
const CHART_COLORS = [
|
||||
'#58a6ff', // blue
|
||||
'#a371f7', // purple
|
||||
'#39d4e8', // cyan
|
||||
'#f0883e', // orange
|
||||
'#db61a2', // pink
|
||||
]
|
||||
|
||||
async function loadComparison() {
|
||||
if (selectedRuns.value.length < 2) {
|
||||
router.push('/')
|
||||
return
|
||||
}
|
||||
|
||||
loading.value = true
|
||||
error.value = null
|
||||
|
||||
try {
|
||||
compareResult.value = await compareRuns(selectedRuns.value)
|
||||
renderChart()
|
||||
} catch (e) {
|
||||
error.value = `Failed to load comparison: ${e}`
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function renderChart() {
|
||||
if (!chartRef.value || !compareResult.value) return
|
||||
|
||||
const traces: Plotly.Data[] = compareResult.value.runs.map((run, idx) => {
|
||||
// Normalize equity curves to start at 100 for comparison
|
||||
const startValue = run.equity_curve[0]?.value || 1
|
||||
const normalizedValues = run.equity_curve.map(p => (p.value / startValue) * 100)
|
||||
|
||||
return {
|
||||
x: run.equity_curve.map(p => p.timestamp),
|
||||
y: normalizedValues,
|
||||
type: 'scatter',
|
||||
mode: 'lines',
|
||||
name: `${run.strategy} (${run.params.period || ''})`,
|
||||
line: { color: CHART_COLORS[idx % CHART_COLORS.length], width: 2 },
|
||||
hovertemplate: `%{x}<br>${run.strategy}: %{y:.2f}<extra></extra>`,
|
||||
}
|
||||
})
|
||||
|
||||
const layout: Partial<Plotly.Layout> = {
|
||||
title: {
|
||||
text: 'Normalized Equity Comparison (Base 100)',
|
||||
font: { color: '#8b949e', size: 14 },
|
||||
},
|
||||
paper_bgcolor: 'transparent',
|
||||
plot_bgcolor: 'transparent',
|
||||
margin: { l: 60, r: 20, t: 50, b: 40 },
|
||||
xaxis: {
|
||||
showgrid: true,
|
||||
gridcolor: '#30363d',
|
||||
tickfont: { color: '#8b949e', size: 10 },
|
||||
linecolor: '#30363d',
|
||||
},
|
||||
yaxis: {
|
||||
showgrid: true,
|
||||
gridcolor: '#30363d',
|
||||
tickfont: { color: '#8b949e', size: 10 },
|
||||
linecolor: '#30363d',
|
||||
title: { text: 'Normalized Value', font: { color: '#8b949e' } },
|
||||
},
|
||||
legend: {
|
||||
orientation: 'h',
|
||||
yanchor: 'bottom',
|
||||
y: 1.02,
|
||||
xanchor: 'left',
|
||||
x: 0,
|
||||
font: { color: '#8b949e' },
|
||||
},
|
||||
hovermode: 'x unified',
|
||||
}
|
||||
|
||||
Plotly.react(chartRef.value, traces, layout, { responsive: true, displayModeBar: false })
|
||||
}
|
||||
|
||||
function formatPercent(val: number): string {
|
||||
return (val >= 0 ? '+' : '') + val.toFixed(2) + '%'
|
||||
}
|
||||
|
||||
function formatNumber(val: number | null): string {
|
||||
if (val === null) return '-'
|
||||
return val.toFixed(2)
|
||||
}
|
||||
|
||||
function getBestIndex(runs: BacktestResult[], metric: keyof BacktestResult['metrics'], higher = true): number {
|
||||
let bestIdx = 0
|
||||
let bestVal = runs[0]?.metrics[metric] ?? 0
|
||||
|
||||
for (let i = 1; i < runs.length; i++) {
|
||||
const val = runs[i]?.metrics[metric] ?? 0
|
||||
const isBetter = higher ? (val as number) > (bestVal as number) : (val as number) < (bestVal as number)
|
||||
if (isBetter) {
|
||||
bestIdx = i
|
||||
bestVal = val
|
||||
}
|
||||
}
|
||||
|
||||
return bestIdx
|
||||
}
|
||||
|
||||
function handleClearAndBack() {
|
||||
clearSelections()
|
||||
router.push('/')
|
||||
}
|
||||
|
||||
onMounted(() => {
|
||||
loadComparison()
|
||||
window.addEventListener('resize', renderChart)
|
||||
})
|
||||
|
||||
onUnmounted(() => {
|
||||
window.removeEventListener('resize', renderChart)
|
||||
if (chartRef.value) {
|
||||
Plotly.purge(chartRef.value)
|
||||
}
|
||||
})
|
||||
|
||||
watch(selectedRuns, () => {
|
||||
if (selectedRuns.value.length >= 2) {
|
||||
loadComparison()
|
||||
}
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="p-6 space-y-6">
|
||||
<!-- Header -->
|
||||
<div class="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 class="text-2xl font-bold">Compare Runs</h1>
|
||||
<p class="text-text-secondary text-sm mt-1">
|
||||
Comparing {{ selectedRuns.length }} backtest runs
|
||||
</p>
|
||||
</div>
|
||||
<button @click="handleClearAndBack" class="btn btn-secondary">
|
||||
Clear & Back
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<!-- Error -->
|
||||
<div v-if="error" class="p-4 rounded-lg bg-loss/10 border border-loss/30 text-loss">
|
||||
{{ error }}
|
||||
</div>
|
||||
|
||||
<!-- Loading -->
|
||||
<div v-if="loading" class="card flex items-center justify-center h-[400px]">
|
||||
<div class="spinner" style="width: 40px; height: 40px;"></div>
|
||||
</div>
|
||||
|
||||
<!-- Comparison Results -->
|
||||
<template v-else-if="compareResult">
|
||||
<!-- Equity Curve Comparison -->
|
||||
<div class="card">
|
||||
<div ref="chartRef" class="h-[400px]"></div>
|
||||
</div>
|
||||
|
||||
<!-- Metrics Comparison Table -->
|
||||
<div class="card overflow-x-auto">
|
||||
<h3 class="text-sm font-semibold text-text-secondary uppercase tracking-wide mb-4">
|
||||
Metrics Comparison
|
||||
</h3>
|
||||
<table class="min-w-full">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Metric</th>
|
||||
<th
|
||||
v-for="(run, idx) in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
>
|
||||
<span
|
||||
class="inline-block w-3 h-3 rounded-full mr-2"
|
||||
:style="{ backgroundColor: CHART_COLORS[idx] }"
|
||||
></span>
|
||||
{{ run.strategy }}
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<!-- Total Return -->
|
||||
<tr>
|
||||
<td class="font-medium">Strategy Return</td>
|
||||
<td
|
||||
v-for="(run, idx) in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
:class="[
|
||||
run.metrics.total_return >= 0 ? 'profit' : 'loss',
|
||||
idx === getBestIndex(compareResult.runs, 'total_return') ? 'font-bold' : ''
|
||||
]"
|
||||
>
|
||||
{{ formatPercent(run.metrics.total_return) }}
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
<!-- Benchmark Return -->
|
||||
<tr>
|
||||
<td class="font-medium">Benchmark (B&H)</td>
|
||||
<td
|
||||
v-for="run in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
:class="run.metrics.benchmark_return >= 0 ? 'profit' : 'loss'"
|
||||
>
|
||||
{{ formatPercent(run.metrics.benchmark_return) }}
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
<!-- Alpha -->
|
||||
<tr>
|
||||
<td class="font-medium">Alpha</td>
|
||||
<td
|
||||
v-for="(run, idx) in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
:class="[
|
||||
run.metrics.alpha >= 0 ? 'profit' : 'loss',
|
||||
idx === getBestIndex(compareResult.runs, 'alpha') ? 'font-bold' : ''
|
||||
]"
|
||||
>
|
||||
{{ formatPercent(run.metrics.alpha) }}
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
<!-- Sharpe Ratio -->
|
||||
<tr>
|
||||
<td class="font-medium">Sharpe Ratio</td>
|
||||
<td
|
||||
v-for="(run, idx) in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
:class="idx === getBestIndex(compareResult.runs, 'sharpe_ratio') ? 'font-bold text-accent-blue' : ''"
|
||||
>
|
||||
{{ formatNumber(run.metrics.sharpe_ratio) }}
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
<!-- Max Drawdown -->
|
||||
<tr>
|
||||
<td class="font-medium">Max Drawdown</td>
|
||||
<td
|
||||
v-for="(run, idx) in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center loss"
|
||||
:class="idx === getBestIndex(compareResult.runs, 'max_drawdown', false) ? 'font-bold' : ''"
|
||||
>
|
||||
{{ formatPercent(run.metrics.max_drawdown) }}
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
<!-- Win Rate -->
|
||||
<tr>
|
||||
<td class="font-medium">Win Rate</td>
|
||||
<td
|
||||
v-for="(run, idx) in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
:class="idx === getBestIndex(compareResult.runs, 'win_rate') ? 'font-bold text-profit' : ''"
|
||||
>
|
||||
{{ formatNumber(run.metrics.win_rate) }}%
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
<!-- Total Trades -->
|
||||
<tr>
|
||||
<td class="font-medium">Total Trades</td>
|
||||
<td
|
||||
v-for="run in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
>
|
||||
{{ run.metrics.total_trades }}
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
<!-- Profit Factor -->
|
||||
<tr>
|
||||
<td class="font-medium">Profit Factor</td>
|
||||
<td
|
||||
v-for="(run, idx) in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
:class="idx === getBestIndex(compareResult.runs, 'profit_factor') ? 'font-bold text-profit' : ''"
|
||||
>
|
||||
{{ formatNumber(run.metrics.profit_factor) }}
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<!-- Parameter Differences -->
|
||||
<div v-if="Object.keys(compareResult.param_diff).length > 0" class="card">
|
||||
<h3 class="text-sm font-semibold text-text-secondary uppercase tracking-wide mb-4">
|
||||
Parameter Differences
|
||||
</h3>
|
||||
<table class="min-w-full">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th
|
||||
v-for="(run, idx) in compareResult.runs"
|
||||
:key="run.run_id"
|
||||
class="text-center"
|
||||
>
|
||||
<span
|
||||
class="inline-block w-3 h-3 rounded-full mr-2"
|
||||
:style="{ backgroundColor: CHART_COLORS[idx] }"
|
||||
></span>
|
||||
Run {{ idx + 1 }}
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr v-for="(values, key) in compareResult.param_diff" :key="key">
|
||||
<td class="font-medium font-mono">{{ key }}</td>
|
||||
<td
|
||||
v-for="(val, idx) in values"
|
||||
:key="idx"
|
||||
class="text-center font-mono"
|
||||
>
|
||||
{{ val ?? '-' }}
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<!-- No Selection -->
|
||||
<div v-else class="card flex items-center justify-center h-[400px]">
|
||||
<div class="text-center text-text-muted">
|
||||
<p>Select at least 2 runs from the history to compare.</p>
|
||||
<button @click="router.push('/')" class="btn btn-secondary mt-4">
|
||||
Go to Dashboard
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
136
frontend/src/views/DashboardView.vue
Normal file
136
frontend/src/views/DashboardView.vue
Normal file
@@ -0,0 +1,136 @@
|
||||
<script setup lang="ts">
|
||||
import { useBacktest } from '@/composables/useBacktest'
|
||||
import BacktestConfig from '@/components/BacktestConfig.vue'
|
||||
import EquityCurve from '@/components/EquityCurve.vue'
|
||||
import MetricsPanel from '@/components/MetricsPanel.vue'
|
||||
import TradeLog from '@/components/TradeLog.vue'
|
||||
|
||||
const { currentResult, loading, error } = useBacktest()
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<div class="p-6 space-y-6">
|
||||
<!-- Header -->
|
||||
<div class="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 class="text-2xl font-bold">Lowkey Backtest</h1>
|
||||
<p class="text-text-secondary text-sm mt-1">
|
||||
Run and analyze trading strategy backtests
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Error Banner -->
|
||||
<div
|
||||
v-if="error"
|
||||
class="p-4 rounded-lg bg-loss/10 border border-loss/30 text-loss"
|
||||
>
|
||||
{{ error }}
|
||||
</div>
|
||||
|
||||
<!-- Main Grid -->
|
||||
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6">
|
||||
<!-- Config Panel (Left) -->
|
||||
<div class="lg:col-span-1">
|
||||
<BacktestConfig />
|
||||
</div>
|
||||
|
||||
<!-- Results (Right) -->
|
||||
<div class="lg:col-span-2 space-y-6">
|
||||
<!-- Loading State -->
|
||||
<div
|
||||
v-if="loading"
|
||||
class="card flex items-center justify-center h-[400px]"
|
||||
>
|
||||
<div class="text-center">
|
||||
<div class="spinner mx-auto mb-4" style="width: 40px; height: 40px;"></div>
|
||||
<p class="text-text-secondary">Running backtest...</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Results Display -->
|
||||
<template v-else-if="currentResult">
|
||||
<!-- Result Header -->
|
||||
<div class="card">
|
||||
<div class="flex items-center justify-between">
|
||||
<div>
|
||||
<h2 class="text-lg font-semibold">
|
||||
{{ currentResult.strategy }} on {{ currentResult.symbol }}
|
||||
</h2>
|
||||
<p class="text-sm text-text-secondary mt-1">
|
||||
{{ currentResult.start_date }} - {{ currentResult.end_date }}
|
||||
<span class="mx-2">|</span>
|
||||
{{ currentResult.market_type.toUpperCase() }}
|
||||
<span v-if="currentResult.leverage > 1" class="ml-2">
|
||||
{{ currentResult.leverage }}x
|
||||
</span>
|
||||
</p>
|
||||
</div>
|
||||
<div class="text-right">
|
||||
<div
|
||||
class="text-2xl font-bold"
|
||||
:class="currentResult.metrics.total_return >= 0 ? 'profit' : 'loss'"
|
||||
>
|
||||
{{ currentResult.metrics.total_return >= 0 ? '+' : '' }}{{ currentResult.metrics.total_return.toFixed(2) }}%
|
||||
</div>
|
||||
<div class="text-sm text-text-secondary">
|
||||
Sharpe: {{ currentResult.metrics.sharpe_ratio.toFixed(2) }}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Equity Curve -->
|
||||
<div class="card">
|
||||
<h3 class="text-sm font-semibold text-text-secondary uppercase tracking-wide mb-4">
|
||||
Equity Curve
|
||||
</h3>
|
||||
<div class="h-[350px]">
|
||||
<EquityCurve :data="currentResult.equity_curve" />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Metrics -->
|
||||
<MetricsPanel
|
||||
:metrics="currentResult.metrics"
|
||||
:leverage="currentResult.leverage"
|
||||
:market-type="currentResult.market_type"
|
||||
/>
|
||||
|
||||
<!-- Trade Log -->
|
||||
<TradeLog :trades="currentResult.trades" />
|
||||
|
||||
<!-- Parameters Used -->
|
||||
<div class="card">
|
||||
<h3 class="text-sm font-semibold text-text-secondary uppercase tracking-wide mb-3">
|
||||
Parameters
|
||||
</h3>
|
||||
<div class="flex flex-wrap gap-2">
|
||||
<span
|
||||
v-for="(value, key) in currentResult.params"
|
||||
:key="key"
|
||||
class="px-2 py-1 rounded bg-bg-tertiary text-sm font-mono"
|
||||
>
|
||||
{{ key }}: {{ value }}
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
|
||||
<!-- Empty State -->
|
||||
<div
|
||||
v-else
|
||||
class="card flex items-center justify-center h-[400px]"
|
||||
>
|
||||
<div class="text-center text-text-muted">
|
||||
<svg class="w-16 h-16 mx-auto mb-4 opacity-50" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="1" d="M9 19v-6a2 2 0 00-2-2H5a2 2 0 00-2 2v6a2 2 0 002 2h2a2 2 0 002-2zm0 0V9a2 2 0 012-2h2a2 2 0 012 2v10m-6 0a2 2 0 002 2h2a2 2 0 002-2m0 0V5a2 2 0 012-2h2a2 2 0 012 2v14a2 2 0 01-2 2h-2a2 2 0 01-2-2z" />
|
||||
</svg>
|
||||
<p>Configure and run a backtest to see results.</p>
|
||||
<p class="text-xs mt-2">Or select a run from history.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
20
frontend/tsconfig.app.json
Normal file
20
frontend/tsconfig.app.json
Normal file
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"extends": "@vue/tsconfig/tsconfig.dom.json",
|
||||
"compilerOptions": {
|
||||
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
|
||||
"types": ["vite/client"],
|
||||
"baseUrl": ".",
|
||||
"paths": {
|
||||
"@/*": ["./src/*"]
|
||||
},
|
||||
|
||||
/* Linting */
|
||||
"strict": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"erasableSyntaxOnly": true,
|
||||
"noFallthroughCasesInSwitch": true,
|
||||
"noUncheckedSideEffectImports": true
|
||||
},
|
||||
"include": ["src/**/*.ts", "src/**/*.tsx", "src/**/*.vue"]
|
||||
}
|
||||
7
frontend/tsconfig.json
Normal file
7
frontend/tsconfig.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"files": [],
|
||||
"references": [
|
||||
{ "path": "./tsconfig.app.json" },
|
||||
{ "path": "./tsconfig.node.json" }
|
||||
]
|
||||
}
|
||||
26
frontend/tsconfig.node.json
Normal file
26
frontend/tsconfig.node.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
|
||||
"target": "ES2023",
|
||||
"lib": ["ES2023"],
|
||||
"module": "ESNext",
|
||||
"types": ["node"],
|
||||
"skipLibCheck": true,
|
||||
|
||||
/* Bundler mode */
|
||||
"moduleResolution": "bundler",
|
||||
"allowImportingTsExtensions": true,
|
||||
"verbatimModuleSyntax": true,
|
||||
"moduleDetection": "force",
|
||||
"noEmit": true,
|
||||
|
||||
/* Linting */
|
||||
"strict": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"erasableSyntaxOnly": true,
|
||||
"noFallthroughCasesInSwitch": true,
|
||||
"noUncheckedSideEffectImports": true
|
||||
},
|
||||
"include": ["vite.config.ts"]
|
||||
}
|
||||
22
frontend/vite.config.ts
Normal file
22
frontend/vite.config.ts
Normal file
@@ -0,0 +1,22 @@
|
||||
import { defineConfig } from 'vite'
|
||||
import vue from '@vitejs/plugin-vue'
|
||||
import tailwindcss from '@tailwindcss/vite'
|
||||
import { fileURLToPath, URL } from 'node:url'
|
||||
|
||||
// https://vite.dev/config/
|
||||
export default defineConfig({
|
||||
plugins: [vue(), tailwindcss()],
|
||||
resolve: {
|
||||
alias: {
|
||||
'@': fileURLToPath(new URL('./src', import.meta.url))
|
||||
}
|
||||
},
|
||||
server: {
|
||||
proxy: {
|
||||
'/api': {
|
||||
target: 'http://127.0.0.1:8000',
|
||||
changeOrigin: true
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
@@ -1,3 +0,0 @@
|
||||
from .supertrend import add_supertrends, compute_meta_trend
|
||||
|
||||
__all__ = ["add_supertrends", "compute_meta_trend"]
|
||||
@@ -1,58 +0,0 @@
|
||||
from __future__ import annotations
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
|
||||
|
||||
def _atr(high: pd.Series, low: pd.Series, close: pd.Series, period: int) -> pd.Series:
|
||||
hl = (high - low).abs()
|
||||
hc = (high - close.shift()).abs()
|
||||
lc = (low - close.shift()).abs()
|
||||
tr = pd.concat([hl, hc, lc], axis=1).max(axis=1)
|
||||
return tr.rolling(period, min_periods=period).mean()
|
||||
|
||||
|
||||
def supertrend_series(df: pd.DataFrame, length: int, multiplier: float) -> pd.Series:
|
||||
atr = _atr(df["High"], df["Low"], df["Close"], length)
|
||||
hl2 = (df["High"] + df["Low"]) / 2
|
||||
upper = hl2 + multiplier * atr
|
||||
lower = hl2 - multiplier * atr
|
||||
|
||||
trend = pd.Series(index=df.index, dtype=float)
|
||||
dir_up = True
|
||||
prev_upper = np.nan
|
||||
prev_lower = np.nan
|
||||
|
||||
for i in range(len(df)):
|
||||
if i == 0 or pd.isna(atr.iat[i]):
|
||||
trend.iat[i] = np.nan
|
||||
prev_upper = upper.iat[i]
|
||||
prev_lower = lower.iat[i]
|
||||
continue
|
||||
|
||||
cu = min(upper.iat[i], prev_upper) if dir_up else upper.iat[i]
|
||||
cl = max(lower.iat[i], prev_lower) if not dir_up else lower.iat[i]
|
||||
|
||||
if df["Close"].iat[i] > cu:
|
||||
dir_up = True
|
||||
elif df["Close"].iat[i] < cl:
|
||||
dir_up = False
|
||||
|
||||
prev_upper = cu if dir_up else upper.iat[i]
|
||||
prev_lower = lower.iat[i] if dir_up else cl
|
||||
trend.iat[i] = cl if dir_up else cu
|
||||
|
||||
return trend
|
||||
|
||||
|
||||
def add_supertrends(df: pd.DataFrame, settings: list[tuple[int, float]]) -> pd.DataFrame:
|
||||
out = df.copy()
|
||||
for length, mult in settings:
|
||||
col = f"supertrend_{length}_{mult}"
|
||||
out[col] = supertrend_series(out, length, mult)
|
||||
out[f"bull_{length}_{mult}"] = (out["Close"] >= out[col]).astype(int)
|
||||
return out
|
||||
|
||||
|
||||
def compute_meta_trend(df: pd.DataFrame, settings: list[tuple[int, float]]) -> pd.Series:
|
||||
bull_cols = [f"bull_{l}_{m}" for l, m in settings]
|
||||
return (df[bull_cols].sum(axis=1) == len(bull_cols)).astype(int)
|
||||
10
intrabar.py
10
intrabar.py
@@ -1,10 +0,0 @@
|
||||
from __future__ import annotations
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def precompute_slices(df: pd.DataFrame) -> pd.DataFrame:
|
||||
return df # hook for future use
|
||||
|
||||
|
||||
def entry_slippage_row(price: float, qty: float, slippage_bps: float) -> float:
|
||||
return price + price * (slippage_bps / 1e4)
|
||||
135
live_trading/README.md
Normal file
135
live_trading/README.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# Live Trading - Regime Reversion Strategy
|
||||
|
||||
This module implements live trading for the ML-based regime detection and mean reversion strategy on OKX perpetual futures.
|
||||
|
||||
## Overview
|
||||
|
||||
The strategy trades ETH perpetual futures based on:
|
||||
1. **BTC/ETH Spread Z-Score**: Identifies when ETH is cheap or expensive relative to BTC
|
||||
2. **Random Forest ML Model**: Predicts probability of successful mean reversion
|
||||
3. **Funding Rate Filter**: Avoids trades in overheated/oversold market conditions
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. API Keys
|
||||
|
||||
The bot loads OKX API credentials from `../BTC_spot_MVRV/.env`.
|
||||
|
||||
**IMPORTANT: OKX uses SEPARATE API keys for live vs demo trading!**
|
||||
|
||||
#### Option A: Demo Trading (Recommended for Testing)
|
||||
1. Go to [OKX Demo Trading](https://www.okx.com/demo-trading)
|
||||
2. Create a demo account if you haven't
|
||||
3. Generate API keys from the demo environment
|
||||
4. Set in `.env`:
|
||||
```env
|
||||
OKX_API_KEY=your_demo_api_key
|
||||
OKX_SECRET=your_demo_secret
|
||||
OKX_PASSWORD=your_demo_passphrase
|
||||
OKX_DEMO_MODE=true
|
||||
```
|
||||
|
||||
#### Option B: Live Trading (Real Funds)
|
||||
Use your existing live API keys with:
|
||||
```env
|
||||
OKX_API_KEY=your_live_api_key
|
||||
OKX_SECRET=your_live_secret
|
||||
OKX_PASSWORD=your_live_passphrase
|
||||
OKX_DEMO_MODE=false
|
||||
```
|
||||
|
||||
**Note:** You cannot use live API keys with `OKX_DEMO_MODE=true` or vice versa.
|
||||
OKX will return error `50101: APIKey does not match current environment`.
|
||||
|
||||
### 2. Dependencies
|
||||
|
||||
All dependencies are already in the project's `pyproject.toml`. No additional installation needed.
|
||||
|
||||
## Usage
|
||||
|
||||
### Run with Demo Account (Recommended First)
|
||||
|
||||
```bash
|
||||
cd /path/to/lowkey_backtest
|
||||
uv run python -m live_trading.main
|
||||
```
|
||||
|
||||
### Command Line Options
|
||||
|
||||
```bash
|
||||
# Custom position size
|
||||
uv run python -m live_trading.main --max-position 500
|
||||
|
||||
# Custom leverage
|
||||
uv run python -m live_trading.main --leverage 2
|
||||
|
||||
# Custom cycle interval (in seconds)
|
||||
uv run python -m live_trading.main --interval 1800
|
||||
|
||||
# Combine options
|
||||
uv run python -m live_trading.main --max-position 1000 --leverage 3 --interval 3600
|
||||
```
|
||||
|
||||
### Live Trading (Use with Caution)
|
||||
|
||||
```bash
|
||||
# Requires OKX_DEMO_MODE=false in .env
|
||||
uv run python -m live_trading.main --live
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
live_trading/
|
||||
__init__.py # Module initialization
|
||||
config.py # Configuration loading
|
||||
okx_client.py # OKX API wrapper
|
||||
data_feed.py # Real-time OHLCV data
|
||||
position_manager.py # Position tracking
|
||||
live_regime_strategy.py # Strategy logic
|
||||
main.py # Entry point
|
||||
.env.example # Environment template
|
||||
README.md # This file
|
||||
```
|
||||
|
||||
## Strategy Parameters
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `z_entry_threshold` | 1.0 | Enter when \|Z-Score\| > threshold |
|
||||
| `z_window` | 24 | Rolling window for Z-Score (hours) |
|
||||
| `model_prob_threshold` | 0.5 | ML probability threshold for entry |
|
||||
| `funding_threshold` | 0.005 | Funding rate filter threshold |
|
||||
| `stop_loss_pct` | 6% | Stop-loss percentage |
|
||||
| `take_profit_pct` | 5% | Take-profit percentage |
|
||||
|
||||
## Files Generated
|
||||
|
||||
- `live_trading/positions.json` - Open positions persistence
|
||||
- `live_trading/trade_log.csv` - Trade history
|
||||
- `live_trading/regime_model.pkl` - Trained ML model
|
||||
- `logs/live_trading.log` - Trading logs
|
||||
|
||||
## Risk Warning
|
||||
|
||||
This is experimental trading software. Use at your own risk:
|
||||
- Always start with demo trading
|
||||
- Never risk more than you can afford to lose
|
||||
- Monitor the bot regularly
|
||||
- Have a kill switch ready (Ctrl+C)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### API Key Issues
|
||||
- Ensure API keys have trading permissions
|
||||
- For demo trading, use demo-specific API keys
|
||||
- Check that passphrase matches exactly
|
||||
|
||||
### No Signals Generated
|
||||
- The strategy requires the ML model to be trained
|
||||
- Need at least 200 candles of data
|
||||
- Model trains automatically on first run
|
||||
|
||||
### Position Sync Issues
|
||||
- The bot syncs with exchange positions on each cycle
|
||||
- If positions are closed manually, the bot will detect this
|
||||
6
live_trading/__init__.py
Normal file
6
live_trading/__init__.py
Normal file
@@ -0,0 +1,6 @@
|
||||
"""
|
||||
Live Trading Module for Regime Reversion Strategy on OKX.
|
||||
|
||||
This module implements live trading using the ML-based regime detection
|
||||
and mean reversion strategy on OKX perpetual futures.
|
||||
"""
|
||||
114
live_trading/config.py
Normal file
114
live_trading/config.py
Normal file
@@ -0,0 +1,114 @@
|
||||
"""
|
||||
Configuration for Live Trading.
|
||||
|
||||
Loads OKX API credentials from environment variables.
|
||||
Uses demo/sandbox mode by default for paper trading.
|
||||
"""
|
||||
import os
|
||||
from pathlib import Path
|
||||
from dataclasses import dataclass, field
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
@dataclass
|
||||
class OKXConfig:
|
||||
"""OKX API configuration."""
|
||||
api_key: str = field(default_factory=lambda: "")
|
||||
secret: str = field(default_factory=lambda: "")
|
||||
password: str = field(default_factory=lambda: "")
|
||||
demo_mode: bool = field(default_factory=lambda: True)
|
||||
|
||||
def __post_init__(self):
|
||||
"""Load credentials based on demo mode setting."""
|
||||
# Check demo mode first
|
||||
self.demo_mode = os.getenv("OKX_DEMO_MODE", "true").lower() in ("true", "1", "yes")
|
||||
|
||||
if self.demo_mode:
|
||||
# Load demo-specific credentials if available
|
||||
self.api_key = os.getenv("OKX_DEMO_API_KEY", os.getenv("OKX_API_KEY", ""))
|
||||
self.secret = os.getenv("OKX_DEMO_SECRET", os.getenv("OKX_SECRET", ""))
|
||||
self.password = os.getenv("OKX_DEMO_PASSWORD", os.getenv("OKX_PASSWORD", ""))
|
||||
else:
|
||||
# Load live credentials
|
||||
self.api_key = os.getenv("OKX_API_KEY", "")
|
||||
self.secret = os.getenv("OKX_SECRET", "")
|
||||
self.password = os.getenv("OKX_PASSWORD", "")
|
||||
|
||||
def validate(self) -> None:
|
||||
"""Validate that required credentials are present."""
|
||||
mode = "demo" if self.demo_mode else "live"
|
||||
if not self.api_key:
|
||||
raise ValueError(f"OKX API key not set for {mode} mode")
|
||||
if not self.secret:
|
||||
raise ValueError(f"OKX secret not set for {mode} mode")
|
||||
if not self.password:
|
||||
raise ValueError(f"OKX password not set for {mode} mode")
|
||||
|
||||
|
||||
@dataclass
|
||||
class TradingConfig:
|
||||
"""Trading parameters configuration."""
|
||||
# Trading pairs
|
||||
eth_symbol: str = "ETH/USDT:USDT" # ETH perpetual (primary trading asset)
|
||||
btc_symbol: str = "BTC/USDT:USDT" # BTC perpetual (context asset)
|
||||
|
||||
# Timeframe
|
||||
timeframe: str = "1h"
|
||||
candles_to_fetch: int = 500 # Enough for feature calculation
|
||||
|
||||
# Position sizing
|
||||
max_position_usdt: float = -1.0 # Max position size in USDT. If <= 0, use all available funds
|
||||
min_position_usdt: float = 1.0 # Min position size in USDT
|
||||
leverage: int = 1 # Leverage (1x = no leverage)
|
||||
margin_mode: str = "cross" # "cross" or "isolated"
|
||||
|
||||
# Risk management
|
||||
stop_loss_pct: float = 0.06 # 6% stop loss
|
||||
take_profit_pct: float = 0.05 # 5% take profit
|
||||
max_concurrent_positions: int = 1 # Max open positions
|
||||
|
||||
# Strategy parameters (from regime_strategy.py)
|
||||
z_entry_threshold: float = 1.0 # Enter when |Z| > 1.0
|
||||
z_window: int = 24 # 24h rolling Z-score window
|
||||
model_prob_threshold: float = 0.5 # ML model probability threshold
|
||||
funding_threshold: float = 0.005 # Funding rate filter threshold
|
||||
|
||||
# Execution
|
||||
sleep_seconds: int = 3600 # Run every hour (1h candles)
|
||||
slippage_pct: float = 0.001 # 0.1% slippage buffer
|
||||
|
||||
|
||||
@dataclass
|
||||
class PathConfig:
|
||||
"""File paths configuration."""
|
||||
base_dir: Path = field(
|
||||
default_factory=lambda: Path(__file__).parent.parent
|
||||
)
|
||||
data_dir: Path = field(default=None)
|
||||
logs_dir: Path = field(default=None)
|
||||
model_path: Path = field(default=None)
|
||||
positions_file: Path = field(default=None)
|
||||
trade_log_file: Path = field(default=None)
|
||||
cq_data_path: Path = field(default=None)
|
||||
|
||||
def __post_init__(self):
|
||||
self.data_dir = self.base_dir / "data"
|
||||
self.logs_dir = self.base_dir / "logs"
|
||||
self.model_path = self.base_dir / "live_trading" / "regime_model.pkl"
|
||||
self.positions_file = self.base_dir / "live_trading" / "positions.json"
|
||||
self.trade_log_file = self.base_dir / "live_trading" / "trade_log.csv"
|
||||
self.cq_data_path = self.data_dir / "cq_training_data.csv"
|
||||
|
||||
# Ensure directories exist
|
||||
self.data_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.logs_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def get_config():
|
||||
"""Get all configuration objects."""
|
||||
okx = OKXConfig()
|
||||
trading = TradingConfig()
|
||||
paths = PathConfig()
|
||||
return okx, trading, paths
|
||||
216
live_trading/data_feed.py
Normal file
216
live_trading/data_feed.py
Normal file
@@ -0,0 +1,216 @@
|
||||
"""
|
||||
Data Feed for Live Trading.
|
||||
|
||||
Fetches real-time OHLCV data from OKX and prepares features
|
||||
for the regime strategy.
|
||||
"""
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
import ta
|
||||
|
||||
from .okx_client import OKXClient
|
||||
from .config import TradingConfig, PathConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class DataFeed:
|
||||
"""
|
||||
Real-time data feed for the regime strategy.
|
||||
|
||||
Fetches BTC and ETH OHLCV data from OKX and calculates
|
||||
the spread-based features required by the ML model.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
okx_client: OKXClient,
|
||||
trading_config: TradingConfig,
|
||||
path_config: PathConfig
|
||||
):
|
||||
self.client = okx_client
|
||||
self.config = trading_config
|
||||
self.paths = path_config
|
||||
self.cq_data: Optional[pd.DataFrame] = None
|
||||
self._load_cq_data()
|
||||
|
||||
def _load_cq_data(self) -> None:
|
||||
"""Load CryptoQuant on-chain data if available."""
|
||||
try:
|
||||
if self.paths.cq_data_path.exists():
|
||||
self.cq_data = pd.read_csv(
|
||||
self.paths.cq_data_path,
|
||||
index_col='timestamp',
|
||||
parse_dates=True
|
||||
)
|
||||
if self.cq_data.index.tz is None:
|
||||
self.cq_data.index = self.cq_data.index.tz_localize('UTC')
|
||||
logger.info(f"Loaded CryptoQuant data: {len(self.cq_data)} rows")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not load CryptoQuant data: {e}")
|
||||
self.cq_data = None
|
||||
|
||||
def fetch_ohlcv_data(self) -> tuple[pd.DataFrame, pd.DataFrame]:
|
||||
"""
|
||||
Fetch OHLCV data for BTC and ETH.
|
||||
|
||||
Returns:
|
||||
Tuple of (btc_df, eth_df) DataFrames
|
||||
"""
|
||||
# Fetch BTC data
|
||||
btc_ohlcv = self.client.fetch_ohlcv(
|
||||
self.config.btc_symbol,
|
||||
self.config.timeframe,
|
||||
self.config.candles_to_fetch
|
||||
)
|
||||
btc_df = self._ohlcv_to_dataframe(btc_ohlcv)
|
||||
|
||||
# Fetch ETH data
|
||||
eth_ohlcv = self.client.fetch_ohlcv(
|
||||
self.config.eth_symbol,
|
||||
self.config.timeframe,
|
||||
self.config.candles_to_fetch
|
||||
)
|
||||
eth_df = self._ohlcv_to_dataframe(eth_ohlcv)
|
||||
|
||||
logger.info(
|
||||
f"Fetched {len(btc_df)} BTC candles and {len(eth_df)} ETH candles"
|
||||
)
|
||||
|
||||
return btc_df, eth_df
|
||||
|
||||
def _ohlcv_to_dataframe(self, ohlcv: list) -> pd.DataFrame:
|
||||
"""Convert OHLCV list to DataFrame."""
|
||||
df = pd.DataFrame(
|
||||
ohlcv,
|
||||
columns=['timestamp', 'open', 'high', 'low', 'close', 'volume']
|
||||
)
|
||||
df['timestamp'] = pd.to_datetime(df['timestamp'], unit='ms', utc=True)
|
||||
df.set_index('timestamp', inplace=True)
|
||||
return df
|
||||
|
||||
def calculate_features(
|
||||
self,
|
||||
btc_df: pd.DataFrame,
|
||||
eth_df: pd.DataFrame
|
||||
) -> pd.DataFrame:
|
||||
"""
|
||||
Calculate spread-based features for the regime strategy.
|
||||
|
||||
Args:
|
||||
btc_df: BTC OHLCV DataFrame
|
||||
eth_df: ETH OHLCV DataFrame
|
||||
|
||||
Returns:
|
||||
DataFrame with calculated features
|
||||
"""
|
||||
# Align indices
|
||||
common_idx = btc_df.index.intersection(eth_df.index)
|
||||
df_btc = btc_df.loc[common_idx].copy()
|
||||
df_eth = eth_df.loc[common_idx].copy()
|
||||
|
||||
# Calculate spread (ETH/BTC ratio)
|
||||
spread = df_eth['close'] / df_btc['close']
|
||||
|
||||
# Z-Score of spread
|
||||
z_window = self.config.z_window
|
||||
rolling_mean = spread.rolling(window=z_window).mean()
|
||||
rolling_std = spread.rolling(window=z_window).std()
|
||||
z_score = (spread - rolling_mean) / rolling_std
|
||||
|
||||
# Spread technicals
|
||||
spread_rsi = ta.momentum.RSIIndicator(spread, window=14).rsi()
|
||||
spread_roc = spread.pct_change(periods=5) * 100
|
||||
spread_change_1h = spread.pct_change(periods=1)
|
||||
|
||||
# Volume ratio
|
||||
vol_ratio = df_eth['volume'] / df_btc['volume']
|
||||
vol_ratio_ma = vol_ratio.rolling(window=12).mean()
|
||||
|
||||
# Volatility
|
||||
ret_btc = df_btc['close'].pct_change()
|
||||
ret_eth = df_eth['close'].pct_change()
|
||||
vol_btc = ret_btc.rolling(window=z_window).std()
|
||||
vol_eth = ret_eth.rolling(window=z_window).std()
|
||||
vol_spread_ratio = vol_eth / vol_btc
|
||||
|
||||
# Build features DataFrame
|
||||
features = pd.DataFrame(index=spread.index)
|
||||
features['spread'] = spread
|
||||
features['z_score'] = z_score
|
||||
features['spread_rsi'] = spread_rsi
|
||||
features['spread_roc'] = spread_roc
|
||||
features['spread_change_1h'] = spread_change_1h
|
||||
features['vol_ratio'] = vol_ratio
|
||||
features['vol_ratio_rel'] = vol_ratio / vol_ratio_ma
|
||||
features['vol_diff_ratio'] = vol_spread_ratio
|
||||
|
||||
# Add price data for reference
|
||||
features['btc_close'] = df_btc['close']
|
||||
features['eth_close'] = df_eth['close']
|
||||
features['eth_volume'] = df_eth['volume']
|
||||
|
||||
# Merge CryptoQuant data if available
|
||||
if self.cq_data is not None:
|
||||
cq_aligned = self.cq_data.reindex(features.index, method='ffill')
|
||||
|
||||
# Calculate derived features
|
||||
if 'btc_funding' in cq_aligned.columns and 'eth_funding' in cq_aligned.columns:
|
||||
cq_aligned['funding_diff'] = (
|
||||
cq_aligned['eth_funding'] - cq_aligned['btc_funding']
|
||||
)
|
||||
if 'btc_inflow' in cq_aligned.columns and 'eth_inflow' in cq_aligned.columns:
|
||||
cq_aligned['inflow_ratio'] = (
|
||||
cq_aligned['eth_inflow'] / (cq_aligned['btc_inflow'] + 1)
|
||||
)
|
||||
|
||||
features = features.join(cq_aligned)
|
||||
|
||||
return features.dropna()
|
||||
|
||||
def get_latest_data(self) -> Optional[pd.DataFrame]:
|
||||
"""
|
||||
Fetch and process latest market data.
|
||||
|
||||
Returns:
|
||||
DataFrame with features or None on error
|
||||
"""
|
||||
try:
|
||||
btc_df, eth_df = self.fetch_ohlcv_data()
|
||||
features = self.calculate_features(btc_df, eth_df)
|
||||
|
||||
if features.empty:
|
||||
logger.warning("No valid features calculated")
|
||||
return None
|
||||
|
||||
logger.info(
|
||||
f"Latest data: ETH={features['eth_close'].iloc[-1]:.2f}, "
|
||||
f"BTC={features['btc_close'].iloc[-1]:.2f}, "
|
||||
f"Z-Score={features['z_score'].iloc[-1]:.3f}"
|
||||
)
|
||||
|
||||
return features
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error fetching market data: {e}", exc_info=True)
|
||||
return None
|
||||
|
||||
def get_current_funding_rates(self) -> dict:
|
||||
"""
|
||||
Get current funding rates for BTC and ETH.
|
||||
|
||||
Returns:
|
||||
Dictionary with 'btc_funding' and 'eth_funding' rates
|
||||
"""
|
||||
btc_funding = self.client.get_funding_rate(self.config.btc_symbol)
|
||||
eth_funding = self.client.get_funding_rate(self.config.eth_symbol)
|
||||
|
||||
return {
|
||||
'btc_funding': btc_funding,
|
||||
'eth_funding': eth_funding,
|
||||
'funding_diff': eth_funding - btc_funding,
|
||||
}
|
||||
13
live_trading/db/__init__.py
Normal file
13
live_trading/db/__init__.py
Normal file
@@ -0,0 +1,13 @@
|
||||
"""Database module for live trading persistence."""
|
||||
from .database import get_db, init_db
|
||||
from .models import Trade, DailySummary, Session
|
||||
from .metrics import MetricsCalculator
|
||||
|
||||
__all__ = [
|
||||
"get_db",
|
||||
"init_db",
|
||||
"Trade",
|
||||
"DailySummary",
|
||||
"Session",
|
||||
"MetricsCalculator",
|
||||
]
|
||||
325
live_trading/db/database.py
Normal file
325
live_trading/db/database.py
Normal file
@@ -0,0 +1,325 @@
|
||||
"""SQLite database connection and operations."""
|
||||
import sqlite3
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from contextlib import contextmanager
|
||||
|
||||
from .models import Trade, DailySummary, Session
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Database schema
|
||||
SCHEMA = """
|
||||
-- Trade history table
|
||||
CREATE TABLE IF NOT EXISTS trades (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
trade_id TEXT UNIQUE NOT NULL,
|
||||
symbol TEXT NOT NULL,
|
||||
side TEXT NOT NULL,
|
||||
entry_price REAL NOT NULL,
|
||||
exit_price REAL,
|
||||
size REAL NOT NULL,
|
||||
size_usdt REAL NOT NULL,
|
||||
pnl_usd REAL,
|
||||
pnl_pct REAL,
|
||||
entry_time TEXT NOT NULL,
|
||||
exit_time TEXT,
|
||||
hold_duration_hours REAL,
|
||||
reason TEXT,
|
||||
order_id_entry TEXT,
|
||||
order_id_exit TEXT,
|
||||
created_at TEXT DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Daily summary table
|
||||
CREATE TABLE IF NOT EXISTS daily_summary (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
date TEXT UNIQUE NOT NULL,
|
||||
total_trades INTEGER DEFAULT 0,
|
||||
winning_trades INTEGER DEFAULT 0,
|
||||
total_pnl_usd REAL DEFAULT 0,
|
||||
max_drawdown_usd REAL DEFAULT 0,
|
||||
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Session metadata
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
start_time TEXT NOT NULL,
|
||||
end_time TEXT,
|
||||
starting_balance REAL,
|
||||
ending_balance REAL,
|
||||
total_pnl REAL,
|
||||
total_trades INTEGER DEFAULT 0
|
||||
);
|
||||
|
||||
-- Indexes for common queries
|
||||
CREATE INDEX IF NOT EXISTS idx_trades_entry_time ON trades(entry_time);
|
||||
CREATE INDEX IF NOT EXISTS idx_trades_exit_time ON trades(exit_time);
|
||||
CREATE INDEX IF NOT EXISTS idx_daily_summary_date ON daily_summary(date);
|
||||
"""
|
||||
|
||||
_db_instance: Optional["TradingDatabase"] = None
|
||||
|
||||
|
||||
class TradingDatabase:
|
||||
"""SQLite database for trade persistence."""
|
||||
|
||||
def __init__(self, db_path: Path):
|
||||
self.db_path = db_path
|
||||
self._connection: Optional[sqlite3.Connection] = None
|
||||
|
||||
@property
|
||||
def connection(self) -> sqlite3.Connection:
|
||||
"""Get or create database connection."""
|
||||
if self._connection is None:
|
||||
self._connection = sqlite3.connect(
|
||||
str(self.db_path),
|
||||
check_same_thread=False,
|
||||
)
|
||||
self._connection.row_factory = sqlite3.Row
|
||||
return self._connection
|
||||
|
||||
def init_schema(self) -> None:
|
||||
"""Initialize database schema."""
|
||||
with self.connection:
|
||||
self.connection.executescript(SCHEMA)
|
||||
logger.info(f"Database initialized at {self.db_path}")
|
||||
|
||||
def close(self) -> None:
|
||||
"""Close database connection."""
|
||||
if self._connection:
|
||||
self._connection.close()
|
||||
self._connection = None
|
||||
|
||||
@contextmanager
|
||||
def transaction(self):
|
||||
"""Context manager for database transactions."""
|
||||
try:
|
||||
yield self.connection
|
||||
self.connection.commit()
|
||||
except Exception:
|
||||
self.connection.rollback()
|
||||
raise
|
||||
|
||||
def insert_trade(self, trade: Trade) -> int:
|
||||
"""
|
||||
Insert a new trade record.
|
||||
|
||||
Args:
|
||||
trade: Trade object to insert
|
||||
|
||||
Returns:
|
||||
Row ID of inserted trade
|
||||
"""
|
||||
sql = """
|
||||
INSERT INTO trades (
|
||||
trade_id, symbol, side, entry_price, exit_price,
|
||||
size, size_usdt, pnl_usd, pnl_pct, entry_time,
|
||||
exit_time, hold_duration_hours, reason,
|
||||
order_id_entry, order_id_exit
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
"""
|
||||
with self.transaction():
|
||||
cursor = self.connection.execute(
|
||||
sql,
|
||||
(
|
||||
trade.trade_id,
|
||||
trade.symbol,
|
||||
trade.side,
|
||||
trade.entry_price,
|
||||
trade.exit_price,
|
||||
trade.size,
|
||||
trade.size_usdt,
|
||||
trade.pnl_usd,
|
||||
trade.pnl_pct,
|
||||
trade.entry_time,
|
||||
trade.exit_time,
|
||||
trade.hold_duration_hours,
|
||||
trade.reason,
|
||||
trade.order_id_entry,
|
||||
trade.order_id_exit,
|
||||
),
|
||||
)
|
||||
return cursor.lastrowid
|
||||
|
||||
def update_trade(self, trade_id: str, **kwargs) -> bool:
|
||||
"""
|
||||
Update an existing trade record.
|
||||
|
||||
Args:
|
||||
trade_id: Trade ID to update
|
||||
**kwargs: Fields to update
|
||||
|
||||
Returns:
|
||||
True if trade was updated
|
||||
"""
|
||||
if not kwargs:
|
||||
return False
|
||||
|
||||
set_clause = ", ".join(f"{k} = ?" for k in kwargs.keys())
|
||||
sql = f"UPDATE trades SET {set_clause} WHERE trade_id = ?"
|
||||
|
||||
with self.transaction():
|
||||
cursor = self.connection.execute(
|
||||
sql, (*kwargs.values(), trade_id)
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def get_trade(self, trade_id: str) -> Optional[Trade]:
|
||||
"""Get a trade by ID."""
|
||||
sql = "SELECT * FROM trades WHERE trade_id = ?"
|
||||
row = self.connection.execute(sql, (trade_id,)).fetchone()
|
||||
if row:
|
||||
return Trade(**dict(row))
|
||||
return None
|
||||
|
||||
def get_trades(
|
||||
self,
|
||||
start_time: Optional[str] = None,
|
||||
end_time: Optional[str] = None,
|
||||
limit: Optional[int] = None,
|
||||
) -> list[Trade]:
|
||||
"""
|
||||
Get trades within a time range.
|
||||
|
||||
Args:
|
||||
start_time: ISO format start time filter
|
||||
end_time: ISO format end time filter
|
||||
limit: Maximum number of trades to return
|
||||
|
||||
Returns:
|
||||
List of Trade objects
|
||||
"""
|
||||
conditions = []
|
||||
params = []
|
||||
|
||||
if start_time:
|
||||
conditions.append("entry_time >= ?")
|
||||
params.append(start_time)
|
||||
if end_time:
|
||||
conditions.append("entry_time <= ?")
|
||||
params.append(end_time)
|
||||
|
||||
where_clause = " AND ".join(conditions) if conditions else "1=1"
|
||||
limit_clause = f"LIMIT {limit}" if limit else ""
|
||||
|
||||
sql = f"""
|
||||
SELECT * FROM trades
|
||||
WHERE {where_clause}
|
||||
ORDER BY entry_time DESC
|
||||
{limit_clause}
|
||||
"""
|
||||
|
||||
rows = self.connection.execute(sql, params).fetchall()
|
||||
return [Trade(**dict(row)) for row in rows]
|
||||
|
||||
def get_all_trades(self) -> list[Trade]:
|
||||
"""Get all trades."""
|
||||
sql = "SELECT * FROM trades ORDER BY entry_time DESC"
|
||||
rows = self.connection.execute(sql).fetchall()
|
||||
return [Trade(**dict(row)) for row in rows]
|
||||
|
||||
def count_trades(self) -> int:
|
||||
"""Get total number of trades."""
|
||||
sql = "SELECT COUNT(*) FROM trades WHERE exit_time IS NOT NULL"
|
||||
return self.connection.execute(sql).fetchone()[0]
|
||||
|
||||
def upsert_daily_summary(self, summary: DailySummary) -> None:
|
||||
"""Insert or update daily summary."""
|
||||
sql = """
|
||||
INSERT INTO daily_summary (
|
||||
date, total_trades, winning_trades, total_pnl_usd, max_drawdown_usd
|
||||
) VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT(date) DO UPDATE SET
|
||||
total_trades = excluded.total_trades,
|
||||
winning_trades = excluded.winning_trades,
|
||||
total_pnl_usd = excluded.total_pnl_usd,
|
||||
max_drawdown_usd = excluded.max_drawdown_usd,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
"""
|
||||
with self.transaction():
|
||||
self.connection.execute(
|
||||
sql,
|
||||
(
|
||||
summary.date,
|
||||
summary.total_trades,
|
||||
summary.winning_trades,
|
||||
summary.total_pnl_usd,
|
||||
summary.max_drawdown_usd,
|
||||
),
|
||||
)
|
||||
|
||||
def get_daily_summary(self, date: str) -> Optional[DailySummary]:
|
||||
"""Get daily summary for a specific date."""
|
||||
sql = "SELECT * FROM daily_summary WHERE date = ?"
|
||||
row = self.connection.execute(sql, (date,)).fetchone()
|
||||
if row:
|
||||
return DailySummary(**dict(row))
|
||||
return None
|
||||
|
||||
def insert_session(self, session: Session) -> int:
|
||||
"""Insert a new session record."""
|
||||
sql = """
|
||||
INSERT INTO sessions (
|
||||
start_time, end_time, starting_balance,
|
||||
ending_balance, total_pnl, total_trades
|
||||
) VALUES (?, ?, ?, ?, ?, ?)
|
||||
"""
|
||||
with self.transaction():
|
||||
cursor = self.connection.execute(
|
||||
sql,
|
||||
(
|
||||
session.start_time,
|
||||
session.end_time,
|
||||
session.starting_balance,
|
||||
session.ending_balance,
|
||||
session.total_pnl,
|
||||
session.total_trades,
|
||||
),
|
||||
)
|
||||
return cursor.lastrowid
|
||||
|
||||
def update_session(self, session_id: int, **kwargs) -> bool:
|
||||
"""Update an existing session."""
|
||||
if not kwargs:
|
||||
return False
|
||||
|
||||
set_clause = ", ".join(f"{k} = ?" for k in kwargs.keys())
|
||||
sql = f"UPDATE sessions SET {set_clause} WHERE id = ?"
|
||||
|
||||
with self.transaction():
|
||||
cursor = self.connection.execute(
|
||||
sql, (*kwargs.values(), session_id)
|
||||
)
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def get_latest_session(self) -> Optional[Session]:
|
||||
"""Get the most recent session."""
|
||||
sql = "SELECT * FROM sessions ORDER BY id DESC LIMIT 1"
|
||||
row = self.connection.execute(sql).fetchone()
|
||||
if row:
|
||||
return Session(**dict(row))
|
||||
return None
|
||||
|
||||
|
||||
def init_db(db_path: Path) -> TradingDatabase:
|
||||
"""
|
||||
Initialize the database.
|
||||
|
||||
Args:
|
||||
db_path: Path to the SQLite database file
|
||||
|
||||
Returns:
|
||||
TradingDatabase instance
|
||||
"""
|
||||
global _db_instance
|
||||
_db_instance = TradingDatabase(db_path)
|
||||
_db_instance.init_schema()
|
||||
return _db_instance
|
||||
|
||||
|
||||
def get_db() -> Optional[TradingDatabase]:
|
||||
"""Get the global database instance."""
|
||||
return _db_instance
|
||||
235
live_trading/db/metrics.py
Normal file
235
live_trading/db/metrics.py
Normal file
@@ -0,0 +1,235 @@
|
||||
"""Metrics calculation from trade database."""
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone, timedelta
|
||||
from typing import Optional
|
||||
|
||||
from .database import TradingDatabase
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class PeriodMetrics:
|
||||
"""Trading metrics for a time period."""
|
||||
|
||||
period_name: str
|
||||
start_time: Optional[str]
|
||||
end_time: Optional[str]
|
||||
total_pnl: float = 0.0
|
||||
total_trades: int = 0
|
||||
winning_trades: int = 0
|
||||
losing_trades: int = 0
|
||||
win_rate: float = 0.0
|
||||
avg_trade_duration_hours: float = 0.0
|
||||
max_drawdown: float = 0.0
|
||||
max_drawdown_pct: float = 0.0
|
||||
best_trade: float = 0.0
|
||||
worst_trade: float = 0.0
|
||||
avg_win: float = 0.0
|
||||
avg_loss: float = 0.0
|
||||
|
||||
|
||||
class MetricsCalculator:
|
||||
"""Calculate trading metrics from database."""
|
||||
|
||||
def __init__(self, db: TradingDatabase):
|
||||
self.db = db
|
||||
|
||||
def get_all_time_metrics(self) -> PeriodMetrics:
|
||||
"""Get metrics for all trades ever."""
|
||||
return self._calculate_metrics("All Time", None, None)
|
||||
|
||||
def get_monthly_metrics(self) -> PeriodMetrics:
|
||||
"""Get metrics for current month."""
|
||||
now = datetime.now(timezone.utc)
|
||||
start = now.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
|
||||
return self._calculate_metrics(
|
||||
"Monthly",
|
||||
start.isoformat(),
|
||||
now.isoformat(),
|
||||
)
|
||||
|
||||
def get_weekly_metrics(self) -> PeriodMetrics:
|
||||
"""Get metrics for current week (Monday to now)."""
|
||||
now = datetime.now(timezone.utc)
|
||||
days_since_monday = now.weekday()
|
||||
start = now - timedelta(days=days_since_monday)
|
||||
start = start.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
return self._calculate_metrics(
|
||||
"Weekly",
|
||||
start.isoformat(),
|
||||
now.isoformat(),
|
||||
)
|
||||
|
||||
def get_daily_metrics(self) -> PeriodMetrics:
|
||||
"""Get metrics for today (UTC)."""
|
||||
now = datetime.now(timezone.utc)
|
||||
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
return self._calculate_metrics(
|
||||
"Daily",
|
||||
start.isoformat(),
|
||||
now.isoformat(),
|
||||
)
|
||||
|
||||
def _calculate_metrics(
|
||||
self,
|
||||
period_name: str,
|
||||
start_time: Optional[str],
|
||||
end_time: Optional[str],
|
||||
) -> PeriodMetrics:
|
||||
"""
|
||||
Calculate metrics for a time period.
|
||||
|
||||
Args:
|
||||
period_name: Name of the period
|
||||
start_time: ISO format start time (None for all time)
|
||||
end_time: ISO format end time (None for all time)
|
||||
|
||||
Returns:
|
||||
PeriodMetrics object
|
||||
"""
|
||||
metrics = PeriodMetrics(
|
||||
period_name=period_name,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
)
|
||||
|
||||
# Build query conditions
|
||||
conditions = ["exit_time IS NOT NULL"]
|
||||
params = []
|
||||
|
||||
if start_time:
|
||||
conditions.append("exit_time >= ?")
|
||||
params.append(start_time)
|
||||
if end_time:
|
||||
conditions.append("exit_time <= ?")
|
||||
params.append(end_time)
|
||||
|
||||
where_clause = " AND ".join(conditions)
|
||||
|
||||
# Get aggregate metrics
|
||||
sql = f"""
|
||||
SELECT
|
||||
COUNT(*) as total_trades,
|
||||
SUM(CASE WHEN pnl_usd > 0 THEN 1 ELSE 0 END) as winning_trades,
|
||||
SUM(CASE WHEN pnl_usd < 0 THEN 1 ELSE 0 END) as losing_trades,
|
||||
COALESCE(SUM(pnl_usd), 0) as total_pnl,
|
||||
COALESCE(AVG(hold_duration_hours), 0) as avg_duration,
|
||||
COALESCE(MAX(pnl_usd), 0) as best_trade,
|
||||
COALESCE(MIN(pnl_usd), 0) as worst_trade,
|
||||
COALESCE(AVG(CASE WHEN pnl_usd > 0 THEN pnl_usd END), 0) as avg_win,
|
||||
COALESCE(AVG(CASE WHEN pnl_usd < 0 THEN pnl_usd END), 0) as avg_loss
|
||||
FROM trades
|
||||
WHERE {where_clause}
|
||||
"""
|
||||
|
||||
row = self.db.connection.execute(sql, params).fetchone()
|
||||
|
||||
if row and row["total_trades"] > 0:
|
||||
metrics.total_trades = row["total_trades"]
|
||||
metrics.winning_trades = row["winning_trades"] or 0
|
||||
metrics.losing_trades = row["losing_trades"] or 0
|
||||
metrics.total_pnl = row["total_pnl"]
|
||||
metrics.avg_trade_duration_hours = row["avg_duration"]
|
||||
metrics.best_trade = row["best_trade"]
|
||||
metrics.worst_trade = row["worst_trade"]
|
||||
metrics.avg_win = row["avg_win"]
|
||||
metrics.avg_loss = row["avg_loss"]
|
||||
|
||||
if metrics.total_trades > 0:
|
||||
metrics.win_rate = (
|
||||
metrics.winning_trades / metrics.total_trades * 100
|
||||
)
|
||||
|
||||
# Calculate max drawdown
|
||||
metrics.max_drawdown = self._calculate_max_drawdown(
|
||||
start_time, end_time
|
||||
)
|
||||
|
||||
return metrics
|
||||
|
||||
def _calculate_max_drawdown(
|
||||
self,
|
||||
start_time: Optional[str],
|
||||
end_time: Optional[str],
|
||||
) -> float:
|
||||
"""Calculate maximum drawdown for a period."""
|
||||
conditions = ["exit_time IS NOT NULL"]
|
||||
params = []
|
||||
|
||||
if start_time:
|
||||
conditions.append("exit_time >= ?")
|
||||
params.append(start_time)
|
||||
if end_time:
|
||||
conditions.append("exit_time <= ?")
|
||||
params.append(end_time)
|
||||
|
||||
where_clause = " AND ".join(conditions)
|
||||
|
||||
sql = f"""
|
||||
SELECT pnl_usd
|
||||
FROM trades
|
||||
WHERE {where_clause}
|
||||
ORDER BY exit_time
|
||||
"""
|
||||
|
||||
rows = self.db.connection.execute(sql, params).fetchall()
|
||||
|
||||
if not rows:
|
||||
return 0.0
|
||||
|
||||
cumulative = 0.0
|
||||
peak = 0.0
|
||||
max_drawdown = 0.0
|
||||
|
||||
for row in rows:
|
||||
pnl = row["pnl_usd"] or 0.0
|
||||
cumulative += pnl
|
||||
peak = max(peak, cumulative)
|
||||
drawdown = peak - cumulative
|
||||
max_drawdown = max(max_drawdown, drawdown)
|
||||
|
||||
return max_drawdown
|
||||
|
||||
def has_monthly_data(self) -> bool:
|
||||
"""Check if we have data spanning more than current month."""
|
||||
sql = """
|
||||
SELECT MIN(exit_time) as first_trade
|
||||
FROM trades
|
||||
WHERE exit_time IS NOT NULL
|
||||
"""
|
||||
row = self.db.connection.execute(sql).fetchone()
|
||||
if not row or not row["first_trade"]:
|
||||
return False
|
||||
|
||||
first_trade = datetime.fromisoformat(row["first_trade"])
|
||||
now = datetime.now(timezone.utc)
|
||||
month_start = now.replace(day=1, hour=0, minute=0, second=0)
|
||||
|
||||
return first_trade < month_start
|
||||
|
||||
def has_weekly_data(self) -> bool:
|
||||
"""Check if we have data spanning more than current week."""
|
||||
sql = """
|
||||
SELECT MIN(exit_time) as first_trade
|
||||
FROM trades
|
||||
WHERE exit_time IS NOT NULL
|
||||
"""
|
||||
row = self.db.connection.execute(sql).fetchone()
|
||||
if not row or not row["first_trade"]:
|
||||
return False
|
||||
|
||||
first_trade = datetime.fromisoformat(row["first_trade"])
|
||||
now = datetime.now(timezone.utc)
|
||||
days_since_monday = now.weekday()
|
||||
week_start = now - timedelta(days=days_since_monday)
|
||||
week_start = week_start.replace(hour=0, minute=0, second=0)
|
||||
|
||||
return first_trade < week_start
|
||||
|
||||
def get_session_start_balance(self) -> Optional[float]:
|
||||
"""Get starting balance from latest session."""
|
||||
sql = "SELECT starting_balance FROM sessions ORDER BY id DESC LIMIT 1"
|
||||
row = self.db.connection.execute(sql).fetchone()
|
||||
return row["starting_balance"] if row else None
|
||||
191
live_trading/db/migrations.py
Normal file
191
live_trading/db/migrations.py
Normal file
@@ -0,0 +1,191 @@
|
||||
"""Database migrations and CSV import."""
|
||||
import csv
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
|
||||
from .database import TradingDatabase
|
||||
from .models import Trade, DailySummary
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def migrate_csv_to_db(db: TradingDatabase, csv_path: Path) -> int:
|
||||
"""
|
||||
Migrate trades from CSV file to SQLite database.
|
||||
|
||||
Args:
|
||||
db: TradingDatabase instance
|
||||
csv_path: Path to trade_log.csv
|
||||
|
||||
Returns:
|
||||
Number of trades migrated
|
||||
"""
|
||||
if not csv_path.exists():
|
||||
logger.info("No CSV file to migrate")
|
||||
return 0
|
||||
|
||||
# Check if database already has trades
|
||||
existing_count = db.count_trades()
|
||||
if existing_count > 0:
|
||||
logger.info(
|
||||
f"Database already has {existing_count} trades, skipping migration"
|
||||
)
|
||||
return 0
|
||||
|
||||
migrated = 0
|
||||
try:
|
||||
with open(csv_path, "r", newline="") as f:
|
||||
reader = csv.DictReader(f)
|
||||
|
||||
for row in reader:
|
||||
trade = _csv_row_to_trade(row)
|
||||
if trade:
|
||||
try:
|
||||
db.insert_trade(trade)
|
||||
migrated += 1
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"Failed to migrate trade {row.get('trade_id')}: {e}"
|
||||
)
|
||||
|
||||
logger.info(f"Migrated {migrated} trades from CSV to database")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"CSV migration failed: {e}")
|
||||
|
||||
return migrated
|
||||
|
||||
|
||||
def _csv_row_to_trade(row: dict) -> Trade | None:
|
||||
"""Convert a CSV row to a Trade object."""
|
||||
try:
|
||||
return Trade(
|
||||
trade_id=row["trade_id"],
|
||||
symbol=row["symbol"],
|
||||
side=row["side"],
|
||||
entry_price=float(row["entry_price"]),
|
||||
exit_price=_safe_float(row.get("exit_price")),
|
||||
size=float(row["size"]),
|
||||
size_usdt=float(row["size_usdt"]),
|
||||
pnl_usd=_safe_float(row.get("pnl_usd")),
|
||||
pnl_pct=_safe_float(row.get("pnl_pct")),
|
||||
entry_time=row["entry_time"],
|
||||
exit_time=row.get("exit_time") or None,
|
||||
hold_duration_hours=_safe_float(row.get("hold_duration_hours")),
|
||||
reason=row.get("reason") or None,
|
||||
order_id_entry=row.get("order_id_entry") or None,
|
||||
order_id_exit=row.get("order_id_exit") or None,
|
||||
)
|
||||
except (KeyError, ValueError) as e:
|
||||
logger.warning(f"Invalid CSV row: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _safe_float(value: str | None) -> float | None:
|
||||
"""Safely convert string to float."""
|
||||
if value is None or value == "":
|
||||
return None
|
||||
try:
|
||||
return float(value)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def rebuild_daily_summaries(db: TradingDatabase) -> int:
|
||||
"""
|
||||
Rebuild daily summary table from trades.
|
||||
|
||||
Args:
|
||||
db: TradingDatabase instance
|
||||
|
||||
Returns:
|
||||
Number of daily summaries created
|
||||
"""
|
||||
sql = """
|
||||
SELECT
|
||||
DATE(exit_time) as date,
|
||||
COUNT(*) as total_trades,
|
||||
SUM(CASE WHEN pnl_usd > 0 THEN 1 ELSE 0 END) as winning_trades,
|
||||
SUM(pnl_usd) as total_pnl_usd
|
||||
FROM trades
|
||||
WHERE exit_time IS NOT NULL
|
||||
GROUP BY DATE(exit_time)
|
||||
ORDER BY date
|
||||
"""
|
||||
|
||||
rows = db.connection.execute(sql).fetchall()
|
||||
count = 0
|
||||
|
||||
for row in rows:
|
||||
summary = DailySummary(
|
||||
date=row["date"],
|
||||
total_trades=row["total_trades"],
|
||||
winning_trades=row["winning_trades"],
|
||||
total_pnl_usd=row["total_pnl_usd"] or 0.0,
|
||||
max_drawdown_usd=0.0, # Calculated separately
|
||||
)
|
||||
db.upsert_daily_summary(summary)
|
||||
count += 1
|
||||
|
||||
# Calculate max drawdowns
|
||||
_calculate_daily_drawdowns(db)
|
||||
|
||||
logger.info(f"Rebuilt {count} daily summaries")
|
||||
return count
|
||||
|
||||
|
||||
def _calculate_daily_drawdowns(db: TradingDatabase) -> None:
|
||||
"""Calculate and update max drawdown for each day."""
|
||||
sql = """
|
||||
SELECT trade_id, DATE(exit_time) as date, pnl_usd
|
||||
FROM trades
|
||||
WHERE exit_time IS NOT NULL
|
||||
ORDER BY exit_time
|
||||
"""
|
||||
|
||||
rows = db.connection.execute(sql).fetchall()
|
||||
|
||||
# Track cumulative PnL and drawdown per day
|
||||
daily_drawdowns: dict[str, float] = {}
|
||||
cumulative_pnl = 0.0
|
||||
peak_pnl = 0.0
|
||||
|
||||
for row in rows:
|
||||
date = row["date"]
|
||||
pnl = row["pnl_usd"] or 0.0
|
||||
|
||||
cumulative_pnl += pnl
|
||||
peak_pnl = max(peak_pnl, cumulative_pnl)
|
||||
drawdown = peak_pnl - cumulative_pnl
|
||||
|
||||
if date not in daily_drawdowns:
|
||||
daily_drawdowns[date] = 0.0
|
||||
daily_drawdowns[date] = max(daily_drawdowns[date], drawdown)
|
||||
|
||||
# Update daily summaries with drawdown
|
||||
for date, drawdown in daily_drawdowns.items():
|
||||
db.connection.execute(
|
||||
"UPDATE daily_summary SET max_drawdown_usd = ? WHERE date = ?",
|
||||
(drawdown, date),
|
||||
)
|
||||
db.connection.commit()
|
||||
|
||||
|
||||
def run_migrations(db: TradingDatabase, csv_path: Path) -> None:
|
||||
"""
|
||||
Run all migrations.
|
||||
|
||||
Args:
|
||||
db: TradingDatabase instance
|
||||
csv_path: Path to trade_log.csv for migration
|
||||
"""
|
||||
logger.info("Running database migrations...")
|
||||
|
||||
# Migrate CSV data if exists
|
||||
migrate_csv_to_db(db, csv_path)
|
||||
|
||||
# Rebuild daily summaries
|
||||
rebuild_daily_summaries(db)
|
||||
|
||||
logger.info("Migrations complete")
|
||||
69
live_trading/db/models.py
Normal file
69
live_trading/db/models.py
Normal file
@@ -0,0 +1,69 @@
|
||||
"""Data models for trade persistence."""
|
||||
from dataclasses import dataclass, asdict
|
||||
from typing import Optional
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
@dataclass
|
||||
class Trade:
|
||||
"""Represents a completed trade."""
|
||||
|
||||
trade_id: str
|
||||
symbol: str
|
||||
side: str
|
||||
entry_price: float
|
||||
size: float
|
||||
size_usdt: float
|
||||
entry_time: str
|
||||
exit_price: Optional[float] = None
|
||||
pnl_usd: Optional[float] = None
|
||||
pnl_pct: Optional[float] = None
|
||||
exit_time: Optional[str] = None
|
||||
hold_duration_hours: Optional[float] = None
|
||||
reason: Optional[str] = None
|
||||
order_id_entry: Optional[str] = None
|
||||
order_id_exit: Optional[str] = None
|
||||
id: Optional[int] = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary."""
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_row(cls, row: tuple, columns: list[str]) -> "Trade":
|
||||
"""Create Trade from database row."""
|
||||
data = dict(zip(columns, row))
|
||||
return cls(**data)
|
||||
|
||||
|
||||
@dataclass
|
||||
class DailySummary:
|
||||
"""Daily trading summary."""
|
||||
|
||||
date: str
|
||||
total_trades: int = 0
|
||||
winning_trades: int = 0
|
||||
total_pnl_usd: float = 0.0
|
||||
max_drawdown_usd: float = 0.0
|
||||
id: Optional[int] = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary."""
|
||||
return asdict(self)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Session:
|
||||
"""Trading session metadata."""
|
||||
|
||||
start_time: str
|
||||
end_time: Optional[str] = None
|
||||
starting_balance: Optional[float] = None
|
||||
ending_balance: Optional[float] = None
|
||||
total_pnl: Optional[float] = None
|
||||
total_trades: int = 0
|
||||
id: Optional[int] = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary."""
|
||||
return asdict(self)
|
||||
15
live_trading/env.template
Normal file
15
live_trading/env.template
Normal file
@@ -0,0 +1,15 @@
|
||||
# OKX API Credentials Template
|
||||
# Copy this file to .env and fill in your credentials
|
||||
# For demo trading, use your OKX demo account API keys
|
||||
# Generate keys at: https://www.okx.com/account/my-api (Demo Trading section)
|
||||
|
||||
OKX_API_KEY=your_api_key_here
|
||||
OKX_SECRET=your_secret_key_here
|
||||
OKX_PASSWORD=your_passphrase_here
|
||||
|
||||
# Demo Mode: Set to "true" for paper trading (sandbox)
|
||||
# Set to "false" for live trading with real funds
|
||||
OKX_DEMO_MODE=true
|
||||
|
||||
# CryptoQuant API (optional, for on-chain features)
|
||||
CRYPTOQUANT_API_KEY=your_cryptoquant_api_key_here
|
||||
395
live_trading/live_regime_strategy.py
Normal file
395
live_trading/live_regime_strategy.py
Normal file
@@ -0,0 +1,395 @@
|
||||
"""
|
||||
Live Regime Reversion Strategy.
|
||||
|
||||
Adapts the backtest regime strategy for live trading.
|
||||
Uses a pre-trained ML model or trains on historical data.
|
||||
"""
|
||||
import logging
|
||||
import pickle
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from sklearn.ensemble import RandomForestClassifier
|
||||
|
||||
from .config import TradingConfig, PathConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class LiveRegimeStrategy:
|
||||
"""
|
||||
Live trading implementation of the ML-based regime detection
|
||||
and mean reversion strategy.
|
||||
|
||||
Logic:
|
||||
1. Calculates BTC/ETH spread Z-Score
|
||||
2. Uses Random Forest to predict reversion probability
|
||||
3. Applies funding rate filter
|
||||
4. Generates long/short signals on ETH perpetual
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
trading_config: TradingConfig,
|
||||
path_config: PathConfig
|
||||
):
|
||||
self.config = trading_config
|
||||
self.paths = path_config
|
||||
self.model: Optional[RandomForestClassifier] = None
|
||||
self.feature_cols: Optional[list] = None
|
||||
self.horizon: int = 54 # Default horizon
|
||||
self._last_model_load_time: float = 0.0
|
||||
self._last_train_time: float = 0.0
|
||||
self._load_or_train_model()
|
||||
|
||||
def reload_model_if_changed(self) -> None:
|
||||
"""Check if model file has changed and reload if necessary."""
|
||||
if not self.paths.model_path.exists():
|
||||
return
|
||||
|
||||
try:
|
||||
mtime = self.paths.model_path.stat().st_mtime
|
||||
if mtime > self._last_model_load_time:
|
||||
logger.info(f"Model file changed, reloading... (last: {self._last_model_load_time}, new: {mtime})")
|
||||
self._load_or_train_model()
|
||||
except Exception as e:
|
||||
logger.warning(f"Error checking model file: {e}")
|
||||
|
||||
def _load_or_train_model(self) -> None:
|
||||
"""Load pre-trained model or train a new one."""
|
||||
if self.paths.model_path.exists():
|
||||
try:
|
||||
self._last_model_load_time = self.paths.model_path.stat().st_mtime
|
||||
with open(self.paths.model_path, 'rb') as f:
|
||||
saved = pickle.load(f)
|
||||
self.model = saved['model']
|
||||
self.feature_cols = saved['feature_cols']
|
||||
|
||||
# Load horizon from metrics if available
|
||||
if 'metrics' in saved and 'horizon' in saved['metrics']:
|
||||
self.horizon = saved['metrics']['horizon']
|
||||
logger.info(f"Loaded model from {self.paths.model_path} (horizon={self.horizon})")
|
||||
else:
|
||||
logger.info(f"Loaded model from {self.paths.model_path} (default horizon={self.horizon})")
|
||||
|
||||
# Load timestamp if available
|
||||
if 'timestamp' in saved:
|
||||
self._last_train_time = saved['timestamp']
|
||||
else:
|
||||
self._last_train_time = self._last_model_load_time
|
||||
|
||||
return
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not load model: {e}")
|
||||
|
||||
logger.info("No pre-trained model found. Will train on first data batch.")
|
||||
|
||||
def save_model(self) -> None:
|
||||
"""Save trained model to file."""
|
||||
if self.model is None:
|
||||
return
|
||||
|
||||
try:
|
||||
with open(self.paths.model_path, 'wb') as f:
|
||||
pickle.dump({
|
||||
'model': self.model,
|
||||
'feature_cols': self.feature_cols,
|
||||
'metrics': {'horizon': self.horizon}, # Save horizon
|
||||
'timestamp': time.time()
|
||||
}, f)
|
||||
logger.info(f"Saved model to {self.paths.model_path}")
|
||||
except Exception as e:
|
||||
logger.error(f"Could not save model: {e}")
|
||||
|
||||
def check_retrain(self, features: pd.DataFrame) -> None:
|
||||
"""Check if model needs retraining (older than 24h)."""
|
||||
if time.time() - self._last_train_time > 24 * 3600:
|
||||
logger.info("Model is older than 24h. Retraining...")
|
||||
self.train_model(features)
|
||||
self._last_train_time = time.time()
|
||||
|
||||
def train_model(self, features: pd.DataFrame) -> None:
|
||||
"""
|
||||
Train the Random Forest model on historical data.
|
||||
|
||||
Args:
|
||||
features: DataFrame with calculated features
|
||||
"""
|
||||
logger.info(f"Training model on {len(features)} samples...")
|
||||
|
||||
z_thresh = self.config.z_entry_threshold
|
||||
horizon = self.horizon
|
||||
profit_target = 0.005 # 0.5% profit threshold
|
||||
stop_loss_pct = self.config.stop_loss_pct
|
||||
|
||||
# Calculate targets path-dependently
|
||||
spread = features['spread'].values
|
||||
z_score = features['z_score'].values
|
||||
n = len(spread)
|
||||
|
||||
targets = np.zeros(n, dtype=int)
|
||||
|
||||
candidates = np.where((z_score > z_thresh) | (z_score < -z_thresh))[0]
|
||||
|
||||
for i in candidates:
|
||||
if i + horizon >= n:
|
||||
continue
|
||||
|
||||
entry_price = spread[i]
|
||||
future_prices = spread[i+1 : i+1+horizon]
|
||||
|
||||
if z_score[i] > z_thresh: # Short
|
||||
target_price = entry_price * (1 - profit_target)
|
||||
stop_price = entry_price * (1 + stop_loss_pct)
|
||||
|
||||
hit_tp = future_prices <= target_price
|
||||
hit_sl = future_prices >= stop_price
|
||||
|
||||
if not np.any(hit_tp):
|
||||
targets[i] = 0
|
||||
elif not np.any(hit_sl):
|
||||
targets[i] = 1
|
||||
else:
|
||||
first_tp_idx = np.argmax(hit_tp)
|
||||
first_sl_idx = np.argmax(hit_sl)
|
||||
if first_tp_idx < first_sl_idx:
|
||||
targets[i] = 1
|
||||
else:
|
||||
targets[i] = 0
|
||||
|
||||
else: # Long
|
||||
target_price = entry_price * (1 + profit_target)
|
||||
stop_price = entry_price * (1 - stop_loss_pct)
|
||||
|
||||
hit_tp = future_prices >= target_price
|
||||
hit_sl = future_prices <= stop_price
|
||||
|
||||
if not np.any(hit_tp):
|
||||
targets[i] = 0
|
||||
elif not np.any(hit_sl):
|
||||
targets[i] = 1
|
||||
else:
|
||||
first_tp_idx = np.argmax(hit_tp)
|
||||
first_sl_idx = np.argmax(hit_sl)
|
||||
if first_tp_idx < first_sl_idx:
|
||||
targets[i] = 1
|
||||
else:
|
||||
targets[i] = 0
|
||||
|
||||
# Exclude non-feature columns
|
||||
exclude = ['spread', 'btc_close', 'eth_close', 'eth_volume']
|
||||
self.feature_cols = [c for c in features.columns if c not in exclude]
|
||||
|
||||
# Clean features
|
||||
X = features[self.feature_cols].fillna(0)
|
||||
X = X.replace([np.inf, -np.inf], 0)
|
||||
|
||||
# Use rows where we had enough data to look ahead
|
||||
valid_mask = np.zeros(n, dtype=bool)
|
||||
valid_mask[:n-horizon] = True
|
||||
|
||||
X_clean = X[valid_mask]
|
||||
y_clean = targets[valid_mask]
|
||||
|
||||
if len(X_clean) < 100:
|
||||
logger.warning("Not enough data to train model")
|
||||
return
|
||||
|
||||
# Train model
|
||||
self.model = RandomForestClassifier(
|
||||
n_estimators=300,
|
||||
max_depth=5,
|
||||
min_samples_leaf=30,
|
||||
class_weight={0: 1, 1: 3},
|
||||
random_state=42
|
||||
)
|
||||
self.model.fit(X_clean, y_clean)
|
||||
|
||||
logger.info(f"Model trained on {len(X_clean)} samples")
|
||||
self.save_model()
|
||||
|
||||
def generate_signal(
|
||||
self,
|
||||
features: pd.DataFrame,
|
||||
current_funding: dict,
|
||||
position_side: Optional[str] = None
|
||||
) -> dict:
|
||||
"""
|
||||
Generate trading signal from latest features.
|
||||
|
||||
Args:
|
||||
features: DataFrame with calculated features
|
||||
current_funding: Dictionary with funding rate data
|
||||
position_side: Current position side ('long', 'short', or None)
|
||||
|
||||
Returns:
|
||||
Signal dictionary with action, side, confidence, etc.
|
||||
"""
|
||||
# Check if retraining is needed
|
||||
self.check_retrain(features)
|
||||
|
||||
if self.model is None:
|
||||
# Train model if not available
|
||||
if len(features) >= 200:
|
||||
self.train_model(features)
|
||||
else:
|
||||
return {'action': 'hold', 'reason': 'model_not_trained'}
|
||||
|
||||
if self.model is None:
|
||||
return {'action': 'hold', 'reason': 'insufficient_data_for_training'}
|
||||
|
||||
# Get latest row
|
||||
latest = features.iloc[-1]
|
||||
z_score = latest['z_score']
|
||||
eth_price = latest['eth_close']
|
||||
btc_price = latest['btc_close']
|
||||
|
||||
# Prepare features for prediction
|
||||
X = features[self.feature_cols].iloc[[-1]].fillna(0)
|
||||
X = X.replace([np.inf, -np.inf], 0)
|
||||
|
||||
# Get prediction probability
|
||||
prob = self.model.predict_proba(X)[0, 1]
|
||||
|
||||
# Apply thresholds
|
||||
z_thresh = self.config.z_entry_threshold
|
||||
prob_thresh = self.config.model_prob_threshold
|
||||
|
||||
# Determine signal direction
|
||||
signal = {
|
||||
'action': 'hold',
|
||||
'side': None,
|
||||
'probability': prob,
|
||||
'z_score': z_score,
|
||||
'eth_price': eth_price,
|
||||
'btc_price': btc_price,
|
||||
'reason': '',
|
||||
}
|
||||
|
||||
# Check for entry conditions
|
||||
if prob > prob_thresh:
|
||||
if z_score > z_thresh:
|
||||
# Spread high (ETH expensive relative to BTC) -> Short ETH
|
||||
signal['action'] = 'entry'
|
||||
signal['side'] = 'short'
|
||||
signal['reason'] = f'z_score={z_score:.2f}>threshold, prob={prob:.2f}'
|
||||
elif z_score < -z_thresh:
|
||||
# Spread low (ETH cheap relative to BTC) -> Long ETH
|
||||
signal['action'] = 'entry'
|
||||
signal['side'] = 'long'
|
||||
signal['reason'] = f'z_score={z_score:.2f}<-threshold, prob={prob:.2f}'
|
||||
else:
|
||||
signal['reason'] = f'z_score={z_score:.2f} within threshold'
|
||||
else:
|
||||
signal['reason'] = f'prob={prob:.2f}<threshold'
|
||||
|
||||
# Apply funding rate filter
|
||||
if signal['action'] == 'entry':
|
||||
btc_funding = current_funding.get('btc_funding', 0)
|
||||
funding_thresh = self.config.funding_threshold
|
||||
|
||||
if signal['side'] == 'long' and btc_funding > funding_thresh:
|
||||
# High positive funding = overheated, don't go long
|
||||
signal['action'] = 'hold'
|
||||
signal['reason'] = f'funding_filter_blocked_long (funding={btc_funding:.4f})'
|
||||
elif signal['side'] == 'short' and btc_funding < -funding_thresh:
|
||||
# High negative funding = oversold, don't go short
|
||||
signal['action'] = 'hold'
|
||||
signal['reason'] = f'funding_filter_blocked_short (funding={btc_funding:.4f})'
|
||||
|
||||
# Check for exit conditions (Overshoot Logic)
|
||||
if signal['action'] == 'hold' and position_side:
|
||||
# Overshoot Logic
|
||||
# If Long, exit if Z > 0.5 (Reverted past 0 to +0.5)
|
||||
if position_side == 'long' and z_score > 0.5:
|
||||
signal['action'] = 'check_exit'
|
||||
signal['reason'] = f'overshoot_exit_long (z={z_score:.2f} > 0.5)'
|
||||
# If Short, exit if Z < -0.5 (Reverted past 0 to -0.5)
|
||||
elif position_side == 'short' and z_score < -0.5:
|
||||
signal['action'] = 'check_exit'
|
||||
signal['reason'] = f'overshoot_exit_short (z={z_score:.2f} < -0.5)'
|
||||
|
||||
logger.info(
|
||||
f"Signal: {signal['action']} {signal['side'] or ''} "
|
||||
f"(prob={prob:.2f}, z={z_score:.2f}, reason={signal['reason']})"
|
||||
)
|
||||
|
||||
return signal
|
||||
|
||||
def calculate_position_size(
|
||||
self,
|
||||
signal: dict,
|
||||
available_usdt: float
|
||||
) -> float:
|
||||
"""
|
||||
Calculate position size based on signal confidence.
|
||||
|
||||
Args:
|
||||
signal: Signal dictionary with probability
|
||||
available_usdt: Available USDT balance
|
||||
|
||||
Returns:
|
||||
Position size in USDT
|
||||
"""
|
||||
prob = signal.get('probability', 0.5)
|
||||
|
||||
# Base size: if max_position_usdt <= 0, use all available funds
|
||||
if self.config.max_position_usdt <= 0:
|
||||
base_size = available_usdt
|
||||
else:
|
||||
base_size = min(available_usdt, self.config.max_position_usdt)
|
||||
|
||||
# Scale by probability (1.0x at 0.5 prob, up to 1.6x at 0.8 prob)
|
||||
scale = 1.0 + (prob - 0.5) * 2.0
|
||||
scale = max(1.0, min(scale, 2.0)) # Clamp between 1x and 2x
|
||||
|
||||
size = base_size * scale
|
||||
|
||||
# Ensure minimum position size
|
||||
if size < self.config.min_position_usdt:
|
||||
return 0.0
|
||||
|
||||
return min(size, available_usdt * 0.95) # Leave 5% buffer
|
||||
|
||||
def calculate_sl_tp(
|
||||
self,
|
||||
entry_price: Optional[float],
|
||||
side: str
|
||||
) -> tuple[Optional[float], Optional[float]]:
|
||||
"""
|
||||
Calculate stop-loss and take-profit prices.
|
||||
|
||||
Args:
|
||||
entry_price: Entry price
|
||||
side: "long" or "short"
|
||||
|
||||
Returns:
|
||||
Tuple of (stop_loss_price, take_profit_price), or (None, None) if
|
||||
entry_price is invalid
|
||||
|
||||
Raises:
|
||||
ValueError: If side is not "long" or "short"
|
||||
"""
|
||||
if entry_price is None or entry_price <= 0:
|
||||
logger.error(
|
||||
f"Invalid entry_price for SL/TP calculation: {entry_price}"
|
||||
)
|
||||
return None, None
|
||||
|
||||
if side not in ("long", "short"):
|
||||
raise ValueError(f"Invalid side: {side}. Must be 'long' or 'short'")
|
||||
|
||||
sl_pct = self.config.stop_loss_pct
|
||||
tp_pct = self.config.take_profit_pct
|
||||
|
||||
if side == "long":
|
||||
stop_loss = entry_price * (1 - sl_pct)
|
||||
take_profit = entry_price * (1 + tp_pct)
|
||||
else: # short
|
||||
stop_loss = entry_price * (1 + sl_pct)
|
||||
take_profit = entry_price * (1 - tp_pct)
|
||||
|
||||
return stop_loss, take_profit
|
||||
576
live_trading/main.py
Normal file
576
live_trading/main.py
Normal file
@@ -0,0 +1,576 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Live Trading Bot for Regime Reversion Strategy on OKX.
|
||||
|
||||
This script runs the regime-based mean reversion strategy
|
||||
on ETH perpetual futures using OKX exchange.
|
||||
|
||||
Usage:
|
||||
# Run with demo account (default)
|
||||
uv run python -m live_trading.main
|
||||
|
||||
# Run with specific settings
|
||||
uv run python -m live_trading.main --max-position 500 --leverage 2
|
||||
|
||||
# Run without UI (headless mode)
|
||||
uv run python -m live_trading.main --no-ui
|
||||
"""
|
||||
import argparse
|
||||
import logging
|
||||
import queue
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
# Add parent directory to path for imports
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from live_trading.config import get_config, OKXConfig, TradingConfig, PathConfig
|
||||
from live_trading.okx_client import OKXClient
|
||||
from live_trading.data_feed import DataFeed
|
||||
from live_trading.position_manager import PositionManager
|
||||
from live_trading.live_regime_strategy import LiveRegimeStrategy
|
||||
from live_trading.db.database import init_db, TradingDatabase
|
||||
from live_trading.db.migrations import run_migrations
|
||||
from live_trading.ui.state import SharedState, PositionState
|
||||
from live_trading.ui.dashboard import TradingDashboard, setup_ui_logging
|
||||
|
||||
|
||||
def setup_logging(
|
||||
log_dir: Path,
|
||||
log_queue: Optional[queue.Queue] = None,
|
||||
) -> logging.Logger:
|
||||
"""
|
||||
Configure logging for the trading bot.
|
||||
|
||||
Args:
|
||||
log_dir: Directory for log files
|
||||
log_queue: Optional queue for UI log handler
|
||||
|
||||
Returns:
|
||||
Logger instance
|
||||
"""
|
||||
log_file = log_dir / "live_trading.log"
|
||||
|
||||
handlers = [
|
||||
logging.FileHandler(log_file),
|
||||
]
|
||||
|
||||
# Only add StreamHandler if no UI (log_queue is None)
|
||||
if log_queue is None:
|
||||
handlers.append(logging.StreamHandler(sys.stdout))
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s [%(levelname)s] %(name)s: %(message)s',
|
||||
handlers=handlers,
|
||||
force=True,
|
||||
)
|
||||
|
||||
# Add UI log handler if queue provided
|
||||
if log_queue is not None:
|
||||
setup_ui_logging(log_queue)
|
||||
|
||||
return logging.getLogger(__name__)
|
||||
|
||||
|
||||
class LiveTradingBot:
|
||||
"""
|
||||
Main trading bot orchestrator.
|
||||
|
||||
Coordinates data fetching, signal generation, and order execution
|
||||
in a continuous loop.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
okx_config: OKXConfig,
|
||||
trading_config: TradingConfig,
|
||||
path_config: PathConfig,
|
||||
database: Optional[TradingDatabase] = None,
|
||||
shared_state: Optional[SharedState] = None,
|
||||
):
|
||||
self.okx_config = okx_config
|
||||
self.trading_config = trading_config
|
||||
self.path_config = path_config
|
||||
self.db = database
|
||||
self.state = shared_state
|
||||
|
||||
self.logger = logging.getLogger(__name__)
|
||||
self.running = True
|
||||
|
||||
# Initialize components
|
||||
self.logger.info("Initializing trading bot components...")
|
||||
|
||||
self.okx_client = OKXClient(okx_config, trading_config)
|
||||
self.data_feed = DataFeed(self.okx_client, trading_config, path_config)
|
||||
self.position_manager = PositionManager(
|
||||
self.okx_client, trading_config, path_config, database
|
||||
)
|
||||
self.strategy = LiveRegimeStrategy(trading_config, path_config)
|
||||
|
||||
# Register signal handlers for graceful shutdown
|
||||
signal.signal(signal.SIGINT, self._handle_shutdown)
|
||||
signal.signal(signal.SIGTERM, self._handle_shutdown)
|
||||
|
||||
# Initialize shared state if provided
|
||||
if self.state:
|
||||
mode = "DEMO" if okx_config.demo_mode else "LIVE"
|
||||
self.state.set_mode(mode)
|
||||
self.state.set_symbols(
|
||||
trading_config.eth_symbol,
|
||||
trading_config.btc_symbol,
|
||||
)
|
||||
self.state.update_account(0.0, 0.0, trading_config.leverage)
|
||||
|
||||
self._print_startup_banner()
|
||||
|
||||
def _print_startup_banner(self) -> None:
|
||||
"""Print startup information."""
|
||||
mode = "DEMO/SANDBOX" if self.okx_config.demo_mode else "LIVE"
|
||||
|
||||
print("=" * 60)
|
||||
print(f" Regime Reversion Strategy - Live Trading Bot")
|
||||
print("=" * 60)
|
||||
print(f" Mode: {mode}")
|
||||
print(f" Trading Pair: {self.trading_config.eth_symbol}")
|
||||
print(f" Context Pair: {self.trading_config.btc_symbol}")
|
||||
print(f" Timeframe: {self.trading_config.timeframe}")
|
||||
print(f" Max Position: ${self.trading_config.max_position_usdt if self.trading_config.max_position_usdt > 0 else 'All available'}")
|
||||
print(f" Leverage: {self.trading_config.leverage}x")
|
||||
print(f" Stop Loss: {self.trading_config.stop_loss_pct * 100:.1f}%")
|
||||
print(f" Take Profit: {self.trading_config.take_profit_pct * 100:.1f}%")
|
||||
print(f" Cycle Interval: {self.trading_config.sleep_seconds // 60} minutes")
|
||||
print("=" * 60)
|
||||
|
||||
if not self.okx_config.demo_mode:
|
||||
print("\n *** WARNING: LIVE TRADING MODE - REAL FUNDS AT RISK ***\n")
|
||||
|
||||
def _handle_shutdown(self, signum, frame) -> None:
|
||||
"""Handle shutdown signals gracefully."""
|
||||
self.logger.info("Shutdown signal received, stopping...")
|
||||
self.running = False
|
||||
if self.state:
|
||||
self.state.stop()
|
||||
|
||||
def run_trading_cycle(self) -> None:
|
||||
"""
|
||||
Execute one trading cycle.
|
||||
|
||||
1. Fetch latest market data
|
||||
2. Update open positions
|
||||
3. Generate trading signal
|
||||
4. Execute trades if signal triggers
|
||||
5. Update shared state for UI
|
||||
"""
|
||||
# Reload model if it has changed (e.g. daily training)
|
||||
try:
|
||||
self.strategy.reload_model_if_changed()
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Failed to reload model: {e}")
|
||||
|
||||
cycle_start = datetime.now(timezone.utc)
|
||||
self.logger.info(f"--- Trading Cycle Start: {cycle_start.isoformat()} ---")
|
||||
|
||||
if self.state:
|
||||
self.state.set_last_cycle_time(cycle_start.isoformat())
|
||||
|
||||
try:
|
||||
# 1. Fetch market data
|
||||
features = self.data_feed.get_latest_data()
|
||||
if features is None or features.empty:
|
||||
self.logger.warning("No market data available, skipping cycle")
|
||||
return
|
||||
|
||||
# Get current prices
|
||||
eth_price = features['eth_close'].iloc[-1]
|
||||
btc_price = features['btc_close'].iloc[-1]
|
||||
|
||||
current_prices = {
|
||||
self.trading_config.eth_symbol: eth_price,
|
||||
self.trading_config.btc_symbol: btc_price,
|
||||
}
|
||||
|
||||
# 2. Update existing positions (check SL/TP)
|
||||
closed_trades = self.position_manager.update_positions(current_prices)
|
||||
if closed_trades:
|
||||
for trade in closed_trades:
|
||||
self.logger.info(
|
||||
f"Trade closed: {trade['trade_id']} "
|
||||
f"PnL=${trade['pnl_usd']:.2f} ({trade['reason']})"
|
||||
)
|
||||
|
||||
# 3. Sync with exchange positions
|
||||
self.position_manager.sync_with_exchange()
|
||||
|
||||
# Get current position side for signal generation
|
||||
symbol = self.trading_config.eth_symbol
|
||||
position = self.position_manager.get_position_for_symbol(symbol)
|
||||
position_side = position.side if position else None
|
||||
|
||||
# 4. Get current funding rates
|
||||
funding = self.data_feed.get_current_funding_rates()
|
||||
|
||||
# 5. Generate trading signal
|
||||
sig = self.strategy.generate_signal(features, funding, position_side=position_side)
|
||||
|
||||
# 6. Update shared state with strategy info
|
||||
self._update_strategy_state(sig, funding)
|
||||
|
||||
# 7. Execute trades based on signal
|
||||
if sig['action'] == 'entry':
|
||||
self._execute_entry(sig, eth_price)
|
||||
elif sig['action'] == 'check_exit':
|
||||
self._execute_exit(sig)
|
||||
|
||||
# 8. Update shared state with position and account
|
||||
self._update_position_state(eth_price)
|
||||
self._update_account_state()
|
||||
|
||||
# 9. Log portfolio summary
|
||||
summary = self.position_manager.get_portfolio_summary()
|
||||
self.logger.info(
|
||||
f"Portfolio: {summary['open_positions']} positions, "
|
||||
f"exposure=${summary['total_exposure_usdt']:.2f}, "
|
||||
f"unrealized_pnl=${summary['total_unrealized_pnl']:.2f}"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Trading cycle error: {e}", exc_info=True)
|
||||
# Save positions on error
|
||||
self.position_manager.save_positions()
|
||||
|
||||
cycle_duration = (datetime.now(timezone.utc) - cycle_start).total_seconds()
|
||||
self.logger.info(f"--- Cycle completed in {cycle_duration:.1f}s ---")
|
||||
|
||||
def _update_strategy_state(self, sig: dict, funding: dict) -> None:
|
||||
"""Update shared state with strategy information."""
|
||||
if not self.state:
|
||||
return
|
||||
|
||||
self.state.update_strategy(
|
||||
z_score=sig.get('z_score', 0.0),
|
||||
probability=sig.get('probability', 0.0),
|
||||
funding_rate=funding.get('btc_funding', 0.0),
|
||||
action=sig.get('action', 'hold'),
|
||||
reason=sig.get('reason', ''),
|
||||
)
|
||||
|
||||
def _update_position_state(self, current_price: float) -> None:
|
||||
"""Update shared state with current position."""
|
||||
if not self.state:
|
||||
return
|
||||
|
||||
symbol = self.trading_config.eth_symbol
|
||||
position = self.position_manager.get_position_for_symbol(symbol)
|
||||
|
||||
if position is None:
|
||||
self.state.clear_position()
|
||||
return
|
||||
|
||||
pos_state = PositionState(
|
||||
trade_id=position.trade_id,
|
||||
symbol=position.symbol,
|
||||
side=position.side,
|
||||
entry_price=position.entry_price,
|
||||
current_price=position.current_price,
|
||||
size=position.size,
|
||||
size_usdt=position.size_usdt,
|
||||
unrealized_pnl=position.unrealized_pnl,
|
||||
unrealized_pnl_pct=position.unrealized_pnl_pct,
|
||||
stop_loss_price=position.stop_loss_price,
|
||||
take_profit_price=position.take_profit_price,
|
||||
)
|
||||
self.state.set_position(pos_state)
|
||||
|
||||
def _update_account_state(self) -> None:
|
||||
"""Update shared state with account information."""
|
||||
if not self.state:
|
||||
return
|
||||
|
||||
try:
|
||||
balance = self.okx_client.get_balance()
|
||||
self.state.update_account(
|
||||
balance=balance.get('total', 0.0),
|
||||
available=balance.get('free', 0.0),
|
||||
leverage=self.trading_config.leverage,
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Failed to update account state: {e}")
|
||||
|
||||
def _execute_entry(self, signal: dict, current_price: float) -> None:
|
||||
"""Execute entry trade."""
|
||||
symbol = self.trading_config.eth_symbol
|
||||
side = signal['side']
|
||||
|
||||
# Check if we can open a position
|
||||
if not self.position_manager.can_open_position():
|
||||
self.logger.info("Cannot open position: max positions reached")
|
||||
return
|
||||
|
||||
# Get account balance
|
||||
balance = self.okx_client.get_balance()
|
||||
available_usdt = balance['free']
|
||||
self.logger.info(f"Account balance: ${available_usdt:.2f} USDT available")
|
||||
|
||||
# Calculate position size
|
||||
size_usdt = self.strategy.calculate_position_size(signal, available_usdt)
|
||||
if size_usdt <= 0:
|
||||
self.logger.info(
|
||||
f"Position size too small (${size_usdt:.2f}), skipping entry. "
|
||||
f"Min required: ${self.strategy.config.min_position_usdt:.2f}"
|
||||
)
|
||||
return
|
||||
|
||||
size_eth = size_usdt / current_price
|
||||
|
||||
# Calculate SL/TP for logging
|
||||
stop_loss, take_profit = self.strategy.calculate_sl_tp(current_price, side)
|
||||
|
||||
sl_str = f"{stop_loss:.2f}" if stop_loss else "N/A"
|
||||
tp_str = f"{take_profit:.2f}" if take_profit else "N/A"
|
||||
self.logger.info(
|
||||
f"Executing {side.upper()} entry: {size_eth:.4f} ETH @ {current_price:.2f} "
|
||||
f"(${size_usdt:.2f}), SL={sl_str}, TP={tp_str}"
|
||||
)
|
||||
|
||||
try:
|
||||
# Place market order (guaranteed to have fill price or raises)
|
||||
order_side = "buy" if side == "long" else "sell"
|
||||
order = self.okx_client.place_market_order(symbol, order_side, size_eth)
|
||||
|
||||
# Get filled price and amount (guaranteed by OKX client)
|
||||
filled_price = order['average']
|
||||
filled_amount = order.get('filled') or size_eth
|
||||
|
||||
# Calculate SL/TP with filled price
|
||||
stop_loss, take_profit = self.strategy.calculate_sl_tp(filled_price, side)
|
||||
|
||||
if stop_loss is None or take_profit is None:
|
||||
raise RuntimeError(
|
||||
f"Failed to calculate SL/TP: filled_price={filled_price}, side={side}"
|
||||
)
|
||||
|
||||
# Get order ID from response
|
||||
order_id = order.get('id', '')
|
||||
|
||||
# Record position locally
|
||||
position = self.position_manager.open_position(
|
||||
symbol=symbol,
|
||||
side=side,
|
||||
entry_price=filled_price,
|
||||
size=filled_amount,
|
||||
stop_loss_price=stop_loss,
|
||||
take_profit_price=take_profit,
|
||||
order_id=order_id,
|
||||
)
|
||||
|
||||
if position:
|
||||
self.logger.info(
|
||||
f"Position opened: {position.trade_id}, "
|
||||
f"{filled_amount:.4f} ETH @ {filled_price:.2f}"
|
||||
)
|
||||
|
||||
# Try to set SL/TP on exchange
|
||||
try:
|
||||
self.okx_client.set_stop_loss_take_profit(
|
||||
symbol, side, filled_amount, stop_loss, take_profit
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Could not set SL/TP on exchange: {e}")
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Order execution failed: {e}", exc_info=True)
|
||||
|
||||
def _execute_exit(self, signal: dict) -> None:
|
||||
"""Execute exit based on mean reversion signal."""
|
||||
symbol = self.trading_config.eth_symbol
|
||||
|
||||
# Get position for ETH
|
||||
position = self.position_manager.get_position_for_symbol(symbol)
|
||||
if not position:
|
||||
return
|
||||
|
||||
current_price = signal.get('eth_price', position.current_price)
|
||||
|
||||
self.logger.info(
|
||||
f"Mean reversion exit signal: closing {position.trade_id} "
|
||||
f"@ {current_price:.2f}"
|
||||
)
|
||||
|
||||
try:
|
||||
# Close position on exchange
|
||||
exit_order = self.okx_client.close_position(symbol)
|
||||
exit_order_id = exit_order.get('id', '') if exit_order else ''
|
||||
|
||||
# Record closure locally
|
||||
self.position_manager.close_position(
|
||||
position.trade_id,
|
||||
current_price,
|
||||
reason="mean_reversion_complete",
|
||||
exit_order_id=exit_order_id,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"Exit execution failed: {e}", exc_info=True)
|
||||
|
||||
def _is_running(self) -> bool:
|
||||
"""Check if bot should continue running."""
|
||||
if not self.running:
|
||||
return False
|
||||
if self.state and not self.state.is_running():
|
||||
return False
|
||||
return True
|
||||
|
||||
def run(self) -> None:
|
||||
"""Main trading loop."""
|
||||
self.logger.info("Starting trading loop...")
|
||||
|
||||
while self._is_running():
|
||||
try:
|
||||
self.run_trading_cycle()
|
||||
|
||||
if self._is_running():
|
||||
sleep_seconds = self.trading_config.sleep_seconds
|
||||
minutes = sleep_seconds // 60
|
||||
self.logger.info(f"Sleeping for {minutes} minutes...")
|
||||
|
||||
# Sleep in smaller chunks to allow faster shutdown
|
||||
for _ in range(sleep_seconds):
|
||||
if not self._is_running():
|
||||
break
|
||||
time.sleep(1)
|
||||
|
||||
except KeyboardInterrupt:
|
||||
self.logger.info("Keyboard interrupt received")
|
||||
break
|
||||
except Exception as e:
|
||||
self.logger.error(f"Unexpected error in main loop: {e}", exc_info=True)
|
||||
time.sleep(60) # Wait before retry
|
||||
|
||||
# Cleanup
|
||||
self.logger.info("Shutting down...")
|
||||
self.position_manager.save_positions()
|
||||
if self.db:
|
||||
self.db.close()
|
||||
self.logger.info("Shutdown complete")
|
||||
|
||||
|
||||
def parse_args():
|
||||
"""Parse command line arguments."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Live Trading Bot for Regime Reversion Strategy"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-position",
|
||||
type=float,
|
||||
default=None,
|
||||
help="Maximum position size in USDT"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--leverage",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Trading leverage (1-125)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--interval",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Trading cycle interval in seconds"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--live",
|
||||
action="store_true",
|
||||
help="Use live trading mode (requires OKX_DEMO_MODE=false)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-ui",
|
||||
action="store_true",
|
||||
help="Run in headless mode without terminal UI"
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point."""
|
||||
args = parse_args()
|
||||
|
||||
# Load configuration
|
||||
okx_config, trading_config, path_config = get_config()
|
||||
|
||||
# Apply command line overrides
|
||||
if args.max_position is not None:
|
||||
trading_config.max_position_usdt = args.max_position
|
||||
if args.leverage is not None:
|
||||
trading_config.leverage = args.leverage
|
||||
if args.interval is not None:
|
||||
trading_config.sleep_seconds = args.interval
|
||||
if args.live:
|
||||
okx_config.demo_mode = False
|
||||
|
||||
# Determine if UI should be enabled
|
||||
use_ui = not args.no_ui and sys.stdin.isatty()
|
||||
|
||||
# Initialize database
|
||||
db_path = path_config.base_dir / "live_trading" / "trading.db"
|
||||
db = init_db(db_path)
|
||||
|
||||
# Run migrations (imports CSV if exists)
|
||||
run_migrations(db, path_config.trade_log_file)
|
||||
|
||||
# Initialize UI components if enabled
|
||||
log_queue: Optional[queue.Queue] = None
|
||||
shared_state: Optional[SharedState] = None
|
||||
dashboard: Optional[TradingDashboard] = None
|
||||
|
||||
if use_ui:
|
||||
log_queue = queue.Queue(maxsize=1000)
|
||||
shared_state = SharedState()
|
||||
|
||||
# Setup logging
|
||||
logger = setup_logging(path_config.logs_dir, log_queue)
|
||||
|
||||
try:
|
||||
# Create bot
|
||||
bot = LiveTradingBot(
|
||||
okx_config,
|
||||
trading_config,
|
||||
path_config,
|
||||
database=db,
|
||||
shared_state=shared_state,
|
||||
)
|
||||
|
||||
# Start dashboard if UI enabled
|
||||
if use_ui and shared_state and log_queue:
|
||||
dashboard = TradingDashboard(
|
||||
state=shared_state,
|
||||
db=db,
|
||||
log_queue=log_queue,
|
||||
on_quit=lambda: setattr(bot, 'running', False),
|
||||
)
|
||||
dashboard.start()
|
||||
logger.info("Dashboard started")
|
||||
|
||||
# Run bot
|
||||
bot.run()
|
||||
|
||||
except ValueError as e:
|
||||
logger.error(f"Configuration error: {e}")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
logger.error(f"Fatal error: {e}", exc_info=True)
|
||||
sys.exit(1)
|
||||
finally:
|
||||
# Cleanup
|
||||
if dashboard:
|
||||
dashboard.stop()
|
||||
if db:
|
||||
db.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
379
live_trading/okx_client.py
Normal file
379
live_trading/okx_client.py
Normal file
@@ -0,0 +1,379 @@
|
||||
"""
|
||||
OKX Exchange Client for Live Trading.
|
||||
|
||||
Handles connection to OKX API, order execution, and account management.
|
||||
Supports demo/sandbox mode for paper trading.
|
||||
"""
|
||||
import logging
|
||||
from typing import Optional
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import ccxt
|
||||
|
||||
from .config import OKXConfig, TradingConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class OKXClient:
|
||||
"""
|
||||
OKX Exchange client wrapper using CCXT.
|
||||
|
||||
Supports both live and demo (sandbox) trading modes.
|
||||
Demo mode uses OKX's official sandbox environment.
|
||||
"""
|
||||
|
||||
def __init__(self, okx_config: OKXConfig, trading_config: TradingConfig):
|
||||
self.okx_config = okx_config
|
||||
self.trading_config = trading_config
|
||||
self.exchange: Optional[ccxt.okx] = None
|
||||
self._setup_exchange()
|
||||
|
||||
def _setup_exchange(self) -> None:
|
||||
"""Initialize CCXT OKX exchange instance."""
|
||||
self.okx_config.validate()
|
||||
|
||||
config = {
|
||||
'apiKey': self.okx_config.api_key,
|
||||
'secret': self.okx_config.secret,
|
||||
'password': self.okx_config.password,
|
||||
'sandbox': self.okx_config.demo_mode,
|
||||
'options': {
|
||||
'defaultType': 'swap', # Perpetual futures
|
||||
},
|
||||
'timeout': 30000,
|
||||
'enableRateLimit': True,
|
||||
}
|
||||
|
||||
self.exchange = ccxt.okx(config)
|
||||
|
||||
mode_str = "DEMO/SANDBOX" if self.okx_config.demo_mode else "LIVE"
|
||||
logger.info(f"OKX Exchange initialized in {mode_str} mode")
|
||||
|
||||
# Configure trading settings
|
||||
self._configure_trading_settings()
|
||||
|
||||
def _configure_trading_settings(self) -> None:
|
||||
"""Configure leverage and margin mode."""
|
||||
symbol = self.trading_config.eth_symbol
|
||||
leverage = self.trading_config.leverage
|
||||
margin_mode = self.trading_config.margin_mode
|
||||
|
||||
try:
|
||||
# Set position mode to one-way (net) first
|
||||
self.exchange.set_position_mode(False) # False = one-way mode
|
||||
logger.info("Position mode set to One-Way (Net)")
|
||||
except Exception as e:
|
||||
# Position mode might already be set
|
||||
logger.debug(f"Position mode setting: {e}")
|
||||
|
||||
try:
|
||||
# Set margin mode with leverage parameter (required by OKX)
|
||||
self.exchange.set_margin_mode(
|
||||
margin_mode,
|
||||
symbol,
|
||||
params={'lever': leverage}
|
||||
)
|
||||
logger.info(
|
||||
f"Margin mode set to {margin_mode} with {leverage}x leverage "
|
||||
f"for {symbol}"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not set margin mode: {e}")
|
||||
# Try setting leverage separately
|
||||
try:
|
||||
self.exchange.set_leverage(leverage, symbol)
|
||||
logger.info(f"Leverage set to {leverage}x for {symbol}")
|
||||
except Exception as e2:
|
||||
logger.warning(f"Could not set leverage: {e2}")
|
||||
|
||||
def fetch_ohlcv(
|
||||
self,
|
||||
symbol: str,
|
||||
timeframe: str = "1h",
|
||||
limit: int = 500
|
||||
) -> list:
|
||||
"""
|
||||
Fetch OHLCV candle data.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol (e.g., "ETH/USDT:USDT")
|
||||
timeframe: Candle timeframe (e.g., "1h")
|
||||
limit: Number of candles to fetch
|
||||
|
||||
Returns:
|
||||
List of OHLCV data
|
||||
"""
|
||||
return self.exchange.fetch_ohlcv(symbol, timeframe, limit=limit)
|
||||
|
||||
def get_balance(self) -> dict:
|
||||
"""
|
||||
Get account balance.
|
||||
|
||||
Returns:
|
||||
Balance dictionary with 'total' and 'free' USDT amounts
|
||||
"""
|
||||
balance = self.exchange.fetch_balance()
|
||||
return {
|
||||
'total': balance.get('USDT', {}).get('total', 0),
|
||||
'free': balance.get('USDT', {}).get('free', 0),
|
||||
}
|
||||
|
||||
def get_positions(self) -> list:
|
||||
"""
|
||||
Get open positions.
|
||||
|
||||
Returns:
|
||||
List of open position dictionaries
|
||||
"""
|
||||
positions = self.exchange.fetch_positions()
|
||||
return [p for p in positions if float(p.get('contracts', 0)) != 0]
|
||||
|
||||
def get_position(self, symbol: str) -> Optional[dict]:
|
||||
"""
|
||||
Get position for a specific symbol.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol
|
||||
|
||||
Returns:
|
||||
Position dictionary or None if no position
|
||||
"""
|
||||
positions = self.get_positions()
|
||||
for pos in positions:
|
||||
if pos.get('symbol') == symbol:
|
||||
return pos
|
||||
return None
|
||||
|
||||
def place_market_order(
|
||||
self,
|
||||
symbol: str,
|
||||
side: str,
|
||||
amount: float,
|
||||
reduce_only: bool = False
|
||||
) -> dict:
|
||||
"""
|
||||
Place a market order and fetch the fill price.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol
|
||||
side: "buy" or "sell"
|
||||
amount: Order amount in base currency
|
||||
reduce_only: If True, only reduce existing position
|
||||
|
||||
Returns:
|
||||
Order result dictionary with guaranteed 'average' fill price
|
||||
|
||||
Raises:
|
||||
RuntimeError: If order placement fails or fill price unavailable
|
||||
"""
|
||||
params = {
|
||||
'tdMode': self.trading_config.margin_mode,
|
||||
}
|
||||
if reduce_only:
|
||||
params['reduceOnly'] = True
|
||||
|
||||
order = self.exchange.create_market_order(
|
||||
symbol, side, amount, params=params
|
||||
)
|
||||
|
||||
order_id = order.get('id')
|
||||
if not order_id:
|
||||
raise RuntimeError(f"Order placement failed: no order ID returned")
|
||||
|
||||
logger.info(
|
||||
f"Market {side.upper()} order placed: {amount} {symbol} "
|
||||
f"@ market price, order_id={order_id}"
|
||||
)
|
||||
|
||||
# Fetch order to get actual fill price if not in initial response
|
||||
fill_price = order.get('average')
|
||||
if fill_price is None or fill_price == 0:
|
||||
logger.info(f"Fetching order {order_id} for fill price...")
|
||||
try:
|
||||
fetched_order = self.exchange.fetch_order(order_id, symbol)
|
||||
fill_price = fetched_order.get('average')
|
||||
order['average'] = fill_price
|
||||
order['filled'] = fetched_order.get('filled', order.get('filled'))
|
||||
order['status'] = fetched_order.get('status', order.get('status'))
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not fetch order details: {e}")
|
||||
|
||||
# Final fallback: use current ticker price
|
||||
if fill_price is None or fill_price == 0:
|
||||
logger.warning(
|
||||
f"No fill price from order response, fetching ticker..."
|
||||
)
|
||||
try:
|
||||
ticker = self.get_ticker(symbol)
|
||||
fill_price = ticker.get('last')
|
||||
order['average'] = fill_price
|
||||
except Exception as e:
|
||||
logger.error(f"Could not fetch ticker: {e}")
|
||||
|
||||
if fill_price is None or fill_price <= 0:
|
||||
raise RuntimeError(
|
||||
f"Could not determine fill price for order {order_id}. "
|
||||
f"Order response: {order}"
|
||||
)
|
||||
|
||||
logger.info(f"Order {order_id} filled at {fill_price}")
|
||||
return order
|
||||
|
||||
def place_limit_order(
|
||||
self,
|
||||
symbol: str,
|
||||
side: str,
|
||||
amount: float,
|
||||
price: float,
|
||||
reduce_only: bool = False
|
||||
) -> dict:
|
||||
"""
|
||||
Place a limit order.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol
|
||||
side: "buy" or "sell"
|
||||
amount: Order amount in base currency
|
||||
price: Limit price
|
||||
reduce_only: If True, only reduce existing position
|
||||
|
||||
Returns:
|
||||
Order result dictionary
|
||||
"""
|
||||
params = {
|
||||
'tdMode': self.trading_config.margin_mode,
|
||||
}
|
||||
if reduce_only:
|
||||
params['reduceOnly'] = True
|
||||
|
||||
order = self.exchange.create_limit_order(
|
||||
symbol, side, amount, price, params=params
|
||||
)
|
||||
logger.info(
|
||||
f"Limit {side.upper()} order placed: {amount} {symbol} "
|
||||
f"@ {price}, order_id={order['id']}"
|
||||
)
|
||||
return order
|
||||
|
||||
def set_stop_loss_take_profit(
|
||||
self,
|
||||
symbol: str,
|
||||
side: str,
|
||||
amount: float,
|
||||
stop_loss_price: float,
|
||||
take_profit_price: float
|
||||
) -> tuple:
|
||||
"""
|
||||
Set stop-loss and take-profit orders.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol
|
||||
side: Position side ("long" or "short")
|
||||
amount: Position size
|
||||
stop_loss_price: Stop-loss trigger price
|
||||
take_profit_price: Take-profit trigger price
|
||||
|
||||
Returns:
|
||||
Tuple of (sl_order, tp_order)
|
||||
"""
|
||||
# For long position: SL sells, TP sells
|
||||
# For short position: SL buys, TP buys
|
||||
close_side = "sell" if side == "long" else "buy"
|
||||
|
||||
# Stop-loss order
|
||||
sl_params = {
|
||||
'tdMode': self.trading_config.margin_mode,
|
||||
'reduceOnly': True,
|
||||
'stopLossPrice': stop_loss_price,
|
||||
}
|
||||
|
||||
try:
|
||||
sl_order = self.exchange.create_order(
|
||||
symbol, 'market', close_side, amount,
|
||||
params={
|
||||
'tdMode': self.trading_config.margin_mode,
|
||||
'reduceOnly': True,
|
||||
'slTriggerPx': str(stop_loss_price),
|
||||
'slOrdPx': '-1', # Market price
|
||||
}
|
||||
)
|
||||
logger.info(f"Stop-loss set at {stop_loss_price}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not set stop-loss: {e}")
|
||||
sl_order = None
|
||||
|
||||
# Take-profit order
|
||||
try:
|
||||
tp_order = self.exchange.create_order(
|
||||
symbol, 'market', close_side, amount,
|
||||
params={
|
||||
'tdMode': self.trading_config.margin_mode,
|
||||
'reduceOnly': True,
|
||||
'tpTriggerPx': str(take_profit_price),
|
||||
'tpOrdPx': '-1', # Market price
|
||||
}
|
||||
)
|
||||
logger.info(f"Take-profit set at {take_profit_price}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not set take-profit: {e}")
|
||||
tp_order = None
|
||||
|
||||
return sl_order, tp_order
|
||||
|
||||
def close_position(self, symbol: str) -> Optional[dict]:
|
||||
"""
|
||||
Close an open position.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol
|
||||
|
||||
Returns:
|
||||
Order result or None if no position
|
||||
"""
|
||||
position = self.get_position(symbol)
|
||||
if not position:
|
||||
logger.info(f"No open position for {symbol}")
|
||||
return None
|
||||
|
||||
contracts = abs(float(position.get('contracts', 0)))
|
||||
if contracts == 0:
|
||||
return None
|
||||
|
||||
side = position.get('side', 'long')
|
||||
close_side = "sell" if side == "long" else "buy"
|
||||
|
||||
order = self.place_market_order(
|
||||
symbol, close_side, contracts, reduce_only=True
|
||||
)
|
||||
logger.info(f"Position closed for {symbol}")
|
||||
return order
|
||||
|
||||
def get_ticker(self, symbol: str) -> dict:
|
||||
"""
|
||||
Get current ticker/price for a symbol.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol
|
||||
|
||||
Returns:
|
||||
Ticker dictionary with 'last', 'bid', 'ask' prices
|
||||
"""
|
||||
return self.exchange.fetch_ticker(symbol)
|
||||
|
||||
def get_funding_rate(self, symbol: str) -> float:
|
||||
"""
|
||||
Get current funding rate for a perpetual symbol.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol
|
||||
|
||||
Returns:
|
||||
Current funding rate as decimal
|
||||
"""
|
||||
try:
|
||||
funding = self.exchange.fetch_funding_rate(symbol)
|
||||
return float(funding.get('fundingRate', 0))
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not fetch funding rate: {e}")
|
||||
return 0.0
|
||||
415
live_trading/position_manager.py
Normal file
415
live_trading/position_manager.py
Normal file
@@ -0,0 +1,415 @@
|
||||
"""
|
||||
Position Manager for Live Trading.
|
||||
|
||||
Tracks open positions, manages risk, and handles SL/TP logic.
|
||||
"""
|
||||
import csv
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional, TYPE_CHECKING
|
||||
from dataclasses import dataclass, asdict
|
||||
|
||||
from .okx_client import OKXClient
|
||||
from .config import TradingConfig, PathConfig
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .db.database import TradingDatabase
|
||||
from .db.models import Trade
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Position:
|
||||
"""Represents an open trading position."""
|
||||
trade_id: str
|
||||
symbol: str
|
||||
side: str # "long" or "short"
|
||||
entry_price: float
|
||||
entry_time: str # ISO format
|
||||
size: float # Amount in base currency (e.g., ETH)
|
||||
size_usdt: float # Notional value in USDT
|
||||
stop_loss_price: float
|
||||
take_profit_price: float
|
||||
current_price: float = 0.0
|
||||
unrealized_pnl: float = 0.0
|
||||
unrealized_pnl_pct: float = 0.0
|
||||
order_id: str = "" # Entry order ID from exchange
|
||||
|
||||
def update_pnl(self, current_price: float) -> None:
|
||||
"""Update unrealized PnL based on current price."""
|
||||
self.current_price = current_price
|
||||
|
||||
if self.side == "long":
|
||||
self.unrealized_pnl = (current_price - self.entry_price) * self.size
|
||||
self.unrealized_pnl_pct = (current_price / self.entry_price - 1) * 100
|
||||
else: # short
|
||||
self.unrealized_pnl = (self.entry_price - current_price) * self.size
|
||||
self.unrealized_pnl_pct = (1 - current_price / self.entry_price) * 100
|
||||
|
||||
def should_stop_loss(self, current_price: float) -> bool:
|
||||
"""Check if stop-loss should trigger."""
|
||||
if self.side == "long":
|
||||
return current_price <= self.stop_loss_price
|
||||
return current_price >= self.stop_loss_price
|
||||
|
||||
def should_take_profit(self, current_price: float) -> bool:
|
||||
"""Check if take-profit should trigger."""
|
||||
if self.side == "long":
|
||||
return current_price >= self.take_profit_price
|
||||
return current_price <= self.take_profit_price
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
"""Convert to dictionary for JSON serialization."""
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict) -> 'Position':
|
||||
"""Create Position from dictionary."""
|
||||
return cls(**data)
|
||||
|
||||
|
||||
class PositionManager:
|
||||
"""
|
||||
Manages trading positions with persistence.
|
||||
|
||||
Tracks open positions, enforces risk limits, and handles
|
||||
position lifecycle (open, update, close).
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
okx_client: OKXClient,
|
||||
trading_config: TradingConfig,
|
||||
path_config: PathConfig,
|
||||
database: Optional["TradingDatabase"] = None,
|
||||
):
|
||||
self.client = okx_client
|
||||
self.config = trading_config
|
||||
self.paths = path_config
|
||||
self.db = database
|
||||
self.positions: dict[str, Position] = {}
|
||||
self.trade_log: list[dict] = []
|
||||
self._load_positions()
|
||||
|
||||
def _load_positions(self) -> None:
|
||||
"""Load positions from file."""
|
||||
if self.paths.positions_file.exists():
|
||||
try:
|
||||
with open(self.paths.positions_file, 'r') as f:
|
||||
data = json.load(f)
|
||||
for trade_id, pos_data in data.items():
|
||||
self.positions[trade_id] = Position.from_dict(pos_data)
|
||||
logger.info(f"Loaded {len(self.positions)} positions from file")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not load positions: {e}")
|
||||
|
||||
def save_positions(self) -> None:
|
||||
"""Save positions to file."""
|
||||
try:
|
||||
data = {
|
||||
trade_id: pos.to_dict()
|
||||
for trade_id, pos in self.positions.items()
|
||||
}
|
||||
with open(self.paths.positions_file, 'w') as f:
|
||||
json.dump(data, f, indent=2)
|
||||
logger.debug(f"Saved {len(self.positions)} positions")
|
||||
except Exception as e:
|
||||
logger.error(f"Could not save positions: {e}")
|
||||
|
||||
def can_open_position(self) -> bool:
|
||||
"""Check if we can open a new position."""
|
||||
return len(self.positions) < self.config.max_concurrent_positions
|
||||
|
||||
def get_position_for_symbol(self, symbol: str) -> Optional[Position]:
|
||||
"""Get position for a specific symbol."""
|
||||
for pos in self.positions.values():
|
||||
if pos.symbol == symbol:
|
||||
return pos
|
||||
return None
|
||||
|
||||
def open_position(
|
||||
self,
|
||||
symbol: str,
|
||||
side: str,
|
||||
entry_price: float,
|
||||
size: float,
|
||||
stop_loss_price: float,
|
||||
take_profit_price: float,
|
||||
order_id: str = ""
|
||||
) -> Optional[Position]:
|
||||
"""
|
||||
Open a new position.
|
||||
|
||||
Args:
|
||||
symbol: Trading pair symbol
|
||||
side: "long" or "short"
|
||||
entry_price: Entry price
|
||||
size: Position size in base currency
|
||||
stop_loss_price: Stop-loss price
|
||||
take_profit_price: Take-profit price
|
||||
order_id: Entry order ID from exchange
|
||||
|
||||
Returns:
|
||||
Position object or None if failed
|
||||
"""
|
||||
if not self.can_open_position():
|
||||
logger.warning("Cannot open position: max concurrent positions reached")
|
||||
return None
|
||||
|
||||
# Check if already have position for this symbol
|
||||
existing = self.get_position_for_symbol(symbol)
|
||||
if existing:
|
||||
logger.warning(f"Already have position for {symbol}")
|
||||
return None
|
||||
|
||||
# Generate trade ID
|
||||
now = datetime.now(timezone.utc)
|
||||
trade_id = f"{symbol}_{now.strftime('%Y%m%d_%H%M%S')}"
|
||||
|
||||
position = Position(
|
||||
trade_id=trade_id,
|
||||
symbol=symbol,
|
||||
side=side,
|
||||
entry_price=entry_price,
|
||||
entry_time=now.isoformat(),
|
||||
size=size,
|
||||
size_usdt=entry_price * size,
|
||||
stop_loss_price=stop_loss_price,
|
||||
take_profit_price=take_profit_price,
|
||||
current_price=entry_price,
|
||||
order_id=order_id,
|
||||
)
|
||||
|
||||
self.positions[trade_id] = position
|
||||
self.save_positions()
|
||||
|
||||
logger.info(
|
||||
f"Opened {side.upper()} position: {size} {symbol} @ {entry_price}, "
|
||||
f"SL={stop_loss_price}, TP={take_profit_price}"
|
||||
)
|
||||
|
||||
return position
|
||||
|
||||
def close_position(
|
||||
self,
|
||||
trade_id: str,
|
||||
exit_price: float,
|
||||
reason: str = "manual",
|
||||
exit_order_id: str = ""
|
||||
) -> Optional[dict]:
|
||||
"""
|
||||
Close a position and record the trade.
|
||||
|
||||
Args:
|
||||
trade_id: Position trade ID
|
||||
exit_price: Exit price
|
||||
reason: Reason for closing (e.g., "stop_loss", "take_profit", "signal")
|
||||
exit_order_id: Exit order ID from exchange
|
||||
|
||||
Returns:
|
||||
Trade record dictionary
|
||||
"""
|
||||
if trade_id not in self.positions:
|
||||
logger.warning(f"Position {trade_id} not found")
|
||||
return None
|
||||
|
||||
position = self.positions[trade_id]
|
||||
position.update_pnl(exit_price)
|
||||
|
||||
# Calculate final PnL
|
||||
entry_time = datetime.fromisoformat(position.entry_time)
|
||||
exit_time = datetime.now(timezone.utc)
|
||||
hold_duration = (exit_time - entry_time).total_seconds() / 3600 # hours
|
||||
|
||||
trade_record = {
|
||||
'trade_id': trade_id,
|
||||
'symbol': position.symbol,
|
||||
'side': position.side,
|
||||
'entry_price': position.entry_price,
|
||||
'exit_price': exit_price,
|
||||
'size': position.size,
|
||||
'size_usdt': position.size_usdt,
|
||||
'pnl_usd': position.unrealized_pnl,
|
||||
'pnl_pct': position.unrealized_pnl_pct,
|
||||
'entry_time': position.entry_time,
|
||||
'exit_time': exit_time.isoformat(),
|
||||
'hold_duration_hours': hold_duration,
|
||||
'reason': reason,
|
||||
'order_id_entry': position.order_id,
|
||||
'order_id_exit': exit_order_id,
|
||||
}
|
||||
|
||||
self.trade_log.append(trade_record)
|
||||
del self.positions[trade_id]
|
||||
self.save_positions()
|
||||
self._append_trade_log(trade_record)
|
||||
|
||||
logger.info(
|
||||
f"Closed {position.side.upper()} position: {position.size} {position.symbol} "
|
||||
f"@ {exit_price}, PnL=${position.unrealized_pnl:.2f} ({position.unrealized_pnl_pct:.2f}%), "
|
||||
f"reason={reason}"
|
||||
)
|
||||
|
||||
return trade_record
|
||||
|
||||
def _append_trade_log(self, trade_record: dict) -> None:
|
||||
"""Append trade record to CSV and SQLite database."""
|
||||
# Write to CSV (backup/compatibility)
|
||||
self._append_trade_csv(trade_record)
|
||||
|
||||
# Write to SQLite (primary)
|
||||
self._append_trade_db(trade_record)
|
||||
|
||||
def _append_trade_csv(self, trade_record: dict) -> None:
|
||||
"""Append trade record to CSV log file."""
|
||||
file_exists = self.paths.trade_log_file.exists()
|
||||
|
||||
try:
|
||||
with open(self.paths.trade_log_file, 'a', newline='') as f:
|
||||
writer = csv.DictWriter(f, fieldnames=trade_record.keys())
|
||||
if not file_exists:
|
||||
writer.writeheader()
|
||||
writer.writerow(trade_record)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to write trade to CSV: {e}")
|
||||
|
||||
def _append_trade_db(self, trade_record: dict) -> None:
|
||||
"""Append trade record to SQLite database."""
|
||||
if self.db is None:
|
||||
return
|
||||
|
||||
try:
|
||||
from .db.models import Trade
|
||||
|
||||
trade = Trade(
|
||||
trade_id=trade_record['trade_id'],
|
||||
symbol=trade_record['symbol'],
|
||||
side=trade_record['side'],
|
||||
entry_price=trade_record['entry_price'],
|
||||
exit_price=trade_record.get('exit_price'),
|
||||
size=trade_record['size'],
|
||||
size_usdt=trade_record['size_usdt'],
|
||||
pnl_usd=trade_record.get('pnl_usd'),
|
||||
pnl_pct=trade_record.get('pnl_pct'),
|
||||
entry_time=trade_record['entry_time'],
|
||||
exit_time=trade_record.get('exit_time'),
|
||||
hold_duration_hours=trade_record.get('hold_duration_hours'),
|
||||
reason=trade_record.get('reason'),
|
||||
order_id_entry=trade_record.get('order_id_entry'),
|
||||
order_id_exit=trade_record.get('order_id_exit'),
|
||||
)
|
||||
self.db.insert_trade(trade)
|
||||
logger.debug(f"Trade {trade.trade_id} saved to database")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to write trade to database: {e}")
|
||||
|
||||
def update_positions(self, current_prices: dict[str, float]) -> list[dict]:
|
||||
"""
|
||||
Update all positions with current prices and check SL/TP.
|
||||
|
||||
Args:
|
||||
current_prices: Dictionary of symbol -> current price
|
||||
|
||||
Returns:
|
||||
List of closed trade records
|
||||
"""
|
||||
closed_trades = []
|
||||
|
||||
for trade_id in list(self.positions.keys()):
|
||||
position = self.positions[trade_id]
|
||||
|
||||
if position.symbol not in current_prices:
|
||||
continue
|
||||
|
||||
current_price = current_prices[position.symbol]
|
||||
position.update_pnl(current_price)
|
||||
|
||||
# Check stop-loss
|
||||
if position.should_stop_loss(current_price):
|
||||
logger.warning(
|
||||
f"Stop-loss triggered for {trade_id} at {current_price}"
|
||||
)
|
||||
# Close position on exchange
|
||||
exit_order_id = ""
|
||||
try:
|
||||
exit_order = self.client.close_position(position.symbol)
|
||||
exit_order_id = exit_order.get('id', '') if exit_order else ''
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to close position on exchange: {e}")
|
||||
|
||||
record = self.close_position(trade_id, current_price, "stop_loss", exit_order_id)
|
||||
if record:
|
||||
closed_trades.append(record)
|
||||
continue
|
||||
|
||||
# Check take-profit
|
||||
if position.should_take_profit(current_price):
|
||||
logger.info(
|
||||
f"Take-profit triggered for {trade_id} at {current_price}"
|
||||
)
|
||||
# Close position on exchange
|
||||
exit_order_id = ""
|
||||
try:
|
||||
exit_order = self.client.close_position(position.symbol)
|
||||
exit_order_id = exit_order.get('id', '') if exit_order else ''
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to close position on exchange: {e}")
|
||||
|
||||
record = self.close_position(trade_id, current_price, "take_profit", exit_order_id)
|
||||
if record:
|
||||
closed_trades.append(record)
|
||||
|
||||
self.save_positions()
|
||||
return closed_trades
|
||||
|
||||
def sync_with_exchange(self) -> None:
|
||||
"""
|
||||
Sync local positions with exchange positions.
|
||||
|
||||
Reconciles any discrepancies between local tracking
|
||||
and actual exchange positions.
|
||||
"""
|
||||
try:
|
||||
exchange_positions = self.client.get_positions()
|
||||
exchange_symbols = {p['symbol'] for p in exchange_positions}
|
||||
|
||||
# Check for positions we have locally but not on exchange
|
||||
for trade_id in list(self.positions.keys()):
|
||||
pos = self.positions[trade_id]
|
||||
if pos.symbol not in exchange_symbols:
|
||||
logger.warning(
|
||||
f"Position {trade_id} not found on exchange, removing"
|
||||
)
|
||||
# Get last price and close
|
||||
try:
|
||||
ticker = self.client.get_ticker(pos.symbol)
|
||||
exit_price = ticker['last']
|
||||
except Exception:
|
||||
exit_price = pos.current_price
|
||||
|
||||
self.close_position(trade_id, exit_price, "sync_removed")
|
||||
|
||||
logger.info(f"Position sync complete: {len(self.positions)} local positions")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Position sync failed: {e}")
|
||||
|
||||
def get_portfolio_summary(self) -> dict:
|
||||
"""
|
||||
Get portfolio summary.
|
||||
|
||||
Returns:
|
||||
Dictionary with portfolio statistics
|
||||
"""
|
||||
total_exposure = sum(p.size_usdt for p in self.positions.values())
|
||||
total_unrealized_pnl = sum(p.unrealized_pnl for p in self.positions.values())
|
||||
|
||||
return {
|
||||
'open_positions': len(self.positions),
|
||||
'total_exposure_usdt': total_exposure,
|
||||
'total_unrealized_pnl': total_unrealized_pnl,
|
||||
'positions': [p.to_dict() for p in self.positions.values()],
|
||||
}
|
||||
10
live_trading/ui/__init__.py
Normal file
10
live_trading/ui/__init__.py
Normal file
@@ -0,0 +1,10 @@
|
||||
"""Terminal UI module for live trading dashboard."""
|
||||
from .dashboard import TradingDashboard
|
||||
from .state import SharedState
|
||||
from .log_handler import UILogHandler
|
||||
|
||||
__all__ = [
|
||||
"TradingDashboard",
|
||||
"SharedState",
|
||||
"UILogHandler",
|
||||
]
|
||||
240
live_trading/ui/dashboard.py
Normal file
240
live_trading/ui/dashboard.py
Normal file
@@ -0,0 +1,240 @@
|
||||
"""Main trading dashboard UI orchestration."""
|
||||
import logging
|
||||
import queue
|
||||
import threading
|
||||
import time
|
||||
from typing import Optional, Callable
|
||||
|
||||
from rich.console import Console
|
||||
from rich.layout import Layout
|
||||
from rich.live import Live
|
||||
from rich.panel import Panel
|
||||
from rich.text import Text
|
||||
|
||||
from .state import SharedState
|
||||
from .log_handler import LogBuffer, UILogHandler
|
||||
from .keyboard import KeyboardHandler
|
||||
from .panels import (
|
||||
HeaderPanel,
|
||||
TabBar,
|
||||
LogPanel,
|
||||
HelpBar,
|
||||
build_summary_panel,
|
||||
)
|
||||
from ..db.database import TradingDatabase
|
||||
from ..db.metrics import MetricsCalculator, PeriodMetrics
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class TradingDashboard:
|
||||
"""
|
||||
Main trading dashboard orchestrator.
|
||||
|
||||
Runs in a separate thread and provides real-time UI updates
|
||||
while the trading loop runs in the main thread.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
state: SharedState,
|
||||
db: TradingDatabase,
|
||||
log_queue: queue.Queue,
|
||||
on_quit: Optional[Callable] = None,
|
||||
):
|
||||
self.state = state
|
||||
self.db = db
|
||||
self.log_queue = log_queue
|
||||
self.on_quit = on_quit
|
||||
|
||||
self.console = Console()
|
||||
self.log_buffer = LogBuffer(max_entries=1000)
|
||||
self.keyboard = KeyboardHandler()
|
||||
self.metrics_calculator = MetricsCalculator(db)
|
||||
|
||||
self._running = False
|
||||
self._thread: Optional[threading.Thread] = None
|
||||
self._active_tab = 0
|
||||
self._cached_metrics: dict[int, PeriodMetrics] = {}
|
||||
self._last_metrics_refresh = 0.0
|
||||
|
||||
def start(self) -> None:
|
||||
"""Start the dashboard in a separate thread."""
|
||||
if self._running:
|
||||
return
|
||||
|
||||
self._running = True
|
||||
self._thread = threading.Thread(target=self._run, daemon=True)
|
||||
self._thread.start()
|
||||
logger.debug("Dashboard thread started")
|
||||
|
||||
def stop(self) -> None:
|
||||
"""Stop the dashboard."""
|
||||
self._running = False
|
||||
if self._thread:
|
||||
self._thread.join(timeout=2.0)
|
||||
logger.debug("Dashboard thread stopped")
|
||||
|
||||
def _run(self) -> None:
|
||||
"""Main dashboard loop."""
|
||||
try:
|
||||
with self.keyboard:
|
||||
with Live(
|
||||
self._build_layout(),
|
||||
console=self.console,
|
||||
refresh_per_second=1,
|
||||
screen=True,
|
||||
) as live:
|
||||
while self._running and self.state.is_running():
|
||||
# Process keyboard input
|
||||
action = self.keyboard.get_action(timeout=0.1)
|
||||
if action:
|
||||
self._handle_action(action)
|
||||
|
||||
# Drain log queue
|
||||
self.log_buffer.drain_queue(self.log_queue)
|
||||
|
||||
# Refresh metrics periodically (every 5 seconds)
|
||||
now = time.time()
|
||||
if now - self._last_metrics_refresh > 5.0:
|
||||
self._refresh_metrics()
|
||||
self._last_metrics_refresh = now
|
||||
|
||||
# Update display
|
||||
live.update(self._build_layout())
|
||||
|
||||
# Small sleep to prevent CPU spinning
|
||||
time.sleep(0.1)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Dashboard error: {e}", exc_info=True)
|
||||
finally:
|
||||
self._running = False
|
||||
|
||||
def _handle_action(self, action: str) -> None:
|
||||
"""Handle keyboard action."""
|
||||
if action == "quit":
|
||||
logger.info("Quit requested from UI")
|
||||
self.state.stop()
|
||||
if self.on_quit:
|
||||
self.on_quit()
|
||||
|
||||
elif action == "refresh":
|
||||
self._refresh_metrics()
|
||||
logger.debug("Manual refresh triggered")
|
||||
|
||||
elif action == "filter":
|
||||
new_filter = self.log_buffer.cycle_filter()
|
||||
logger.debug(f"Log filter changed to: {new_filter}")
|
||||
|
||||
elif action == "filter_trades":
|
||||
self.log_buffer.set_filter(LogBuffer.FILTER_TRADES)
|
||||
logger.debug("Log filter set to: trades")
|
||||
|
||||
elif action == "filter_all":
|
||||
self.log_buffer.set_filter(LogBuffer.FILTER_ALL)
|
||||
logger.debug("Log filter set to: all")
|
||||
|
||||
elif action == "filter_errors":
|
||||
self.log_buffer.set_filter(LogBuffer.FILTER_ERRORS)
|
||||
logger.debug("Log filter set to: errors")
|
||||
|
||||
elif action == "tab_general":
|
||||
self._active_tab = 0
|
||||
elif action == "tab_monthly":
|
||||
if self._has_monthly_data():
|
||||
self._active_tab = 1
|
||||
elif action == "tab_weekly":
|
||||
if self._has_weekly_data():
|
||||
self._active_tab = 2
|
||||
elif action == "tab_daily":
|
||||
self._active_tab = 3
|
||||
|
||||
def _refresh_metrics(self) -> None:
|
||||
"""Refresh metrics from database."""
|
||||
try:
|
||||
self._cached_metrics[0] = self.metrics_calculator.get_all_time_metrics()
|
||||
self._cached_metrics[1] = self.metrics_calculator.get_monthly_metrics()
|
||||
self._cached_metrics[2] = self.metrics_calculator.get_weekly_metrics()
|
||||
self._cached_metrics[3] = self.metrics_calculator.get_daily_metrics()
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to refresh metrics: {e}")
|
||||
|
||||
def _has_monthly_data(self) -> bool:
|
||||
"""Check if monthly tab should be shown."""
|
||||
try:
|
||||
return self.metrics_calculator.has_monthly_data()
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def _has_weekly_data(self) -> bool:
|
||||
"""Check if weekly tab should be shown."""
|
||||
try:
|
||||
return self.metrics_calculator.has_weekly_data()
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def _build_layout(self) -> Layout:
|
||||
"""Build the complete dashboard layout."""
|
||||
layout = Layout()
|
||||
|
||||
# Calculate available height
|
||||
term_height = self.console.height or 40
|
||||
|
||||
# Header takes 3 lines
|
||||
# Help bar takes 1 line
|
||||
# Summary panel takes about 12-14 lines
|
||||
# Rest goes to logs
|
||||
log_height = max(8, term_height - 20)
|
||||
|
||||
layout.split_column(
|
||||
Layout(name="header", size=3),
|
||||
Layout(name="summary", size=14),
|
||||
Layout(name="logs", size=log_height),
|
||||
Layout(name="help", size=1),
|
||||
)
|
||||
|
||||
# Header
|
||||
layout["header"].update(HeaderPanel(self.state).render())
|
||||
|
||||
# Summary panel with tabs
|
||||
current_metrics = self._cached_metrics.get(self._active_tab)
|
||||
tab_bar = TabBar(active_tab=self._active_tab)
|
||||
|
||||
layout["summary"].update(
|
||||
build_summary_panel(
|
||||
state=self.state,
|
||||
metrics=current_metrics,
|
||||
tab_bar=tab_bar,
|
||||
has_monthly=self._has_monthly_data(),
|
||||
has_weekly=self._has_weekly_data(),
|
||||
)
|
||||
)
|
||||
|
||||
# Log panel
|
||||
layout["logs"].update(LogPanel(self.log_buffer).render(height=log_height))
|
||||
|
||||
# Help bar
|
||||
layout["help"].update(HelpBar().render())
|
||||
|
||||
return layout
|
||||
|
||||
|
||||
def setup_ui_logging(log_queue: queue.Queue) -> UILogHandler:
|
||||
"""
|
||||
Set up logging to capture messages for UI.
|
||||
|
||||
Args:
|
||||
log_queue: Queue to send log messages to
|
||||
|
||||
Returns:
|
||||
UILogHandler instance
|
||||
"""
|
||||
handler = UILogHandler(log_queue)
|
||||
handler.setLevel(logging.INFO)
|
||||
|
||||
# Add handler to root logger
|
||||
root_logger = logging.getLogger()
|
||||
root_logger.addHandler(handler)
|
||||
|
||||
return handler
|
||||
128
live_trading/ui/keyboard.py
Normal file
128
live_trading/ui/keyboard.py
Normal file
@@ -0,0 +1,128 @@
|
||||
"""Keyboard input handling for terminal UI."""
|
||||
import sys
|
||||
import select
|
||||
import termios
|
||||
import tty
|
||||
from typing import Optional, Callable
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyAction:
|
||||
"""Represents a keyboard action."""
|
||||
|
||||
key: str
|
||||
action: str
|
||||
description: str
|
||||
|
||||
|
||||
class KeyboardHandler:
|
||||
"""
|
||||
Non-blocking keyboard input handler.
|
||||
|
||||
Uses terminal raw mode to capture single keypresses
|
||||
without waiting for Enter.
|
||||
"""
|
||||
|
||||
# Key mappings
|
||||
ACTIONS = {
|
||||
"q": "quit",
|
||||
"Q": "quit",
|
||||
"\x03": "quit", # Ctrl+C
|
||||
"r": "refresh",
|
||||
"R": "refresh",
|
||||
"f": "filter",
|
||||
"F": "filter",
|
||||
"t": "filter_trades",
|
||||
"T": "filter_trades",
|
||||
"l": "filter_all",
|
||||
"L": "filter_all",
|
||||
"e": "filter_errors",
|
||||
"E": "filter_errors",
|
||||
"1": "tab_general",
|
||||
"2": "tab_monthly",
|
||||
"3": "tab_weekly",
|
||||
"4": "tab_daily",
|
||||
}
|
||||
|
||||
def __init__(self):
|
||||
self._old_settings = None
|
||||
self._enabled = False
|
||||
|
||||
def enable(self) -> bool:
|
||||
"""
|
||||
Enable raw keyboard input mode.
|
||||
|
||||
Returns:
|
||||
True if enabled successfully
|
||||
"""
|
||||
try:
|
||||
if not sys.stdin.isatty():
|
||||
return False
|
||||
|
||||
self._old_settings = termios.tcgetattr(sys.stdin)
|
||||
tty.setcbreak(sys.stdin.fileno())
|
||||
self._enabled = True
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def disable(self) -> None:
|
||||
"""Restore normal terminal mode."""
|
||||
if self._enabled and self._old_settings:
|
||||
try:
|
||||
termios.tcsetattr(
|
||||
sys.stdin,
|
||||
termios.TCSADRAIN,
|
||||
self._old_settings,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
self._enabled = False
|
||||
|
||||
def get_key(self, timeout: float = 0.1) -> Optional[str]:
|
||||
"""
|
||||
Get a keypress if available (non-blocking).
|
||||
|
||||
Args:
|
||||
timeout: Seconds to wait for input
|
||||
|
||||
Returns:
|
||||
Key character or None if no input
|
||||
"""
|
||||
if not self._enabled:
|
||||
return None
|
||||
|
||||
try:
|
||||
readable, _, _ = select.select([sys.stdin], [], [], timeout)
|
||||
if readable:
|
||||
return sys.stdin.read(1)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return None
|
||||
|
||||
def get_action(self, timeout: float = 0.1) -> Optional[str]:
|
||||
"""
|
||||
Get action name for pressed key.
|
||||
|
||||
Args:
|
||||
timeout: Seconds to wait for input
|
||||
|
||||
Returns:
|
||||
Action name or None
|
||||
"""
|
||||
key = self.get_key(timeout)
|
||||
if key:
|
||||
return self.ACTIONS.get(key)
|
||||
return None
|
||||
|
||||
def __enter__(self):
|
||||
"""Context manager entry."""
|
||||
self.enable()
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_val, exc_tb):
|
||||
"""Context manager exit."""
|
||||
self.disable()
|
||||
return False
|
||||
178
live_trading/ui/log_handler.py
Normal file
178
live_trading/ui/log_handler.py
Normal file
@@ -0,0 +1,178 @@
|
||||
"""Custom logging handler for UI integration."""
|
||||
import logging
|
||||
import queue
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
from collections import deque
|
||||
|
||||
|
||||
@dataclass
|
||||
class LogEntry:
|
||||
"""A single log entry."""
|
||||
|
||||
timestamp: str
|
||||
level: str
|
||||
message: str
|
||||
logger_name: str
|
||||
|
||||
@property
|
||||
def level_color(self) -> str:
|
||||
"""Get Rich color for log level."""
|
||||
colors = {
|
||||
"DEBUG": "dim",
|
||||
"INFO": "white",
|
||||
"WARNING": "yellow",
|
||||
"ERROR": "red",
|
||||
"CRITICAL": "bold red",
|
||||
}
|
||||
return colors.get(self.level, "white")
|
||||
|
||||
|
||||
class UILogHandler(logging.Handler):
|
||||
"""
|
||||
Custom logging handler that sends logs to UI.
|
||||
|
||||
Uses a thread-safe queue to pass log entries from the trading
|
||||
thread to the UI thread.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
log_queue: queue.Queue,
|
||||
max_entries: int = 1000,
|
||||
):
|
||||
super().__init__()
|
||||
self.log_queue = log_queue
|
||||
self.max_entries = max_entries
|
||||
self.setFormatter(
|
||||
logging.Formatter("%(asctime)s [%(levelname)s] %(name)s: %(message)s")
|
||||
)
|
||||
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
"""Emit a log record to the queue."""
|
||||
try:
|
||||
entry = LogEntry(
|
||||
timestamp=datetime.fromtimestamp(record.created).strftime(
|
||||
"%H:%M:%S"
|
||||
),
|
||||
level=record.levelname,
|
||||
message=self.format_message(record),
|
||||
logger_name=record.name,
|
||||
)
|
||||
# Non-blocking put, drop if queue is full
|
||||
try:
|
||||
self.log_queue.put_nowait(entry)
|
||||
except queue.Full:
|
||||
pass
|
||||
except Exception:
|
||||
self.handleError(record)
|
||||
|
||||
def format_message(self, record: logging.LogRecord) -> str:
|
||||
"""Format the log message."""
|
||||
return record.getMessage()
|
||||
|
||||
|
||||
class LogBuffer:
|
||||
"""
|
||||
Thread-safe buffer for log entries with filtering support.
|
||||
|
||||
Maintains a fixed-size buffer of log entries and supports
|
||||
filtering by log type.
|
||||
"""
|
||||
|
||||
FILTER_ALL = "all"
|
||||
FILTER_ERRORS = "errors"
|
||||
FILTER_TRADES = "trades"
|
||||
FILTER_SIGNALS = "signals"
|
||||
|
||||
FILTERS = [FILTER_ALL, FILTER_ERRORS, FILTER_TRADES, FILTER_SIGNALS]
|
||||
|
||||
def __init__(self, max_entries: int = 1000):
|
||||
self.max_entries = max_entries
|
||||
self._entries: deque[LogEntry] = deque(maxlen=max_entries)
|
||||
self._current_filter = self.FILTER_ALL
|
||||
|
||||
def add(self, entry: LogEntry) -> None:
|
||||
"""Add a log entry to the buffer."""
|
||||
self._entries.append(entry)
|
||||
|
||||
def get_filtered(self, limit: int = 50) -> list[LogEntry]:
|
||||
"""
|
||||
Get filtered log entries.
|
||||
|
||||
Args:
|
||||
limit: Maximum number of entries to return
|
||||
|
||||
Returns:
|
||||
List of filtered LogEntry objects (most recent first)
|
||||
"""
|
||||
entries = list(self._entries)
|
||||
|
||||
if self._current_filter == self.FILTER_ERRORS:
|
||||
entries = [e for e in entries if e.level in ("ERROR", "CRITICAL")]
|
||||
elif self._current_filter == self.FILTER_TRADES:
|
||||
# Key terms indicating actual trading activity
|
||||
include_keywords = [
|
||||
"order", "entry", "exit", "executed", "filled",
|
||||
"opening", "closing", "position opened", "position closed"
|
||||
]
|
||||
# Terms to exclude (noise)
|
||||
exclude_keywords = [
|
||||
"sync complete", "0 positions", "portfolio: 0 positions"
|
||||
]
|
||||
|
||||
entries = [
|
||||
e for e in entries
|
||||
if any(kw in e.message.lower() for kw in include_keywords)
|
||||
and not any(ex in e.message.lower() for ex in exclude_keywords)
|
||||
]
|
||||
elif self._current_filter == self.FILTER_SIGNALS:
|
||||
signal_keywords = ["signal", "z_score", "prob", "z="]
|
||||
entries = [
|
||||
e for e in entries
|
||||
if any(kw in e.message.lower() for kw in signal_keywords)
|
||||
]
|
||||
|
||||
# Return most recent entries
|
||||
return list(reversed(entries[-limit:]))
|
||||
|
||||
def set_filter(self, filter_name: str) -> None:
|
||||
"""Set a specific filter."""
|
||||
if filter_name in self.FILTERS:
|
||||
self._current_filter = filter_name
|
||||
|
||||
def cycle_filter(self) -> str:
|
||||
"""Cycle to next filter and return its name."""
|
||||
current_idx = self.FILTERS.index(self._current_filter)
|
||||
next_idx = (current_idx + 1) % len(self.FILTERS)
|
||||
self._current_filter = self.FILTERS[next_idx]
|
||||
return self._current_filter
|
||||
|
||||
def get_current_filter(self) -> str:
|
||||
"""Get current filter name."""
|
||||
return self._current_filter
|
||||
|
||||
def clear(self) -> None:
|
||||
"""Clear all log entries."""
|
||||
self._entries.clear()
|
||||
|
||||
def drain_queue(self, log_queue: queue.Queue) -> int:
|
||||
"""
|
||||
Drain log entries from queue into buffer.
|
||||
|
||||
Args:
|
||||
log_queue: Queue to drain from
|
||||
|
||||
Returns:
|
||||
Number of entries drained
|
||||
"""
|
||||
count = 0
|
||||
while True:
|
||||
try:
|
||||
entry = log_queue.get_nowait()
|
||||
self.add(entry)
|
||||
count += 1
|
||||
except queue.Empty:
|
||||
break
|
||||
return count
|
||||
399
live_trading/ui/panels.py
Normal file
399
live_trading/ui/panels.py
Normal file
@@ -0,0 +1,399 @@
|
||||
"""UI panel components using Rich."""
|
||||
from typing import Optional
|
||||
|
||||
from rich.console import Console, Group
|
||||
from rich.panel import Panel
|
||||
from rich.table import Table
|
||||
from rich.text import Text
|
||||
from rich.layout import Layout
|
||||
|
||||
from .state import SharedState, PositionState, StrategyState, AccountState
|
||||
from .log_handler import LogBuffer, LogEntry
|
||||
from ..db.metrics import PeriodMetrics
|
||||
|
||||
|
||||
def format_pnl(value: float, include_sign: bool = True) -> Text:
|
||||
"""Format PnL value with color."""
|
||||
if value > 0:
|
||||
sign = "+" if include_sign else ""
|
||||
return Text(f"{sign}${value:.2f}", style="green")
|
||||
elif value < 0:
|
||||
return Text(f"${value:.2f}", style="red")
|
||||
else:
|
||||
return Text(f"${value:.2f}", style="white")
|
||||
|
||||
|
||||
def format_pct(value: float, include_sign: bool = True) -> Text:
|
||||
"""Format percentage value with color."""
|
||||
if value > 0:
|
||||
sign = "+" if include_sign else ""
|
||||
return Text(f"{sign}{value:.2f}%", style="green")
|
||||
elif value < 0:
|
||||
return Text(f"{value:.2f}%", style="red")
|
||||
else:
|
||||
return Text(f"{value:.2f}%", style="white")
|
||||
|
||||
|
||||
def format_side(side: str) -> Text:
|
||||
"""Format position side with color."""
|
||||
if side.lower() == "long":
|
||||
return Text("LONG", style="bold green")
|
||||
else:
|
||||
return Text("SHORT", style="bold red")
|
||||
|
||||
|
||||
class HeaderPanel:
|
||||
"""Header panel with title and mode indicator."""
|
||||
|
||||
def __init__(self, state: SharedState):
|
||||
self.state = state
|
||||
|
||||
def render(self) -> Panel:
|
||||
"""Render the header panel."""
|
||||
mode = self.state.get_mode()
|
||||
eth_symbol, _ = self.state.get_symbols()
|
||||
|
||||
mode_style = "yellow" if mode == "DEMO" else "bold red"
|
||||
mode_text = Text(f"[{mode}]", style=mode_style)
|
||||
|
||||
title = Text()
|
||||
title.append("REGIME REVERSION STRATEGY - LIVE TRADING", style="bold white")
|
||||
title.append(" ")
|
||||
title.append(mode_text)
|
||||
title.append(" ")
|
||||
title.append(eth_symbol, style="cyan")
|
||||
|
||||
return Panel(title, style="blue", height=3)
|
||||
|
||||
|
||||
class TabBar:
|
||||
"""Tab bar for period selection."""
|
||||
|
||||
TABS = ["1:General", "2:Monthly", "3:Weekly", "4:Daily"]
|
||||
|
||||
def __init__(self, active_tab: int = 0):
|
||||
self.active_tab = active_tab
|
||||
|
||||
def render(
|
||||
self,
|
||||
has_monthly: bool = True,
|
||||
has_weekly: bool = True,
|
||||
) -> Text:
|
||||
"""Render the tab bar."""
|
||||
text = Text()
|
||||
text.append(" ")
|
||||
|
||||
for i, tab in enumerate(self.TABS):
|
||||
# Check if tab should be shown
|
||||
if i == 1 and not has_monthly:
|
||||
continue
|
||||
if i == 2 and not has_weekly:
|
||||
continue
|
||||
|
||||
if i == self.active_tab:
|
||||
text.append(f"[{tab}]", style="bold white on blue")
|
||||
else:
|
||||
text.append(f"[{tab}]", style="dim")
|
||||
text.append(" ")
|
||||
|
||||
return text
|
||||
|
||||
|
||||
class MetricsPanel:
|
||||
"""Panel showing trading metrics."""
|
||||
|
||||
def __init__(self, metrics: Optional[PeriodMetrics] = None):
|
||||
self.metrics = metrics
|
||||
|
||||
def render(self) -> Table:
|
||||
"""Render metrics as a table."""
|
||||
table = Table(
|
||||
show_header=False,
|
||||
show_edge=False,
|
||||
box=None,
|
||||
padding=(0, 1),
|
||||
)
|
||||
table.add_column("Label", style="dim")
|
||||
table.add_column("Value")
|
||||
|
||||
if self.metrics is None or self.metrics.total_trades == 0:
|
||||
table.add_row("Status", Text("No trade data", style="dim"))
|
||||
return table
|
||||
|
||||
m = self.metrics
|
||||
|
||||
table.add_row("Total PnL:", format_pnl(m.total_pnl))
|
||||
table.add_row("Win Rate:", Text(f"{m.win_rate:.1f}%", style="white"))
|
||||
table.add_row("Total Trades:", Text(str(m.total_trades), style="white"))
|
||||
table.add_row(
|
||||
"Win/Loss:",
|
||||
Text(f"{m.winning_trades}/{m.losing_trades}", style="white"),
|
||||
)
|
||||
table.add_row(
|
||||
"Avg Duration:",
|
||||
Text(f"{m.avg_trade_duration_hours:.1f}h", style="white"),
|
||||
)
|
||||
table.add_row("Max Drawdown:", format_pnl(-m.max_drawdown))
|
||||
table.add_row("Best Trade:", format_pnl(m.best_trade))
|
||||
table.add_row("Worst Trade:", format_pnl(m.worst_trade))
|
||||
|
||||
return table
|
||||
|
||||
|
||||
class PositionPanel:
|
||||
"""Panel showing current position."""
|
||||
|
||||
def __init__(self, position: Optional[PositionState] = None):
|
||||
self.position = position
|
||||
|
||||
def render(self) -> Table:
|
||||
"""Render position as a table."""
|
||||
table = Table(
|
||||
show_header=False,
|
||||
show_edge=False,
|
||||
box=None,
|
||||
padding=(0, 1),
|
||||
)
|
||||
table.add_column("Label", style="dim")
|
||||
table.add_column("Value")
|
||||
|
||||
if self.position is None:
|
||||
table.add_row("Status", Text("No open position", style="dim"))
|
||||
return table
|
||||
|
||||
p = self.position
|
||||
|
||||
table.add_row("Side:", format_side(p.side))
|
||||
table.add_row("Entry:", Text(f"${p.entry_price:.2f}", style="white"))
|
||||
table.add_row("Current:", Text(f"${p.current_price:.2f}", style="white"))
|
||||
|
||||
# Unrealized PnL
|
||||
pnl_text = Text()
|
||||
pnl_text.append_text(format_pnl(p.unrealized_pnl))
|
||||
pnl_text.append(" (")
|
||||
pnl_text.append_text(format_pct(p.unrealized_pnl_pct))
|
||||
pnl_text.append(")")
|
||||
table.add_row("Unrealized:", pnl_text)
|
||||
|
||||
table.add_row("Size:", Text(f"${p.size_usdt:.2f}", style="white"))
|
||||
|
||||
# SL/TP
|
||||
if p.side == "long":
|
||||
sl_dist = (p.stop_loss_price / p.entry_price - 1) * 100
|
||||
tp_dist = (p.take_profit_price / p.entry_price - 1) * 100
|
||||
else:
|
||||
sl_dist = (1 - p.stop_loss_price / p.entry_price) * 100
|
||||
tp_dist = (1 - p.take_profit_price / p.entry_price) * 100
|
||||
|
||||
sl_text = Text(f"${p.stop_loss_price:.2f} ({sl_dist:+.1f}%)", style="red")
|
||||
tp_text = Text(f"${p.take_profit_price:.2f} ({tp_dist:+.1f}%)", style="green")
|
||||
|
||||
table.add_row("Stop Loss:", sl_text)
|
||||
table.add_row("Take Profit:", tp_text)
|
||||
|
||||
return table
|
||||
|
||||
|
||||
class AccountPanel:
|
||||
"""Panel showing account information."""
|
||||
|
||||
def __init__(self, account: Optional[AccountState] = None):
|
||||
self.account = account
|
||||
|
||||
def render(self) -> Table:
|
||||
"""Render account info as a table."""
|
||||
table = Table(
|
||||
show_header=False,
|
||||
show_edge=False,
|
||||
box=None,
|
||||
padding=(0, 1),
|
||||
)
|
||||
table.add_column("Label", style="dim")
|
||||
table.add_column("Value")
|
||||
|
||||
if self.account is None:
|
||||
table.add_row("Status", Text("Loading...", style="dim"))
|
||||
return table
|
||||
|
||||
a = self.account
|
||||
|
||||
table.add_row("Balance:", Text(f"${a.balance:.2f}", style="white"))
|
||||
table.add_row("Available:", Text(f"${a.available:.2f}", style="white"))
|
||||
table.add_row("Leverage:", Text(f"{a.leverage}x", style="cyan"))
|
||||
|
||||
return table
|
||||
|
||||
|
||||
class StrategyPanel:
|
||||
"""Panel showing strategy state."""
|
||||
|
||||
def __init__(self, strategy: Optional[StrategyState] = None):
|
||||
self.strategy = strategy
|
||||
|
||||
def render(self) -> Table:
|
||||
"""Render strategy state as a table."""
|
||||
table = Table(
|
||||
show_header=False,
|
||||
show_edge=False,
|
||||
box=None,
|
||||
padding=(0, 1),
|
||||
)
|
||||
table.add_column("Label", style="dim")
|
||||
table.add_column("Value")
|
||||
|
||||
if self.strategy is None:
|
||||
table.add_row("Status", Text("Waiting...", style="dim"))
|
||||
return table
|
||||
|
||||
s = self.strategy
|
||||
|
||||
# Z-score with color based on threshold
|
||||
z_style = "white"
|
||||
if abs(s.z_score) > 1.0:
|
||||
z_style = "yellow"
|
||||
if abs(s.z_score) > 1.5:
|
||||
z_style = "bold yellow"
|
||||
table.add_row("Z-Score:", Text(f"{s.z_score:.2f}", style=z_style))
|
||||
|
||||
# Probability with color
|
||||
prob_style = "white"
|
||||
if s.probability > 0.5:
|
||||
prob_style = "green"
|
||||
if s.probability > 0.7:
|
||||
prob_style = "bold green"
|
||||
table.add_row("Probability:", Text(f"{s.probability:.2f}", style=prob_style))
|
||||
|
||||
# Funding rate
|
||||
funding_style = "green" if s.funding_rate >= 0 else "red"
|
||||
table.add_row(
|
||||
"Funding:",
|
||||
Text(f"{s.funding_rate:.4f}", style=funding_style),
|
||||
)
|
||||
|
||||
# Last action
|
||||
action_style = "white"
|
||||
if s.last_action == "entry":
|
||||
action_style = "bold cyan"
|
||||
elif s.last_action == "check_exit":
|
||||
action_style = "yellow"
|
||||
table.add_row("Last Action:", Text(s.last_action, style=action_style))
|
||||
|
||||
return table
|
||||
|
||||
|
||||
class LogPanel:
|
||||
"""Panel showing log entries."""
|
||||
|
||||
def __init__(self, log_buffer: LogBuffer):
|
||||
self.log_buffer = log_buffer
|
||||
|
||||
def render(self, height: int = 10) -> Panel:
|
||||
"""Render log panel."""
|
||||
filter_name = self.log_buffer.get_current_filter().title()
|
||||
entries = self.log_buffer.get_filtered(limit=height - 2)
|
||||
|
||||
lines = []
|
||||
for entry in entries:
|
||||
line = Text()
|
||||
line.append(f"{entry.timestamp} ", style="dim")
|
||||
line.append(f"[{entry.level}] ", style=entry.level_color)
|
||||
line.append(entry.message)
|
||||
lines.append(line)
|
||||
|
||||
if not lines:
|
||||
lines.append(Text("No logs to display", style="dim"))
|
||||
|
||||
content = Group(*lines)
|
||||
|
||||
# Build "tabbed" title
|
||||
tabs = []
|
||||
|
||||
# All Logs tab
|
||||
if filter_name == "All":
|
||||
tabs.append("[bold white on blue] [L]ogs [/]")
|
||||
else:
|
||||
tabs.append("[dim] [L]ogs [/]")
|
||||
|
||||
# Trades tab
|
||||
if filter_name == "Trades":
|
||||
tabs.append("[bold white on blue] [T]rades [/]")
|
||||
else:
|
||||
tabs.append("[dim] [T]rades [/]")
|
||||
|
||||
# Errors tab
|
||||
if filter_name == "Errors":
|
||||
tabs.append("[bold white on blue] [E]rrors [/]")
|
||||
else:
|
||||
tabs.append("[dim] [E]rrors [/]")
|
||||
|
||||
title = " ".join(tabs)
|
||||
subtitle = "Press 'l', 't', 'e' to switch tabs"
|
||||
|
||||
return Panel(
|
||||
content,
|
||||
title=title,
|
||||
subtitle=subtitle,
|
||||
title_align="left",
|
||||
subtitle_align="right",
|
||||
border_style="blue",
|
||||
)
|
||||
|
||||
|
||||
class HelpBar:
|
||||
"""Bottom help bar with keyboard shortcuts."""
|
||||
|
||||
def render(self) -> Text:
|
||||
"""Render help bar."""
|
||||
text = Text()
|
||||
text.append(" [q]", style="bold")
|
||||
text.append("Quit ", style="dim")
|
||||
text.append("[r]", style="bold")
|
||||
text.append("Refresh ", style="dim")
|
||||
text.append("[1-4]", style="bold")
|
||||
text.append("Tabs ", style="dim")
|
||||
text.append("[l/t/e]", style="bold")
|
||||
text.append("LogView", style="dim")
|
||||
|
||||
return text
|
||||
|
||||
|
||||
def build_summary_panel(
|
||||
state: SharedState,
|
||||
metrics: Optional[PeriodMetrics],
|
||||
tab_bar: TabBar,
|
||||
has_monthly: bool,
|
||||
has_weekly: bool,
|
||||
) -> Panel:
|
||||
"""Build the complete summary panel with all sections."""
|
||||
# Create layout for summary content
|
||||
layout = Layout()
|
||||
|
||||
# Tab bar at top
|
||||
tabs = tab_bar.render(has_monthly, has_weekly)
|
||||
|
||||
# Create tables for each section
|
||||
metrics_table = MetricsPanel(metrics).render()
|
||||
position_table = PositionPanel(state.get_position()).render()
|
||||
account_table = AccountPanel(state.get_account()).render()
|
||||
strategy_table = StrategyPanel(state.get_strategy()).render()
|
||||
|
||||
# Build two-column layout
|
||||
left_col = Table(show_header=True, show_edge=False, box=None, padding=(0, 2))
|
||||
left_col.add_column("PERFORMANCE", style="bold cyan")
|
||||
left_col.add_column("ACCOUNT", style="bold cyan")
|
||||
left_col.add_row(metrics_table, account_table)
|
||||
|
||||
right_col = Table(show_header=True, show_edge=False, box=None, padding=(0, 2))
|
||||
right_col.add_column("CURRENT POSITION", style="bold cyan")
|
||||
right_col.add_column("STRATEGY STATE", style="bold cyan")
|
||||
right_col.add_row(position_table, strategy_table)
|
||||
|
||||
# Combine into main table
|
||||
main_table = Table(show_header=False, show_edge=False, box=None, expand=True)
|
||||
main_table.add_column(ratio=1)
|
||||
main_table.add_column(ratio=1)
|
||||
main_table.add_row(left_col, right_col)
|
||||
|
||||
content = Group(tabs, Text(""), main_table)
|
||||
|
||||
return Panel(content, border_style="blue")
|
||||
195
live_trading/ui/state.py
Normal file
195
live_trading/ui/state.py
Normal file
@@ -0,0 +1,195 @@
|
||||
"""Thread-safe shared state for UI and trading loop."""
|
||||
import threading
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
@dataclass
|
||||
class PositionState:
|
||||
"""Current position information."""
|
||||
|
||||
trade_id: str = ""
|
||||
symbol: str = ""
|
||||
side: str = ""
|
||||
entry_price: float = 0.0
|
||||
current_price: float = 0.0
|
||||
size: float = 0.0
|
||||
size_usdt: float = 0.0
|
||||
unrealized_pnl: float = 0.0
|
||||
unrealized_pnl_pct: float = 0.0
|
||||
stop_loss_price: float = 0.0
|
||||
take_profit_price: float = 0.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class StrategyState:
|
||||
"""Current strategy signal state."""
|
||||
|
||||
z_score: float = 0.0
|
||||
probability: float = 0.0
|
||||
funding_rate: float = 0.0
|
||||
last_action: str = "hold"
|
||||
last_reason: str = ""
|
||||
last_signal_time: str = ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class AccountState:
|
||||
"""Account balance information."""
|
||||
|
||||
balance: float = 0.0
|
||||
available: float = 0.0
|
||||
leverage: int = 1
|
||||
|
||||
|
||||
class SharedState:
|
||||
"""
|
||||
Thread-safe shared state between trading loop and UI.
|
||||
|
||||
All access to state fields should go through the getter/setter methods
|
||||
which use a lock for thread safety.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self._lock = threading.Lock()
|
||||
self._position: Optional[PositionState] = None
|
||||
self._strategy = StrategyState()
|
||||
self._account = AccountState()
|
||||
self._is_running = True
|
||||
self._last_cycle_time: Optional[str] = None
|
||||
self._mode = "DEMO"
|
||||
self._eth_symbol = "ETH/USDT:USDT"
|
||||
self._btc_symbol = "BTC/USDT:USDT"
|
||||
|
||||
# Position methods
|
||||
def get_position(self) -> Optional[PositionState]:
|
||||
"""Get current position state."""
|
||||
with self._lock:
|
||||
return self._position
|
||||
|
||||
def set_position(self, position: Optional[PositionState]) -> None:
|
||||
"""Set current position state."""
|
||||
with self._lock:
|
||||
self._position = position
|
||||
|
||||
def update_position_price(self, current_price: float) -> None:
|
||||
"""Update current price and recalculate PnL."""
|
||||
with self._lock:
|
||||
if self._position is None:
|
||||
return
|
||||
|
||||
self._position.current_price = current_price
|
||||
|
||||
if self._position.side == "long":
|
||||
pnl = (current_price - self._position.entry_price)
|
||||
self._position.unrealized_pnl = pnl * self._position.size
|
||||
pnl_pct = (current_price / self._position.entry_price - 1) * 100
|
||||
else:
|
||||
pnl = (self._position.entry_price - current_price)
|
||||
self._position.unrealized_pnl = pnl * self._position.size
|
||||
pnl_pct = (1 - current_price / self._position.entry_price) * 100
|
||||
|
||||
self._position.unrealized_pnl_pct = pnl_pct
|
||||
|
||||
def clear_position(self) -> None:
|
||||
"""Clear current position."""
|
||||
with self._lock:
|
||||
self._position = None
|
||||
|
||||
# Strategy methods
|
||||
def get_strategy(self) -> StrategyState:
|
||||
"""Get current strategy state."""
|
||||
with self._lock:
|
||||
return StrategyState(
|
||||
z_score=self._strategy.z_score,
|
||||
probability=self._strategy.probability,
|
||||
funding_rate=self._strategy.funding_rate,
|
||||
last_action=self._strategy.last_action,
|
||||
last_reason=self._strategy.last_reason,
|
||||
last_signal_time=self._strategy.last_signal_time,
|
||||
)
|
||||
|
||||
def update_strategy(
|
||||
self,
|
||||
z_score: float,
|
||||
probability: float,
|
||||
funding_rate: float,
|
||||
action: str,
|
||||
reason: str,
|
||||
) -> None:
|
||||
"""Update strategy state."""
|
||||
with self._lock:
|
||||
self._strategy.z_score = z_score
|
||||
self._strategy.probability = probability
|
||||
self._strategy.funding_rate = funding_rate
|
||||
self._strategy.last_action = action
|
||||
self._strategy.last_reason = reason
|
||||
self._strategy.last_signal_time = datetime.now(
|
||||
timezone.utc
|
||||
).isoformat()
|
||||
|
||||
# Account methods
|
||||
def get_account(self) -> AccountState:
|
||||
"""Get current account state."""
|
||||
with self._lock:
|
||||
return AccountState(
|
||||
balance=self._account.balance,
|
||||
available=self._account.available,
|
||||
leverage=self._account.leverage,
|
||||
)
|
||||
|
||||
def update_account(
|
||||
self,
|
||||
balance: float,
|
||||
available: float,
|
||||
leverage: int,
|
||||
) -> None:
|
||||
"""Update account state."""
|
||||
with self._lock:
|
||||
self._account.balance = balance
|
||||
self._account.available = available
|
||||
self._account.leverage = leverage
|
||||
|
||||
# Control methods
|
||||
def is_running(self) -> bool:
|
||||
"""Check if trading loop is running."""
|
||||
with self._lock:
|
||||
return self._is_running
|
||||
|
||||
def stop(self) -> None:
|
||||
"""Signal to stop trading loop."""
|
||||
with self._lock:
|
||||
self._is_running = False
|
||||
|
||||
def get_last_cycle_time(self) -> Optional[str]:
|
||||
"""Get last trading cycle time."""
|
||||
with self._lock:
|
||||
return self._last_cycle_time
|
||||
|
||||
def set_last_cycle_time(self, time_str: str) -> None:
|
||||
"""Set last trading cycle time."""
|
||||
with self._lock:
|
||||
self._last_cycle_time = time_str
|
||||
|
||||
# Config methods
|
||||
def get_mode(self) -> str:
|
||||
"""Get trading mode (DEMO/LIVE)."""
|
||||
with self._lock:
|
||||
return self._mode
|
||||
|
||||
def set_mode(self, mode: str) -> None:
|
||||
"""Set trading mode."""
|
||||
with self._lock:
|
||||
self._mode = mode
|
||||
|
||||
def get_symbols(self) -> tuple[str, str]:
|
||||
"""Get trading symbols (eth, btc)."""
|
||||
with self._lock:
|
||||
return self._eth_symbol, self._btc_symbol
|
||||
|
||||
def set_symbols(self, eth_symbol: str, btc_symbol: str) -> None:
|
||||
"""Set trading symbols."""
|
||||
with self._lock:
|
||||
self._eth_symbol = eth_symbol
|
||||
self._btc_symbol = btc_symbol
|
||||
@@ -1,11 +0,0 @@
|
||||
from __future__ import annotations
|
||||
from pathlib import Path
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def write_trade_log(trades: list[dict], path: Path) -> None:
|
||||
if not trades:
|
||||
return
|
||||
df = pd.DataFrame(trades)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
df.to_csv(path, index=False)
|
||||
10
main.py
Normal file
10
main.py
Normal file
@@ -0,0 +1,10 @@
|
||||
"""
|
||||
Lowkey Backtest CLI - VectorBT Edition
|
||||
|
||||
A backtesting framework supporting multiple market types (spot, perpetual)
|
||||
with realistic trading simulation including leverage, funding, and shorts.
|
||||
"""
|
||||
from engine.cli import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,11 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
TAKER_FEE_BPS_DEFAULT = 10.0 # 0.10%
|
||||
|
||||
|
||||
def okx_fee(fee_bps: float, notional_usd: float) -> float:
|
||||
return notional_usd * (fee_bps / 1e4)
|
||||
|
||||
|
||||
def estimate_slippage_rate(slippage_bps: float, notional_usd: float) -> float:
|
||||
return notional_usd * (slippage_bps / 1e4)
|
||||
54
metrics.py
54
metrics.py
@@ -1,54 +0,0 @@
|
||||
from __future__ import annotations
|
||||
from dataclasses import dataclass
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
|
||||
@dataclass
|
||||
class Perf:
|
||||
total_return: float
|
||||
max_drawdown: float
|
||||
sharpe_ratio: float
|
||||
win_rate: float
|
||||
num_trades: int
|
||||
final_equity: float
|
||||
initial_equity: float
|
||||
num_stop_losses: int
|
||||
total_fees: float
|
||||
total_slippage_usd: float
|
||||
avg_slippage_bps: float
|
||||
|
||||
|
||||
def compute_metrics(equity_curve: pd.Series, trades: list[dict]) -> Perf:
|
||||
ret = equity_curve.pct_change().fillna(0.0)
|
||||
total_return = equity_curve.iat[-1] / equity_curve.iat[0] - 1.0
|
||||
cummax = equity_curve.cummax()
|
||||
dd = (equity_curve / cummax - 1.0).min()
|
||||
max_drawdown = dd
|
||||
|
||||
if ret.std(ddof=0) > 0:
|
||||
sharpe = (ret.mean() / ret.std(ddof=0)) * np.sqrt(252 * 24 * 60) # minute bars -> annualized
|
||||
else:
|
||||
sharpe = 0.0
|
||||
|
||||
closes = [t for t in trades if t.get("side") == "SELL"]
|
||||
wins = [t for t in closes if t.get("pnl", 0.0) > 0]
|
||||
win_rate = (len(wins) / len(closes)) if closes else 0.0
|
||||
|
||||
fees = sum(t.get("fee", 0.0) for t in trades)
|
||||
slip = sum(t.get("slippage", 0.0) for t in trades)
|
||||
slippage_bps = [t.get("slippage_bps", 0.0) for t in trades if "slippage_bps" in t]
|
||||
|
||||
return Perf(
|
||||
total_return=total_return,
|
||||
max_drawdown=max_drawdown,
|
||||
sharpe_ratio=sharpe,
|
||||
win_rate=win_rate,
|
||||
num_trades=len(closes),
|
||||
final_equity=float(equity_curve.iat[-1]),
|
||||
initial_equity=float(equity_curve.iat[0]),
|
||||
num_stop_losses=sum(1 for t in closes if t.get("reason") == "stop"),
|
||||
total_fees=fees,
|
||||
total_slippage_usd=slip,
|
||||
avg_slippage_bps=float(np.mean(slippage_bps)) if slippage_bps else 0.0,
|
||||
)
|
||||
@@ -5,5 +5,31 @@ description = "Add your description here"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"ccxt>=4.5.32",
|
||||
"numpy>=2.3.2",
|
||||
"pandas>=2.3.1",
|
||||
"ta>=0.11.0",
|
||||
"vectorbt>=0.28.2",
|
||||
"scikit-learn>=1.6.0",
|
||||
"matplotlib>=3.10.0",
|
||||
"plotly>=5.24.0",
|
||||
"requests>=2.32.5",
|
||||
"python-dotenv>=1.2.1",
|
||||
# Terminal UI
|
||||
"rich>=13.0.0",
|
||||
# API dependencies
|
||||
"fastapi>=0.115.0",
|
||||
"uvicorn[standard]>=0.34.0",
|
||||
"sqlalchemy>=2.0.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.0.0",
|
||||
]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
pythonpath = ["."]
|
||||
markers = [
|
||||
"network: marks tests as requiring network access",
|
||||
]
|
||||
|
||||
532
research/regime_detection.py
Normal file
532
research/regime_detection.py
Normal file
@@ -0,0 +1,532 @@
|
||||
"""
|
||||
Regime Detection Research Script with Walk-Forward Training.
|
||||
|
||||
Tests multiple holding horizons to find optimal parameters
|
||||
without look-ahead bias.
|
||||
|
||||
Usage:
|
||||
uv run python research/regime_detection.py [options]
|
||||
|
||||
Options:
|
||||
--days DAYS Number of days of data (default: 90)
|
||||
--start DATE Start date (YYYY-MM-DD), overrides --days
|
||||
--end DATE End date (YYYY-MM-DD), defaults to now
|
||||
"""
|
||||
import argparse
|
||||
import sys
|
||||
import os
|
||||
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
import ta
|
||||
from sklearn.ensemble import RandomForestClassifier
|
||||
from sklearn.metrics import classification_report, f1_score
|
||||
|
||||
from engine.data_manager import DataManager
|
||||
from engine.market import MarketType
|
||||
from engine.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
# Configuration
|
||||
TRAIN_RATIO = 0.7 # 70% train, 30% test
|
||||
PROFIT_THRESHOLD = 0.005 # 0.5% profit target
|
||||
STOP_LOSS_PCT = 0.06 # 6% stop loss
|
||||
Z_WINDOW = 24
|
||||
FEE_RATE = 0.001 # 0.1% round-trip fee
|
||||
DEFAULT_DAYS = 90 # Default lookback period in days
|
||||
|
||||
|
||||
def load_data(days: int = DEFAULT_DAYS, start_date: str = None, end_date: str = None):
|
||||
"""
|
||||
Load and align BTC/ETH data.
|
||||
|
||||
Args:
|
||||
days: Number of days of historical data (default: 90)
|
||||
start_date: Optional start date (YYYY-MM-DD), overrides days
|
||||
end_date: Optional end date (YYYY-MM-DD), defaults to now
|
||||
|
||||
Returns:
|
||||
Tuple of (df_btc, df_eth) DataFrames
|
||||
"""
|
||||
dm = DataManager()
|
||||
|
||||
df_btc = dm.load_data("okx", "BTC-USDT", "1h", MarketType.SPOT)
|
||||
df_eth = dm.load_data("okx", "ETH-USDT", "1h", MarketType.SPOT)
|
||||
|
||||
# Determine date range
|
||||
if end_date:
|
||||
end = pd.Timestamp(end_date, tz="UTC")
|
||||
else:
|
||||
end = pd.Timestamp.now(tz="UTC")
|
||||
|
||||
if start_date:
|
||||
start = pd.Timestamp(start_date, tz="UTC")
|
||||
else:
|
||||
start = end - pd.Timedelta(days=days)
|
||||
|
||||
df_btc = df_btc[(df_btc.index >= start) & (df_btc.index <= end)]
|
||||
df_eth = df_eth[(df_eth.index >= start) & (df_eth.index <= end)]
|
||||
|
||||
# Align indices
|
||||
common = df_btc.index.intersection(df_eth.index)
|
||||
df_btc = df_btc.loc[common]
|
||||
df_eth = df_eth.loc[common]
|
||||
|
||||
logger.info(f"Loaded {len(common)} aligned hourly bars from {start} to {end}")
|
||||
return df_btc, df_eth
|
||||
|
||||
|
||||
def load_cryptoquant_data():
|
||||
"""Load CryptoQuant on-chain data if available."""
|
||||
try:
|
||||
cq_path = "data/cq_training_data.csv"
|
||||
cq_df = pd.read_csv(cq_path, index_col='timestamp', parse_dates=True)
|
||||
if cq_df.index.tz is None:
|
||||
cq_df.index = cq_df.index.tz_localize('UTC')
|
||||
logger.info(f"Loaded CryptoQuant data: {len(cq_df)} rows")
|
||||
return cq_df
|
||||
except Exception as e:
|
||||
logger.warning(f"CryptoQuant data not available: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def calculate_features(df_btc, df_eth, cq_df=None):
|
||||
"""Calculate all features for the model."""
|
||||
spread = df_eth['close'] / df_btc['close']
|
||||
|
||||
# Z-Score
|
||||
rolling_mean = spread.rolling(window=Z_WINDOW).mean()
|
||||
rolling_std = spread.rolling(window=Z_WINDOW).std()
|
||||
z_score = (spread - rolling_mean) / rolling_std
|
||||
|
||||
# Technicals
|
||||
spread_rsi = ta.momentum.RSIIndicator(spread, window=14).rsi()
|
||||
spread_roc = spread.pct_change(periods=5) * 100
|
||||
spread_change_1h = spread.pct_change(periods=1)
|
||||
|
||||
# Volume
|
||||
vol_ratio = df_eth['volume'] / df_btc['volume']
|
||||
vol_ratio_ma = vol_ratio.rolling(window=12).mean()
|
||||
|
||||
# Volatility
|
||||
ret_btc = df_btc['close'].pct_change()
|
||||
ret_eth = df_eth['close'].pct_change()
|
||||
vol_btc = ret_btc.rolling(window=Z_WINDOW).std()
|
||||
vol_eth = ret_eth.rolling(window=Z_WINDOW).std()
|
||||
vol_spread_ratio = vol_eth / vol_btc
|
||||
|
||||
features = pd.DataFrame(index=spread.index)
|
||||
features['spread'] = spread
|
||||
features['z_score'] = z_score
|
||||
features['spread_rsi'] = spread_rsi
|
||||
features['spread_roc'] = spread_roc
|
||||
features['spread_change_1h'] = spread_change_1h
|
||||
features['vol_ratio'] = vol_ratio
|
||||
features['vol_ratio_rel'] = vol_ratio / vol_ratio_ma
|
||||
features['vol_diff_ratio'] = vol_spread_ratio
|
||||
|
||||
# Add CQ features if available
|
||||
if cq_df is not None:
|
||||
cq_aligned = cq_df.reindex(features.index, method='ffill')
|
||||
if 'btc_funding' in cq_aligned.columns and 'eth_funding' in cq_aligned.columns:
|
||||
cq_aligned['funding_diff'] = cq_aligned['eth_funding'] - cq_aligned['btc_funding']
|
||||
if 'btc_inflow' in cq_aligned.columns and 'eth_inflow' in cq_aligned.columns:
|
||||
cq_aligned['inflow_ratio'] = cq_aligned['eth_inflow'] / (cq_aligned['btc_inflow'] + 1)
|
||||
features = features.join(cq_aligned)
|
||||
|
||||
return features.dropna()
|
||||
|
||||
|
||||
def calculate_targets(features, horizon):
|
||||
"""
|
||||
Calculate target labels for a given horizon.
|
||||
|
||||
Uses path-dependent labeling: Success is hitting Profit Target BEFORE Stop Loss.
|
||||
"""
|
||||
spread = features['spread'].values
|
||||
z_score = features['z_score'].values
|
||||
n = len(spread)
|
||||
|
||||
targets = np.zeros(n, dtype=int)
|
||||
|
||||
# Create valid mask (rows with complete future data)
|
||||
valid_mask = np.zeros(n, dtype=bool)
|
||||
valid_mask[:n-horizon] = True
|
||||
|
||||
# Only iterate relevant rows for efficiency
|
||||
candidates = np.where((z_score > 1.0) | (z_score < -1.0))[0]
|
||||
|
||||
for i in candidates:
|
||||
if i + horizon >= n:
|
||||
continue
|
||||
|
||||
entry_price = spread[i]
|
||||
future_prices = spread[i+1 : i+1+horizon]
|
||||
|
||||
if z_score[i] > 1.0: # Short
|
||||
target_price = entry_price * (1 - PROFIT_THRESHOLD)
|
||||
stop_price = entry_price * (1 + STOP_LOSS_PCT)
|
||||
|
||||
# Identify first hit indices
|
||||
hit_tp = future_prices <= target_price
|
||||
hit_sl = future_prices >= stop_price
|
||||
|
||||
if not np.any(hit_tp):
|
||||
targets[i] = 0 # Target never hit
|
||||
elif not np.any(hit_sl):
|
||||
targets[i] = 1 # Target hit, SL never hit
|
||||
else:
|
||||
first_tp_idx = np.argmax(hit_tp)
|
||||
first_sl_idx = np.argmax(hit_sl)
|
||||
|
||||
# Success if TP hit before SL
|
||||
if first_tp_idx < first_sl_idx:
|
||||
targets[i] = 1
|
||||
else:
|
||||
targets[i] = 0
|
||||
|
||||
else: # Long
|
||||
target_price = entry_price * (1 + PROFIT_THRESHOLD)
|
||||
stop_price = entry_price * (1 - STOP_LOSS_PCT)
|
||||
|
||||
hit_tp = future_prices >= target_price
|
||||
hit_sl = future_prices <= stop_price
|
||||
|
||||
if not np.any(hit_tp):
|
||||
targets[i] = 0
|
||||
elif not np.any(hit_sl):
|
||||
targets[i] = 1
|
||||
else:
|
||||
first_tp_idx = np.argmax(hit_tp)
|
||||
first_sl_idx = np.argmax(hit_sl)
|
||||
|
||||
if first_tp_idx < first_sl_idx:
|
||||
targets[i] = 1
|
||||
else:
|
||||
targets[i] = 0
|
||||
|
||||
return targets, pd.Series(valid_mask, index=features.index), None, None
|
||||
|
||||
|
||||
def calculate_mae(features, predictions, test_idx, horizon):
|
||||
"""Calculate Maximum Adverse Excursion for predicted trades."""
|
||||
test_features = features.loc[test_idx]
|
||||
spread = test_features['spread']
|
||||
z_score = test_features['z_score']
|
||||
|
||||
mae_values = []
|
||||
|
||||
for i, (idx, pred) in enumerate(zip(test_idx, predictions)):
|
||||
if pred != 1:
|
||||
continue
|
||||
|
||||
entry_spread = spread.loc[idx]
|
||||
z = z_score.loc[idx]
|
||||
|
||||
# Get future spread values
|
||||
future_idx = features.index.get_loc(idx)
|
||||
future_end = min(future_idx + horizon, len(features))
|
||||
future_spreads = features['spread'].iloc[future_idx:future_end]
|
||||
|
||||
if len(future_spreads) < 2:
|
||||
continue
|
||||
|
||||
if z > 1.0: # Short trade
|
||||
max_adverse = (future_spreads.max() - entry_spread) / entry_spread
|
||||
else: # Long trade
|
||||
max_adverse = (entry_spread - future_spreads.min()) / entry_spread
|
||||
|
||||
mae_values.append(max_adverse * 100) # As percentage
|
||||
|
||||
return np.mean(mae_values) if mae_values else 0.0
|
||||
|
||||
|
||||
def calculate_net_profit(features, predictions, test_idx, horizon):
|
||||
"""
|
||||
Calculate estimated net profit including fees.
|
||||
Enforces 'one trade at a time' and simulates SL/TP exits.
|
||||
"""
|
||||
test_features = features.loc[test_idx]
|
||||
spread = test_features['spread']
|
||||
z_score = test_features['z_score']
|
||||
|
||||
total_pnl = 0.0
|
||||
n_trades = 0
|
||||
|
||||
# Track when we are free to trade again
|
||||
next_trade_idx = 0
|
||||
|
||||
# Pre-calculate indices for speed
|
||||
all_indices = features.index
|
||||
|
||||
for i, (idx, pred) in enumerate(zip(test_idx, predictions)):
|
||||
# Skip if we are still in a trade
|
||||
if i < next_trade_idx:
|
||||
continue
|
||||
|
||||
if pred != 1:
|
||||
continue
|
||||
|
||||
entry_spread = spread.loc[idx]
|
||||
z = z_score.loc[idx]
|
||||
|
||||
# Get future spread values
|
||||
current_loc = features.index.get_loc(idx)
|
||||
future_end_loc = min(current_loc + horizon, len(features))
|
||||
future_spreads = features['spread'].iloc[current_loc+1 : future_end_loc]
|
||||
|
||||
if len(future_spreads) < 1:
|
||||
continue
|
||||
|
||||
pnl = 0.0
|
||||
trade_duration = len(future_spreads)
|
||||
|
||||
if z > 1.0: # Short trade
|
||||
tp_price = entry_spread * (1 - PROFIT_THRESHOLD)
|
||||
sl_price = entry_spread * (1 + STOP_LOSS_PCT)
|
||||
|
||||
hit_tp = future_spreads <= tp_price
|
||||
hit_sl = future_spreads >= sl_price
|
||||
|
||||
# Check what happened first
|
||||
first_tp = np.argmax(hit_tp.values) if hit_tp.any() else 99999
|
||||
first_sl = np.argmax(hit_sl.values) if hit_sl.any() else 99999
|
||||
|
||||
if first_sl < first_tp and first_sl < 99999:
|
||||
# Stopped out
|
||||
exit_price = future_spreads.iloc[first_sl] # Approx SL price
|
||||
# Use exact SL price for realistic simulation? Or close
|
||||
# Let's use the close price of the bar where it crossed
|
||||
pnl = (entry_spread - exit_price) / entry_spread
|
||||
trade_duration = first_sl + 1
|
||||
elif first_tp < first_sl and first_tp < 99999:
|
||||
# Take profit
|
||||
exit_price = future_spreads.iloc[first_tp]
|
||||
pnl = (entry_spread - exit_price) / entry_spread
|
||||
trade_duration = first_tp + 1
|
||||
else:
|
||||
# Held to horizon
|
||||
exit_price = future_spreads.iloc[-1]
|
||||
pnl = (entry_spread - exit_price) / entry_spread
|
||||
|
||||
else: # Long trade
|
||||
tp_price = entry_spread * (1 + PROFIT_THRESHOLD)
|
||||
sl_price = entry_spread * (1 - STOP_LOSS_PCT)
|
||||
|
||||
hit_tp = future_spreads >= tp_price
|
||||
hit_sl = future_spreads <= sl_price
|
||||
|
||||
first_tp = np.argmax(hit_tp.values) if hit_tp.any() else 99999
|
||||
first_sl = np.argmax(hit_sl.values) if hit_sl.any() else 99999
|
||||
|
||||
if first_sl < first_tp and first_sl < 99999:
|
||||
# Stopped out
|
||||
exit_price = future_spreads.iloc[first_sl]
|
||||
pnl = (exit_price - entry_spread) / entry_spread
|
||||
trade_duration = first_sl + 1
|
||||
elif first_tp < first_sl and first_tp < 99999:
|
||||
# Take profit
|
||||
exit_price = future_spreads.iloc[first_tp]
|
||||
pnl = (exit_price - entry_spread) / entry_spread
|
||||
trade_duration = first_tp + 1
|
||||
else:
|
||||
# Held to horizon
|
||||
exit_price = future_spreads.iloc[-1]
|
||||
pnl = (exit_price - entry_spread) / entry_spread
|
||||
|
||||
# Subtract fees
|
||||
net_pnl = pnl - FEE_RATE
|
||||
total_pnl += net_pnl
|
||||
n_trades += 1
|
||||
|
||||
# Set next available trade index
|
||||
next_trade_idx = i + trade_duration
|
||||
|
||||
return total_pnl, n_trades
|
||||
|
||||
|
||||
def test_horizon(features, horizon):
|
||||
"""Test a single horizon with walk-forward training."""
|
||||
# Calculate targets
|
||||
targets, valid_mask, _, _ = calculate_targets(features, horizon)
|
||||
|
||||
# Walk-forward split
|
||||
n_samples = len(features)
|
||||
train_size = int(n_samples * TRAIN_RATIO)
|
||||
|
||||
train_features = features.iloc[:train_size]
|
||||
test_features = features.iloc[train_size:]
|
||||
|
||||
train_targets = targets[:train_size]
|
||||
test_targets = targets[train_size:]
|
||||
|
||||
train_valid = valid_mask.iloc[:train_size]
|
||||
test_valid = valid_mask.iloc[train_size:]
|
||||
|
||||
# Prepare training data (only valid rows)
|
||||
exclude = ['spread']
|
||||
cols = [c for c in features.columns if c not in exclude]
|
||||
|
||||
X_train = train_features[cols].fillna(0).replace([np.inf, -np.inf], 0)
|
||||
X_train_valid = X_train[train_valid]
|
||||
y_train_valid = train_targets[train_valid]
|
||||
|
||||
if len(X_train_valid) < 50:
|
||||
return None # Not enough training data
|
||||
|
||||
# Train model
|
||||
model = RandomForestClassifier(
|
||||
n_estimators=300, max_depth=5, min_samples_leaf=30,
|
||||
class_weight={0: 1, 1: 3}, random_state=42
|
||||
)
|
||||
model.fit(X_train_valid, y_train_valid)
|
||||
|
||||
# Predict on test set
|
||||
X_test = test_features[cols].fillna(0).replace([np.inf, -np.inf], 0)
|
||||
predictions = model.predict(X_test)
|
||||
|
||||
# Only evaluate on valid test rows (those with complete future data)
|
||||
test_valid_mask = test_valid.values
|
||||
y_test_valid = test_targets[test_valid_mask]
|
||||
pred_valid = predictions[test_valid_mask]
|
||||
|
||||
if len(y_test_valid) < 10:
|
||||
return None
|
||||
|
||||
# Calculate metrics
|
||||
f1 = f1_score(y_test_valid, pred_valid, zero_division=0)
|
||||
|
||||
# Calculate MAE and Net Profit on ALL test predictions (not just valid targets)
|
||||
test_idx = test_features.index
|
||||
avg_mae = calculate_mae(features, predictions, test_idx, horizon)
|
||||
net_pnl, n_trades = calculate_net_profit(features, predictions, test_idx, horizon)
|
||||
|
||||
return {
|
||||
'horizon': horizon,
|
||||
'f1_score': f1,
|
||||
'avg_mae': avg_mae,
|
||||
'net_pnl': net_pnl,
|
||||
'n_trades': n_trades,
|
||||
'train_samples': len(X_train_valid),
|
||||
'test_samples': len(X_test)
|
||||
}
|
||||
|
||||
|
||||
def test_horizons(features, horizons):
|
||||
"""Test multiple horizons and return comparison."""
|
||||
results = []
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
print("WALK-FORWARD HORIZON OPTIMIZATION")
|
||||
print(f"Train Ratio: {TRAIN_RATIO*100:.0f}% | Profit Target: {PROFIT_THRESHOLD*100:.1f}% | Stop Loss: {STOP_LOSS_PCT*100:.1f}% | Fee Rate: {FEE_RATE*100:.2f}%")
|
||||
print("=" * 80)
|
||||
|
||||
for h in horizons:
|
||||
result = test_horizon(features, h)
|
||||
if result:
|
||||
results.append(result)
|
||||
print(f"Horizon {h:3d}h: F1={result['f1_score']:.3f}, "
|
||||
f"MAE={result['avg_mae']:.2f}%, "
|
||||
f"Net PnL={result['net_pnl']*100:.2f}%, "
|
||||
f"Trades={result['n_trades']}")
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def parse_args():
|
||||
"""Parse command line arguments."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Regime detection research - test multiple horizons"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--days",
|
||||
type=int,
|
||||
default=DEFAULT_DAYS,
|
||||
help=f"Number of days of data (default: {DEFAULT_DAYS})"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--start",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Start date (YYYY-MM-DD), overrides --days"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--end",
|
||||
type=str,
|
||||
default=None,
|
||||
help="End date (YYYY-MM-DD), defaults to now"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output",
|
||||
type=str,
|
||||
default="research/horizon_optimization_results.csv",
|
||||
help="Output CSV path"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-horizon",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Path to save the best horizon (integer) to a file"
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main():
|
||||
"""Main research function."""
|
||||
args = parse_args()
|
||||
|
||||
# Load data with dynamic date range
|
||||
df_btc, df_eth = load_data(
|
||||
days=args.days,
|
||||
start_date=args.start,
|
||||
end_date=args.end
|
||||
)
|
||||
cq_df = load_cryptoquant_data()
|
||||
|
||||
# Calculate features
|
||||
features = calculate_features(df_btc, df_eth, cq_df)
|
||||
logger.info(f"Calculated {len(features)} feature rows with {len(features.columns)} columns")
|
||||
|
||||
# Test horizons from 6h to 150h
|
||||
horizons = list(range(6, 151, 6)) # 6, 12, 18, ..., 150
|
||||
|
||||
results = test_horizons(features, horizons)
|
||||
|
||||
if not results:
|
||||
print("No valid results!")
|
||||
return None
|
||||
|
||||
# Find best by different metrics
|
||||
results_df = pd.DataFrame(results)
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
print("BEST HORIZONS BY METRIC")
|
||||
print("=" * 80)
|
||||
|
||||
best_f1 = results_df.loc[results_df['f1_score'].idxmax()]
|
||||
print(f"Best F1 Score: {best_f1['horizon']:.0f}h (F1={best_f1['f1_score']:.3f})")
|
||||
|
||||
best_pnl = results_df.loc[results_df['net_pnl'].idxmax()]
|
||||
print(f"Best Net PnL: {best_pnl['horizon']:.0f}h (PnL={best_pnl['net_pnl']*100:.2f}%)")
|
||||
|
||||
lowest_mae = results_df.loc[results_df['avg_mae'].idxmin()]
|
||||
print(f"Lowest MAE: {lowest_mae['horizon']:.0f}h (MAE={lowest_mae['avg_mae']:.2f}%)")
|
||||
|
||||
# Save results
|
||||
results_df.to_csv(args.output, index=False)
|
||||
print(f"\nResults saved to {args.output}")
|
||||
|
||||
# Save best horizon if requested
|
||||
if args.output_horizon:
|
||||
best_h = int(best_pnl['horizon'])
|
||||
with open(args.output_horizon, 'w') as f:
|
||||
f.write(str(best_h))
|
||||
print(f"Best horizon {best_h}h saved to {args.output_horizon}")
|
||||
|
||||
return results_df
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
BIN
research/regime_results.png
Normal file
BIN
research/regime_results.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 289 KiB |
80
strategies/base.py
Normal file
80
strategies/base.py
Normal file
@@ -0,0 +1,80 @@
|
||||
"""
|
||||
Base strategy class for all trading strategies.
|
||||
|
||||
Strategies should inherit from BaseStrategy and implement the run() method.
|
||||
"""
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from engine.market import MarketType
|
||||
|
||||
|
||||
class BaseStrategy(ABC):
|
||||
"""
|
||||
Abstract base class for trading strategies.
|
||||
|
||||
Class Attributes:
|
||||
default_market_type: Default market type for this strategy
|
||||
default_leverage: Default leverage (only applies to perpetuals)
|
||||
default_sl_stop: Default stop-loss percentage
|
||||
default_tp_stop: Default take-profit percentage
|
||||
default_sl_trail: Whether stop-loss is trailing by default
|
||||
"""
|
||||
# Market configuration defaults
|
||||
default_market_type: MarketType = MarketType.SPOT
|
||||
default_leverage: int = 1
|
||||
|
||||
# Risk management defaults (can be overridden per strategy)
|
||||
default_sl_stop: float | None = None
|
||||
default_tp_stop: float | None = None
|
||||
default_sl_trail: bool = False
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
self.params = kwargs
|
||||
|
||||
@abstractmethod
|
||||
def run(
|
||||
self,
|
||||
close: pd.Series,
|
||||
**kwargs
|
||||
) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]:
|
||||
"""
|
||||
Run the strategy logic.
|
||||
|
||||
Args:
|
||||
close: Price series (can be multiple columns for grid search)
|
||||
**kwargs: Additional data (high, low, open, volume) and parameters
|
||||
|
||||
Returns:
|
||||
Tuple of 4 DataFrames/Series:
|
||||
- long_entries: Boolean signals to open long positions
|
||||
- long_exits: Boolean signals to close long positions
|
||||
- short_entries: Boolean signals to open short positions
|
||||
- short_exits: Boolean signals to close short positions
|
||||
|
||||
Note:
|
||||
For spot markets, short signals will be ignored.
|
||||
For backward compatibility, strategies can return 2-tuple (entries, exits)
|
||||
which will be interpreted as long-only signals.
|
||||
"""
|
||||
pass
|
||||
|
||||
def get_indicator(self, ind_cls, *args, **kwargs):
|
||||
"""Helper to run a vectorbt indicator."""
|
||||
return ind_cls.run(*args, **kwargs)
|
||||
|
||||
@staticmethod
|
||||
def create_empty_signals(reference: pd.Series | pd.DataFrame) -> pd.DataFrame:
|
||||
"""
|
||||
Create an empty (all False) signal DataFrame matching the reference shape.
|
||||
|
||||
Args:
|
||||
reference: Series or DataFrame to match shape/index
|
||||
|
||||
Returns:
|
||||
DataFrame of False values with same shape as reference
|
||||
"""
|
||||
if isinstance(reference, pd.DataFrame):
|
||||
return pd.DataFrame(False, index=reference.index, columns=reference.columns)
|
||||
return pd.Series(False, index=reference.index)
|
||||
97
strategies/examples.py
Normal file
97
strategies/examples.py
Normal file
@@ -0,0 +1,97 @@
|
||||
"""
|
||||
Example trading strategies for backtesting.
|
||||
|
||||
These are simple strategies demonstrating the framework usage.
|
||||
"""
|
||||
import pandas as pd
|
||||
import vectorbt as vbt
|
||||
|
||||
from engine.market import MarketType
|
||||
from strategies.base import BaseStrategy
|
||||
|
||||
|
||||
class RsiStrategy(BaseStrategy):
|
||||
"""
|
||||
RSI mean-reversion strategy.
|
||||
|
||||
Long entry when RSI crosses below oversold level.
|
||||
Long exit when RSI crosses above overbought level.
|
||||
"""
|
||||
default_market_type = MarketType.SPOT
|
||||
default_leverage = 1
|
||||
|
||||
def run(
|
||||
self,
|
||||
close: pd.Series,
|
||||
period: int = 14,
|
||||
rsi_lower: int = 30,
|
||||
rsi_upper: int = 70,
|
||||
**kwargs
|
||||
) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]:
|
||||
"""
|
||||
Generate RSI-based trading signals.
|
||||
|
||||
Args:
|
||||
close: Price series
|
||||
period: RSI calculation period
|
||||
rsi_lower: Oversold threshold (buy signal)
|
||||
rsi_upper: Overbought threshold (sell signal)
|
||||
|
||||
Returns:
|
||||
4-tuple of (long_entries, long_exits, short_entries, short_exits)
|
||||
"""
|
||||
# Calculate RSI
|
||||
rsi = vbt.RSI.run(close, window=period)
|
||||
|
||||
# Long signals: buy oversold, sell overbought
|
||||
long_entries = rsi.rsi_crossed_below(rsi_lower)
|
||||
long_exits = rsi.rsi_crossed_above(rsi_upper)
|
||||
|
||||
# No short signals for this strategy (spot-focused)
|
||||
short_entries = BaseStrategy.create_empty_signals(long_entries)
|
||||
short_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
|
||||
return long_entries, long_exits, short_entries, short_exits
|
||||
|
||||
|
||||
class MaCrossStrategy(BaseStrategy):
|
||||
"""
|
||||
Moving Average crossover strategy.
|
||||
|
||||
Long entry when fast MA crosses above slow MA.
|
||||
Long exit when fast MA crosses below slow MA.
|
||||
"""
|
||||
default_market_type = MarketType.SPOT
|
||||
default_leverage = 1
|
||||
|
||||
def run(
|
||||
self,
|
||||
close: pd.Series,
|
||||
fast_window: int = 10,
|
||||
slow_window: int = 20,
|
||||
**kwargs
|
||||
) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]:
|
||||
"""
|
||||
Generate MA crossover trading signals.
|
||||
|
||||
Args:
|
||||
close: Price series
|
||||
fast_window: Fast MA period
|
||||
slow_window: Slow MA period
|
||||
|
||||
Returns:
|
||||
4-tuple of (long_entries, long_exits, short_entries, short_exits)
|
||||
"""
|
||||
# Calculate Moving Averages
|
||||
fast_ma = vbt.MA.run(close, window=fast_window)
|
||||
slow_ma = vbt.MA.run(close, window=slow_window)
|
||||
|
||||
# Long signals
|
||||
long_entries = fast_ma.ma_crossed_above(slow_ma)
|
||||
long_exits = fast_ma.ma_crossed_below(slow_ma)
|
||||
|
||||
# No short signals for this strategy
|
||||
short_entries = BaseStrategy.create_empty_signals(long_entries)
|
||||
short_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
|
||||
return long_entries, long_exits, short_entries, short_exits
|
||||
151
strategies/factory.py
Normal file
151
strategies/factory.py
Normal file
@@ -0,0 +1,151 @@
|
||||
"""
|
||||
Strategy factory for creating strategy instances with their parameters.
|
||||
|
||||
Centralizes strategy creation and parameter configuration.
|
||||
"""
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
from strategies.base import BaseStrategy
|
||||
|
||||
|
||||
@dataclass
|
||||
class StrategyConfig:
|
||||
"""
|
||||
Configuration for a strategy including default and grid parameters.
|
||||
|
||||
Attributes:
|
||||
strategy_class: The strategy class to instantiate
|
||||
default_params: Parameters for single backtest runs
|
||||
grid_params: Parameters for grid search optimization
|
||||
"""
|
||||
strategy_class: type[BaseStrategy]
|
||||
default_params: dict[str, Any] = field(default_factory=dict)
|
||||
grid_params: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
def _build_registry() -> dict[str, StrategyConfig]:
|
||||
"""
|
||||
Build the strategy registry lazily to avoid circular imports.
|
||||
|
||||
Returns:
|
||||
Dictionary mapping strategy names to their configurations
|
||||
"""
|
||||
# Import here to avoid circular imports
|
||||
from strategies.examples import MaCrossStrategy, RsiStrategy
|
||||
from strategies.supertrend import MetaSupertrendStrategy
|
||||
from strategies.regime_strategy import RegimeReversionStrategy
|
||||
|
||||
return {
|
||||
"rsi": StrategyConfig(
|
||||
strategy_class=RsiStrategy,
|
||||
default_params={
|
||||
'period': 14,
|
||||
'rsi_lower': 30,
|
||||
'rsi_upper': 70
|
||||
},
|
||||
grid_params={
|
||||
'period': np.arange(10, 25, 2),
|
||||
'rsi_lower': [20, 30, 40],
|
||||
'rsi_upper': [60, 70, 80]
|
||||
}
|
||||
),
|
||||
"macross": StrategyConfig(
|
||||
strategy_class=MaCrossStrategy,
|
||||
default_params={
|
||||
'fast_window': 10,
|
||||
'slow_window': 20
|
||||
},
|
||||
grid_params={
|
||||
'fast_window': np.arange(5, 20, 5),
|
||||
'slow_window': np.arange(20, 60, 10)
|
||||
}
|
||||
),
|
||||
"meta_st": StrategyConfig(
|
||||
strategy_class=MetaSupertrendStrategy,
|
||||
default_params={
|
||||
'period1': 12, 'multiplier1': 3.0,
|
||||
'period2': 10, 'multiplier2': 1.0,
|
||||
'period3': 11, 'multiplier3': 2.0
|
||||
},
|
||||
grid_params={
|
||||
'multiplier1': [2.0, 3.0, 4.0],
|
||||
'period1': [10, 12, 14],
|
||||
'period2': 11, 'multiplier2': 2.0,
|
||||
'period3': 12, 'multiplier3': 1.0
|
||||
}
|
||||
),
|
||||
"regime": StrategyConfig(
|
||||
strategy_class=RegimeReversionStrategy,
|
||||
default_params={
|
||||
# Optimal from walk-forward research (research/horizon_optimization_results.csv)
|
||||
'horizon': 102, # 4.25 days - best Net PnL
|
||||
'z_window': 24, # 24h rolling Z-score window
|
||||
'z_entry_threshold': 1.0, # Enter when |Z| > 1.0
|
||||
'profit_target': 0.005, # 0.5% target for ML labels
|
||||
'stop_loss': 0.06, # 6% stop loss
|
||||
'take_profit': 0.05, # 5% take profit
|
||||
'train_ratio': 0.7, # 70% train / 30% test
|
||||
'trend_window': 0, # Disabled SMA filter
|
||||
'use_funding_filter': True, # Enabled Funding filter
|
||||
'funding_threshold': 0.005 # 0.005% threshold (Proven profitable)
|
||||
},
|
||||
grid_params={
|
||||
'horizon': [84, 96, 102, 108, 120],
|
||||
'z_entry_threshold': [0.8, 1.0, 1.2],
|
||||
'stop_loss': [0.04, 0.06, 0.08],
|
||||
'funding_threshold': [0.005, 0.01, 0.02]
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
# Module-level cache for the registry
|
||||
_REGISTRY_CACHE: dict[str, StrategyConfig] | None = None
|
||||
|
||||
|
||||
def get_registry() -> dict[str, StrategyConfig]:
|
||||
"""Get the strategy registry, building it on first access."""
|
||||
global _REGISTRY_CACHE
|
||||
if _REGISTRY_CACHE is None:
|
||||
_REGISTRY_CACHE = _build_registry()
|
||||
return _REGISTRY_CACHE
|
||||
|
||||
|
||||
def get_strategy_names() -> list[str]:
|
||||
"""
|
||||
Get list of available strategy names.
|
||||
|
||||
Returns:
|
||||
List of strategy name strings
|
||||
"""
|
||||
return list(get_registry().keys())
|
||||
|
||||
|
||||
def get_strategy(name: str, is_grid: bool = False) -> tuple[BaseStrategy, dict[str, Any]]:
|
||||
"""
|
||||
Create a strategy instance with appropriate parameters.
|
||||
|
||||
Args:
|
||||
name: Strategy identifier (e.g., 'rsi', 'macross', 'meta_st')
|
||||
is_grid: If True, return grid search parameters
|
||||
|
||||
Returns:
|
||||
Tuple of (strategy instance, parameters dict)
|
||||
|
||||
Raises:
|
||||
KeyError: If strategy name is not found in registry
|
||||
"""
|
||||
registry = get_registry()
|
||||
|
||||
if name not in registry:
|
||||
available = ", ".join(registry.keys())
|
||||
raise KeyError(f"Unknown strategy '{name}'. Available: {available}")
|
||||
|
||||
config = registry[name]
|
||||
strategy = config.strategy_class()
|
||||
params = config.grid_params if is_grid else config.default_params
|
||||
|
||||
return strategy, params.copy()
|
||||
407
strategies/regime_strategy.py
Normal file
407
strategies/regime_strategy.py
Normal file
@@ -0,0 +1,407 @@
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
import ta
|
||||
import vectorbt as vbt
|
||||
from sklearn.ensemble import RandomForestClassifier
|
||||
|
||||
from strategies.base import BaseStrategy
|
||||
from engine.market import MarketType
|
||||
from engine.data_manager import DataManager
|
||||
from engine.logging_config import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
class RegimeReversionStrategy(BaseStrategy):
|
||||
"""
|
||||
ML-Based Regime Detection & Mean Reversion Strategy.
|
||||
|
||||
Logic:
|
||||
1. Tracks the BTC/ETH Spread and its Z-Score (24h window).
|
||||
2. Uses a Random Forest model to predict if an extreme Z-Score will revert profitably.
|
||||
3. Features: Spread Technicals (RSI, ROC) + On-Chain Flows (Inflow, Funding).
|
||||
4. Entry: When Model Probability > 0.5.
|
||||
5. Exit: Z-Score reversion to 0 or SL/TP.
|
||||
|
||||
Walk-Forward Training:
|
||||
- Trains on first `train_ratio` of data (default 70%)
|
||||
- Generates signals only for remaining test period (30%)
|
||||
- Eliminates look-ahead bias for realistic backtest results
|
||||
"""
|
||||
|
||||
# Optimal parameters from walk-forward research (2025-10 to 2025-12)
|
||||
# Research: research/horizon_optimization_results.csv
|
||||
OPTIMAL_HORIZON = 54 # Updated from 102h based on corrected labeling
|
||||
OPTIMAL_Z_WINDOW = 24 # 24h rolling window for spread Z-score
|
||||
OPTIMAL_TRAIN_RATIO = 0.7 # 70% train / 30% test split
|
||||
OPTIMAL_PROFIT_TARGET = 0.005 # 0.5% profit threshold for target definition
|
||||
OPTIMAL_Z_ENTRY = 1.0 # Enter when |Z| > 1.0
|
||||
|
||||
def __init__(self,
|
||||
model_path: str = "data/regime_model.pkl",
|
||||
horizon: int = OPTIMAL_HORIZON,
|
||||
z_window: int = OPTIMAL_Z_WINDOW,
|
||||
z_entry_threshold: float = OPTIMAL_Z_ENTRY,
|
||||
profit_target: float = OPTIMAL_PROFIT_TARGET,
|
||||
stop_loss: float = 0.06, # 6% - accommodates 1.95% avg MAE
|
||||
take_profit: float = 0.05, # 5% swing target
|
||||
train_ratio: float = OPTIMAL_TRAIN_RATIO,
|
||||
trend_window: int = 0, # Disable SMA filter
|
||||
use_funding_filter: bool = True, # Enable Funding Rate filter
|
||||
funding_threshold: float = 0.005 # Tightened to 0.005%
|
||||
):
|
||||
super().__init__()
|
||||
self.model_path = model_path
|
||||
self.horizon = horizon
|
||||
self.z_window = z_window
|
||||
self.z_entry_threshold = z_entry_threshold
|
||||
self.profit_target = profit_target
|
||||
self.stop_loss = stop_loss
|
||||
self.take_profit = take_profit
|
||||
self.train_ratio = train_ratio
|
||||
self.trend_window = trend_window
|
||||
self.use_funding_filter = use_funding_filter
|
||||
self.funding_threshold = funding_threshold
|
||||
|
||||
# Default Strategy Config
|
||||
self.default_market_type = MarketType.PERPETUAL
|
||||
self.default_leverage = 1
|
||||
|
||||
self.dm = DataManager()
|
||||
self.model = None
|
||||
self.feature_cols = None
|
||||
self.train_end_idx = None # Will store the training cutoff point
|
||||
|
||||
def run(self, close, **kwargs):
|
||||
"""
|
||||
Execute the strategy logic.
|
||||
We assume this strategy is run on ETH-USDT (the active asset).
|
||||
We will fetch BTC-USDT internally to calculate the spread.
|
||||
"""
|
||||
# 1. Identify Context
|
||||
# We need BTC data aligned with the incoming ETH 'close' series
|
||||
start_date = close.index.min()
|
||||
end_date = close.index.max()
|
||||
|
||||
logger.info("Fetching BTC context data...")
|
||||
try:
|
||||
# Load BTC data (Context) - Must match the timeframe of the backtest
|
||||
# Research was done on 1h candles, so strategy should be run on 1h
|
||||
# Use PERPETUAL data to match the trading instrument (ETH Perp)
|
||||
df_btc = self.dm.load_data("okx", "BTC-USDT", "1h", MarketType.PERPETUAL)
|
||||
|
||||
# Align BTC to ETH (close)
|
||||
df_btc = df_btc.reindex(close.index, method='ffill')
|
||||
btc_close = df_btc['close']
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to load BTC context: {e}")
|
||||
empty = self.create_empty_signals(close)
|
||||
return empty, empty, empty, empty
|
||||
|
||||
# 2. Construct DataFrames for Feature Engineering
|
||||
# We need volume/high/low for features, but 'run' signature primarily gives 'close'.
|
||||
# kwargs might have high/low/volume if passed by Backtester.run_strategy
|
||||
eth_vol = kwargs.get('volume')
|
||||
|
||||
if eth_vol is None:
|
||||
logger.warning("Volume data missing. Feature calculation might fail.")
|
||||
# Fallback or error handling
|
||||
eth_vol = pd.Series(0, index=close.index)
|
||||
|
||||
# Construct dummy dfs for prepare_features
|
||||
# We only really need Close and Volume for the current feature set
|
||||
df_a = pd.DataFrame({'close': btc_close, 'volume': df_btc['volume']})
|
||||
df_b = pd.DataFrame({'close': close, 'volume': eth_vol})
|
||||
|
||||
# 3. Load On-Chain Data (CryptoQuant)
|
||||
# We use the saved CSV for training/inference
|
||||
# In a live setting, this would query the API for recent data
|
||||
cq_df = None
|
||||
try:
|
||||
cq_path = "data/cq_training_data.csv"
|
||||
cq_df = pd.read_csv(cq_path, index_col='timestamp', parse_dates=True)
|
||||
if cq_df.index.tz is None:
|
||||
cq_df.index = cq_df.index.tz_localize('UTC')
|
||||
except Exception:
|
||||
logger.warning("CryptoQuant data not found. Running without on-chain features.")
|
||||
|
||||
# 4. Calculate Features
|
||||
features = self.prepare_features(df_a, df_b, cq_df)
|
||||
|
||||
# 5. Walk-Forward Split
|
||||
# Train on first `train_ratio` of data, test on remainder
|
||||
n_samples = len(features)
|
||||
train_size = int(n_samples * self.train_ratio)
|
||||
|
||||
train_features = features.iloc[:train_size]
|
||||
test_features = features.iloc[train_size:]
|
||||
|
||||
train_end_date = train_features.index[-1]
|
||||
test_start_date = test_features.index[0]
|
||||
|
||||
logger.info(
|
||||
f"Walk-Forward Split: Train={len(train_features)} bars "
|
||||
f"(until {train_end_date.strftime('%Y-%m-%d')}), "
|
||||
f"Test={len(test_features)} bars "
|
||||
f"(from {test_start_date.strftime('%Y-%m-%d')})"
|
||||
)
|
||||
|
||||
# 6. Train Model on Training Period ONLY
|
||||
if self.model is None:
|
||||
logger.info("Training Regime Model on training period only...")
|
||||
self.model, self.feature_cols = self.train_model(train_features)
|
||||
|
||||
# 7. Predict on TEST Period ONLY
|
||||
# Use valid columns only
|
||||
X_test = test_features[self.feature_cols].fillna(0)
|
||||
X_test = X_test.replace([np.inf, -np.inf], 0)
|
||||
|
||||
# Predict Probabilities for test period
|
||||
probs = self.model.predict_proba(X_test)[:, 1]
|
||||
|
||||
# 8. Generate Entry Signals (TEST period only)
|
||||
# If Z > threshold (Spread High, ETH Expensive) -> Short ETH
|
||||
# If Z < -threshold (Spread Low, ETH Cheap) -> Long ETH
|
||||
z_thresh = self.z_entry_threshold
|
||||
|
||||
short_signal_test = (probs > 0.5) & (test_features['z_score'].values > z_thresh)
|
||||
long_signal_test = (probs > 0.5) & (test_features['z_score'].values < -z_thresh)
|
||||
|
||||
# 8b. Apply Trend Filter (Macro Regime)
|
||||
# Rule: Long only if BTC > SMA (Bull), Short only if BTC < SMA (Bear)
|
||||
if self.trend_window > 0:
|
||||
# Calculate SMA on full BTC history first
|
||||
btc_sma = btc_close.rolling(window=self.trend_window).mean()
|
||||
|
||||
# Align with test period
|
||||
test_btc_close = btc_close.reindex(test_features.index)
|
||||
test_btc_sma = btc_sma.reindex(test_features.index)
|
||||
|
||||
# Define Regimes
|
||||
is_bull = (test_btc_close > test_btc_sma).values
|
||||
is_bear = (test_btc_close < test_btc_sma).values
|
||||
|
||||
# Apply Filter
|
||||
long_signal_test = long_signal_test & is_bull
|
||||
short_signal_test = short_signal_test & is_bear
|
||||
|
||||
# 8c. Apply Funding Rate Filter
|
||||
# Rule: If Funding > Threshold (Greedy) -> No Longs.
|
||||
# If Funding < -Threshold (Fearful) -> No Shorts.
|
||||
if self.use_funding_filter and 'btc_funding' in test_features.columns:
|
||||
funding = test_features['btc_funding'].values
|
||||
thresh = self.funding_threshold
|
||||
|
||||
# Greedy Market (High Positive Funding) -> Risk of Long Squeeze -> Block Longs
|
||||
# (Or implies trend is up? Actually for Mean Reversion, high funding often marks tops)
|
||||
# We block Longs because we don't want to buy into an overheated market?
|
||||
# Actually, "Greedy" means Longs are paying Shorts.
|
||||
# If we Long, we pay funding.
|
||||
# If we Short, we receive funding.
|
||||
# So High Funding = Good for Shorts (receive yield + reversion).
|
||||
# Bad for Longs (pay yield + likely top).
|
||||
|
||||
is_overheated = funding > thresh
|
||||
is_oversold = funding < -thresh
|
||||
|
||||
# Block Longs if Overheated
|
||||
long_signal_test = long_signal_test & (~is_overheated)
|
||||
|
||||
# Block Shorts if Oversold (Negative Funding) -> Risk of Short Squeeze
|
||||
short_signal_test = short_signal_test & (~is_oversold)
|
||||
|
||||
n_blocked_long = (is_overheated & (probs > 0.5) & (test_features['z_score'].values < -z_thresh)).sum()
|
||||
n_blocked_short = (is_oversold & (probs > 0.5) & (test_features['z_score'].values > z_thresh)).sum()
|
||||
|
||||
if n_blocked_long > 0 or n_blocked_short > 0:
|
||||
logger.info(f"Funding Filter: Blocked {n_blocked_long} Longs, {n_blocked_short} Shorts")
|
||||
|
||||
# 9. Calculate Position Sizing (Probability-Based)
|
||||
# Base size = 1.0 (100% of equity)
|
||||
# Scale: 1.0 + (Prob - 0.5) * 2
|
||||
# Example: Prob=0.6 -> Size=1.2, Prob=0.8 -> Size=1.6
|
||||
|
||||
# Align probabilities to close index
|
||||
probs_series = pd.Series(0.0, index=test_features.index)
|
||||
probs_series[:] = probs
|
||||
probs_aligned = probs_series.reindex(close.index, fill_value=0.0)
|
||||
|
||||
# Calculate dynamic size
|
||||
dynamic_size = 1.0 + (probs_aligned - 0.5) * 2.0
|
||||
# Cap leverage between 1x and 2x
|
||||
size = dynamic_size.clip(lower=1.0, upper=2.0)
|
||||
|
||||
# Create full-length signal series (False for training period)
|
||||
long_entries = pd.Series(False, index=close.index)
|
||||
short_entries = pd.Series(False, index=close.index)
|
||||
|
||||
# Map test signals to their correct indices
|
||||
test_idx = test_features.index
|
||||
for i, idx in enumerate(test_idx):
|
||||
if idx in close.index:
|
||||
long_entries.loc[idx] = bool(long_signal_test[i])
|
||||
short_entries.loc[idx] = bool(short_signal_test[i])
|
||||
|
||||
# 9. Generate Exits
|
||||
# Exit when Z-Score crosses back through 0 (mean reversion complete)
|
||||
z_reindexed = features['z_score'].reindex(close.index, fill_value=0)
|
||||
|
||||
# Exit Long when Z > 0, Exit Short when Z < 0
|
||||
long_exits = z_reindexed > 0
|
||||
short_exits = z_reindexed < 0
|
||||
|
||||
# Log signal counts for verification
|
||||
n_long = long_entries.sum()
|
||||
n_short = short_entries.sum()
|
||||
logger.info(f"Generated {n_long} long signals, {n_short} short signals (test period only)")
|
||||
|
||||
return long_entries, long_exits, short_entries, short_exits, size
|
||||
|
||||
def prepare_features(self, df_btc, df_eth, cq_df=None):
|
||||
"""Replicate research feature engineering"""
|
||||
# Align
|
||||
common = df_btc.index.intersection(df_eth.index)
|
||||
df_a = df_btc.loc[common].copy()
|
||||
df_b = df_eth.loc[common].copy()
|
||||
|
||||
# Spread
|
||||
spread = df_b['close'] / df_a['close']
|
||||
|
||||
# Z-Score
|
||||
rolling_mean = spread.rolling(window=self.z_window).mean()
|
||||
rolling_std = spread.rolling(window=self.z_window).std()
|
||||
z_score = (spread - rolling_mean) / rolling_std
|
||||
|
||||
# Technicals
|
||||
spread_rsi = ta.momentum.RSIIndicator(spread, window=14).rsi()
|
||||
spread_roc = spread.pct_change(periods=5) * 100
|
||||
spread_change_1h = spread.pct_change(periods=1)
|
||||
|
||||
# Volume
|
||||
vol_ratio = df_b['volume'] / df_a['volume']
|
||||
vol_ratio_ma = vol_ratio.rolling(window=12).mean()
|
||||
|
||||
# Volatility
|
||||
ret_a = df_a['close'].pct_change()
|
||||
ret_b = df_b['close'].pct_change()
|
||||
vol_a = ret_a.rolling(window=self.z_window).std()
|
||||
vol_b = ret_b.rolling(window=self.z_window).std()
|
||||
vol_spread_ratio = vol_b / vol_a
|
||||
|
||||
features = pd.DataFrame(index=spread.index)
|
||||
features['spread'] = spread
|
||||
features['z_score'] = z_score
|
||||
features['spread_rsi'] = spread_rsi
|
||||
features['spread_roc'] = spread_roc
|
||||
features['spread_change_1h'] = spread_change_1h
|
||||
features['vol_ratio'] = vol_ratio
|
||||
features['vol_ratio_rel'] = vol_ratio / vol_ratio_ma
|
||||
features['vol_diff_ratio'] = vol_spread_ratio
|
||||
|
||||
# CQ Merge
|
||||
if cq_df is not None:
|
||||
cq_aligned = cq_df.reindex(features.index, method='ffill')
|
||||
if 'btc_funding' in cq_aligned.columns and 'eth_funding' in cq_aligned.columns:
|
||||
cq_aligned['funding_diff'] = cq_aligned['eth_funding'] - cq_aligned['btc_funding']
|
||||
if 'btc_inflow' in cq_aligned.columns and 'eth_inflow' in cq_aligned.columns:
|
||||
cq_aligned['inflow_ratio'] = cq_aligned['eth_inflow'] / (cq_aligned['btc_inflow'] + 1)
|
||||
features = features.join(cq_aligned)
|
||||
|
||||
return features.dropna()
|
||||
|
||||
def train_model(self, train_features):
|
||||
"""
|
||||
Train Random Forest on training data only.
|
||||
|
||||
This method receives ONLY the training subset of features,
|
||||
ensuring no look-ahead bias. The model learns from historical
|
||||
patterns and is then applied to unseen test data.
|
||||
|
||||
Args:
|
||||
train_features: DataFrame containing features for training period only
|
||||
"""
|
||||
threshold = self.profit_target
|
||||
stop_loss_pct = self.stop_loss
|
||||
horizon = self.horizon
|
||||
z_thresh = self.z_entry_threshold
|
||||
|
||||
# Calculate targets path-dependently (checking SL before TP)
|
||||
spread = train_features['spread'].values
|
||||
z_score = train_features['z_score'].values
|
||||
n = len(spread)
|
||||
|
||||
targets = np.zeros(n, dtype=int)
|
||||
|
||||
# Only iterate relevant rows for efficiency
|
||||
candidates = np.where((z_score > z_thresh) | (z_score < -z_thresh))[0]
|
||||
|
||||
for i in candidates:
|
||||
if i + horizon >= n:
|
||||
continue
|
||||
|
||||
entry_price = spread[i]
|
||||
future_prices = spread[i+1 : i+1+horizon]
|
||||
|
||||
if z_score[i] > z_thresh: # Short
|
||||
target_price = entry_price * (1 - threshold)
|
||||
stop_price = entry_price * (1 + stop_loss_pct)
|
||||
|
||||
hit_tp = future_prices <= target_price
|
||||
hit_sl = future_prices >= stop_price
|
||||
|
||||
if not np.any(hit_tp):
|
||||
targets[i] = 0
|
||||
elif not np.any(hit_sl):
|
||||
targets[i] = 1
|
||||
else:
|
||||
first_tp_idx = np.argmax(hit_tp)
|
||||
first_sl_idx = np.argmax(hit_sl)
|
||||
if first_tp_idx < first_sl_idx:
|
||||
targets[i] = 1
|
||||
else:
|
||||
targets[i] = 0
|
||||
|
||||
else: # Long
|
||||
target_price = entry_price * (1 + threshold)
|
||||
stop_price = entry_price * (1 - stop_loss_pct)
|
||||
|
||||
hit_tp = future_prices >= target_price
|
||||
hit_sl = future_prices <= stop_price
|
||||
|
||||
if not np.any(hit_tp):
|
||||
targets[i] = 0
|
||||
elif not np.any(hit_sl):
|
||||
targets[i] = 1
|
||||
else:
|
||||
first_tp_idx = np.argmax(hit_tp)
|
||||
first_sl_idx = np.argmax(hit_sl)
|
||||
if first_tp_idx < first_sl_idx:
|
||||
targets[i] = 1
|
||||
else:
|
||||
targets[i] = 0
|
||||
|
||||
# Build model
|
||||
model = RandomForestClassifier(
|
||||
n_estimators=300, max_depth=5, min_samples_leaf=30,
|
||||
class_weight={0: 1, 1: 3}, random_state=42
|
||||
)
|
||||
|
||||
# Exclude non-feature columns
|
||||
exclude = ['spread']
|
||||
cols = [c for c in train_features.columns if c not in exclude]
|
||||
|
||||
# Clean features
|
||||
X_train = train_features[cols].fillna(0)
|
||||
X_train = X_train.replace([np.inf, -np.inf], 0)
|
||||
|
||||
# Use rows where we had enough data to look ahead
|
||||
valid_mask = np.zeros(n, dtype=bool)
|
||||
valid_mask[:n-horizon] = True
|
||||
|
||||
X_train_clean = X_train[valid_mask]
|
||||
targets_clean = targets[valid_mask]
|
||||
|
||||
logger.info(f"Training on {len(X_train_clean)} valid samples (removed {len(X_train) - len(X_train_clean)} with incomplete future data)")
|
||||
|
||||
model.fit(X_train_clean, targets_clean)
|
||||
return model, cols
|
||||
6
strategies/supertrend/__init__.py
Normal file
6
strategies/supertrend/__init__.py
Normal file
@@ -0,0 +1,6 @@
|
||||
"""
|
||||
Meta Supertrend strategy package.
|
||||
"""
|
||||
from .strategy import MetaSupertrendStrategy
|
||||
|
||||
__all__ = ['MetaSupertrendStrategy']
|
||||
128
strategies/supertrend/indicators.py
Normal file
128
strategies/supertrend/indicators.py
Normal file
@@ -0,0 +1,128 @@
|
||||
"""
|
||||
Supertrend indicators and helper functions.
|
||||
"""
|
||||
import numpy as np
|
||||
import vectorbt as vbt
|
||||
from numba import njit
|
||||
|
||||
# --- Numba Compiled Helper Functions ---
|
||||
|
||||
@njit(cache=False) # Disable cache to avoid stale compilation issues
|
||||
def get_tr_nb(high, low, close):
|
||||
"""Calculate True Range (Numba compiled)."""
|
||||
# Ensure 1D arrays
|
||||
high = high.ravel()
|
||||
low = low.ravel()
|
||||
close = close.ravel()
|
||||
|
||||
tr = np.empty_like(close)
|
||||
tr[0] = high[0] - low[0]
|
||||
for i in range(1, len(close)):
|
||||
tr[i] = max(high[i] - low[i], abs(high[i] - close[i-1]), abs(low[i] - close[i-1]))
|
||||
return tr
|
||||
|
||||
@njit(cache=False)
|
||||
def get_atr_nb(high, low, close, period):
|
||||
"""Calculate ATR using Wilder's Smoothing (Numba compiled)."""
|
||||
# Ensure 1D arrays
|
||||
high = high.ravel()
|
||||
low = low.ravel()
|
||||
close = close.ravel()
|
||||
|
||||
# Ensure period is native Python int (critical for Numba array indexing)
|
||||
n = len(close)
|
||||
p = int(period)
|
||||
|
||||
tr = get_tr_nb(high, low, close)
|
||||
atr = np.full(n, np.nan, dtype=np.float64)
|
||||
|
||||
if n < p:
|
||||
return atr
|
||||
|
||||
# Initial ATR is simple average of TR
|
||||
sum_tr = 0.0
|
||||
for i in range(p):
|
||||
sum_tr += tr[i]
|
||||
atr[p - 1] = sum_tr / p
|
||||
|
||||
# Subsequent ATR is Wilder's smoothed
|
||||
for i in range(p, n):
|
||||
atr[i] = (atr[i - 1] * (p - 1) + tr[i]) / p
|
||||
|
||||
return atr
|
||||
|
||||
@njit(cache=False)
|
||||
def get_supertrend_nb(high, low, close, period, multiplier):
|
||||
"""Calculate SuperTrend completely in Numba."""
|
||||
# Ensure 1D arrays
|
||||
high = high.ravel()
|
||||
low = low.ravel()
|
||||
close = close.ravel()
|
||||
|
||||
# Ensure params are native Python types (critical for Numba)
|
||||
n = len(close)
|
||||
p = int(period)
|
||||
m = float(multiplier)
|
||||
|
||||
atr = get_atr_nb(high, low, close, p)
|
||||
|
||||
final_upper = np.full(n, np.nan, dtype=np.float64)
|
||||
final_lower = np.full(n, np.nan, dtype=np.float64)
|
||||
trend = np.ones(n, dtype=np.int8) # 1 Bull, -1 Bear
|
||||
|
||||
# Skip until we have valid ATR
|
||||
start_idx = p
|
||||
if start_idx >= n:
|
||||
return trend
|
||||
|
||||
# Init first valid point
|
||||
hl2 = (high[start_idx] + low[start_idx]) / 2
|
||||
final_upper[start_idx] = hl2 + m * atr[start_idx]
|
||||
final_lower[start_idx] = hl2 - m * atr[start_idx]
|
||||
|
||||
# Loop
|
||||
for i in range(start_idx + 1, n):
|
||||
cur_hl2 = (high[i] + low[i]) / 2
|
||||
cur_atr = atr[i]
|
||||
basic_upper = cur_hl2 + m * cur_atr
|
||||
basic_lower = cur_hl2 - m * cur_atr
|
||||
|
||||
# Upper Band Logic
|
||||
if basic_upper < final_upper[i-1] or close[i-1] > final_upper[i-1]:
|
||||
final_upper[i] = basic_upper
|
||||
else:
|
||||
final_upper[i] = final_upper[i-1]
|
||||
|
||||
# Lower Band Logic
|
||||
if basic_lower > final_lower[i-1] or close[i-1] < final_lower[i-1]:
|
||||
final_lower[i] = basic_lower
|
||||
else:
|
||||
final_lower[i] = final_lower[i-1]
|
||||
|
||||
# Trend Logic
|
||||
if trend[i-1] == 1:
|
||||
if close[i] < final_lower[i-1]:
|
||||
trend[i] = -1
|
||||
else:
|
||||
trend[i] = 1
|
||||
else:
|
||||
if close[i] > final_upper[i-1]:
|
||||
trend[i] = 1
|
||||
else:
|
||||
trend[i] = -1
|
||||
|
||||
return trend
|
||||
|
||||
# --- VectorBT Indicator Factory ---
|
||||
|
||||
SuperTrendIndicator = vbt.IndicatorFactory(
|
||||
class_name='SuperTrend',
|
||||
short_name='st',
|
||||
input_names=['high', 'low', 'close'],
|
||||
param_names=['period', 'multiplier'],
|
||||
output_names=['trend']
|
||||
).from_apply_func(
|
||||
get_supertrend_nb,
|
||||
keep_pd=False, # Disable automatic Pandas wrapping of inputs
|
||||
param_product=True # Enable Cartesian product for list params
|
||||
)
|
||||
142
strategies/supertrend/strategy.py
Normal file
142
strategies/supertrend/strategy.py
Normal file
@@ -0,0 +1,142 @@
|
||||
"""
|
||||
Meta Supertrend strategy implementation.
|
||||
"""
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from engine.market import MarketType
|
||||
from strategies.base import BaseStrategy
|
||||
from .indicators import SuperTrendIndicator
|
||||
|
||||
|
||||
class MetaSupertrendStrategy(BaseStrategy):
|
||||
"""
|
||||
Meta Supertrend Strategy using 3 Supertrend indicators.
|
||||
|
||||
Enters long when all 3 Supertrends are bullish.
|
||||
Enters short when all 3 Supertrends are bearish.
|
||||
|
||||
Designed for perpetual futures with leverage and short-selling support.
|
||||
"""
|
||||
# Market configuration
|
||||
default_market_type = MarketType.PERPETUAL
|
||||
default_leverage = 5
|
||||
|
||||
# Risk management parameters
|
||||
default_sl_stop = 0.02 # 2% stop loss
|
||||
default_sl_trail = True # Trailing stop enabled
|
||||
default_exit_on_bearish_flip = False # Rely on SL/TP, not bearish flip
|
||||
|
||||
def run(
|
||||
self,
|
||||
close: pd.Series,
|
||||
high: pd.Series = None,
|
||||
low: pd.Series = None,
|
||||
period1: int = 10,
|
||||
multiplier1: float = 3.0,
|
||||
period2: int = 11,
|
||||
multiplier2: float = 2.0,
|
||||
period3: int = 12,
|
||||
multiplier3: float = 1.0,
|
||||
exit_on_bearish_flip: bool = None,
|
||||
enable_short: bool = True,
|
||||
**kwargs
|
||||
) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]:
|
||||
|
||||
# 1. Validation & Setup
|
||||
if exit_on_bearish_flip is None:
|
||||
exit_on_bearish_flip = self.default_exit_on_bearish_flip
|
||||
|
||||
if high is None or low is None:
|
||||
raise ValueError("MetaSupertrendStrategy requires High and Low prices.")
|
||||
|
||||
# 2. Calculate Supertrends
|
||||
t1, t2, t3 = self._calculate_supertrends(
|
||||
high, low, close,
|
||||
period1, multiplier1,
|
||||
period2, multiplier2,
|
||||
period3, multiplier3
|
||||
)
|
||||
|
||||
# 3. Meta Signals
|
||||
bullish, bearish = self._calculate_meta_signals(t1, t2, t3, close)
|
||||
|
||||
# 4. Generate Entry/Exit Signals
|
||||
return self._generate_signals(bullish, bearish, exit_on_bearish_flip, enable_short)
|
||||
|
||||
def _calculate_supertrends(
|
||||
self, high, low, close, p1, m1, p2, m2, p3, m3
|
||||
):
|
||||
"""Run the 3 Supertrend indicators."""
|
||||
# Pass NumPy arrays explicitly to avoid Numba typing errors
|
||||
h_vals = high.values
|
||||
l_vals = low.values
|
||||
c_vals = close.values
|
||||
|
||||
def run_st(p, m):
|
||||
st = SuperTrendIndicator.run(h_vals, l_vals, c_vals, period=p, multiplier=m)
|
||||
trend = st.trend
|
||||
if isinstance(trend, pd.DataFrame):
|
||||
trend.index = close.index
|
||||
if trend.shape[1] == 1:
|
||||
trend = trend.iloc[:, 0]
|
||||
elif isinstance(trend, pd.Series):
|
||||
trend.index = close.index
|
||||
return trend
|
||||
|
||||
t1 = run_st(p1, m1)
|
||||
t2 = run_st(p2, m2)
|
||||
t3 = run_st(p3, m3)
|
||||
return t1, t2, t3
|
||||
|
||||
def _calculate_meta_signals(self, t1, t2, t3, close_series):
|
||||
"""Combine 3 Supertrends into boolean Bullish/Bearish signals."""
|
||||
# Use NumPy broadcasting
|
||||
t1_vals = t1.values if isinstance(t1, pd.DataFrame) else t1.values.reshape(-1, 1)
|
||||
# Force column vectors for broadcasting if scalar result
|
||||
t2_vals = t2.values.reshape(-1, 1)
|
||||
t3_vals = t3.values.reshape(-1, 1)
|
||||
|
||||
# Boolean logic on numpy arrays (1 = Bull, -1 = Bear)
|
||||
bullish_vals = (t1_vals == 1) & (t2_vals == 1) & (t3_vals == 1)
|
||||
bearish_vals = (t1_vals == -1) & (t2_vals == -1) & (t3_vals == -1)
|
||||
|
||||
# Reconstruct Pandas objects
|
||||
if isinstance(t1, pd.DataFrame):
|
||||
bullish = pd.DataFrame(bullish_vals, index=t1.index, columns=t1.columns)
|
||||
bearish = pd.DataFrame(bearish_vals, index=t1.index, columns=t1.columns)
|
||||
else:
|
||||
bullish = pd.Series(bullish_vals.flatten(), index=t1.index)
|
||||
bearish = pd.Series(bearish_vals.flatten(), index=t1.index)
|
||||
|
||||
return bullish, bearish
|
||||
|
||||
def _generate_signals(
|
||||
self, bullish, bearish, exit_on_bearish_flip, enable_short
|
||||
):
|
||||
"""Generate long/short entry/exit signals based on meta trend."""
|
||||
# Long Entries: Change from Not Bullish to Bullish
|
||||
prev_bullish = bullish.shift(1).fillna(False)
|
||||
long_entries = bullish & (~prev_bullish)
|
||||
|
||||
# Long Exits
|
||||
if exit_on_bearish_flip:
|
||||
prev_bearish = bearish.shift(1).fillna(False)
|
||||
long_exits = bearish & (~prev_bearish)
|
||||
else:
|
||||
long_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
|
||||
# Short signals
|
||||
if enable_short:
|
||||
prev_bearish = bearish.shift(1).fillna(False)
|
||||
short_entries = bearish & (~prev_bearish)
|
||||
|
||||
if exit_on_bearish_flip:
|
||||
short_exits = bullish & (~prev_bullish)
|
||||
else:
|
||||
short_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
else:
|
||||
short_entries = BaseStrategy.create_empty_signals(long_entries)
|
||||
short_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
|
||||
return long_entries, long_exits, short_entries, short_exits
|
||||
6
strategies/supertrend_pkg/__init__.py
Normal file
6
strategies/supertrend_pkg/__init__.py
Normal file
@@ -0,0 +1,6 @@
|
||||
"""
|
||||
Meta Supertrend strategy package.
|
||||
"""
|
||||
from .strategy import MetaSupertrendStrategy
|
||||
|
||||
__all__ = ['MetaSupertrendStrategy']
|
||||
128
strategies/supertrend_pkg/indicators.py
Normal file
128
strategies/supertrend_pkg/indicators.py
Normal file
@@ -0,0 +1,128 @@
|
||||
"""
|
||||
Supertrend indicators and helper functions.
|
||||
"""
|
||||
import numpy as np
|
||||
import vectorbt as vbt
|
||||
from numba import njit
|
||||
|
||||
# --- Numba Compiled Helper Functions ---
|
||||
|
||||
@njit(cache=False) # Disable cache to avoid stale compilation issues
|
||||
def get_tr_nb(high, low, close):
|
||||
"""Calculate True Range (Numba compiled)."""
|
||||
# Ensure 1D arrays
|
||||
high = high.ravel()
|
||||
low = low.ravel()
|
||||
close = close.ravel()
|
||||
|
||||
tr = np.empty_like(close)
|
||||
tr[0] = high[0] - low[0]
|
||||
for i in range(1, len(close)):
|
||||
tr[i] = max(high[i] - low[i], abs(high[i] - close[i-1]), abs(low[i] - close[i-1]))
|
||||
return tr
|
||||
|
||||
@njit(cache=False)
|
||||
def get_atr_nb(high, low, close, period):
|
||||
"""Calculate ATR using Wilder's Smoothing (Numba compiled)."""
|
||||
# Ensure 1D arrays
|
||||
high = high.ravel()
|
||||
low = low.ravel()
|
||||
close = close.ravel()
|
||||
|
||||
# Ensure period is native Python int (critical for Numba array indexing)
|
||||
n = len(close)
|
||||
p = int(period)
|
||||
|
||||
tr = get_tr_nb(high, low, close)
|
||||
atr = np.full(n, np.nan, dtype=np.float64)
|
||||
|
||||
if n < p:
|
||||
return atr
|
||||
|
||||
# Initial ATR is simple average of TR
|
||||
sum_tr = 0.0
|
||||
for i in range(p):
|
||||
sum_tr += tr[i]
|
||||
atr[p - 1] = sum_tr / p
|
||||
|
||||
# Subsequent ATR is Wilder's smoothed
|
||||
for i in range(p, n):
|
||||
atr[i] = (atr[i - 1] * (p - 1) + tr[i]) / p
|
||||
|
||||
return atr
|
||||
|
||||
@njit(cache=False)
|
||||
def get_supertrend_nb(high, low, close, period, multiplier):
|
||||
"""Calculate SuperTrend completely in Numba."""
|
||||
# Ensure 1D arrays
|
||||
high = high.ravel()
|
||||
low = low.ravel()
|
||||
close = close.ravel()
|
||||
|
||||
# Ensure params are native Python types (critical for Numba)
|
||||
n = len(close)
|
||||
p = int(period)
|
||||
m = float(multiplier)
|
||||
|
||||
atr = get_atr_nb(high, low, close, p)
|
||||
|
||||
final_upper = np.full(n, np.nan, dtype=np.float64)
|
||||
final_lower = np.full(n, np.nan, dtype=np.float64)
|
||||
trend = np.ones(n, dtype=np.int8) # 1 Bull, -1 Bear
|
||||
|
||||
# Skip until we have valid ATR
|
||||
start_idx = p
|
||||
if start_idx >= n:
|
||||
return trend
|
||||
|
||||
# Init first valid point
|
||||
hl2 = (high[start_idx] + low[start_idx]) / 2
|
||||
final_upper[start_idx] = hl2 + m * atr[start_idx]
|
||||
final_lower[start_idx] = hl2 - m * atr[start_idx]
|
||||
|
||||
# Loop
|
||||
for i in range(start_idx + 1, n):
|
||||
cur_hl2 = (high[i] + low[i]) / 2
|
||||
cur_atr = atr[i]
|
||||
basic_upper = cur_hl2 + m * cur_atr
|
||||
basic_lower = cur_hl2 - m * cur_atr
|
||||
|
||||
# Upper Band Logic
|
||||
if basic_upper < final_upper[i-1] or close[i-1] > final_upper[i-1]:
|
||||
final_upper[i] = basic_upper
|
||||
else:
|
||||
final_upper[i] = final_upper[i-1]
|
||||
|
||||
# Lower Band Logic
|
||||
if basic_lower > final_lower[i-1] or close[i-1] < final_lower[i-1]:
|
||||
final_lower[i] = basic_lower
|
||||
else:
|
||||
final_lower[i] = final_lower[i-1]
|
||||
|
||||
# Trend Logic
|
||||
if trend[i-1] == 1:
|
||||
if close[i] < final_lower[i-1]:
|
||||
trend[i] = -1
|
||||
else:
|
||||
trend[i] = 1
|
||||
else:
|
||||
if close[i] > final_upper[i-1]:
|
||||
trend[i] = 1
|
||||
else:
|
||||
trend[i] = -1
|
||||
|
||||
return trend
|
||||
|
||||
# --- VectorBT Indicator Factory ---
|
||||
|
||||
SuperTrendIndicator = vbt.IndicatorFactory(
|
||||
class_name='SuperTrend',
|
||||
short_name='st',
|
||||
input_names=['high', 'low', 'close'],
|
||||
param_names=['period', 'multiplier'],
|
||||
output_names=['trend']
|
||||
).from_apply_func(
|
||||
get_supertrend_nb,
|
||||
keep_pd=False, # Disable automatic Pandas wrapping of inputs
|
||||
param_product=True # Enable Cartesian product for list params
|
||||
)
|
||||
142
strategies/supertrend_pkg/strategy.py
Normal file
142
strategies/supertrend_pkg/strategy.py
Normal file
@@ -0,0 +1,142 @@
|
||||
"""
|
||||
Meta Supertrend strategy implementation.
|
||||
"""
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from engine.market import MarketType
|
||||
from strategies.base import BaseStrategy
|
||||
from .indicators import SuperTrendIndicator
|
||||
|
||||
|
||||
class MetaSupertrendStrategy(BaseStrategy):
|
||||
"""
|
||||
Meta Supertrend Strategy using 3 Supertrend indicators.
|
||||
|
||||
Enters long when all 3 Supertrends are bullish.
|
||||
Enters short when all 3 Supertrends are bearish.
|
||||
|
||||
Designed for perpetual futures with leverage and short-selling support.
|
||||
"""
|
||||
# Market configuration
|
||||
default_market_type = MarketType.PERPETUAL
|
||||
default_leverage = 5
|
||||
|
||||
# Risk management parameters
|
||||
default_sl_stop = 0.02 # 2% stop loss
|
||||
default_sl_trail = True # Trailing stop enabled
|
||||
default_exit_on_bearish_flip = False # Rely on SL/TP, not bearish flip
|
||||
|
||||
def run(
|
||||
self,
|
||||
close: pd.Series,
|
||||
high: pd.Series = None,
|
||||
low: pd.Series = None,
|
||||
period1: int = 10,
|
||||
multiplier1: float = 3.0,
|
||||
period2: int = 11,
|
||||
multiplier2: float = 2.0,
|
||||
period3: int = 12,
|
||||
multiplier3: float = 1.0,
|
||||
exit_on_bearish_flip: bool = None,
|
||||
enable_short: bool = True,
|
||||
**kwargs
|
||||
) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]:
|
||||
|
||||
# 1. Validation & Setup
|
||||
if exit_on_bearish_flip is None:
|
||||
exit_on_bearish_flip = self.default_exit_on_bearish_flip
|
||||
|
||||
if high is None or low is None:
|
||||
raise ValueError("MetaSupertrendStrategy requires High and Low prices.")
|
||||
|
||||
# 2. Calculate Supertrends
|
||||
t1, t2, t3 = self._calculate_supertrends(
|
||||
high, low, close,
|
||||
period1, multiplier1,
|
||||
period2, multiplier2,
|
||||
period3, multiplier3
|
||||
)
|
||||
|
||||
# 3. Meta Signals
|
||||
bullish, bearish = self._calculate_meta_signals(t1, t2, t3, close)
|
||||
|
||||
# 4. Generate Entry/Exit Signals
|
||||
return self._generate_signals(bullish, bearish, exit_on_bearish_flip, enable_short)
|
||||
|
||||
def _calculate_supertrends(
|
||||
self, high, low, close, p1, m1, p2, m2, p3, m3
|
||||
):
|
||||
"""Run the 3 Supertrend indicators."""
|
||||
# Pass NumPy arrays explicitly to avoid Numba typing errors
|
||||
h_vals = high.values
|
||||
l_vals = low.values
|
||||
c_vals = close.values
|
||||
|
||||
def run_st(p, m):
|
||||
st = SuperTrendIndicator.run(h_vals, l_vals, c_vals, period=p, multiplier=m)
|
||||
trend = st.trend
|
||||
if isinstance(trend, pd.DataFrame):
|
||||
trend.index = close.index
|
||||
if trend.shape[1] == 1:
|
||||
trend = trend.iloc[:, 0]
|
||||
elif isinstance(trend, pd.Series):
|
||||
trend.index = close.index
|
||||
return trend
|
||||
|
||||
t1 = run_st(p1, m1)
|
||||
t2 = run_st(p2, m2)
|
||||
t3 = run_st(p3, m3)
|
||||
return t1, t2, t3
|
||||
|
||||
def _calculate_meta_signals(self, t1, t2, t3, close_series):
|
||||
"""Combine 3 Supertrends into boolean Bullish/Bearish signals."""
|
||||
# Use NumPy broadcasting
|
||||
t1_vals = t1.values if isinstance(t1, pd.DataFrame) else t1.values.reshape(-1, 1)
|
||||
# Force column vectors for broadcasting if scalar result
|
||||
t2_vals = t2.values.reshape(-1, 1)
|
||||
t3_vals = t3.values.reshape(-1, 1)
|
||||
|
||||
# Boolean logic on numpy arrays (1 = Bull, -1 = Bear)
|
||||
bullish_vals = (t1_vals == 1) & (t2_vals == 1) & (t3_vals == 1)
|
||||
bearish_vals = (t1_vals == -1) & (t2_vals == -1) & (t3_vals == -1)
|
||||
|
||||
# Reconstruct Pandas objects
|
||||
if isinstance(t1, pd.DataFrame):
|
||||
bullish = pd.DataFrame(bullish_vals, index=t1.index, columns=t1.columns)
|
||||
bearish = pd.DataFrame(bearish_vals, index=t1.index, columns=t1.columns)
|
||||
else:
|
||||
bullish = pd.Series(bullish_vals.flatten(), index=t1.index)
|
||||
bearish = pd.Series(bearish_vals.flatten(), index=t1.index)
|
||||
|
||||
return bullish, bearish
|
||||
|
||||
def _generate_signals(
|
||||
self, bullish, bearish, exit_on_bearish_flip, enable_short
|
||||
):
|
||||
"""Generate long/short entry/exit signals based on meta trend."""
|
||||
# Long Entries: Change from Not Bullish to Bullish
|
||||
prev_bullish = bullish.shift(1).fillna(False)
|
||||
long_entries = bullish & (~prev_bullish)
|
||||
|
||||
# Long Exits
|
||||
if exit_on_bearish_flip:
|
||||
prev_bearish = bearish.shift(1).fillna(False)
|
||||
long_exits = bearish & (~prev_bearish)
|
||||
else:
|
||||
long_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
|
||||
# Short signals
|
||||
if enable_short:
|
||||
prev_bearish = bearish.shift(1).fillna(False)
|
||||
short_entries = bearish & (~prev_bearish)
|
||||
|
||||
if exit_on_bearish_flip:
|
||||
short_exits = bullish & (~prev_bullish)
|
||||
else:
|
||||
short_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
else:
|
||||
short_entries = BaseStrategy.create_empty_signals(long_entries)
|
||||
short_exits = BaseStrategy.create_empty_signals(long_entries)
|
||||
|
||||
return long_entries, long_exits, short_entries, short_exits
|
||||
351
tasks/prd-terminal-ui.md
Normal file
351
tasks/prd-terminal-ui.md
Normal file
@@ -0,0 +1,351 @@
|
||||
# PRD: Terminal UI for Live Trading Bot
|
||||
|
||||
## Introduction/Overview
|
||||
|
||||
The live trading bot currently uses basic console logging for output, making it difficult to monitor trading activity, track performance, and understand the system state at a glance. This feature introduces a Rich-based terminal UI that provides a professional, real-time dashboard for monitoring the live trading bot.
|
||||
|
||||
The UI will display a horizontal split layout with a **summary panel** at the top (with tabbed time-period views) and a **scrollable log panel** at the bottom. The interface will update every second and support keyboard navigation.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Provide real-time visibility into trading performance (PnL, win rate, trade count)
|
||||
2. Enable monitoring of current position state (entry, SL/TP, unrealized PnL)
|
||||
3. Display strategy signals (Z-score, model probability) for transparency
|
||||
4. Support historical performance tracking across time periods (daily, weekly, monthly, all-time)
|
||||
5. Improve operational experience with keyboard shortcuts and log filtering
|
||||
6. Create a responsive design that works across different terminal sizes
|
||||
|
||||
## User Stories
|
||||
|
||||
1. **As a trader**, I want to see my total PnL and daily PnL at a glance so I can quickly assess performance.
|
||||
2. **As a trader**, I want to see my current position details (entry price, unrealized PnL, SL/TP levels) so I can monitor risk.
|
||||
3. **As a trader**, I want to view performance metrics by time period (daily, weekly, monthly) so I can track trends.
|
||||
4. **As a trader**, I want to filter logs by type (errors, trades, signals) so I can focus on relevant information.
|
||||
5. **As a trader**, I want keyboard shortcuts to navigate the UI without using a mouse.
|
||||
6. **As a trader**, I want the UI to show strategy state (Z-score, probability) so I understand why signals are generated.
|
||||
|
||||
## Functional Requirements
|
||||
|
||||
### FR1: Layout Structure
|
||||
|
||||
1.1. The UI must use a horizontal split layout with the summary panel at the top and logs panel at the bottom.
|
||||
|
||||
1.2. The summary panel must contain tabbed views accessible via number keys:
|
||||
- Tab 1 (`1`): **General** - Overall metrics since bot started
|
||||
- Tab 2 (`2`): **Monthly** - Current month metrics (shown only if data spans > 1 month)
|
||||
- Tab 3 (`3`): **Weekly** - Current week metrics (shown only if data spans > 1 week)
|
||||
- Tab 4 (`4`): **Daily** - Today's metrics
|
||||
|
||||
1.3. The logs panel must be a scrollable area showing recent log entries.
|
||||
|
||||
1.4. The UI must be responsive and adapt to terminal size (minimum 80x24).
|
||||
|
||||
### FR2: Metrics Display
|
||||
|
||||
The summary panel must display the following metrics:
|
||||
|
||||
**Performance Metrics:**
|
||||
2.1. Total PnL (USD) - cumulative profit/loss since tracking began
|
||||
2.2. Period PnL (USD) - profit/loss for selected time period (daily/weekly/monthly)
|
||||
2.3. Win Rate (%) - percentage of winning trades
|
||||
2.4. Total Number of Trades
|
||||
2.5. Average Trade Duration (hours)
|
||||
2.6. Max Drawdown (USD and %)
|
||||
|
||||
**Current Position (if open):**
|
||||
2.7. Symbol and side (long/short)
|
||||
2.8. Entry price
|
||||
2.9. Current price
|
||||
2.10. Unrealized PnL (USD and %)
|
||||
2.11. Stop-loss price and distance (%)
|
||||
2.12. Take-profit price and distance (%)
|
||||
2.13. Position size (USD)
|
||||
|
||||
**Account Status:**
|
||||
2.14. Account balance / available margin (USDT)
|
||||
2.15. Current leverage setting
|
||||
|
||||
**Strategy State:**
|
||||
2.16. Current Z-score
|
||||
2.17. Model probability
|
||||
2.18. Current funding rate (BTC)
|
||||
2.19. Last signal action and reason
|
||||
|
||||
### FR3: Historical Data Loading
|
||||
|
||||
3.1. On startup, the system must initialize SQLite database at `live_trading/trading.db`.
|
||||
|
||||
3.2. If `trade_log.csv` exists and database is empty, migrate CSV data to SQLite.
|
||||
|
||||
3.3. The UI must load current positions from `live_trading/positions.json` (kept for compatibility with existing position manager).
|
||||
|
||||
3.4. Metrics must be calculated via SQL aggregation queries for each time period.
|
||||
|
||||
3.5. If no historical data exists, the UI must show "No data" gracefully.
|
||||
|
||||
3.6. New trades must be written to both SQLite (primary) and CSV (backup/compatibility).
|
||||
|
||||
### FR4: Real-Time Updates
|
||||
|
||||
4.1. The UI must refresh every 1 second.
|
||||
|
||||
4.2. Position unrealized PnL must update based on latest price data.
|
||||
|
||||
4.3. New log entries must appear in real-time.
|
||||
|
||||
4.4. Metrics must recalculate when trades are opened/closed.
|
||||
|
||||
### FR5: Log Panel
|
||||
|
||||
5.1. The log panel must display log entries with timestamp, level, and message.
|
||||
|
||||
5.2. Log entries must be color-coded by level:
|
||||
- ERROR: Red
|
||||
- WARNING: Yellow
|
||||
- INFO: White/Default
|
||||
- DEBUG: Gray (if shown)
|
||||
|
||||
5.3. The log panel must support filtering by log type:
|
||||
- All logs (default)
|
||||
- Errors only
|
||||
- Trades only (entries containing "position", "trade", "order")
|
||||
- Signals only (entries containing "signal", "z_score", "prob")
|
||||
|
||||
5.4. Filter switching must be available via keyboard shortcut (`f` to cycle filters).
|
||||
|
||||
### FR6: Keyboard Controls
|
||||
|
||||
6.1. `q` or `Ctrl+C` - Graceful shutdown
|
||||
6.2. `r` - Force refresh data
|
||||
6.3. `1` - Switch to General tab
|
||||
6.4. `2` - Switch to Monthly tab
|
||||
6.5. `3` - Switch to Weekly tab
|
||||
6.6. `4` - Switch to Daily tab
|
||||
6.7. `f` - Cycle log filter
|
||||
6.8. Arrow keys - Scroll logs (if supported)
|
||||
|
||||
### FR7: Color Scheme
|
||||
|
||||
7.1. Use dark theme as base.
|
||||
|
||||
7.2. PnL values must be colored:
|
||||
- Positive: Green
|
||||
- Negative: Red
|
||||
- Zero/Neutral: White
|
||||
|
||||
7.3. Position side must be colored:
|
||||
- Long: Green
|
||||
- Short: Red
|
||||
|
||||
7.4. Use consistent color coding for emphasis and warnings.
|
||||
|
||||
## Non-Goals (Out of Scope)
|
||||
|
||||
1. **Mouse support** - This is a keyboard-driven terminal UI
|
||||
2. **Trade execution from UI** - The UI is read-only; trades are executed by the bot
|
||||
3. **Configuration editing** - Config changes require restarting the bot
|
||||
4. **Multi-exchange support** - Only OKX is supported
|
||||
5. **Charts/graphs** - Text-based metrics only (no ASCII charts in v1)
|
||||
6. **Sound alerts** - No audio notifications
|
||||
7. **Remote access** - Local terminal only
|
||||
|
||||
## Design Considerations
|
||||
|
||||
### Technology Choice: Rich
|
||||
|
||||
Use the [Rich](https://github.com/Textualize/rich) Python library for terminal UI:
|
||||
- Rich provides `Live` display for real-time updates
|
||||
- Rich `Layout` for split-screen design
|
||||
- Rich `Table` for metrics display
|
||||
- Rich `Panel` for bordered sections
|
||||
- Rich `Text` for colored output
|
||||
|
||||
Alternative considered: **Textual** (also by Will McGugan) provides more advanced TUI features but adds complexity. Rich is simpler and sufficient for this use case.
|
||||
|
||||
### UI Mockup
|
||||
|
||||
```
|
||||
+==============================================================================+
|
||||
| REGIME REVERSION STRATEGY - LIVE TRADING [DEMO] ETH/USDT |
|
||||
+==============================================================================+
|
||||
| [1:General] [2:Monthly] [3:Weekly] [4:Daily] |
|
||||
+------------------------------------------------------------------------------+
|
||||
| PERFORMANCE | CURRENT POSITION |
|
||||
| Total PnL: $1,234.56 | Side: LONG |
|
||||
| Today PnL: $45.23 | Entry: $3,245.50 |
|
||||
| Win Rate: 67.5% | Current: $3,289.00 |
|
||||
| Total Trades: 24 | Unrealized: +$43.50 (+1.34%) |
|
||||
| Avg Duration: 4.2h | Size: $500.00 |
|
||||
| Max Drawdown: -$156.00 | SL: $3,050.00 (-6.0%) TP: $3,408.00 (+5%)|
|
||||
| | |
|
||||
| ACCOUNT | STRATEGY STATE |
|
||||
| Balance: $5,432.10 | Z-Score: 1.45 |
|
||||
| Available: $4,932.10 | Probability: 0.72 |
|
||||
| Leverage: 2x | Funding: 0.0012 |
|
||||
+------------------------------------------------------------------------------+
|
||||
| LOGS [Filter: All] Press 'f' cycle |
|
||||
+------------------------------------------------------------------------------+
|
||||
| 14:32:15 [INFO] Trading Cycle Start: 2026-01-16T14:32:15+00:00 |
|
||||
| 14:32:16 [INFO] Signal: entry long (prob=0.72, z=-1.45, reason=z_score...) |
|
||||
| 14:32:17 [INFO] Executing LONG entry: 0.1540 ETH @ 3245.50 ($500.00) |
|
||||
| 14:32:18 [INFO] Position opened: ETH/USDT:USDT_20260116_143217 |
|
||||
| 14:32:18 [INFO] Portfolio: 1 positions, exposure=$500.00, unrealized=$0.00 |
|
||||
| 14:32:18 [INFO] --- Cycle completed in 3.2s --- |
|
||||
| 14:32:18 [INFO] Sleeping for 60 minutes... |
|
||||
| |
|
||||
+------------------------------------------------------------------------------+
|
||||
| [q]Quit [r]Refresh [1-4]Tabs [f]Filter |
|
||||
+==============================================================================+
|
||||
```
|
||||
|
||||
### File Structure
|
||||
|
||||
```
|
||||
live_trading/
|
||||
ui/
|
||||
__init__.py
|
||||
dashboard.py # Main UI orchestration and threading
|
||||
panels.py # Panel components (metrics, logs, position)
|
||||
state.py # Thread-safe shared state
|
||||
log_handler.py # Custom logging handler for UI queue
|
||||
keyboard.py # Keyboard input handling
|
||||
db/
|
||||
__init__.py
|
||||
database.py # SQLite connection and queries
|
||||
models.py # Data models (Trade, DailySummary, Session)
|
||||
migrations.py # CSV migration and schema setup
|
||||
metrics.py # Metrics aggregation queries
|
||||
trading.db # SQLite database file (created at runtime)
|
||||
```
|
||||
|
||||
## Technical Considerations
|
||||
|
||||
### Integration with Existing Code
|
||||
|
||||
1. **Logging Integration**: Create a custom `logging.Handler` that captures log messages and forwards them to the UI log panel while still writing to file.
|
||||
|
||||
2. **Data Access**: The UI needs access to:
|
||||
- `PositionManager` for current positions
|
||||
- `TradingConfig` for settings display
|
||||
- SQLite database for historical metrics
|
||||
- Real-time data from `DataFeed` for current prices
|
||||
|
||||
3. **Main Loop Modification**: The `LiveTradingBot.run()` method needs modification to run the UI in a **separate thread** alongside the trading loop. The UI thread handles rendering and keyboard input while the main thread executes trading logic.
|
||||
|
||||
4. **Graceful Shutdown**: Ensure `SIGINT`/`SIGTERM` handlers work with the UI layer and properly terminate the UI thread.
|
||||
|
||||
### Database Schema (SQLite)
|
||||
|
||||
Create `live_trading/trading.db` with the following schema:
|
||||
|
||||
```sql
|
||||
-- Trade history table
|
||||
CREATE TABLE trades (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
trade_id TEXT UNIQUE NOT NULL,
|
||||
symbol TEXT NOT NULL,
|
||||
side TEXT NOT NULL, -- 'long' or 'short'
|
||||
entry_price REAL NOT NULL,
|
||||
exit_price REAL,
|
||||
size REAL NOT NULL,
|
||||
size_usdt REAL NOT NULL,
|
||||
pnl_usd REAL,
|
||||
pnl_pct REAL,
|
||||
entry_time TEXT NOT NULL, -- ISO format
|
||||
exit_time TEXT,
|
||||
hold_duration_hours REAL,
|
||||
reason TEXT, -- 'stop_loss', 'take_profit', 'signal', etc.
|
||||
order_id_entry TEXT,
|
||||
order_id_exit TEXT,
|
||||
created_at TEXT DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Daily summary table (for faster queries)
|
||||
CREATE TABLE daily_summary (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
date TEXT UNIQUE NOT NULL, -- YYYY-MM-DD
|
||||
total_trades INTEGER DEFAULT 0,
|
||||
winning_trades INTEGER DEFAULT 0,
|
||||
total_pnl_usd REAL DEFAULT 0,
|
||||
max_drawdown_usd REAL DEFAULT 0,
|
||||
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Session metadata
|
||||
CREATE TABLE sessions (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
start_time TEXT NOT NULL,
|
||||
end_time TEXT,
|
||||
starting_balance REAL,
|
||||
ending_balance REAL,
|
||||
total_pnl REAL,
|
||||
total_trades INTEGER DEFAULT 0
|
||||
);
|
||||
|
||||
-- Indexes for common queries
|
||||
CREATE INDEX idx_trades_entry_time ON trades(entry_time);
|
||||
CREATE INDEX idx_trades_exit_time ON trades(exit_time);
|
||||
CREATE INDEX idx_daily_summary_date ON daily_summary(date);
|
||||
```
|
||||
|
||||
### Migration from CSV
|
||||
|
||||
On first run with the new system:
|
||||
1. Check if `trading.db` exists
|
||||
2. If not, create database with schema
|
||||
3. If `trade_log.csv` exists, migrate data to `trades` table
|
||||
4. Rebuild `daily_summary` from migrated trades
|
||||
|
||||
### Dependencies
|
||||
|
||||
Add to `pyproject.toml`:
|
||||
```toml
|
||||
dependencies = [
|
||||
# ... existing deps
|
||||
"rich>=13.0.0",
|
||||
]
|
||||
```
|
||||
|
||||
Note: SQLite is part of Python's standard library (`sqlite3`), no additional dependency needed.
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
- UI runs in a separate thread to avoid blocking trading logic
|
||||
- Log buffer limited to 1000 entries in memory to prevent growth
|
||||
- SQLite queries should use indexes for fast period-based aggregations
|
||||
- Historical data loading happens once at startup, incremental updates thereafter
|
||||
|
||||
### Threading Model
|
||||
|
||||
```
|
||||
Main Thread UI Thread
|
||||
| |
|
||||
v v
|
||||
[Trading Loop] [Rich Live Display]
|
||||
| |
|
||||
+---> SharedState <------------+
|
||||
(thread-safe)
|
||||
| |
|
||||
+---> LogQueue <---------------+
|
||||
(thread-safe)
|
||||
```
|
||||
|
||||
- Use `threading.Lock` for shared state access
|
||||
- Use `queue.Queue` for log message passing
|
||||
- UI thread polls for updates every 1 second
|
||||
|
||||
## Success Metrics
|
||||
|
||||
1. UI starts successfully and displays all required metrics
|
||||
2. UI updates in real-time (1-second refresh) without impacting trading performance
|
||||
3. All keyboard shortcuts function correctly
|
||||
4. Historical data loads and displays accurately from SQLite
|
||||
5. Log filtering works as expected
|
||||
6. UI gracefully handles edge cases (no data, no position, terminal resize)
|
||||
7. CSV migration completes successfully on first run
|
||||
8. Database queries complete within 100ms
|
||||
|
||||
---
|
||||
|
||||
*Generated: 2026-01-16*
|
||||
*Decisions: Threading model, 1000 log buffer, SQLite database, no fallback mode*
|
||||
1
tests/__init__.py
Normal file
1
tests/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Test suite for lowkey_backtest."""
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user