Simon/lowkey_backtest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Cleaning up PRDs files

Browse Source
This commit is contained in:
Simon Moisy
2026-01-15 10:51:42 +08:00
parent c4ecb29d4c
commit 7e4a6874a2
3 changed files with 0 additions and 609 deletions
Download Patch File Download Diff File Expand all files Collapse all files

238
tasks/prd-interactive-ui.md
View File

@@ -1,238 +0,0 @@
# PRD: Interactive Backtest UI
## 1. Introduction / Overview
Lowkey Backtest currently operates exclusively through a CLI. This project adds an interactive web-based UI that allows users to:
- Run backtests with existing code-defined strategies
- Adjust strategy parameters without editing code
- Compare multiple backtest runs visually
- Analyze performance through interactive charts and metrics
The UI will use **FastAPI** for the backend API and **Vue 3** for the frontend, styled with a dark theme inspired by QuantConnect.
---
## 2. Goals
| # | Goal | Success Metric |
|---|------|----------------|
| G1 | Run backtests from UI | User can select strategy, symbol, parameters and execute backtest |
| G2 | Visualize results | Equity curves, trade markers, and metrics displayed interactively |
| G3 | Compare runs | Side-by-side comparison of 2+ backtest results |
| G4 | Parameter experimentation | Adjust parameters via form inputs, re-run, and compare to previous |
| G5 | Self-hostable | Deployable on user's Arch-based NAS with minimal configuration |
---
## 3. User Stories
### US1: Run a Backtest
> As a trader, I want to select a strategy and configure its parameters in the UI so that I can run a backtest without using the terminal.
### US2: View Backtest Results
> As a trader, I want to see an interactive equity curve with trade entry/exit markers so that I can visually understand strategy performance.
### US3: Compare Parameter Changes
> As a trader, I want to compare two or more backtest runs side-by-side so that I can determine if a parameter optimization improves performance.
### US4: Quick Parameter Tweaking
> As a trader, I want to adjust strategy parameters (e.g., period, multiplier, SL/TP) via sliders or inputs and re-run the backtest so that I can iterate quickly.
### US5: Review Historical Runs
> As a trader, I want to see a list of previous backtest runs so that I can reload and compare past results.
---
## 4. Functional Requirements
### 4.1 Backend (FastAPI)
| # | Requirement |
|---|-------------|
| FR1 | `GET /api/strategies` - Return list of available strategies with their default and grid parameters |
| FR2 | `GET /api/symbols` - Return list of available symbols from local data |
| FR3 | `POST /api/backtest` - Execute a backtest with given strategy, symbol, timeframe, and parameters. Return results including portfolio stats, trades, and equity curve data |
| FR4 | `GET /api/backtest/{run_id}` - Retrieve a specific historical backtest result |
| FR5 | `GET /api/backtests` - List all saved backtest runs with summary metadata |
| FR6 | `POST /api/compare` - Accept multiple run IDs and return combined data for comparison |
| FR7 | `GET /api/data/status` - Return available data inventory (symbols, timeframes, date ranges) |
| FR8 | Results must be persisted (SQLite or JSON files) to allow historical retrieval |
### 4.2 Frontend (Vue 3)
| # | Requirement |
|---|-------------|
| FR9 | **Strategy Selector**: Dropdown to select strategy (rsi, macross, meta_st, regime) |
| FR10 | **Parameter Form**: Dynamic form fields based on selected strategy's parameters (inputs, sliders) |
| FR11 | **Symbol/Market Picker**: Select trading pair and market type (spot/perpetual) |
| FR12 | **Date Range Selector**: Pick start and end dates for backtest period |
| FR13 | **Run Button**: Execute backtest with loading state indicator |
| FR14 | **Equity Curve Chart**: Interactive line chart (Plotly.js) showing portfolio value over time with drawdown overlay (shaded area) |
| FR15 | **Trade Markers**: Overlay entry (green) and exit (red) markers on price/equity chart |
| FR16 | **Metrics Panel**: Display key stats (Total Return, Sharpe, Max DD, Win Rate, etc.) |
| FR17 | **Trade Log Table**: Sortable, filterable table of all trades |
| FR18 | **Run History Sidebar**: List of previous runs with quick-load action |
| FR19 | **Comparison View**: Select 2+ runs to overlay equity curves and compare metrics in a table |
| FR20 | **Dark Theme**: QuantConnect-inspired dark color scheme |
### 4.3 Comparison Features
| # | Requirement |
|---|-------------|
| FR21 | Overlay up to 5 equity curves on the same chart with different colors |
| FR22 | Metrics comparison table showing deltas between runs |
| FR23 | Highlight which run performed better for each metric |
| FR24 | Display parameter differences between compared runs |
---
## 5. Non-Goals (Out of Scope)
| # | Non-Goal |
|---|----------|
| NG1 | Strategy code editing in the UI (strategies are created in code editor) |
| NG2 | Live trading or paper trading integration |
| NG3 | Real-time market data streaming |
| NG4 | Multi-user authentication or access control |
| NG5 | Walk-Forward Analysis UI (may be added in future iteration) |
| NG6 | Data download management (use CLI for now) |
---
## 6. Design Considerations
### 6.1 Visual Style
- **Theme**: Dark mode only (similar to QuantConnect, TradingView dark)
- **Colors**:
- Background: `#1a1a2e` (deep navy)
- Cards/Panels: `#16213e`
- Accent: `#0f3460` (muted blue)
- Profit: `#00d26a` (green)
- Loss: `#ff6b6b` (red)
- Text: `#e8e8e8`
- **Typography**: Monospace for numbers/stats, clean sans-serif for labels
- **Layout**: Sidebar navigation + main content area
### 6.2 Component Library
- Consider Tailwind CSS for styling flexibility
- Plotly.js for charts (matches existing VectorBT ecosystem)
- Headless UI or Radix for accessible components
### 6.3 Responsive Design
- Primary target: Desktop (1920x1080+)
- Minimum supported: 1280x720
- Mobile not required
---
## 7. Technical Considerations
### 7.1 Architecture
```
lowkey_backtest/
api/ # FastAPI backend
__init__.py
main.py # FastAPI app entry point
routers/
backtest.py # Backtest endpoints
strategies.py # Strategy info endpoints
data.py # Data status endpoints
models/
schemas.py # Pydantic request/response models
services/
runner.py # Wraps existing Backtester
storage.py # Run persistence (SQLite)
frontend/ # Vue 3 app (Vite)
src/
components/ # Reusable UI components
views/ # Page-level components
composables/ # Vue composition functions
api/ # Axios API client
assets/ # Styles, images
```
### 7.2 Data Flow
1. Frontend requests available strategies via `GET /api/strategies`
2. User configures parameters and clicks "Run Backtest"
3. Frontend sends `POST /api/backtest` with configuration
4. Backend instantiates strategy, runs `Backtester.run_strategy()`, saves result
5. Backend returns serialized results (equity curve as JSON array, trades, stats)
6. Frontend renders charts and metrics
7. Run is saved and appears in history sidebar
### 7.3 Dependencies
**Backend:**
- FastAPI
- Uvicorn (ASGI server)
- SQLAlchemy (run storage)
- Existing engine modules (Backtester, DataManager, etc.)
**Frontend:**
- Vue 3 (Composition API)
- Vite (build tool)
- Plotly.js (charts)
- Tailwind CSS (styling)
- Axios (API calls)
### 7.4 Deployment
- Development: `uvicorn api.main:app --reload` + `npm run dev`
- Production (NAS):
- Build frontend static files
- Serve via FastAPI's `StaticFiles` or nginx
- Run with systemd service
---
## 8. Success Metrics
| Metric | Target |
|--------|--------|
| Backtest execution via UI | Works for all 4 strategies |
| Result visualization | Equity curve + trades render correctly |
| Comparison view | Can compare up to 5 runs with overlaid equity curves |
| Run persistence | Historical runs survive app restart |
| Page load time | < 2s for dashboard |
| Backtest response time | Same as CLI (backend performance unchanged) |
---
## 9. Resolved Questions
| # | Question | Decision |
|---|----------|----------|
| Q1 | React or Vue for frontend? | **Vue 3** - simpler, sufficient for this use case |
| Q2 | Should the equity curve include drawdown overlay by default? | **Yes** - shaded area beneath equity curve |
| Q3 | How many historical runs to keep before auto-cleanup? | **Unlimited** - manual deletion only |
| Q4 | Should comparison support more than 2 runs? | **Yes** - support 3-5 run overlays |
| Q5 | Include WFA visualization in V1 or defer to V2? | Deferred to V2 |
---
## 10. Milestones (Suggested)
| Phase | Deliverable | Effort |
|-------|-------------|--------|
| **Phase 1** | FastAPI backend with core endpoints (strategies, backtest, history) | 2-3 days |
| **Phase 2** | Frontend scaffold with dark theme and strategy selector | 2-3 days |
| **Phase 3** | Backtest execution flow and results visualization | 3-4 days |
| **Phase 4** | Run history and comparison view | 2-3 days |
| **Phase 5** | Polish, testing, NAS deployment | 2 days |
**Total Estimated Effort**: 11-15 days
---
## Appendix: Reference Screenshots
*(QuantConnect-style inspiration)*
- Dark navy background with card-based layout
- Left sidebar for navigation and run history
- Main area split: configuration panel (top/left) + results (bottom/right)
- Charts with grid lines, legend, and interactive tooltips
- Metrics displayed in a compact card grid

295
tasks/prd-market-type-selection.md
View File

@@ -1,295 +0,0 @@
# PRD: Market Type Selection for Backtesting
## Introduction/Overview
Currently, the backtesting system operates with a single, implicit market type assumption. This PRD defines the implementation of **market type selection** (Spot vs. USDT-M Perpetual Futures) to enable realistic simulation of different trading conditions.
**Problem Statement:**
- Strategies cannot be backtested against different market mechanics (leverage, funding, short-selling)
- Fee structures are uniform regardless of market type
- No support for short-selling strategies
- Data fetching doesn't distinguish between spot and futures markets
**Goal:**
Enable users to backtest strategies against specific market types (Spot or USDT-M Perpetual) with realistic trading conditions matching OKX's live environment.
---
## Goals
1. **Support two market types:** Spot and USDT-M Perpetual Futures
2. **Realistic fee simulation:** Match OKX's fee structure per market type
3. **Leverage support:** Per-strategy configurable leverage (perpetuals only)
4. **Funding rate simulation:** Simplified funding rate model for perpetuals
5. **Short-selling support:** Enable strategies to generate short signals
6. **Liquidation awareness:** Warn when positions would be liquidated (no full simulation)
7. **Separate data storage:** Download and store data per market type
8. **Grid search integration:** Allow leverage optimization in parameter grids
---
## User Stories
1. **As a trader**, I want to backtest my strategy on perpetual futures so that I can simulate leveraged trading with funding costs.
2. **As a trader**, I want to backtest on spot markets so that I can compare performance without leverage or funding overhead.
3. **As a strategy developer**, I want to define a default market type for my strategy so that it runs with appropriate settings by default.
4. **As a trader**, I want to test different leverage levels so that I can find the optimal risk/reward balance.
5. **As a trader**, I want to see warnings when my position would have been liquidated so that I can adjust my risk parameters.
6. **As a strategy developer**, I want to create strategies that can go short so that I can profit from downward price movements.
---
## Functional Requirements
### FR1: Market Type Enum and Configuration
1.1. Create a `MarketType` enum with values: `SPOT`, `PERPETUAL`
1.2. Each strategy class must have a `default_market_type` class attribute
1.3. Market type can be overridden via CLI (optional, for testing)
### FR2: Data Management
2.1. Modify `DataManager` to support market type in data paths:
- Spot: `data/ccxt/{exchange}/spot/{symbol}/{timeframe}.csv`
- Perpetual: `data/ccxt/{exchange}/perpetual/{symbol}/{timeframe}.csv`
2.2. Update `download` command to accept `--market` flag:
```bash
uv run python main.py download --pair BTC/USDT --market perpetual
```
2.3. Use CCXT's market type parameter when fetching data:
- Spot: `exchange.fetch_ohlcv(symbol, timeframe, ...)`
- Perpetual: `exchange.fetch_ohlcv(symbol + ':USDT', timeframe, ...)`
### FR3: Fee Structure
3.1. Define default fees per market type (matching OKX):
| Market Type | Maker Fee | Taker Fee | Notes |
|-------------|-----------|-----------|-------|
| Spot | 0.08% | 0.10% | No funding |
| Perpetual | 0.02% | 0.05% | + funding |
3.2. Allow fee override via CLI (existing `--fees` flag)
### FR4: Leverage Support (Perpetual Only)
4.1. Add `default_leverage` class attribute to strategies (default: 1 for spot, configurable for perpetual)
4.2. Add `--leverage` CLI flag for backtest command
4.3. Leverage affects:
- Position sizing (notional = cash * leverage)
- PnL calculation (multiplied by leverage)
- Liquidation threshold calculation
4.4. Support leverage in grid search parameter grids
### FR5: Funding Rate Simulation (Perpetual Only)
5.1. Implement simplified funding rate model:
- Default rate: 0.01% per 8 hours (configurable)
- Applied every 8 hours to open positions
- Positive rate: Longs pay shorts
- Negative rate: Shorts pay longs
5.2. Add `--funding-rate` CLI flag to override default
5.3. Track cumulative funding paid/received in backtest stats
### FR6: Short-Selling Support
6.1. Modify `BaseStrategy.run()` signature to return 4 signal arrays:
```python
def run(self, close, **kwargs) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]:
"""
Returns:
long_entries: Boolean signals to open long positions
long_exits: Boolean signals to close long positions
short_entries: Boolean signals to open short positions
short_exits: Boolean signals to close short positions
"""
```
6.2. Update `Backtester` to use VectorBT's `direction` parameter or dual portfolio simulation
6.3. For spot market: Ignore short signals (log warning if present)
6.4. For perpetual market: Process both long and short signals
### FR7: Liquidation Warning
7.1. Calculate liquidation price based on:
- Entry price
- Leverage
- Maintenance margin rate (OKX: ~0.4% for BTC)
7.2. During backtest, check if price crosses liquidation threshold
7.3. Log warning with details:
```
WARNING: Position would be liquidated at bar 1234 (price: $45,000, liq_price: $44,820)
```
7.4. Include liquidation event count in backtest summary stats
### FR8: Backtester Integration
8.1. Modify `Backtester.run_strategy()` to accept market type from strategy
8.2. Apply market-specific simulation parameters:
- Fees (if not overridden)
- Leverage
- Funding rate calculation
- Short-selling capability
8.3. Update portfolio simulation to handle leveraged positions
### FR9: Reporting Updates
9.1. Add market type to backtest summary output
9.2. Add new stats for perpetual backtests:
- Total funding paid/received
- Number of liquidation warnings
- Effective leverage used
9.3. Update CSV exports to include market-specific columns
---
## Non-Goals (Out of Scope)
- **Coin-M (Inverse) Perpetuals:** Not included in v1
- **Spot Margin Trading:** Not included in v1
- **Expiry Futures:** Not included in v1
- **Full Liquidation Simulation:** Only warnings, no automatic position closure
- **Real Funding Rate Data:** Use simplified model; historical funding API integration is future work
- **Cross-Margin Mode:** Assume isolated margin for simplicity
- **Partial Liquidation:** Assume full liquidation threshold only
---
## Design Considerations
### Data Directory Structure (New)
```
data/ccxt/
okx/
spot/
BTC-USDT/
1m.csv
1d.csv
perpetual/
BTC-USDT/
1m.csv
1d.csv
```
### Strategy Class Example
```python
class MetaSupertrendStrategy(BaseStrategy):
default_market_type = MarketType.PERPETUAL
default_leverage = 5
default_sl_stop = 0.02
def run(self, close, **kwargs):
# ... indicator logic ...
return long_entries, long_exits, short_entries, short_exits
```
### CLI Usage Examples
```bash
# Download perpetual data
uv run python main.py download --pair BTC/USDT --market perpetual
# Backtest with strategy defaults (uses strategy's default_market_type)
uv run python main.py backtest --strategy meta_st --pair BTC/USDT
# Override leverage
uv run python main.py backtest --strategy meta_st --pair BTC/USDT --leverage 10
# Grid search including leverage
uv run python main.py backtest --strategy meta_st --pair BTC/USDT --grid
# (leverage can be part of param grid in strategy factory)
```
---
## Technical Considerations
1. **VectorBT Compatibility:**
- VectorBT's `Portfolio.from_signals()` supports `direction` parameter for long/short
- Alternatively, run two portfolios (long-only, short-only) and combine
- Leverage can be simulated via `size` parameter or post-processing returns
2. **CCXT Market Type Handling:**
- OKX perpetual symbols use format: `BTC/USDT:USDT`
- Need to handle symbol conversion in DataManager
3. **Funding Rate Timing:**
- OKX funding at 00:00, 08:00, 16:00 UTC
- Need to identify these timestamps in the data and apply funding
4. **Backward Compatibility:**
- Existing strategies should work with minimal changes
- Default to `MarketType.SPOT` if not specified
- Existing 2-tuple return from `run()` should be interpreted as long-only
---
## Success Metrics
1. **Functional:** All existing backtests produce same results when run with `MarketType.SPOT`
2. **Functional:** Perpetual backtests correctly apply funding every 8 hours
3. **Functional:** Leverage multiplies both gains and losses correctly
4. **Functional:** Short signals are processed for perpetual, ignored for spot
5. **Usability:** Users can switch market types with minimal configuration
6. **Accuracy:** Fee structures match OKX's published rates
---
## Open Questions
1. **Position Sizing with Leverage:**
- Should leverage affect `init_cash` interpretation (notional value) or position size directly?
- Recommendation: Affect position size; `init_cash` remains the actual margin deposited.
2. **Multiple Positions:**
- Can strategies hold both long and short simultaneously (hedging)?
- Recommendation: No for v1; only one direction at a time.
3. **Funding Rate Sign:**
- When funding is positive, longs pay shorts. Should we assume the user is always the "taker" of funding?
- Recommendation: Yes, apply funding based on position direction.
4. **Migration Path:**
- Should we migrate existing data to new directory structure?
- Recommendation: No auto-migration; users re-download with `--market` flag.
---
## Implementation Priority
| Priority | Component | Complexity |
|----------|-----------|------------|
| 1 | MarketType enum + strategy defaults | Low |
| 2 | DataManager market type support | Medium |
| 3 | Fee structure per market type | Low |
| 4 | Short-selling signal support | Medium |
| 5 | Leverage simulation | Medium |
| 6 | Funding rate simulation | Medium |
| 7 | Liquidation warnings | Low |
| 8 | Reporting updates | Low |
| 9 | Grid search leverage support | Low |

76
tasks/prd-vectorbt-migration.md
View File

@@ -1,76 +0,0 @@
# PRD: VectorBT Migration & CCXT Integration
## 1. Introduction
The goal of this project is to refactor the current backtesting infrastructure to a professional-grade stack using **VectorBT** for high-performance backtesting and **CCXT** for robust historical data acquisition. The system will support rapid prototyping of "many simple strategies," parameter optimization (Grid Search), and stability testing (Walk-Forward Analysis).
## 2. Goals
- **Replace Custom Backtester:** Retire the existing loop-based backtesting logic in favor of vectorized operations using `vectorbt`.
- **Automate Data Collection:** Implement a `ccxt` based downloader to fetch and cache OHLCV data from OKX (and other exchanges) automatically.
- **Enable Optimization:** Built-in support for Grid Search to find optimal strategy parameters.
- **Validation:** Implement Walk-Forward Analysis (WFA) to validate strategy robustness and prevent overfitting.
- **Standardized Reporting:** Generate consistent outputs: Console summaries, CSV logs, and VectorBT interactive plots.
## 3. User Stories
- **Data Acquisition:** "As a user, I want to run a command `download_data --pair BTC/USDT --exchange okx` and have the system fetch historical 1-minute candles and save them to `data/ccxt/okx/BTC-USDT/1m.csv`."
- **Strategy Dev:** "As a researcher, I want to define a new strategy by simply writing a class/function that defines entry/exit signals, without worrying about the backtesting loop."
- **Optimization:** "As a researcher, I want to say 'Optimize RSI period between 10 and 20' and get a heatmap of results."
- **Validation:** "As a researcher, I want to verify if my 'best' parameters work on unseen data using Walk-Forward Analysis."
- **Analysis:** "As a user, I want to see an equity curve and key metrics (Sharpe, Drawdown) immediately after a test run."
## 4. Functional Requirements
### 4.1 Data Module (`data_manager`)
- **Exchange Interface:** Use `ccxt` to connect to exchanges (initially OKX).
- **Fetching Logic:** Fetch OHLCV data in chunks to handle rate limits and long histories.
- **Storage:** Save data to standardized paths: `data/ccxt/{exchange}/{pair}_{timeframe}.csv`.
- **Loading:** Utility to load saved CSVs into a Pandas DataFrame compatible with `vectorbt`.
### 4.2 Strategy Interface (`strategies/`)
- **Base Protocol:** Define a standard structure for strategies. A strategy should return/define:
- Indicator calculations (Vectorized).
- Entry signals (Boolean Series).
- Exit signals (Boolean Series).
- **Parameterization:** Strategies must accept dynamic parameters to support Grid Search.
### 4.3 Backtest Engine (`engine.py`)
- **Simulation:** Use `vectorbt.Portfolio.from_signals` (or similar) for fast simulation.
- **Cost Model:** Support configurable fees (maker/taker) and slippage estimates.
- **Grid Search:** Utilize `vectorbt`'s parameter broadcasting to run many variations simultaneously.
- **Walk-Forward Analysis:**
- Implement a splitting mechanism (e.g., `vectorbt.Splitter`) to divide data into In-Sample (Train) and Out-of-Sample (Test) sets.
- Execute optimization on Train, validate on Test.
### 4.4 Reporting (`reporting.py`)
- **Console:** Print key metrics: Total Return, Sharpe Ratio, Max Drawdown, Win Rate, Count of Trades.
- **Files:** Save detailed trade logs and metrics summaries to `backtest_logs/`.
- **Visuals:** Generate and save/show `vectorbt` plots (Equity curve, Drawdowns).
## 5. Non-Goals
- Real-time live trading execution (this is strictly for research/backtesting).
- Complex Machine Learning models (initially focusing on indicator-based logic).
- High-frequency tick-level backtesting (1-minute granularity is the target).
## 6. Technical Architecture Proposal
```text
project_root/
├── data/
│ └── ccxt/ # New data storage structure
├── strategies/ # Strategy definitions
│ ├── __init__.py
│ ├── base.py # Abstract Base Class
│ └── ma_cross.py # Example strategy
├── engine/
│ ├── data_loader.py # CCXT wrapper
│ ├── backtester.py # VBT runner
│ └── optimizer.py # Grid Search & WFA logic
├── main.py # CLI entry point
└── pyproject.toml
```
## 7. Success Metrics
- Can download 1 year of 1m BTC/USDT data from OKX in < 2 minutes.
- Can run a 100-parameter grid search on 1 year of 1m data in < 10 seconds.
- Walk-forward analysis produces a clear "Robustness Score" or visual comparison of Train vs Test performance.
## 8. Open Questions
- Do we need to handle funding rates for perp futures in the PnL calculation immediately? (Assumed NO for V1, stick to spot/simple futures price action).