From 7e4a6874a2e40e8af41afc2b1bb99790477c0424 Mon Sep 17 00:00:00 2001 From: Simon Moisy Date: Thu, 15 Jan 2026 10:51:42 +0800 Subject: [PATCH] Cleaning up PRDs files --- tasks/prd-interactive-ui.md | 238 ----------------------- tasks/prd-market-type-selection.md | 295 ----------------------------- tasks/prd-vectorbt-migration.md | 76 -------- 3 files changed, 609 deletions(-) delete mode 100644 tasks/prd-interactive-ui.md delete mode 100644 tasks/prd-market-type-selection.md delete mode 100644 tasks/prd-vectorbt-migration.md diff --git a/tasks/prd-interactive-ui.md b/tasks/prd-interactive-ui.md deleted file mode 100644 index 5e07b2e..0000000 --- a/tasks/prd-interactive-ui.md +++ /dev/null @@ -1,238 +0,0 @@ -# PRD: Interactive Backtest UI - -## 1. Introduction / Overview - -Lowkey Backtest currently operates exclusively through a CLI. This project adds an interactive web-based UI that allows users to: - -- Run backtests with existing code-defined strategies -- Adjust strategy parameters without editing code -- Compare multiple backtest runs visually -- Analyze performance through interactive charts and metrics - -The UI will use **FastAPI** for the backend API and **Vue 3** for the frontend, styled with a dark theme inspired by QuantConnect. - ---- - -## 2. Goals - -| # | Goal | Success Metric | -|---|------|----------------| -| G1 | Run backtests from UI | User can select strategy, symbol, parameters and execute backtest | -| G2 | Visualize results | Equity curves, trade markers, and metrics displayed interactively | -| G3 | Compare runs | Side-by-side comparison of 2+ backtest results | -| G4 | Parameter experimentation | Adjust parameters via form inputs, re-run, and compare to previous | -| G5 | Self-hostable | Deployable on user's Arch-based NAS with minimal configuration | - ---- - -## 3. User Stories - -### US1: Run a Backtest -> As a trader, I want to select a strategy and configure its parameters in the UI so that I can run a backtest without using the terminal. - -### US2: View Backtest Results -> As a trader, I want to see an interactive equity curve with trade entry/exit markers so that I can visually understand strategy performance. - -### US3: Compare Parameter Changes -> As a trader, I want to compare two or more backtest runs side-by-side so that I can determine if a parameter optimization improves performance. - -### US4: Quick Parameter Tweaking -> As a trader, I want to adjust strategy parameters (e.g., period, multiplier, SL/TP) via sliders or inputs and re-run the backtest so that I can iterate quickly. - -### US5: Review Historical Runs -> As a trader, I want to see a list of previous backtest runs so that I can reload and compare past results. - ---- - -## 4. Functional Requirements - -### 4.1 Backend (FastAPI) - -| # | Requirement | -|---|-------------| -| FR1 | `GET /api/strategies` - Return list of available strategies with their default and grid parameters | -| FR2 | `GET /api/symbols` - Return list of available symbols from local data | -| FR3 | `POST /api/backtest` - Execute a backtest with given strategy, symbol, timeframe, and parameters. Return results including portfolio stats, trades, and equity curve data | -| FR4 | `GET /api/backtest/{run_id}` - Retrieve a specific historical backtest result | -| FR5 | `GET /api/backtests` - List all saved backtest runs with summary metadata | -| FR6 | `POST /api/compare` - Accept multiple run IDs and return combined data for comparison | -| FR7 | `GET /api/data/status` - Return available data inventory (symbols, timeframes, date ranges) | -| FR8 | Results must be persisted (SQLite or JSON files) to allow historical retrieval | - -### 4.2 Frontend (Vue 3) - -| # | Requirement | -|---|-------------| -| FR9 | **Strategy Selector**: Dropdown to select strategy (rsi, macross, meta_st, regime) | -| FR10 | **Parameter Form**: Dynamic form fields based on selected strategy's parameters (inputs, sliders) | -| FR11 | **Symbol/Market Picker**: Select trading pair and market type (spot/perpetual) | -| FR12 | **Date Range Selector**: Pick start and end dates for backtest period | -| FR13 | **Run Button**: Execute backtest with loading state indicator | -| FR14 | **Equity Curve Chart**: Interactive line chart (Plotly.js) showing portfolio value over time with drawdown overlay (shaded area) | -| FR15 | **Trade Markers**: Overlay entry (green) and exit (red) markers on price/equity chart | -| FR16 | **Metrics Panel**: Display key stats (Total Return, Sharpe, Max DD, Win Rate, etc.) | -| FR17 | **Trade Log Table**: Sortable, filterable table of all trades | -| FR18 | **Run History Sidebar**: List of previous runs with quick-load action | -| FR19 | **Comparison View**: Select 2+ runs to overlay equity curves and compare metrics in a table | -| FR20 | **Dark Theme**: QuantConnect-inspired dark color scheme | - -### 4.3 Comparison Features - -| # | Requirement | -|---|-------------| -| FR21 | Overlay up to 5 equity curves on the same chart with different colors | -| FR22 | Metrics comparison table showing deltas between runs | -| FR23 | Highlight which run performed better for each metric | -| FR24 | Display parameter differences between compared runs | - ---- - -## 5. Non-Goals (Out of Scope) - -| # | Non-Goal | -|---|----------| -| NG1 | Strategy code editing in the UI (strategies are created in code editor) | -| NG2 | Live trading or paper trading integration | -| NG3 | Real-time market data streaming | -| NG4 | Multi-user authentication or access control | -| NG5 | Walk-Forward Analysis UI (may be added in future iteration) | -| NG6 | Data download management (use CLI for now) | - ---- - -## 6. Design Considerations - -### 6.1 Visual Style -- **Theme**: Dark mode only (similar to QuantConnect, TradingView dark) -- **Colors**: - - Background: `#1a1a2e` (deep navy) - - Cards/Panels: `#16213e` - - Accent: `#0f3460` (muted blue) - - Profit: `#00d26a` (green) - - Loss: `#ff6b6b` (red) - - Text: `#e8e8e8` -- **Typography**: Monospace for numbers/stats, clean sans-serif for labels -- **Layout**: Sidebar navigation + main content area - -### 6.2 Component Library -- Consider Tailwind CSS for styling flexibility -- Plotly.js for charts (matches existing VectorBT ecosystem) -- Headless UI or Radix for accessible components - -### 6.3 Responsive Design -- Primary target: Desktop (1920x1080+) -- Minimum supported: 1280x720 -- Mobile not required - ---- - -## 7. Technical Considerations - -### 7.1 Architecture - -``` -lowkey_backtest/ - api/ # FastAPI backend - __init__.py - main.py # FastAPI app entry point - routers/ - backtest.py # Backtest endpoints - strategies.py # Strategy info endpoints - data.py # Data status endpoints - models/ - schemas.py # Pydantic request/response models - services/ - runner.py # Wraps existing Backtester - storage.py # Run persistence (SQLite) - frontend/ # Vue 3 app (Vite) - src/ - components/ # Reusable UI components - views/ # Page-level components - composables/ # Vue composition functions - api/ # Axios API client - assets/ # Styles, images -``` - -### 7.2 Data Flow - -1. Frontend requests available strategies via `GET /api/strategies` -2. User configures parameters and clicks "Run Backtest" -3. Frontend sends `POST /api/backtest` with configuration -4. Backend instantiates strategy, runs `Backtester.run_strategy()`, saves result -5. Backend returns serialized results (equity curve as JSON array, trades, stats) -6. Frontend renders charts and metrics -7. Run is saved and appears in history sidebar - -### 7.3 Dependencies - -**Backend:** -- FastAPI -- Uvicorn (ASGI server) -- SQLAlchemy (run storage) -- Existing engine modules (Backtester, DataManager, etc.) - -**Frontend:** -- Vue 3 (Composition API) -- Vite (build tool) -- Plotly.js (charts) -- Tailwind CSS (styling) -- Axios (API calls) - -### 7.4 Deployment - -- Development: `uvicorn api.main:app --reload` + `npm run dev` -- Production (NAS): - - Build frontend static files - - Serve via FastAPI's `StaticFiles` or nginx - - Run with systemd service - ---- - -## 8. Success Metrics - -| Metric | Target | -|--------|--------| -| Backtest execution via UI | Works for all 4 strategies | -| Result visualization | Equity curve + trades render correctly | -| Comparison view | Can compare up to 5 runs with overlaid equity curves | -| Run persistence | Historical runs survive app restart | -| Page load time | < 2s for dashboard | -| Backtest response time | Same as CLI (backend performance unchanged) | - ---- - -## 9. Resolved Questions - -| # | Question | Decision | -|---|----------|----------| -| Q1 | React or Vue for frontend? | **Vue 3** - simpler, sufficient for this use case | -| Q2 | Should the equity curve include drawdown overlay by default? | **Yes** - shaded area beneath equity curve | -| Q3 | How many historical runs to keep before auto-cleanup? | **Unlimited** - manual deletion only | -| Q4 | Should comparison support more than 2 runs? | **Yes** - support 3-5 run overlays | -| Q5 | Include WFA visualization in V1 or defer to V2? | Deferred to V2 | - ---- - -## 10. Milestones (Suggested) - -| Phase | Deliverable | Effort | -|-------|-------------|--------| -| **Phase 1** | FastAPI backend with core endpoints (strategies, backtest, history) | 2-3 days | -| **Phase 2** | Frontend scaffold with dark theme and strategy selector | 2-3 days | -| **Phase 3** | Backtest execution flow and results visualization | 3-4 days | -| **Phase 4** | Run history and comparison view | 2-3 days | -| **Phase 5** | Polish, testing, NAS deployment | 2 days | - -**Total Estimated Effort**: 11-15 days - ---- - -## Appendix: Reference Screenshots - -*(QuantConnect-style inspiration)* - -- Dark navy background with card-based layout -- Left sidebar for navigation and run history -- Main area split: configuration panel (top/left) + results (bottom/right) -- Charts with grid lines, legend, and interactive tooltips -- Metrics displayed in a compact card grid diff --git a/tasks/prd-market-type-selection.md b/tasks/prd-market-type-selection.md deleted file mode 100644 index e272cd2..0000000 --- a/tasks/prd-market-type-selection.md +++ /dev/null @@ -1,295 +0,0 @@ -# PRD: Market Type Selection for Backtesting - -## Introduction/Overview - -Currently, the backtesting system operates with a single, implicit market type assumption. This PRD defines the implementation of **market type selection** (Spot vs. USDT-M Perpetual Futures) to enable realistic simulation of different trading conditions. - -**Problem Statement:** -- Strategies cannot be backtested against different market mechanics (leverage, funding, short-selling) -- Fee structures are uniform regardless of market type -- No support for short-selling strategies -- Data fetching doesn't distinguish between spot and futures markets - -**Goal:** -Enable users to backtest strategies against specific market types (Spot or USDT-M Perpetual) with realistic trading conditions matching OKX's live environment. - ---- - -## Goals - -1. **Support two market types:** Spot and USDT-M Perpetual Futures -2. **Realistic fee simulation:** Match OKX's fee structure per market type -3. **Leverage support:** Per-strategy configurable leverage (perpetuals only) -4. **Funding rate simulation:** Simplified funding rate model for perpetuals -5. **Short-selling support:** Enable strategies to generate short signals -6. **Liquidation awareness:** Warn when positions would be liquidated (no full simulation) -7. **Separate data storage:** Download and store data per market type -8. **Grid search integration:** Allow leverage optimization in parameter grids - ---- - -## User Stories - -1. **As a trader**, I want to backtest my strategy on perpetual futures so that I can simulate leveraged trading with funding costs. - -2. **As a trader**, I want to backtest on spot markets so that I can compare performance without leverage or funding overhead. - -3. **As a strategy developer**, I want to define a default market type for my strategy so that it runs with appropriate settings by default. - -4. **As a trader**, I want to test different leverage levels so that I can find the optimal risk/reward balance. - -5. **As a trader**, I want to see warnings when my position would have been liquidated so that I can adjust my risk parameters. - -6. **As a strategy developer**, I want to create strategies that can go short so that I can profit from downward price movements. - ---- - -## Functional Requirements - -### FR1: Market Type Enum and Configuration - -1.1. Create a `MarketType` enum with values: `SPOT`, `PERPETUAL` - -1.2. Each strategy class must have a `default_market_type` class attribute - -1.3. Market type can be overridden via CLI (optional, for testing) - -### FR2: Data Management - -2.1. Modify `DataManager` to support market type in data paths: - - Spot: `data/ccxt/{exchange}/spot/{symbol}/{timeframe}.csv` - - Perpetual: `data/ccxt/{exchange}/perpetual/{symbol}/{timeframe}.csv` - -2.2. Update `download` command to accept `--market` flag: - ```bash - uv run python main.py download --pair BTC/USDT --market perpetual - ``` - -2.3. Use CCXT's market type parameter when fetching data: - - Spot: `exchange.fetch_ohlcv(symbol, timeframe, ...)` - - Perpetual: `exchange.fetch_ohlcv(symbol + ':USDT', timeframe, ...)` - -### FR3: Fee Structure - -3.1. Define default fees per market type (matching OKX): - -| Market Type | Maker Fee | Taker Fee | Notes | -|-------------|-----------|-----------|-------| -| Spot | 0.08% | 0.10% | No funding | -| Perpetual | 0.02% | 0.05% | + funding | - -3.2. Allow fee override via CLI (existing `--fees` flag) - -### FR4: Leverage Support (Perpetual Only) - -4.1. Add `default_leverage` class attribute to strategies (default: 1 for spot, configurable for perpetual) - -4.2. Add `--leverage` CLI flag for backtest command - -4.3. Leverage affects: - - Position sizing (notional = cash * leverage) - - PnL calculation (multiplied by leverage) - - Liquidation threshold calculation - -4.4. Support leverage in grid search parameter grids - -### FR5: Funding Rate Simulation (Perpetual Only) - -5.1. Implement simplified funding rate model: - - Default rate: 0.01% per 8 hours (configurable) - - Applied every 8 hours to open positions - - Positive rate: Longs pay shorts - - Negative rate: Shorts pay longs - -5.2. Add `--funding-rate` CLI flag to override default - -5.3. Track cumulative funding paid/received in backtest stats - -### FR6: Short-Selling Support - -6.1. Modify `BaseStrategy.run()` signature to return 4 signal arrays: - ```python - def run(self, close, **kwargs) -> tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, pd.DataFrame]: - """ - Returns: - long_entries: Boolean signals to open long positions - long_exits: Boolean signals to close long positions - short_entries: Boolean signals to open short positions - short_exits: Boolean signals to close short positions - """ - ``` - -6.2. Update `Backtester` to use VectorBT's `direction` parameter or dual portfolio simulation - -6.3. For spot market: Ignore short signals (log warning if present) - -6.4. For perpetual market: Process both long and short signals - -### FR7: Liquidation Warning - -7.1. Calculate liquidation price based on: - - Entry price - - Leverage - - Maintenance margin rate (OKX: ~0.4% for BTC) - -7.2. During backtest, check if price crosses liquidation threshold - -7.3. Log warning with details: - ``` - WARNING: Position would be liquidated at bar 1234 (price: $45,000, liq_price: $44,820) - ``` - -7.4. Include liquidation event count in backtest summary stats - -### FR8: Backtester Integration - -8.1. Modify `Backtester.run_strategy()` to accept market type from strategy - -8.2. Apply market-specific simulation parameters: - - Fees (if not overridden) - - Leverage - - Funding rate calculation - - Short-selling capability - -8.3. Update portfolio simulation to handle leveraged positions - -### FR9: Reporting Updates - -9.1. Add market type to backtest summary output - -9.2. Add new stats for perpetual backtests: - - Total funding paid/received - - Number of liquidation warnings - - Effective leverage used - -9.3. Update CSV exports to include market-specific columns - ---- - -## Non-Goals (Out of Scope) - -- **Coin-M (Inverse) Perpetuals:** Not included in v1 -- **Spot Margin Trading:** Not included in v1 -- **Expiry Futures:** Not included in v1 -- **Full Liquidation Simulation:** Only warnings, no automatic position closure -- **Real Funding Rate Data:** Use simplified model; historical funding API integration is future work -- **Cross-Margin Mode:** Assume isolated margin for simplicity -- **Partial Liquidation:** Assume full liquidation threshold only - ---- - -## Design Considerations - -### Data Directory Structure (New) - -``` -data/ccxt/ - okx/ - spot/ - BTC-USDT/ - 1m.csv - 1d.csv - perpetual/ - BTC-USDT/ - 1m.csv - 1d.csv -``` - -### Strategy Class Example - -```python -class MetaSupertrendStrategy(BaseStrategy): - default_market_type = MarketType.PERPETUAL - default_leverage = 5 - default_sl_stop = 0.02 - - def run(self, close, **kwargs): - # ... indicator logic ... - return long_entries, long_exits, short_entries, short_exits -``` - -### CLI Usage Examples - -```bash -# Download perpetual data -uv run python main.py download --pair BTC/USDT --market perpetual - -# Backtest with strategy defaults (uses strategy's default_market_type) -uv run python main.py backtest --strategy meta_st --pair BTC/USDT - -# Override leverage -uv run python main.py backtest --strategy meta_st --pair BTC/USDT --leverage 10 - -# Grid search including leverage -uv run python main.py backtest --strategy meta_st --pair BTC/USDT --grid -# (leverage can be part of param grid in strategy factory) -``` - ---- - -## Technical Considerations - -1. **VectorBT Compatibility:** - - VectorBT's `Portfolio.from_signals()` supports `direction` parameter for long/short - - Alternatively, run two portfolios (long-only, short-only) and combine - - Leverage can be simulated via `size` parameter or post-processing returns - -2. **CCXT Market Type Handling:** - - OKX perpetual symbols use format: `BTC/USDT:USDT` - - Need to handle symbol conversion in DataManager - -3. **Funding Rate Timing:** - - OKX funding at 00:00, 08:00, 16:00 UTC - - Need to identify these timestamps in the data and apply funding - -4. **Backward Compatibility:** - - Existing strategies should work with minimal changes - - Default to `MarketType.SPOT` if not specified - - Existing 2-tuple return from `run()` should be interpreted as long-only - ---- - -## Success Metrics - -1. **Functional:** All existing backtests produce same results when run with `MarketType.SPOT` -2. **Functional:** Perpetual backtests correctly apply funding every 8 hours -3. **Functional:** Leverage multiplies both gains and losses correctly -4. **Functional:** Short signals are processed for perpetual, ignored for spot -5. **Usability:** Users can switch market types with minimal configuration -6. **Accuracy:** Fee structures match OKX's published rates - ---- - -## Open Questions - -1. **Position Sizing with Leverage:** - - Should leverage affect `init_cash` interpretation (notional value) or position size directly? - - Recommendation: Affect position size; `init_cash` remains the actual margin deposited. - -2. **Multiple Positions:** - - Can strategies hold both long and short simultaneously (hedging)? - - Recommendation: No for v1; only one direction at a time. - -3. **Funding Rate Sign:** - - When funding is positive, longs pay shorts. Should we assume the user is always the "taker" of funding? - - Recommendation: Yes, apply funding based on position direction. - -4. **Migration Path:** - - Should we migrate existing data to new directory structure? - - Recommendation: No auto-migration; users re-download with `--market` flag. - ---- - -## Implementation Priority - -| Priority | Component | Complexity | -|----------|-----------|------------| -| 1 | MarketType enum + strategy defaults | Low | -| 2 | DataManager market type support | Medium | -| 3 | Fee structure per market type | Low | -| 4 | Short-selling signal support | Medium | -| 5 | Leverage simulation | Medium | -| 6 | Funding rate simulation | Medium | -| 7 | Liquidation warnings | Low | -| 8 | Reporting updates | Low | -| 9 | Grid search leverage support | Low | diff --git a/tasks/prd-vectorbt-migration.md b/tasks/prd-vectorbt-migration.md deleted file mode 100644 index 6949b05..0000000 --- a/tasks/prd-vectorbt-migration.md +++ /dev/null @@ -1,76 +0,0 @@ -# PRD: VectorBT Migration & CCXT Integration - -## 1. Introduction -The goal of this project is to refactor the current backtesting infrastructure to a professional-grade stack using **VectorBT** for high-performance backtesting and **CCXT** for robust historical data acquisition. The system will support rapid prototyping of "many simple strategies," parameter optimization (Grid Search), and stability testing (Walk-Forward Analysis). - -## 2. Goals -- **Replace Custom Backtester:** Retire the existing loop-based backtesting logic in favor of vectorized operations using `vectorbt`. -- **Automate Data Collection:** Implement a `ccxt` based downloader to fetch and cache OHLCV data from OKX (and other exchanges) automatically. -- **Enable Optimization:** Built-in support for Grid Search to find optimal strategy parameters. -- **Validation:** Implement Walk-Forward Analysis (WFA) to validate strategy robustness and prevent overfitting. -- **Standardized Reporting:** Generate consistent outputs: Console summaries, CSV logs, and VectorBT interactive plots. - -## 3. User Stories -- **Data Acquisition:** "As a user, I want to run a command `download_data --pair BTC/USDT --exchange okx` and have the system fetch historical 1-minute candles and save them to `data/ccxt/okx/BTC-USDT/1m.csv`." -- **Strategy Dev:** "As a researcher, I want to define a new strategy by simply writing a class/function that defines entry/exit signals, without worrying about the backtesting loop." -- **Optimization:** "As a researcher, I want to say 'Optimize RSI period between 10 and 20' and get a heatmap of results." -- **Validation:** "As a researcher, I want to verify if my 'best' parameters work on unseen data using Walk-Forward Analysis." -- **Analysis:** "As a user, I want to see an equity curve and key metrics (Sharpe, Drawdown) immediately after a test run." - -## 4. Functional Requirements - -### 4.1 Data Module (`data_manager`) -- **Exchange Interface:** Use `ccxt` to connect to exchanges (initially OKX). -- **Fetching Logic:** Fetch OHLCV data in chunks to handle rate limits and long histories. -- **Storage:** Save data to standardized paths: `data/ccxt/{exchange}/{pair}_{timeframe}.csv`. -- **Loading:** Utility to load saved CSVs into a Pandas DataFrame compatible with `vectorbt`. - -### 4.2 Strategy Interface (`strategies/`) -- **Base Protocol:** Define a standard structure for strategies. A strategy should return/define: - - Indicator calculations (Vectorized). - - Entry signals (Boolean Series). - - Exit signals (Boolean Series). -- **Parameterization:** Strategies must accept dynamic parameters to support Grid Search. - -### 4.3 Backtest Engine (`engine.py`) -- **Simulation:** Use `vectorbt.Portfolio.from_signals` (or similar) for fast simulation. -- **Cost Model:** Support configurable fees (maker/taker) and slippage estimates. -- **Grid Search:** Utilize `vectorbt`'s parameter broadcasting to run many variations simultaneously. -- **Walk-Forward Analysis:** - - Implement a splitting mechanism (e.g., `vectorbt.Splitter`) to divide data into In-Sample (Train) and Out-of-Sample (Test) sets. - - Execute optimization on Train, validate on Test. - -### 4.4 Reporting (`reporting.py`) -- **Console:** Print key metrics: Total Return, Sharpe Ratio, Max Drawdown, Win Rate, Count of Trades. -- **Files:** Save detailed trade logs and metrics summaries to `backtest_logs/`. -- **Visuals:** Generate and save/show `vectorbt` plots (Equity curve, Drawdowns). - -## 5. Non-Goals -- Real-time live trading execution (this is strictly for research/backtesting). -- Complex Machine Learning models (initially focusing on indicator-based logic). -- High-frequency tick-level backtesting (1-minute granularity is the target). - -## 6. Technical Architecture Proposal -```text -project_root/ -├── data/ -│ └── ccxt/ # New data storage structure -├── strategies/ # Strategy definitions -│ ├── __init__.py -│ ├── base.py # Abstract Base Class -│ └── ma_cross.py # Example strategy -├── engine/ -│ ├── data_loader.py # CCXT wrapper -│ ├── backtester.py # VBT runner -│ └── optimizer.py # Grid Search & WFA logic -├── main.py # CLI entry point -└── pyproject.toml -``` - -## 7. Success Metrics -- Can download 1 year of 1m BTC/USDT data from OKX in < 2 minutes. -- Can run a 100-parameter grid search on 1 year of 1m data in < 10 seconds. -- Walk-forward analysis produces a clear "Robustness Score" or visual comparison of Train vs Test performance. - -## 8. Open Questions -- Do we need to handle funding rates for perp futures in the PnL calculation immediately? (Assumed NO for V1, stick to spot/simple futures price action).