Files
lowkey_backtest/tasks/prd-terminal-ui.md
Simon Moisy b5550f4ff4 Add daily model training scripts and terminal UI for live trading
- Introduced `train_daily.sh` for automating daily model retraining, including data download and model training steps.
- Added `install_cron.sh` for setting up a cron job to run the daily training script.
- Created `setup_schedule.sh` for configuring Systemd timers for daily training tasks.
- Implemented a terminal UI using Rich for real-time monitoring of trading performance, including metrics display and log handling.
- Updated `pyproject.toml` to include the `rich` dependency for UI functionality.
- Enhanced `.gitignore` to exclude model and log files.
- Added database support for trade persistence and metrics calculation.
- Updated README with installation and usage instructions for the new features.
2026-01-18 11:08:57 +08:00

13 KiB

PRD: Terminal UI for Live Trading Bot

Introduction/Overview

The live trading bot currently uses basic console logging for output, making it difficult to monitor trading activity, track performance, and understand the system state at a glance. This feature introduces a Rich-based terminal UI that provides a professional, real-time dashboard for monitoring the live trading bot.

The UI will display a horizontal split layout with a summary panel at the top (with tabbed time-period views) and a scrollable log panel at the bottom. The interface will update every second and support keyboard navigation.

Goals

  1. Provide real-time visibility into trading performance (PnL, win rate, trade count)
  2. Enable monitoring of current position state (entry, SL/TP, unrealized PnL)
  3. Display strategy signals (Z-score, model probability) for transparency
  4. Support historical performance tracking across time periods (daily, weekly, monthly, all-time)
  5. Improve operational experience with keyboard shortcuts and log filtering
  6. Create a responsive design that works across different terminal sizes

User Stories

  1. As a trader, I want to see my total PnL and daily PnL at a glance so I can quickly assess performance.
  2. As a trader, I want to see my current position details (entry price, unrealized PnL, SL/TP levels) so I can monitor risk.
  3. As a trader, I want to view performance metrics by time period (daily, weekly, monthly) so I can track trends.
  4. As a trader, I want to filter logs by type (errors, trades, signals) so I can focus on relevant information.
  5. As a trader, I want keyboard shortcuts to navigate the UI without using a mouse.
  6. As a trader, I want the UI to show strategy state (Z-score, probability) so I understand why signals are generated.

Functional Requirements

FR1: Layout Structure

1.1. The UI must use a horizontal split layout with the summary panel at the top and logs panel at the bottom.

1.2. The summary panel must contain tabbed views accessible via number keys:

  • Tab 1 (1): General - Overall metrics since bot started
  • Tab 2 (2): Monthly - Current month metrics (shown only if data spans > 1 month)
  • Tab 3 (3): Weekly - Current week metrics (shown only if data spans > 1 week)
  • Tab 4 (4): Daily - Today's metrics

1.3. The logs panel must be a scrollable area showing recent log entries.

1.4. The UI must be responsive and adapt to terminal size (minimum 80x24).

FR2: Metrics Display

The summary panel must display the following metrics:

Performance Metrics: 2.1. Total PnL (USD) - cumulative profit/loss since tracking began 2.2. Period PnL (USD) - profit/loss for selected time period (daily/weekly/monthly) 2.3. Win Rate (%) - percentage of winning trades 2.4. Total Number of Trades 2.5. Average Trade Duration (hours) 2.6. Max Drawdown (USD and %)

Current Position (if open): 2.7. Symbol and side (long/short) 2.8. Entry price 2.9. Current price 2.10. Unrealized PnL (USD and %) 2.11. Stop-loss price and distance (%) 2.12. Take-profit price and distance (%) 2.13. Position size (USD)

Account Status: 2.14. Account balance / available margin (USDT) 2.15. Current leverage setting

Strategy State: 2.16. Current Z-score 2.17. Model probability 2.18. Current funding rate (BTC) 2.19. Last signal action and reason

FR3: Historical Data Loading

3.1. On startup, the system must initialize SQLite database at live_trading/trading.db.

3.2. If trade_log.csv exists and database is empty, migrate CSV data to SQLite.

3.3. The UI must load current positions from live_trading/positions.json (kept for compatibility with existing position manager).

3.4. Metrics must be calculated via SQL aggregation queries for each time period.

3.5. If no historical data exists, the UI must show "No data" gracefully.

3.6. New trades must be written to both SQLite (primary) and CSV (backup/compatibility).

FR4: Real-Time Updates

4.1. The UI must refresh every 1 second.

4.2. Position unrealized PnL must update based on latest price data.

4.3. New log entries must appear in real-time.

4.4. Metrics must recalculate when trades are opened/closed.

FR5: Log Panel

5.1. The log panel must display log entries with timestamp, level, and message.

5.2. Log entries must be color-coded by level:

  • ERROR: Red
  • WARNING: Yellow
  • INFO: White/Default
  • DEBUG: Gray (if shown)

5.3. The log panel must support filtering by log type:

  • All logs (default)
  • Errors only
  • Trades only (entries containing "position", "trade", "order")
  • Signals only (entries containing "signal", "z_score", "prob")

5.4. Filter switching must be available via keyboard shortcut (f to cycle filters).

FR6: Keyboard Controls

6.1. q or Ctrl+C - Graceful shutdown 6.2. r - Force refresh data 6.3. 1 - Switch to General tab 6.4. 2 - Switch to Monthly tab 6.5. 3 - Switch to Weekly tab 6.6. 4 - Switch to Daily tab 6.7. f - Cycle log filter 6.8. Arrow keys - Scroll logs (if supported)

FR7: Color Scheme

7.1. Use dark theme as base.

7.2. PnL values must be colored:

  • Positive: Green
  • Negative: Red
  • Zero/Neutral: White

7.3. Position side must be colored:

  • Long: Green
  • Short: Red

7.4. Use consistent color coding for emphasis and warnings.

Non-Goals (Out of Scope)

  1. Mouse support - This is a keyboard-driven terminal UI
  2. Trade execution from UI - The UI is read-only; trades are executed by the bot
  3. Configuration editing - Config changes require restarting the bot
  4. Multi-exchange support - Only OKX is supported
  5. Charts/graphs - Text-based metrics only (no ASCII charts in v1)
  6. Sound alerts - No audio notifications
  7. Remote access - Local terminal only

Design Considerations

Technology Choice: Rich

Use the Rich Python library for terminal UI:

  • Rich provides Live display for real-time updates
  • Rich Layout for split-screen design
  • Rich Table for metrics display
  • Rich Panel for bordered sections
  • Rich Text for colored output

Alternative considered: Textual (also by Will McGugan) provides more advanced TUI features but adds complexity. Rich is simpler and sufficient for this use case.

UI Mockup

+==============================================================================+
|  REGIME REVERSION STRATEGY - LIVE TRADING                    [DEMO]  ETH/USDT |
+==============================================================================+
| [1:General] [2:Monthly] [3:Weekly] [4:Daily]                                  |
+------------------------------------------------------------------------------+
|  PERFORMANCE                    |  CURRENT POSITION                          |
|  Total PnL:      $1,234.56      |  Side:        LONG                         |
|  Today PnL:        $45.23       |  Entry:       $3,245.50                    |
|  Win Rate:          67.5%       |  Current:     $3,289.00                    |
|  Total Trades:         24       |  Unrealized:  +$43.50 (+1.34%)             |
|  Avg Duration:      4.2h        |  Size:        $500.00                      |
|  Max Drawdown:    -$156.00      |  SL: $3,050.00 (-6.0%)  TP: $3,408.00 (+5%)|
|                                 |                                            |
|  ACCOUNT                        |  STRATEGY STATE                            |
|  Balance:       $5,432.10       |  Z-Score:      1.45                        |
|  Available:     $4,932.10       |  Probability:  0.72                        |
|  Leverage:            2x        |  Funding:      0.0012                      |
+------------------------------------------------------------------------------+
|  LOGS  [Filter: All]                                          Press 'f' cycle |
+------------------------------------------------------------------------------+
|  14:32:15 [INFO] Trading Cycle Start: 2026-01-16T14:32:15+00:00              |
|  14:32:16 [INFO] Signal: entry long (prob=0.72, z=-1.45, reason=z_score...)  |
|  14:32:17 [INFO] Executing LONG entry: 0.1540 ETH @ 3245.50 ($500.00)        |
|  14:32:18 [INFO] Position opened: ETH/USDT:USDT_20260116_143217              |
|  14:32:18 [INFO] Portfolio: 1 positions, exposure=$500.00, unrealized=$0.00  |
|  14:32:18 [INFO] --- Cycle completed in 3.2s ---                             |
|  14:32:18 [INFO] Sleeping for 60 minutes...                                  |
|                                                                              |
+------------------------------------------------------------------------------+
|  [q]Quit  [r]Refresh  [1-4]Tabs  [f]Filter                                   |
+==============================================================================+

File Structure

live_trading/
  ui/
    __init__.py
    dashboard.py      # Main UI orchestration and threading
    panels.py         # Panel components (metrics, logs, position)
    state.py          # Thread-safe shared state
    log_handler.py    # Custom logging handler for UI queue
    keyboard.py       # Keyboard input handling
  db/
    __init__.py
    database.py       # SQLite connection and queries
    models.py         # Data models (Trade, DailySummary, Session)
    migrations.py     # CSV migration and schema setup
    metrics.py        # Metrics aggregation queries
  trading.db          # SQLite database file (created at runtime)

Technical Considerations

Integration with Existing Code

  1. Logging Integration: Create a custom logging.Handler that captures log messages and forwards them to the UI log panel while still writing to file.

  2. Data Access: The UI needs access to:

    • PositionManager for current positions
    • TradingConfig for settings display
    • SQLite database for historical metrics
    • Real-time data from DataFeed for current prices
  3. Main Loop Modification: The LiveTradingBot.run() method needs modification to run the UI in a separate thread alongside the trading loop. The UI thread handles rendering and keyboard input while the main thread executes trading logic.

  4. Graceful Shutdown: Ensure SIGINT/SIGTERM handlers work with the UI layer and properly terminate the UI thread.

Database Schema (SQLite)

Create live_trading/trading.db with the following schema:

-- Trade history table
CREATE TABLE trades (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    trade_id TEXT UNIQUE NOT NULL,
    symbol TEXT NOT NULL,
    side TEXT NOT NULL,  -- 'long' or 'short'
    entry_price REAL NOT NULL,
    exit_price REAL,
    size REAL NOT NULL,
    size_usdt REAL NOT NULL,
    pnl_usd REAL,
    pnl_pct REAL,
    entry_time TEXT NOT NULL,  -- ISO format
    exit_time TEXT,
    hold_duration_hours REAL,
    reason TEXT,  -- 'stop_loss', 'take_profit', 'signal', etc.
    order_id_entry TEXT,
    order_id_exit TEXT,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP
);

-- Daily summary table (for faster queries)
CREATE TABLE daily_summary (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    date TEXT UNIQUE NOT NULL,  -- YYYY-MM-DD
    total_trades INTEGER DEFAULT 0,
    winning_trades INTEGER DEFAULT 0,
    total_pnl_usd REAL DEFAULT 0,
    max_drawdown_usd REAL DEFAULT 0,
    updated_at TEXT DEFAULT CURRENT_TIMESTAMP
);

-- Session metadata
CREATE TABLE sessions (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    start_time TEXT NOT NULL,
    end_time TEXT,
    starting_balance REAL,
    ending_balance REAL,
    total_pnl REAL,
    total_trades INTEGER DEFAULT 0
);

-- Indexes for common queries
CREATE INDEX idx_trades_entry_time ON trades(entry_time);
CREATE INDEX idx_trades_exit_time ON trades(exit_time);
CREATE INDEX idx_daily_summary_date ON daily_summary(date);

Migration from CSV

On first run with the new system:

  1. Check if trading.db exists
  2. If not, create database with schema
  3. If trade_log.csv exists, migrate data to trades table
  4. Rebuild daily_summary from migrated trades

Dependencies

Add to pyproject.toml:

dependencies = [
    # ... existing deps
    "rich>=13.0.0",
]

Note: SQLite is part of Python's standard library (sqlite3), no additional dependency needed.

Performance Considerations

  • UI runs in a separate thread to avoid blocking trading logic
  • Log buffer limited to 1000 entries in memory to prevent growth
  • SQLite queries should use indexes for fast period-based aggregations
  • Historical data loading happens once at startup, incremental updates thereafter

Threading Model

Main Thread                    UI Thread
    |                              |
    v                              v
[Trading Loop]              [Rich Live Display]
    |                              |
    +---> SharedState <------------+
          (thread-safe)
    |                              |
    +---> LogQueue <---------------+
          (thread-safe)
  • Use threading.Lock for shared state access
  • Use queue.Queue for log message passing
  • UI thread polls for updates every 1 second

Success Metrics

  1. UI starts successfully and displays all required metrics
  2. UI updates in real-time (1-second refresh) without impacting trading performance
  3. All keyboard shortcuts function correctly
  4. Historical data loads and displays accurately from SQLite
  5. Log filtering works as expected
  6. UI gracefully handles edge cases (no data, no position, terminal resize)
  7. CSV migration completes successfully on first run
  8. Database queries complete within 100ms

Generated: 2026-01-16 Decisions: Threading model, 1000 log buffer, SQLite database, no fallback mode