Simon Moisy
b5550f4ff4
- Introduced `train_daily.sh` for automating daily model retraining, including data download and model training steps. - Added `install_cron.sh` for setting up a cron job to run the daily training script. - Created `setup_schedule.sh` for configuring Systemd timers for daily training tasks. - Implemented a terminal UI using Rich for real-time monitoring of trading performance, including metrics display and log handling. - Updated `pyproject.toml` to include the `rich` dependency for UI functionality. - Enhanced `.gitignore` to exclude model and log files. - Added database support for trade persistence and metrics calculation. - Updated README with installation and usage instructions for the new features.
352 lines
13 KiB
Markdown
352 lines
13 KiB
Markdown
# PRD: Terminal UI for Live Trading Bot
|
|
|
|
## Introduction/Overview
|
|
|
|
The live trading bot currently uses basic console logging for output, making it difficult to monitor trading activity, track performance, and understand the system state at a glance. This feature introduces a Rich-based terminal UI that provides a professional, real-time dashboard for monitoring the live trading bot.
|
|
|
|
The UI will display a horizontal split layout with a **summary panel** at the top (with tabbed time-period views) and a **scrollable log panel** at the bottom. The interface will update every second and support keyboard navigation.
|
|
|
|
## Goals
|
|
|
|
1. Provide real-time visibility into trading performance (PnL, win rate, trade count)
|
|
2. Enable monitoring of current position state (entry, SL/TP, unrealized PnL)
|
|
3. Display strategy signals (Z-score, model probability) for transparency
|
|
4. Support historical performance tracking across time periods (daily, weekly, monthly, all-time)
|
|
5. Improve operational experience with keyboard shortcuts and log filtering
|
|
6. Create a responsive design that works across different terminal sizes
|
|
|
|
## User Stories
|
|
|
|
1. **As a trader**, I want to see my total PnL and daily PnL at a glance so I can quickly assess performance.
|
|
2. **As a trader**, I want to see my current position details (entry price, unrealized PnL, SL/TP levels) so I can monitor risk.
|
|
3. **As a trader**, I want to view performance metrics by time period (daily, weekly, monthly) so I can track trends.
|
|
4. **As a trader**, I want to filter logs by type (errors, trades, signals) so I can focus on relevant information.
|
|
5. **As a trader**, I want keyboard shortcuts to navigate the UI without using a mouse.
|
|
6. **As a trader**, I want the UI to show strategy state (Z-score, probability) so I understand why signals are generated.
|
|
|
|
## Functional Requirements
|
|
|
|
### FR1: Layout Structure
|
|
|
|
1.1. The UI must use a horizontal split layout with the summary panel at the top and logs panel at the bottom.
|
|
|
|
1.2. The summary panel must contain tabbed views accessible via number keys:
|
|
- Tab 1 (`1`): **General** - Overall metrics since bot started
|
|
- Tab 2 (`2`): **Monthly** - Current month metrics (shown only if data spans > 1 month)
|
|
- Tab 3 (`3`): **Weekly** - Current week metrics (shown only if data spans > 1 week)
|
|
- Tab 4 (`4`): **Daily** - Today's metrics
|
|
|
|
1.3. The logs panel must be a scrollable area showing recent log entries.
|
|
|
|
1.4. The UI must be responsive and adapt to terminal size (minimum 80x24).
|
|
|
|
### FR2: Metrics Display
|
|
|
|
The summary panel must display the following metrics:
|
|
|
|
**Performance Metrics:**
|
|
2.1. Total PnL (USD) - cumulative profit/loss since tracking began
|
|
2.2. Period PnL (USD) - profit/loss for selected time period (daily/weekly/monthly)
|
|
2.3. Win Rate (%) - percentage of winning trades
|
|
2.4. Total Number of Trades
|
|
2.5. Average Trade Duration (hours)
|
|
2.6. Max Drawdown (USD and %)
|
|
|
|
**Current Position (if open):**
|
|
2.7. Symbol and side (long/short)
|
|
2.8. Entry price
|
|
2.9. Current price
|
|
2.10. Unrealized PnL (USD and %)
|
|
2.11. Stop-loss price and distance (%)
|
|
2.12. Take-profit price and distance (%)
|
|
2.13. Position size (USD)
|
|
|
|
**Account Status:**
|
|
2.14. Account balance / available margin (USDT)
|
|
2.15. Current leverage setting
|
|
|
|
**Strategy State:**
|
|
2.16. Current Z-score
|
|
2.17. Model probability
|
|
2.18. Current funding rate (BTC)
|
|
2.19. Last signal action and reason
|
|
|
|
### FR3: Historical Data Loading
|
|
|
|
3.1. On startup, the system must initialize SQLite database at `live_trading/trading.db`.
|
|
|
|
3.2. If `trade_log.csv` exists and database is empty, migrate CSV data to SQLite.
|
|
|
|
3.3. The UI must load current positions from `live_trading/positions.json` (kept for compatibility with existing position manager).
|
|
|
|
3.4. Metrics must be calculated via SQL aggregation queries for each time period.
|
|
|
|
3.5. If no historical data exists, the UI must show "No data" gracefully.
|
|
|
|
3.6. New trades must be written to both SQLite (primary) and CSV (backup/compatibility).
|
|
|
|
### FR4: Real-Time Updates
|
|
|
|
4.1. The UI must refresh every 1 second.
|
|
|
|
4.2. Position unrealized PnL must update based on latest price data.
|
|
|
|
4.3. New log entries must appear in real-time.
|
|
|
|
4.4. Metrics must recalculate when trades are opened/closed.
|
|
|
|
### FR5: Log Panel
|
|
|
|
5.1. The log panel must display log entries with timestamp, level, and message.
|
|
|
|
5.2. Log entries must be color-coded by level:
|
|
- ERROR: Red
|
|
- WARNING: Yellow
|
|
- INFO: White/Default
|
|
- DEBUG: Gray (if shown)
|
|
|
|
5.3. The log panel must support filtering by log type:
|
|
- All logs (default)
|
|
- Errors only
|
|
- Trades only (entries containing "position", "trade", "order")
|
|
- Signals only (entries containing "signal", "z_score", "prob")
|
|
|
|
5.4. Filter switching must be available via keyboard shortcut (`f` to cycle filters).
|
|
|
|
### FR6: Keyboard Controls
|
|
|
|
6.1. `q` or `Ctrl+C` - Graceful shutdown
|
|
6.2. `r` - Force refresh data
|
|
6.3. `1` - Switch to General tab
|
|
6.4. `2` - Switch to Monthly tab
|
|
6.5. `3` - Switch to Weekly tab
|
|
6.6. `4` - Switch to Daily tab
|
|
6.7. `f` - Cycle log filter
|
|
6.8. Arrow keys - Scroll logs (if supported)
|
|
|
|
### FR7: Color Scheme
|
|
|
|
7.1. Use dark theme as base.
|
|
|
|
7.2. PnL values must be colored:
|
|
- Positive: Green
|
|
- Negative: Red
|
|
- Zero/Neutral: White
|
|
|
|
7.3. Position side must be colored:
|
|
- Long: Green
|
|
- Short: Red
|
|
|
|
7.4. Use consistent color coding for emphasis and warnings.
|
|
|
|
## Non-Goals (Out of Scope)
|
|
|
|
1. **Mouse support** - This is a keyboard-driven terminal UI
|
|
2. **Trade execution from UI** - The UI is read-only; trades are executed by the bot
|
|
3. **Configuration editing** - Config changes require restarting the bot
|
|
4. **Multi-exchange support** - Only OKX is supported
|
|
5. **Charts/graphs** - Text-based metrics only (no ASCII charts in v1)
|
|
6. **Sound alerts** - No audio notifications
|
|
7. **Remote access** - Local terminal only
|
|
|
|
## Design Considerations
|
|
|
|
### Technology Choice: Rich
|
|
|
|
Use the [Rich](https://github.com/Textualize/rich) Python library for terminal UI:
|
|
- Rich provides `Live` display for real-time updates
|
|
- Rich `Layout` for split-screen design
|
|
- Rich `Table` for metrics display
|
|
- Rich `Panel` for bordered sections
|
|
- Rich `Text` for colored output
|
|
|
|
Alternative considered: **Textual** (also by Will McGugan) provides more advanced TUI features but adds complexity. Rich is simpler and sufficient for this use case.
|
|
|
|
### UI Mockup
|
|
|
|
```
|
|
+==============================================================================+
|
|
| REGIME REVERSION STRATEGY - LIVE TRADING [DEMO] ETH/USDT |
|
|
+==============================================================================+
|
|
| [1:General] [2:Monthly] [3:Weekly] [4:Daily] |
|
|
+------------------------------------------------------------------------------+
|
|
| PERFORMANCE | CURRENT POSITION |
|
|
| Total PnL: $1,234.56 | Side: LONG |
|
|
| Today PnL: $45.23 | Entry: $3,245.50 |
|
|
| Win Rate: 67.5% | Current: $3,289.00 |
|
|
| Total Trades: 24 | Unrealized: +$43.50 (+1.34%) |
|
|
| Avg Duration: 4.2h | Size: $500.00 |
|
|
| Max Drawdown: -$156.00 | SL: $3,050.00 (-6.0%) TP: $3,408.00 (+5%)|
|
|
| | |
|
|
| ACCOUNT | STRATEGY STATE |
|
|
| Balance: $5,432.10 | Z-Score: 1.45 |
|
|
| Available: $4,932.10 | Probability: 0.72 |
|
|
| Leverage: 2x | Funding: 0.0012 |
|
|
+------------------------------------------------------------------------------+
|
|
| LOGS [Filter: All] Press 'f' cycle |
|
|
+------------------------------------------------------------------------------+
|
|
| 14:32:15 [INFO] Trading Cycle Start: 2026-01-16T14:32:15+00:00 |
|
|
| 14:32:16 [INFO] Signal: entry long (prob=0.72, z=-1.45, reason=z_score...) |
|
|
| 14:32:17 [INFO] Executing LONG entry: 0.1540 ETH @ 3245.50 ($500.00) |
|
|
| 14:32:18 [INFO] Position opened: ETH/USDT:USDT_20260116_143217 |
|
|
| 14:32:18 [INFO] Portfolio: 1 positions, exposure=$500.00, unrealized=$0.00 |
|
|
| 14:32:18 [INFO] --- Cycle completed in 3.2s --- |
|
|
| 14:32:18 [INFO] Sleeping for 60 minutes... |
|
|
| |
|
|
+------------------------------------------------------------------------------+
|
|
| [q]Quit [r]Refresh [1-4]Tabs [f]Filter |
|
|
+==============================================================================+
|
|
```
|
|
|
|
### File Structure
|
|
|
|
```
|
|
live_trading/
|
|
ui/
|
|
__init__.py
|
|
dashboard.py # Main UI orchestration and threading
|
|
panels.py # Panel components (metrics, logs, position)
|
|
state.py # Thread-safe shared state
|
|
log_handler.py # Custom logging handler for UI queue
|
|
keyboard.py # Keyboard input handling
|
|
db/
|
|
__init__.py
|
|
database.py # SQLite connection and queries
|
|
models.py # Data models (Trade, DailySummary, Session)
|
|
migrations.py # CSV migration and schema setup
|
|
metrics.py # Metrics aggregation queries
|
|
trading.db # SQLite database file (created at runtime)
|
|
```
|
|
|
|
## Technical Considerations
|
|
|
|
### Integration with Existing Code
|
|
|
|
1. **Logging Integration**: Create a custom `logging.Handler` that captures log messages and forwards them to the UI log panel while still writing to file.
|
|
|
|
2. **Data Access**: The UI needs access to:
|
|
- `PositionManager` for current positions
|
|
- `TradingConfig` for settings display
|
|
- SQLite database for historical metrics
|
|
- Real-time data from `DataFeed` for current prices
|
|
|
|
3. **Main Loop Modification**: The `LiveTradingBot.run()` method needs modification to run the UI in a **separate thread** alongside the trading loop. The UI thread handles rendering and keyboard input while the main thread executes trading logic.
|
|
|
|
4. **Graceful Shutdown**: Ensure `SIGINT`/`SIGTERM` handlers work with the UI layer and properly terminate the UI thread.
|
|
|
|
### Database Schema (SQLite)
|
|
|
|
Create `live_trading/trading.db` with the following schema:
|
|
|
|
```sql
|
|
-- Trade history table
|
|
CREATE TABLE trades (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
trade_id TEXT UNIQUE NOT NULL,
|
|
symbol TEXT NOT NULL,
|
|
side TEXT NOT NULL, -- 'long' or 'short'
|
|
entry_price REAL NOT NULL,
|
|
exit_price REAL,
|
|
size REAL NOT NULL,
|
|
size_usdt REAL NOT NULL,
|
|
pnl_usd REAL,
|
|
pnl_pct REAL,
|
|
entry_time TEXT NOT NULL, -- ISO format
|
|
exit_time TEXT,
|
|
hold_duration_hours REAL,
|
|
reason TEXT, -- 'stop_loss', 'take_profit', 'signal', etc.
|
|
order_id_entry TEXT,
|
|
order_id_exit TEXT,
|
|
created_at TEXT DEFAULT CURRENT_TIMESTAMP
|
|
);
|
|
|
|
-- Daily summary table (for faster queries)
|
|
CREATE TABLE daily_summary (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
date TEXT UNIQUE NOT NULL, -- YYYY-MM-DD
|
|
total_trades INTEGER DEFAULT 0,
|
|
winning_trades INTEGER DEFAULT 0,
|
|
total_pnl_usd REAL DEFAULT 0,
|
|
max_drawdown_usd REAL DEFAULT 0,
|
|
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
|
|
);
|
|
|
|
-- Session metadata
|
|
CREATE TABLE sessions (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
start_time TEXT NOT NULL,
|
|
end_time TEXT,
|
|
starting_balance REAL,
|
|
ending_balance REAL,
|
|
total_pnl REAL,
|
|
total_trades INTEGER DEFAULT 0
|
|
);
|
|
|
|
-- Indexes for common queries
|
|
CREATE INDEX idx_trades_entry_time ON trades(entry_time);
|
|
CREATE INDEX idx_trades_exit_time ON trades(exit_time);
|
|
CREATE INDEX idx_daily_summary_date ON daily_summary(date);
|
|
```
|
|
|
|
### Migration from CSV
|
|
|
|
On first run with the new system:
|
|
1. Check if `trading.db` exists
|
|
2. If not, create database with schema
|
|
3. If `trade_log.csv` exists, migrate data to `trades` table
|
|
4. Rebuild `daily_summary` from migrated trades
|
|
|
|
### Dependencies
|
|
|
|
Add to `pyproject.toml`:
|
|
```toml
|
|
dependencies = [
|
|
# ... existing deps
|
|
"rich>=13.0.0",
|
|
]
|
|
```
|
|
|
|
Note: SQLite is part of Python's standard library (`sqlite3`), no additional dependency needed.
|
|
|
|
### Performance Considerations
|
|
|
|
- UI runs in a separate thread to avoid blocking trading logic
|
|
- Log buffer limited to 1000 entries in memory to prevent growth
|
|
- SQLite queries should use indexes for fast period-based aggregations
|
|
- Historical data loading happens once at startup, incremental updates thereafter
|
|
|
|
### Threading Model
|
|
|
|
```
|
|
Main Thread UI Thread
|
|
| |
|
|
v v
|
|
[Trading Loop] [Rich Live Display]
|
|
| |
|
|
+---> SharedState <------------+
|
|
(thread-safe)
|
|
| |
|
|
+---> LogQueue <---------------+
|
|
(thread-safe)
|
|
```
|
|
|
|
- Use `threading.Lock` for shared state access
|
|
- Use `queue.Queue` for log message passing
|
|
- UI thread polls for updates every 1 second
|
|
|
|
## Success Metrics
|
|
|
|
1. UI starts successfully and displays all required metrics
|
|
2. UI updates in real-time (1-second refresh) without impacting trading performance
|
|
3. All keyboard shortcuts function correctly
|
|
4. Historical data loads and displays accurately from SQLite
|
|
5. Log filtering works as expected
|
|
6. UI gracefully handles edge cases (no data, no position, terminal resize)
|
|
7. CSV migration completes successfully on first run
|
|
8. Database queries complete within 100ms
|
|
|
|
---
|
|
|
|
*Generated: 2026-01-16*
|
|
*Decisions: Threading model, 1000 log buffer, SQLite database, no fallback mode*
|