TCPDashboard/tasks/tasks-crypto-bot-prd.md

Relevant Files

• app.py - Main Dash application entry point and dashboard interface
• bot_manager.py - Bot lifecycle management and coordination
• database/models.py - PostgreSQL database models and schema definitions (updated to match schema_clean.sql)
• database/schema_clean.sql - Clean database schema without hypertables (actively used, includes raw_trades table)
• database/schema.sql - Complete database schema with TimescaleDB hypertables (for future optimization)
• database/connection.py - Database connection utility with connection pooling, session management, and raw data utilities
• database/init/init.sql - Docker initialization script for automatic database setup
• database/init/schema_clean.sql - Copy of clean schema for Docker initialization
• data/okx_collector.py - OKX API integration for real-time market data collection
• data/aggregator.py - OHLCV candle aggregation and processing
• strategies/base_strategy.py - Base strategy class and interface
• strategies/ema_crossover.py - Example EMA crossover strategy implementation
• components/dashboard.py - Dashboard UI components and layouts
• components/charts.py - Price charts and visualization components
• backtesting/engine.py - Backtesting engine for historical strategy testing
• backtesting/performance.py - Performance metrics calculation
• config/bot_configs/ - Directory for JSON bot configuration files
• config/strategies/ - Directory for JSON strategy parameter files
• config/settings.py - Centralized configuration settings using Pydantic
• scripts/dev.py - Development setup and management script
• scripts/init_database.py - Database initialization and verification script
• requirements.txt - Python dependencies managed by UV
• docker-compose.yml - Docker services configuration with TimescaleDB support
• tests/test_strategies.py - Unit tests for strategy implementations
• tests/test_bot_manager.py - Unit tests for bot management functionality
• tests/test_data_collection.py - Unit tests for data collection and aggregation
• docs/setup.md - Comprehensive setup guide for new machines and environments

Tasks

• 1.0 Database Foundation and Schema Setup

  • 1.1 Install and configure PostgreSQL with Docker
  • 1.2 Create database schema following the PRD specifications (market_data, bots, signals, trades, bot_performance tables)
  • 1.3 Implement database connection utility with connection pooling
  • 1.4 Create database models using SQLAlchemy or similar ORM
  • 1.5 Add proper indexes for time-series data optimization
  • 1.6 Setup Redis for pub/sub messaging
  • 1.7 Create database migration scripts and initial data seeding
  • 1.8 Unit test database models and connection utilities

• 2.0 Market Data Collection and Processing System

  • 2.1 Implement OKX WebSocket API connector for real-time data
  • 2.2 Create OHLCV candle aggregation logic with multiple timeframes (1m, 5m, 15m, 1h, 4h, 1d)
  • 2.3 Build data validation and error handling for market data
  • 2.4 Implement Redis channels for real-time data distribution
  • 2.5 Create data storage layer for OHLCV data in PostgreSQL
  • 2.6 Add technical indicators calculation (SMA, EMA, RSI, MACD, Bollinger Bands)
  • 2.7 Implement data recovery and reconnection logic for API failures
  • 2.8 Create data collection service with proper logging
  • 2.9 Unit test data collection and aggregation logic

• 3.0 Basic Dashboard for Data Visualization and Analysis

  • 3.1 Setup Dash application framework with Mantine UI components
  • 3.2 Create basic layout and navigation structure
  • 3.3 Implement real-time OHLCV price charts with Plotly (candlestick charts)
  • 3.4 Add technical indicators overlay on price charts (SMA, EMA, RSI, MACD)
  • 3.5 Create market data monitoring dashboard (real-time data feed status)
  • 3.6 Build simple data analysis tools (volume analysis, price movement statistics)
  • 3.7 Setup real-time dashboard updates using Redis callbacks
  • 3.8 Add data export functionality for analysis (CSV/JSON export)
  • 3.9 Unit test basic dashboard components and data visualization

• 4.0 Strategy Engine and Bot Management Framework

  • 4.1 Design and implement base strategy interface class
  • 4.2 Create EMA crossover strategy as reference implementation
  • 4.3 Implement JSON-based strategy parameter configuration system
  • 4.4 Build bot lifecycle management (create, start, stop, pause, delete)
  • 4.5 Create signal generation and processing logic
  • 4.6 Implement virtual portfolio management and balance tracking
  • 4.7 Add bot status monitoring and heartbeat system
  • 4.8 Create bot configuration management with JSON files
  • 4.9 Implement multi-bot coordination and resource management
  • 4.10 Unit test strategy engine and bot management functionality

• 5.0 Advanced Dashboard Features and Bot Interface

  • 5.1 Build bot management interface (start/stop controls, status indicators)
  • 5.2 Create bot configuration forms for JSON parameter editing
  • 5.3 Add strategy signal overlay on price charts
  • 5.4 Implement bot status monitoring dashboard
  • 5.5 Create system health and performance monitoring interface
  • 5.6 Unit test advanced dashboard features and bot interface

• 6.0 Backtesting Engine and Performance Analytics

  • 6.1 Implement historical data loading from database or file
  • 6.2 Create vectorized backtesting engine using pandas operations
  • 6.3 Build performance metrics calculation (Sharpe ratio, drawdown, win rate, total return)
  • 6.4 Implement realistic fee modeling (0.1% per trade for OKX)
  • 6.5 Add look-ahead bias prevention with proper timestamp handling
  • 6.6 Create parallel backtesting system for multiple strategies
  • 6.7 Create strategy comparison and reporting functionality
  • 6.8 Build backtesting results visualization and export
  • 6.9 Implement configurable test periods (1 day to 24 months)
  • 6.10 Unit test backtesting engine and performance analytics

• 7.0 Real-Time Trading Simulation

  • 7.1 Implement virtual trading execution engine
  • 7.2 Create order management system (market, limit orders)
  • 7.3 Build trade execution logic with proper timing
  • 7.4 Implement position tracking and balance updates
  • 7.5 Add risk management controls (stop-loss, take-profit, position sizing)
  • 7.6 Create trade reconciliation and confirmation system
  • 7.7 Implement fee calculation and tracking
  • 7.8 Add emergency stop mechanisms for bots
  • 7.9 Unit test real-time trading simulation

• 8.0 Portfolio Visualization and Trade Analytics

  • 8.1 Build portfolio performance visualization charts (equity curve, drawdown, win rate)
  • 8.2 Create trade history table with P&L calculations
  • 8.3 Implement real-time portfolio tracking and updates
  • 8.4 Add performance comparison charts between multiple bots
  • 8.5 Create trade analytics and statistics dashboard
  • 8.6 Unit test portfolio visualization and trade analytics

• 9.0 Documentation and User Guide

  • 9.1 Write comprehensive README with setup instructions
  • 9.2 Create API documentation for all modules
  • 9.3 Document strategy development guidelines
  • 9.4 Write user guide for bot configuration and management
  • 9.5 Create troubleshooting guide for common issues
  • 9.6 Document database schema and data flow
  • 9.7 Add code comments and docstrings throughout codebase

• 10.0 Deployment and Monitoring Setup

  • 10.1 Create Docker containers for all services
  • 10.2 Setup docker-compose for local development environment
  • 10.3 Implement health checks for all services
  • 10.4 Create deployment scripts and configuration
  • 10.5 Setup basic logging and monitoring
  • 10.6 Implement crash recovery and auto-restart mechanisms
  • 10.7 Create backup and restore procedures for database

• 11.0 Security and Error Handling

  • 11.1 Implement secure API key storage and management
  • 11.2 Add input validation for all user inputs and API responses
  • 11.3 Create comprehensive error handling and logging throughout system
  • 11.4 Implement rate limiting for API calls
  • 11.5 Add data encryption for sensitive information
  • 11.6 Create security audit checklist and implementation
  • 11.7 Implement graceful degradation for partial system failures

• 12.0 Final Integration and Testing

  • 12.1 Comprehensive system integration testing
  • 12.2 Performance optimization and bottleneck identification
  • 12.3 Memory leak detection and cleanup
  • 12.4 End-to-end testing with multiple concurrent bots
  • 12.5 Documentation updates and final review
  • 12.6 Prepare for production deployment
  • 12.7 Create maintenance and support procedures

• 13.0 Performance Optimization and Scaling (Future Enhancement)

  • 13.1 Implement TimescaleDB hypertables for time-series optimization
  • 13.2 Optimize database schema for hypertable compatibility (composite primary keys)
  • 13.3 Add database query performance monitoring and analysis
  • 13.4 Implement advanced connection pooling optimization
  • 13.5 Add caching layer for frequently accessed market data
  • 13.6 Optimize data retention and archival strategies
  • 13.7 Implement horizontal scaling for high-volume trading scenarios

Notes

• Automatic Database Setup: Database schema is automatically initialized when Docker containers start via database/init/ scripts
• Environment Configuration: All credentials and settings are managed via .env file with consistent defaults
• Security: No hardcoded passwords exist in the codebase - all credentials must be loaded from environment variables
• Clean Schema Approach: Using schema_clean.sql for simpler setup without TimescaleDB hypertables (can be upgraded later)
• Unit tests should be placed in the tests/ directory with descriptive names
• Use uv run pytest to run all tests or uv run pytest tests/specific_test.py for individual test files
• JSON configuration files allow rapid strategy parameter testing without code changes
• Redis will be used for real-time messaging between components
• Database models now use JSONB instead of JSON for PostgreSQL optimization
• Connection pooling is configured with proper retry logic and monitoring
• Raw data is stored in PostgreSQL with automatic cleanup utilities (configurable retention period)
• Raw data storage includes: ticker data, trade data, orderbook snapshots, candle data, and balance updates