283 lines
12 KiB
Markdown
283 lines
12 KiB
Markdown
# TCP Dashboard Documentation
|
|
|
|
Welcome to the documentation for the TCP Trading Platform. This resource provides comprehensive information for developers, contributors, and anyone interested in the platform's architecture and functionality.
|
|
|
|
## Table of Contents
|
|
|
|
### 1. Project Overview
|
|
- **[Project Context (`CONTEXT.md`)]** - The single source of truth for the project's current state, architecture, and conventions. **Start here.**
|
|
- **[Product Requirements (`crypto-bot-prd.md`)]** - The Product Requirements Document (PRD) outlining the project's goals and scope.
|
|
|
|
### 2. Getting Started
|
|
- **[Setup Guide (`guides/setup.md`)]** - Instructions for setting up the development environment.
|
|
- **[Contributing (`CONTRIBUTING.md`)]** - Guidelines for contributing to the project.
|
|
|
|
### 3. Architecture & Design
|
|
- **[Architecture Overview (`architecture.md`)]** - High-level system architecture, components, and data flow.
|
|
- **[Architecture Decision Records (`decisions/`)](./decisions/)** - Key architectural decisions and their justifications.
|
|
|
|
### 4. Modules Documentation
|
|
This section contains detailed technical documentation for each system module.
|
|
|
|
- **[Chart System (`modules/charts/`)](./modules/charts/)** - Comprehensive documentation for the modular chart system.
|
|
- **[Data Collectors (`modules/data_collectors.md`)]** - Guide to the data collector framework.
|
|
- **[Database Operations (`modules/database_operations.md`)]** - Details on the repository pattern for database interactions.
|
|
- **[Technical Indicators (`modules/technical-indicators.md`)]** - Information on the technical analysis module.
|
|
- **[Exchange Integrations (`modules/exchanges/`)](./modules/exchanges/)** - Exchange-specific implementation details.
|
|
- **[Logging System (`modules/logging.md`)]** - The unified logging framework.
|
|
- **[Data Collection Service (`modules/services/data_collection_service.md`)]** - The high-level service that orchestrates data collectors.
|
|
|
|
### 5. API & Reference
|
|
- **[API Documentation (`API.md`)]** - Placeholder for future REST API documentation.
|
|
- **[Technical Reference (`reference/`)](./reference/)** - Detailed specifications, data formats, and standards.
|
|
- **[Changelog (`CHANGELOG.md`)]** - A log of all notable changes to the project.
|
|
|
|
## How to Use This Documentation
|
|
|
|
- **For a high-level understanding**, start with the `CONTEXT.md` and `architecture.md` files.
|
|
- **For development tasks**, refer to the specific module documentation in the `modules/` directory.
|
|
- **For setup and contribution guidelines**, see the `guides/` and `CONTRIBUTING.md` files.
|
|
|
|
This documentation is intended to be a living document that evolves with the project. Please keep it up-to-date as you make changes.
|
|
|
|
|
|
|
|
### 📖 **[Setup & Guides](guides/)**
|
|
|
|
- **[Setup Guide](guides/setup.md)** - *Comprehensive setup instructions*
|
|
- Environment configuration and prerequisites
|
|
- Database setup with Docker and PostgreSQL
|
|
- Development workflow and best practices
|
|
- Production deployment guidelines
|
|
|
|
### 📋 **[Technical Reference](reference/)**
|
|
|
|
- **[Project Specification](reference/specification.md)** - *Technical specifications and requirements*
|
|
- System requirements and constraints
|
|
- Database schema specifications
|
|
- API endpoint definitions
|
|
- Data format specifications
|
|
|
|
- **[Aggregation Strategy](reference/aggregation-strategy.md)** - *Comprehensive data aggregation documentation*
|
|
- Right-aligned timestamp strategy (industry standard)
|
|
- Future leakage prevention safeguards
|
|
- Real-time vs historical processing
|
|
- Database storage patterns
|
|
- Testing methodology and examples
|
|
|
|
## 🎯 **Quick Start**
|
|
|
|
1. **New to the platform?** Start with the [Setup Guide](guides/setup.md)
|
|
2. **Working with charts and indicators?** See [Chart Layers Documentation](components/charts/)
|
|
3. **Implementing data collectors?** See [Data Collectors Documentation](components/data_collectors.md)
|
|
4. **Understanding the architecture?** Read [Architecture Overview](architecture/architecture.md)
|
|
5. **Modular dashboard development?** Check [Dashboard Structure](dashboard-modular-structure.md)
|
|
6. **Exchange integration?** Check [Exchange Documentation](exchanges/)
|
|
7. **Troubleshooting?** Check component-specific documentation
|
|
|
|
## 🏛️ **System Components**
|
|
|
|
### Core Infrastructure
|
|
- **Database Layer**: PostgreSQL with SQLAlchemy models
|
|
- **Real-time Messaging**: Redis pub/sub for data distribution
|
|
- **Configuration Management**: Pydantic-based settings
|
|
- **Containerization**: Docker and docker-compose setup
|
|
|
|
### Data Collection & Processing
|
|
- **Abstract Base Collectors**: Standardized interface for all exchange connectors
|
|
- **Exchange Factory Pattern**: Unified collector creation across exchanges
|
|
- **Modular Exchange Architecture**: Organized exchange implementations in dedicated folders
|
|
- **Health Monitoring**: Automatic failure detection and recovery
|
|
- **Data Validation**: Comprehensive validation for market data
|
|
- **Multi-Exchange Support**: OKX (production-ready), Binance and other exchanges (planned)
|
|
|
|
### Trading & Strategy Engine
|
|
- **Strategy Framework**: Base strategy classes and implementations
|
|
- **Bot Management**: Lifecycle management with JSON configuration
|
|
- **Backtesting Engine**: Historical strategy testing with performance metrics
|
|
- **Portfolio Management**: Virtual trading with P&L tracking
|
|
|
|
### User Interface & Visualization
|
|
- **Modular Dashboard**: Dash-based web interface with separated layouts and callbacks
|
|
- **Chart Layers System**: Interactive price charts with 26+ technical indicators
|
|
- **Strategy Templates**: 5 pre-configured trading strategies (EMA crossover, momentum, etc.)
|
|
- **User Indicator Management**: Custom indicator creation with JSON persistence
|
|
- **Real-time Updates**: Chart and system health monitoring with auto-refresh
|
|
- **Bot Controls**: Start/stop/configure trading bots (planned)
|
|
- **Performance Analytics**: Portfolio visualization and trade analytics (planned)
|
|
|
|
## 📋 **Task Progress**
|
|
|
|
The platform follows a structured development approach with clearly defined tasks:
|
|
|
|
- ✅ **Database Foundation** - Complete
|
|
- ✅ **Enhanced Data Collectors** - Complete with health monitoring
|
|
- ✅ **OKX Data Collector** - Complete with factory pattern and production testing
|
|
- ✅ **Modular Chart Layers System** - Complete with strategy support
|
|
- ✅ **Dashboard Modular Structure** - Complete with separated concerns
|
|
- ✅ **Custom Indicator Management** - Complete with CRUD operations
|
|
- ⏳ **Multi-Exchange Support** - In progress (Binance connector next)
|
|
- ⏳ **Bot Signal Layer** - Planned for integration
|
|
- ⏳ **Strategy Engine** - Planned
|
|
- ⏳ **Advanced Features** - Planned
|
|
|
|
For detailed task tracking, see [tasks/tasks-crypto-bot-prd.md](../tasks/tasks-crypto-bot-prd.md) and [tasks/3.4. Chart layers.md](../tasks/3.4. Chart layers.md).
|
|
|
|
## 🛠️ **Development Workflow**
|
|
|
|
### Setting Up Development Environment
|
|
|
|
```bash
|
|
# Clone and setup
|
|
git clone <repository>
|
|
cd TCPDashboard
|
|
|
|
# Install dependencies with UV
|
|
uv sync
|
|
|
|
# Setup environment
|
|
cp .env.example .env
|
|
# Edit .env with your configuration
|
|
|
|
# Start services
|
|
docker-compose up -d
|
|
|
|
# Initialize database
|
|
uv run python scripts/init_database.py
|
|
|
|
# Run tests
|
|
uv run pytest
|
|
```
|
|
|
|
### Key Development Tools
|
|
|
|
- **UV**: Modern Python package management
|
|
- **pytest**: Testing framework with async support
|
|
- **SQLAlchemy**: Database ORM with migration support
|
|
- **Dash + Mantine**: Modern web UI framework
|
|
- **Docker**: Containerized development environment
|
|
|
|
## 🔍 **Testing**
|
|
|
|
The platform includes comprehensive test coverage:
|
|
|
|
- **Unit Tests**: Individual component testing
|
|
- **Integration Tests**: Cross-component functionality
|
|
- **Performance Tests**: Load and stress testing
|
|
- **End-to-End Tests**: Full system workflows
|
|
|
|
```bash
|
|
# Run all tests
|
|
uv run pytest
|
|
|
|
# Run specific test files
|
|
uv run pytest tests/test_base_collector.py
|
|
uv run pytest tests/test_collector_manager.py
|
|
|
|
# Run with coverage
|
|
uv run pytest --cov=data --cov-report=html
|
|
```
|
|
|
|
## 📊 **Monitoring & Observability**
|
|
|
|
### Logging
|
|
- **Structured Logging**: JSON-formatted logs with automatic cleanup
|
|
- **Multiple Levels**: Debug, Info, Warning, Error with configurable output
|
|
- **Component Isolation**: Separate loggers for different system components
|
|
|
|
### Health Monitoring
|
|
- **Collector Health**: Real-time status and performance metrics
|
|
- **Auto-Recovery**: Automatic restart on failures
|
|
- **Performance Tracking**: Message rates, uptime, error rates
|
|
|
|
### Metrics Integration
|
|
- **Prometheus Support**: Built-in metrics collection
|
|
- **Custom Dashboards**: System performance visualization
|
|
- **Alerting**: Configurable alerts for system health
|
|
|
|
## 🔐 **Security & Best Practices**
|
|
|
|
### Configuration Management
|
|
- **Environment Variables**: All sensitive data via `.env` files
|
|
- **No Hardcoded Secrets**: Clean separation of configuration and code
|
|
- **Validation**: Pydantic-based configuration validation
|
|
|
|
### Data Handling
|
|
- **Input Validation**: Comprehensive validation for all external data
|
|
- **Error Handling**: Robust error handling with proper logging
|
|
- **Resource Management**: Proper cleanup and resource management
|
|
|
|
### Code Quality
|
|
- **Type Hints**: Full type annotation coverage
|
|
- **Documentation**: Comprehensive docstrings and comments
|
|
- **Testing**: High test coverage with multiple test types
|
|
- **Code Standards**: Consistent formatting and patterns
|
|
|
|
## 🤝 **Contributing**
|
|
|
|
### Development Guidelines
|
|
1. Follow existing code patterns and architecture
|
|
2. Add comprehensive tests for new functionality
|
|
3. Update documentation for API changes
|
|
4. Use type hints and proper error handling
|
|
5. Follow the existing logging patterns
|
|
|
|
### Code Review Process
|
|
1. Create feature branches from main
|
|
2. Write tests before implementing features
|
|
3. Ensure all tests pass and maintain coverage
|
|
4. Update relevant documentation
|
|
5. Submit pull requests with clear descriptions
|
|
|
|
## 📞 **Support**
|
|
|
|
### Getting Help
|
|
1. **Documentation**: Check relevant component documentation
|
|
2. **Logs**: Review system logs in `./logs/` directory
|
|
3. **Status**: Use built-in status and health check methods
|
|
4. **Tests**: Run test suite to verify system integrity
|
|
|
|
### Common Issues
|
|
- **Database Connection**: Check Docker services and environment variables
|
|
- **Collector Failures**: Review collector health status and logs
|
|
- **Performance Issues**: Monitor system resources and optimize accordingly
|
|
|
|
---
|
|
|
|
## 📁 **Documentation File Structure**
|
|
|
|
```
|
|
docs/
|
|
├── README.md # This file - main documentation index
|
|
├── architecture/ # System architecture and design
|
|
│ ├── README.md # Architecture overview
|
|
│ ├── architecture.md # Technical architecture
|
|
│ └── crypto-bot-prd.md # Product requirements
|
|
├── components/ # Core system components
|
|
│ ├── README.md # Component overview
|
|
│ ├── data_collectors.md # Data collection system
|
|
│ └── logging.md # Logging framework
|
|
├── exchanges/ # Exchange integrations
|
|
│ ├── README.md # Exchange overview
|
|
│ └── okx_collector.md # OKX implementation
|
|
├── guides/ # User guides and tutorials
|
|
│ ├── README.md # Guide overview
|
|
│ └── setup.md # Setup instructions
|
|
└── reference/ # Technical reference
|
|
├── README.md # Reference overview
|
|
└── specification.md # Technical specifications
|
|
```
|
|
|
|
## 🔗 **Navigation**
|
|
|
|
- **🏗️ [Architecture & Design](architecture/)** - System design and requirements
|
|
- **🔧 [Core Components](components/)** - Technical implementation details
|
|
- **🌐 [Exchange Integrations](exchanges/)** - Exchange-specific documentation
|
|
- **📖 [Setup & Guides](guides/)** - User guides and tutorials
|
|
- **📋 [Technical Reference](reference/)** - API specifications and schemas
|
|
|
|
---
|
|
|
|
*Last updated: $(date)*
|
|
|
|
For the most current information, refer to the individual component documentation linked above. |