TCPDashboard/docs/README.md

# TCP Dashboard Documentation

Welcome to the **TCP Dashboard** (Trading Crypto Platform) documentation. This platform provides a comprehensive solution for cryptocurrency trading bot development, backtesting, and portfolio management.

## 📚 Documentation Index

### 🏗️ **Architecture & Design**

- **[Architecture Overview](architecture.md)** - High-level system architecture and component design
- **[Project Specification](specification.md)** - Technical specifications and requirements
- **[Crypto Bot PRD](crypto-bot-prd.md)** - Product Requirements Document for the crypto trading bot platform

### 🚀 **Setup & Installation**

- **[Setup Guide](setup.md)** - Comprehensive setup instructions for new machines and environments
  - Environment configuration
  - Database setup with Docker
  - Development workflow
  - Production deployment

### 🔧 **Core Systems**

#### Data Collection System

- **[Data Collectors Documentation](data_collectors.md)** - *Comprehensive guide to the enhanced data collector system*
  - **BaseDataCollector** abstract class with health monitoring
  - **CollectorManager** for centralized management
  - **Exchange Factory Pattern** for standardized collector creation
  - **Modular Exchange Architecture** for scalable implementation
  - Auto-restart and failure recovery
  - Health monitoring and alerting
  - Performance optimization
  - Integration examples
  - Troubleshooting guide

- **[OKX Collector Documentation](okx_collector.md)** - *Complete guide to OKX exchange integration*
  - Real-time trades, orderbook, and ticker data collection
  - WebSocket connection management with OKX-specific ping/pong
  - Factory pattern usage and configuration
  - Data processing and validation
  - Monitoring and troubleshooting
  - Production deployment guide

#### Logging System

- **[Enhanced Logging System](logging.md)** - Unified logging framework
  - Multi-level logging with automatic cleanup
  - Console and file output with formatting
  - Performance monitoring
  - Integration across all components

## 🎯 **Quick Start**

1. **New to the platform?** Start with the [Setup Guide](setup.md)
2. **Implementing data collectors?** See [Data Collectors Documentation](data_collectors.md)
3. **Understanding the architecture?** Read [Architecture Overview](architecture.md)
4. **Troubleshooting?** Check component-specific documentation

## 🏛️ **System Components**

### Core Infrastructure
- **Database Layer**: PostgreSQL with SQLAlchemy models
- **Real-time Messaging**: Redis pub/sub for data distribution
- **Configuration Management**: Pydantic-based settings
- **Containerization**: Docker and docker-compose setup

### Data Collection & Processing
- **Abstract Base Collectors**: Standardized interface for all exchange connectors
- **Exchange Factory Pattern**: Unified collector creation across exchanges
- **Modular Exchange Architecture**: Organized exchange implementations in dedicated folders
- **Health Monitoring**: Automatic failure detection and recovery
- **Data Validation**: Comprehensive validation for market data
- **Multi-Exchange Support**: OKX (production-ready), Binance and other exchanges (planned)

### Trading & Strategy Engine
- **Strategy Framework**: Base strategy classes and implementations
- **Bot Management**: Lifecycle management with JSON configuration
- **Backtesting Engine**: Historical strategy testing with performance metrics
- **Portfolio Management**: Virtual trading with P&L tracking

### User Interface
- **Dashboard**: Dash-based web interface with Mantine UI
- **Real-time Charts**: Interactive price charts with technical indicators
- **Bot Controls**: Start/stop/configure trading bots
- **Performance Analytics**: Portfolio visualization and trade analytics

## 📋 **Task Progress**

The platform follows a structured development approach with clearly defined tasks:

- ✅ **Database Foundation** - Complete
- ✅ **Enhanced Data Collectors** - Complete with health monitoring
- ✅ **OKX Data Collector** - Complete with factory pattern and production testing
- ⏳ **Multi-Exchange Support** - In progress (Binance connector next)
- ⏳ **Basic Dashboard** - Planned
- ⏳ **Strategy Engine** - Planned
- ⏳ **Advanced Features** - Planned

For detailed task tracking, see [tasks/tasks-crypto-bot-prd.md](../tasks/tasks-crypto-bot-prd.md).

## 🛠️ **Development Workflow**

### Setting Up Development Environment

```bash
# Clone and setup
git clone <repository>
cd TCPDashboard

# Install dependencies with UV
uv sync

# Setup environment
cp .env.example .env
# Edit .env with your configuration

# Start services
docker-compose up -d

# Initialize database
uv run python scripts/init_database.py

# Run tests
uv run pytest
```

### Key Development Tools

- **UV**: Modern Python package management
- **pytest**: Testing framework with async support
- **SQLAlchemy**: Database ORM with migration support
- **Dash + Mantine**: Modern web UI framework
- **Docker**: Containerized development environment

## 🔍 **Testing**

The platform includes comprehensive test coverage:

- **Unit Tests**: Individual component testing
- **Integration Tests**: Cross-component functionality
- **Performance Tests**: Load and stress testing
- **End-to-End Tests**: Full system workflows

```bash
# Run all tests
uv run pytest

# Run specific test files
uv run pytest tests/test_base_collector.py
uv run pytest tests/test_collector_manager.py

# Run with coverage
uv run pytest --cov=data --cov-report=html
```

## 📊 **Monitoring & Observability**

### Logging
- **Structured Logging**: JSON-formatted logs with automatic cleanup
- **Multiple Levels**: Debug, Info, Warning, Error with configurable output
- **Component Isolation**: Separate loggers for different system components

### Health Monitoring
- **Collector Health**: Real-time status and performance metrics
- **Auto-Recovery**: Automatic restart on failures
- **Performance Tracking**: Message rates, uptime, error rates

### Metrics Integration
- **Prometheus Support**: Built-in metrics collection
- **Custom Dashboards**: System performance visualization
- **Alerting**: Configurable alerts for system health

## 🔐 **Security & Best Practices**

### Configuration Management
- **Environment Variables**: All sensitive data via `.env` files
- **No Hardcoded Secrets**: Clean separation of configuration and code
- **Validation**: Pydantic-based configuration validation

### Data Handling
- **Input Validation**: Comprehensive validation for all external data
- **Error Handling**: Robust error handling with proper logging
- **Resource Management**: Proper cleanup and resource management

### Code Quality
- **Type Hints**: Full type annotation coverage
- **Documentation**: Comprehensive docstrings and comments
- **Testing**: High test coverage with multiple test types
- **Code Standards**: Consistent formatting and patterns

## 🤝 **Contributing**

### Development Guidelines
1. Follow existing code patterns and architecture
2. Add comprehensive tests for new functionality
3. Update documentation for API changes
4. Use type hints and proper error handling
5. Follow the existing logging patterns

### Code Review Process
1. Create feature branches from main
2. Write tests before implementing features
3. Ensure all tests pass and maintain coverage
4. Update relevant documentation
5. Submit pull requests with clear descriptions

## 📞 **Support**

### Getting Help
1. **Documentation**: Check relevant component documentation
2. **Logs**: Review system logs in `./logs/` directory  
3. **Status**: Use built-in status and health check methods
4. **Tests**: Run test suite to verify system integrity

### Common Issues
- **Database Connection**: Check Docker services and environment variables
- **Collector Failures**: Review collector health status and logs
- **Performance Issues**: Monitor system resources and optimize accordingly

---

## 📁 **File Structure**

```
TCPDashboard/
├── docs/                    # Documentation (you are here)
├── data/                    # Data collection system
├── database/                # Database models and utilities  
├── utils/                   # Shared utilities (logging, etc.)
├── tests/                   # Test suite
├── examples/                # Usage examples
├── config/                  # Configuration files
├── logs/                    # Application logs
└── scripts/                 # Utility scripts
```

---

*Last updated: $(date)*

For the most current information, refer to the individual component documentation linked above.