TCPDashboard/docs/guides/README.md

Guides Documentation

This section contains user guides, tutorials, and setup instructions for the TCP Dashboard platform.

📋 Contents

Setup & Installation

• Setup Guide - Comprehensive setup instructions for new machines and environments
  • Environment configuration and prerequisites
  • Database setup with Docker and PostgreSQL
  • Development workflow and best practices
  • Production deployment guidelines
  • Troubleshooting common setup issues

Quick Start Guides

For Developers

# Quick setup for development
git clone <repository>
cd TCPDashboard
uv sync
cp .env.example .env
docker-compose up -d
uv run python scripts/init_database.py

For Users

# Quick data collection setup
from data.exchanges import create_okx_collector
from data.base_collector import DataType

collector = create_okx_collector(
    symbol='BTC-USDT',
    data_types=[DataType.TRADE]
)
await collector.start()

🚀 Tutorial Series

Getting Started

1. Environment Setup - Setting up your development environment
2. First Data Collector - Creating your first data collector
3. Database Integration - Connecting to the database
4. Adding Monitoring - Setting up logging and monitoring

Advanced Topics

1. Multi-Exchange Setup - Collecting from multiple exchanges
2. Production Deployment - Deploying to production
3. Performance Optimization - Optimizing for high throughput
4. Custom Integrations - Building custom data sources

🛠️ Development Workflow

Daily Development

# Start development environment
docker-compose up -d

# Install new dependencies
uv add package-name

# Run tests
uv run pytest

# Check code quality
uv run black .
uv run isort .

Code Organization

• data/: Data collection and processing
• database/: Database models and utilities
• utils/: Shared utilities and logging
• tests/: Test suite
• docs/: Documentation
• config/: Configuration files

Best Practices

1. Follow existing patterns: Use established code patterns
2. Write tests first: TDD approach for new features
3. Document changes: Update docs with code changes
4. Use type hints: Full type annotation coverage
5. Handle errors: Robust error handling throughout

🔧 Configuration Management

Environment Variables

Key environment variables to configure:

# Database
DATABASE_URL=postgresql://user:pass@localhost:5432/tcp_dashboard

# Logging
LOG_LEVEL=INFO
LOG_CLEANUP=true

# Data Collection
DEFAULT_HEALTH_CHECK_INTERVAL=30
AUTO_RESTART=true

Configuration Files

The platform uses JSON configuration files:

• config/okx_config.json: OKX exchange settings
• config/database_config.json: Database configuration
• config/logging_config.json: Logging settings

Security Best Practices

• Never commit secrets: Use .env files for sensitive data
• Validate inputs: Comprehensive input validation
• Use HTTPS: Secure connections in production
• Regular updates: Keep dependencies updated

📊 Monitoring & Observability

Health Monitoring

The platform includes comprehensive health monitoring:

# Check system health
from data.collector_manager import CollectorManager

manager = CollectorManager()
status = manager.get_status()

print(f"Running collectors: {status['statistics']['running_collectors']}")
print(f"Failed collectors: {status['statistics']['failed_collectors']}")

Logging

Structured logging across all components:

from utils.logger import get_logger

logger = get_logger("my_component")
logger.info("Component started", extra={"component": "my_component"})

Performance Metrics

Built-in performance tracking:

• Message rates: Real-time data processing rates
• Error rates: System health and stability
• Resource usage: Memory and CPU utilization
• Uptime: Component availability metrics

🧪 Testing

Running Tests

# Run all tests
uv run pytest

# Run specific test files
uv run pytest tests/test_base_collector.py

# Run with coverage
uv run pytest --cov=data --cov-report=html

# Run integration tests
uv run pytest tests/integration/

Test Organization

• Unit tests: Individual component testing
• Integration tests: Cross-component functionality
• Performance tests: Load and stress testing
• End-to-end tests: Full system workflows

Writing Tests

Follow these patterns when writing tests:

import pytest
import asyncio
from data.exchanges import create_okx_collector

@pytest.mark.asyncio
async def test_okx_collector():
    collector = create_okx_collector('BTC-USDT')
    assert collector is not None
    
    # Test lifecycle
    await collector.start()
    status = collector.get_status()
    assert status['status'] == 'running'
    
    await collector.stop()

🚀 Deployment

Development Deployment

For local development:

# Start services
docker-compose up -d

# Initialize database
uv run python scripts/init_database.py

# Start data collection
uv run python scripts/start_collectors.py

Production Deployment

For production environments:

# Use production docker-compose
docker-compose -f docker-compose.prod.yml up -d

# Set production environment
export ENV=production
export LOG_LEVEL=INFO

# Start with monitoring
uv run python scripts/production_start.py

Docker Deployment

Using Docker containers:

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .
CMD ["python", "-m", "scripts.production_start"]

🔗 Related Documentation

• Modules Documentation (../modules/) - Technical component details
• [Architecture Overview (../architecture.md)] - System design
• Exchange Documentation (../modules/exchanges/) - Exchange integrations
• Reference (../reference/) - Technical specifications

📞 Support & Troubleshooting

Common Issues

1. Database Connection Errors

   • Check Docker services: docker-compose ps
   • Verify environment variables in .env
   • Test connection: uv run python scripts/test_db_connection.py

2. Collector Failures

   • Check logs: tail -f logs/collector_error.log
   • Verify configuration: Review config/*.json files
   • Test manually: uv run python scripts/test_okx_collector.py

3. Performance Issues

   • Monitor resource usage: docker stats
   • Check message rates: Collector status endpoints
   • Optimize configuration: Adjust health check intervals

Getting Help

1. Check Documentation: Review relevant section documentation
2. Review Logs: System logs in ./logs/ directory
3. Test Components: Use built-in test scripts
4. Check Status: Use status and health check methods

Debug Mode

Enable detailed debugging:

export LOG_LEVEL=DEBUG
uv run python your_script.py

# Check detailed logs
tail -f logs/*_debug.log

---

For the complete documentation index, see the [main documentation README (../README.md)]