TCPDashboard/docs/guides/README.md

# Guides Documentation

This section contains user guides, tutorials, and setup instructions for the TCP Dashboard platform.

## 📋 Contents

### Setup & Installation

- **[Setup Guide](setup.md)** - *Comprehensive setup instructions for new machines and environments*
  - Environment configuration and prerequisites
  - Database setup with Docker and PostgreSQL
  - Development workflow and best practices
  - Production deployment guidelines
  - Troubleshooting common setup issues

### Quick Start Guides

#### For Developers

```bash
# Quick setup for development
git clone <repository>
cd TCPDashboard
uv sync
cp .env.example .env
docker-compose up -d
uv run python scripts/init_database.py
```

#### For Users

```python
# Quick data collection setup
from data.exchanges import create_okx_collector
from data.base_collector import DataType

collector = create_okx_collector(
    symbol='BTC-USDT',
    data_types=[DataType.TRADE]
)
await collector.start()
```

## 🚀 Tutorial Series

### Getting Started

1. **[Environment Setup](setup.md#environment-setup)** - Setting up your development environment
2. **[First Data Collector](setup.md#first-collector)** - Creating your first data collector
3. **[Database Integration](setup.md#database-setup)** - Connecting to the database
4. **[Adding Monitoring](setup.md#monitoring)** - Setting up logging and monitoring

### Advanced Topics

1. **[Multi-Exchange Setup](setup.md#multi-exchange)** - Collecting from multiple exchanges
2. **[Production Deployment](setup.md#production)** - Deploying to production
3. **[Performance Optimization](setup.md#optimization)** - Optimizing for high throughput
4. **[Custom Integrations](setup.md#custom)** - Building custom data sources

## 🛠️ Development Workflow

### Daily Development

```bash
# Start development environment
docker-compose up -d

# Install new dependencies
uv add package-name

# Run tests
uv run pytest

# Check code quality
uv run black .
uv run isort .
```

### Code Organization

- **`data/`**: Data collection and processing
- **`database/`**: Database models and utilities
- **`utils/`**: Shared utilities and logging
- **`tests/`**: Test suite
- **`docs/`**: Documentation
- **`config/`**: Configuration files

### Best Practices

1. **Follow existing patterns**: Use established code patterns
2. **Write tests first**: TDD approach for new features
3. **Document changes**: Update docs with code changes
4. **Use type hints**: Full type annotation coverage
5. **Handle errors**: Robust error handling throughout

## 🔧 Configuration Management

### Environment Variables

Key environment variables to configure:

```bash
# Database
DATABASE_URL=postgresql://user:pass@localhost:5432/tcp_dashboard

# Logging
LOG_LEVEL=INFO
LOG_CLEANUP=true

# Data Collection
DEFAULT_HEALTH_CHECK_INTERVAL=30
AUTO_RESTART=true
```

### Configuration Files

The platform uses JSON configuration files:

- **`config/okx_config.json`**: OKX exchange settings
- **`config/database_config.json`**: Database configuration
- **`config/logging_config.json`**: Logging settings

### Security Best Practices

- **Never commit secrets**: Use `.env` files for sensitive data
- **Validate inputs**: Comprehensive input validation
- **Use HTTPS**: Secure connections in production
- **Regular updates**: Keep dependencies updated

## 📊 Monitoring & Observability

### Health Monitoring

The platform includes comprehensive health monitoring:

```python
# Check system health
from data.collector_manager import CollectorManager

manager = CollectorManager()
status = manager.get_status()

print(f"Running collectors: {status['statistics']['running_collectors']}")
print(f"Failed collectors: {status['statistics']['failed_collectors']}")
```

### Logging

Structured logging across all components:

```python
from utils.logger import get_logger

logger = get_logger("my_component")
logger.info("Component started", extra={"component": "my_component"})
```

### Performance Metrics

Built-in performance tracking:

- **Message rates**: Real-time data processing rates
- **Error rates**: System health and stability
- **Resource usage**: Memory and CPU utilization
- **Uptime**: Component availability metrics

## 🧪 Testing

### Running Tests

```bash
# Run all tests
uv run pytest

# Run specific test files
uv run pytest tests/test_base_collector.py

# Run with coverage
uv run pytest --cov=data --cov-report=html

# Run integration tests
uv run pytest tests/integration/
```

### Test Organization

- **Unit tests**: Individual component testing
- **Integration tests**: Cross-component functionality
- **Performance tests**: Load and stress testing
- **End-to-end tests**: Full system workflows

### Writing Tests

Follow these patterns when writing tests:

```python
import pytest
import asyncio
from data.exchanges import create_okx_collector

@pytest.mark.asyncio
async def test_okx_collector():
    collector = create_okx_collector('BTC-USDT')
    assert collector is not None
    
    # Test lifecycle
    await collector.start()
    status = collector.get_status()
    assert status['status'] == 'running'
    
    await collector.stop()
```

## 🚀 Deployment

### Development Deployment

For local development:

```bash
# Start services
docker-compose up -d

# Initialize database
uv run python scripts/init_database.py

# Start data collection
uv run python scripts/start_collectors.py
```

### Production Deployment

For production environments:

```bash
# Use production docker-compose
docker-compose -f docker-compose.prod.yml up -d

# Set production environment
export ENV=production
export LOG_LEVEL=INFO

# Start with monitoring
uv run python scripts/production_start.py
```

### Docker Deployment

Using Docker containers:

```dockerfile
FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .
CMD ["python", "-m", "scripts.production_start"]
```

## 🔗 Related Documentation

- **[Modules Documentation (`../modules/`)](../modules/)** - Technical component details
- **[Architecture Overview (`../architecture.md`)]** - System design
- **[Exchange Documentation (`../modules/exchanges/`)](../modules/exchanges/)** - Exchange integrations
- **[Reference (`../reference/`)](../reference/)** - Technical specifications

## 📞 Support & Troubleshooting

### Common Issues

1. **Database Connection Errors**
   - Check Docker services: `docker-compose ps`
   - Verify environment variables in `.env`
   - Test connection: `uv run python scripts/test_db_connection.py`

2. **Collector Failures**
   - Check logs: `tail -f logs/collector_error.log`
   - Verify configuration: Review `config/*.json` files
   - Test manually: `uv run python scripts/test_okx_collector.py`

3. **Performance Issues**
   - Monitor resource usage: `docker stats`
   - Check message rates: Collector status endpoints
   - Optimize configuration: Adjust health check intervals

### Getting Help

1. **Check Documentation**: Review relevant section documentation
2. **Review Logs**: System logs in `./logs/` directory
3. **Test Components**: Use built-in test scripts
4. **Check Status**: Use status and health check methods

### Debug Mode

Enable detailed debugging:

```bash
export LOG_LEVEL=DEBUG
uv run python your_script.py

# Check detailed logs
tail -f logs/*_debug.log
```

---

*For the complete documentation index, see the [main documentation README (`../README.md`)]*