TCPDashboard/docs/setup.md

# Development Environment Setup

This guide will help you set up the Crypto Trading Bot Dashboard development environment.

## Prerequisites

- Python 3.10+
- Docker Desktop (for Windows/Mac) or Docker Engine (for Linux)
- UV package manager
- Git

## Quick Start

### 1. Initial Setup

```bash
# Install dependencies (including dev tools)
uv sync --dev

# Set up environment and start services
python scripts/dev.py setup
```

### 2. Start Services

```bash
# Start PostgreSQL and Redis services
python scripts/dev.py start
```

### 3. Configure API Keys

Copy `env.template` to `.env` and update the OKX API credentials:

```bash
# Copy template (Windows)
copy env.template .env

# Copy template (Unix)
cp env.template .env

# Edit .env file with your actual OKX API credentials
# OKX_API_KEY=your_actual_api_key
# OKX_SECRET_KEY=your_actual_secret_key  
# OKX_PASSPHRASE=your_actual_passphrase
```

### 4. Verify Setup

```bash
# Run setup verification tests
uv run python tests/test_setup.py
```

### 5. Start Dashboard with Hot Reload

```bash
# Start with hot reload (recommended for development)
python scripts/dev.py dev-server

# Or start without hot reload
python scripts/dev.py run
```

## Development Commands

### Using the dev.py script:

```bash
# Show available commands
python scripts/dev.py

# Set up environment and install dependencies
python scripts/dev.py setup

# Start all services (Docker)
python scripts/dev.py start

# Stop all services  
python scripts/dev.py stop

# Restart services
python scripts/dev.py restart

# Check service status
python scripts/dev.py status

# Install/update dependencies
python scripts/dev.py install

# Run development server with hot reload
python scripts/dev.py dev-server

# Run application without hot reload
python scripts/dev.py run
```

### Direct Docker commands:

```bash
# Start services in background
docker-compose up -d

# View service logs
docker-compose logs -f

# Stop services
docker-compose down

# Rebuild and restart
docker-compose up -d --build
```

## Hot Reload Development

The development server includes hot reload functionality that automatically restarts the application when Python files change.

### Features:
- 🔥 **Auto-restart** on file changes
- 👀 **Watches multiple directories** (config, database, components, etc.)
- 🚀 **Fast restart** with debouncing (1-second delay)
- 🛑 **Graceful shutdown** with Ctrl+C

### Usage:
```bash
# Start hot reload server
python scripts/dev.py dev-server

# The server will watch these directories:
# - . (root)
# - config/
# - database/
# - components/
# - data/
# - strategies/
# - trader/
```

## Data Persistence

### Database Persistence
✅ **PostgreSQL data persists** across container restarts
- Volume: `postgres_data` mounted to `/var/lib/postgresql/data`
- Data survives `docker-compose down` and `docker-compose up`

### Redis Persistence
✅ **Redis data persists** with AOF (Append-Only File)
- Volume: `redis_data` mounted to `/data`
- AOF sync every second for durability
- Data survives container restarts

### Removing Persistent Data
```bash
# Stop services and remove volumes (CAUTION: This deletes all data)
docker-compose down -v

# Or remove specific volumes
docker volume rm dashboard_postgres_data
docker volume rm dashboard_redis_data
```

## Dependency Management

### Adding New Dependencies

```bash
# Add runtime dependency
uv add "package-name>=1.0.0"

# Add development dependency
uv add --dev "dev-package>=1.0.0"

# Install all dependencies
uv sync --dev
```

### Key Dependencies Included:
- **Web Framework**: Dash, Plotly
- **Database**: SQLAlchemy, psycopg2-binary, Alembic
- **Data Processing**: pandas, numpy
- **Configuration**: pydantic, python-dotenv
- **Development**: watchdog (hot reload), pytest, black, mypy

See `docs/dependency-management.md` for detailed dependency management guide.

## Directory Structure

```
Dashboard/
├── config/                 # Configuration files
│   ├── settings.py         # Application settings
│   └── bot_configs/        # Bot configuration files
├── database/               # Database related files
│   └── init/               # Database initialization scripts
├── scripts/                # Development scripts
│   ├── dev.py              # Main development script
│   ├── setup.sh            # Setup script (Unix)
│   ├── start.sh            # Start script (Unix)
│   └── stop.sh             # Stop script (Unix)
├── tests/                  # Test files
│   └── test_setup.py       # Setup verification tests
├── docs/                   # Documentation
│   ├── setup.md           # This file
│   └── dependency-management.md  # Dependency guide
├── docker-compose.yml      # Docker services configuration
├── env.template            # Environment variables template
├── pyproject.toml          # Dependencies and project config
└── main.py                 # Main application entry point
```

## Services

### PostgreSQL Database
- **Host**: localhost:5432
- **Database**: dashboard
- **User**: dashboard  
- **Password**: dashboard123 (development only)
- **Persistence**: ✅ Data persists across restarts

### Redis Cache
- **Host**: localhost:6379
- **No password** (development only)
- **Persistence**: ✅ AOF enabled, data persists across restarts

## Environment Variables

Key environment variables (see `env.template` for full list):

- `DATABASE_URL` - PostgreSQL connection string
- `OKX_API_KEY` - OKX API key
- `OKX_SECRET_KEY` - OKX secret key
- `OKX_PASSPHRASE` - OKX passphrase
- `OKX_SANDBOX` - Use OKX sandbox (true/false)
- `DEBUG` - Enable debug mode
- `LOG_LEVEL` - Logging level (DEBUG, INFO, WARNING, ERROR)

## Troubleshooting

### Docker Issues

1. **Docker not running**: Start Docker Desktop/Engine
2. **Port conflicts**: Check if ports 5432 or 6379 are already in use
3. **Permission issues**: On Linux, add your user to the docker group
4. **Data persistence issues**: Check if volumes are properly mounted

### Database Connection Issues

1. **Connection refused**: Ensure PostgreSQL container is running
2. **Authentication failed**: Check credentials in `.env` file
3. **Database doesn't exist**: Run the setup script again
4. **Data loss**: Check if volume is mounted correctly

### Dependency Issues

1. **Import errors**: Run `uv sync --dev` to install dependencies
2. **Version conflicts**: Check `pyproject.toml` for compatibility
3. **Hot reload not working**: Ensure `watchdog` is installed

### Hot Reload Issues

1. **Changes not detected**: Check if files are in watched directories
2. **Rapid restarts**: Built-in 1-second debouncing should prevent this
3. **Process not stopping**: Use Ctrl+C to gracefully shutdown

## Performance Tips

1. **Use SSD**: Store Docker volumes on SSD for better database performance
2. **Increase Docker memory**: Allocate more RAM to Docker Desktop
3. **Hot reload**: Use `dev-server` for faster development cycles
4. **Dependency caching**: UV caches dependencies for faster installs

## Next Steps

After successful setup:

1. **Phase 1.0**: Database Infrastructure Setup
2. **Phase 2.0**: Bot Management System Development  
3. **Phase 3.0**: OKX Integration and Data Pipeline
4. **Phase 4.0**: Dashboard UI and Visualization
5. **Phase 5.0**: Backtesting System Implementation

See `tasks/tasks-prd-crypto-bot-dashboard.md` for detailed task list.