282 lines
7.3 KiB
Markdown
282 lines
7.3 KiB
Markdown
# Development Environment Setup
|
|
|
|
This guide will help you set up the Crypto Trading Bot Dashboard development environment.
|
|
|
|
## Prerequisites
|
|
|
|
- Python 3.10+
|
|
- Docker Desktop (for Windows/Mac) or Docker Engine (for Linux)
|
|
- UV package manager
|
|
- Git
|
|
|
|
## Quick Start
|
|
|
|
### 1. Initial Setup
|
|
|
|
```bash
|
|
# Install dependencies (including dev tools)
|
|
uv sync --dev
|
|
|
|
# Set up environment and start services
|
|
python scripts/dev.py setup
|
|
```
|
|
|
|
### 2. Start Services
|
|
|
|
```bash
|
|
# Start PostgreSQL and Redis services
|
|
python scripts/dev.py start
|
|
```
|
|
|
|
### 3. Configure API Keys
|
|
|
|
Copy `env.template` to `.env` and update the OKX API credentials:
|
|
|
|
```bash
|
|
# Copy template (Windows)
|
|
copy env.template .env
|
|
|
|
# Copy template (Unix)
|
|
cp env.template .env
|
|
|
|
# Edit .env file with your actual OKX API credentials
|
|
# OKX_API_KEY=your_actual_api_key
|
|
# OKX_SECRET_KEY=your_actual_secret_key
|
|
# OKX_PASSPHRASE=your_actual_passphrase
|
|
```
|
|
|
|
### 4. Verify Setup
|
|
|
|
```bash
|
|
# Run setup verification tests
|
|
uv run python tests/test_setup.py
|
|
```
|
|
|
|
### 5. Start Dashboard with Hot Reload
|
|
|
|
```bash
|
|
# Start with hot reload (recommended for development)
|
|
python scripts/dev.py dev-server
|
|
|
|
# Or start without hot reload
|
|
python scripts/dev.py run
|
|
```
|
|
|
|
## Development Commands
|
|
|
|
### Using the dev.py script:
|
|
|
|
```bash
|
|
# Show available commands
|
|
python scripts/dev.py
|
|
|
|
# Set up environment and install dependencies
|
|
python scripts/dev.py setup
|
|
|
|
# Start all services (Docker)
|
|
python scripts/dev.py start
|
|
|
|
# Stop all services
|
|
python scripts/dev.py stop
|
|
|
|
# Restart services
|
|
python scripts/dev.py restart
|
|
|
|
# Check service status
|
|
python scripts/dev.py status
|
|
|
|
# Install/update dependencies
|
|
python scripts/dev.py install
|
|
|
|
# Run development server with hot reload
|
|
python scripts/dev.py dev-server
|
|
|
|
# Run application without hot reload
|
|
python scripts/dev.py run
|
|
```
|
|
|
|
### Direct Docker commands:
|
|
|
|
```bash
|
|
# Start services in background
|
|
docker-compose up -d
|
|
|
|
# View service logs
|
|
docker-compose logs -f
|
|
|
|
# Stop services
|
|
docker-compose down
|
|
|
|
# Rebuild and restart
|
|
docker-compose up -d --build
|
|
```
|
|
|
|
## Hot Reload Development
|
|
|
|
The development server includes hot reload functionality that automatically restarts the application when Python files change.
|
|
|
|
### Features:
|
|
- 🔥 **Auto-restart** on file changes
|
|
- 👀 **Watches multiple directories** (config, database, components, etc.)
|
|
- 🚀 **Fast restart** with debouncing (1-second delay)
|
|
- 🛑 **Graceful shutdown** with Ctrl+C
|
|
|
|
### Usage:
|
|
```bash
|
|
# Start hot reload server
|
|
python scripts/dev.py dev-server
|
|
|
|
# The server will watch these directories:
|
|
# - . (root)
|
|
# - config/
|
|
# - database/
|
|
# - components/
|
|
# - data/
|
|
# - strategies/
|
|
# - trader/
|
|
```
|
|
|
|
## Data Persistence
|
|
|
|
### Database Persistence
|
|
✅ **PostgreSQL data persists** across container restarts
|
|
- Volume: `postgres_data` mounted to `/var/lib/postgresql/data`
|
|
- Data survives `docker-compose down` and `docker-compose up`
|
|
|
|
### Redis Persistence
|
|
✅ **Redis data persists** with AOF (Append-Only File)
|
|
- Volume: `redis_data` mounted to `/data`
|
|
- AOF sync every second for durability
|
|
- Data survives container restarts
|
|
|
|
### Removing Persistent Data
|
|
```bash
|
|
# Stop services and remove volumes (CAUTION: This deletes all data)
|
|
docker-compose down -v
|
|
|
|
# Or remove specific volumes
|
|
docker volume rm dashboard_postgres_data
|
|
docker volume rm dashboard_redis_data
|
|
```
|
|
|
|
## Dependency Management
|
|
|
|
### Adding New Dependencies
|
|
|
|
```bash
|
|
# Add runtime dependency
|
|
uv add "package-name>=1.0.0"
|
|
|
|
# Add development dependency
|
|
uv add --dev "dev-package>=1.0.0"
|
|
|
|
# Install all dependencies
|
|
uv sync --dev
|
|
```
|
|
|
|
### Key Dependencies Included:
|
|
- **Web Framework**: Dash, Plotly
|
|
- **Database**: SQLAlchemy, psycopg2-binary, Alembic
|
|
- **Data Processing**: pandas, numpy
|
|
- **Configuration**: pydantic, python-dotenv
|
|
- **Development**: watchdog (hot reload), pytest, black, mypy
|
|
|
|
See `docs/dependency-management.md` for detailed dependency management guide.
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
Dashboard/
|
|
├── config/ # Configuration files
|
|
│ ├── settings.py # Application settings
|
|
│ └── bot_configs/ # Bot configuration files
|
|
├── database/ # Database related files
|
|
│ └── init/ # Database initialization scripts
|
|
├── scripts/ # Development scripts
|
|
│ ├── dev.py # Main development script
|
|
│ ├── setup.sh # Setup script (Unix)
|
|
│ ├── start.sh # Start script (Unix)
|
|
│ └── stop.sh # Stop script (Unix)
|
|
├── tests/ # Test files
|
|
│ └── test_setup.py # Setup verification tests
|
|
├── docs/ # Documentation
|
|
│ ├── setup.md # This file
|
|
│ └── dependency-management.md # Dependency guide
|
|
├── docker-compose.yml # Docker services configuration
|
|
├── env.template # Environment variables template
|
|
├── pyproject.toml # Dependencies and project config
|
|
└── main.py # Main application entry point
|
|
```
|
|
|
|
## Services
|
|
|
|
### PostgreSQL Database
|
|
- **Host**: localhost:5432
|
|
- **Database**: dashboard
|
|
- **User**: dashboard
|
|
- **Password**: dashboard123 (development only)
|
|
- **Persistence**: ✅ Data persists across restarts
|
|
|
|
### Redis Cache
|
|
- **Host**: localhost:6379
|
|
- **No password** (development only)
|
|
- **Persistence**: ✅ AOF enabled, data persists across restarts
|
|
|
|
## Environment Variables
|
|
|
|
Key environment variables (see `env.template` for full list):
|
|
|
|
- `DATABASE_URL` - PostgreSQL connection string
|
|
- `OKX_API_KEY` - OKX API key
|
|
- `OKX_SECRET_KEY` - OKX secret key
|
|
- `OKX_PASSPHRASE` - OKX passphrase
|
|
- `OKX_SANDBOX` - Use OKX sandbox (true/false)
|
|
- `DEBUG` - Enable debug mode
|
|
- `LOG_LEVEL` - Logging level (DEBUG, INFO, WARNING, ERROR)
|
|
|
|
## Troubleshooting
|
|
|
|
### Docker Issues
|
|
|
|
1. **Docker not running**: Start Docker Desktop/Engine
|
|
2. **Port conflicts**: Check if ports 5432 or 6379 are already in use
|
|
3. **Permission issues**: On Linux, add your user to the docker group
|
|
4. **Data persistence issues**: Check if volumes are properly mounted
|
|
|
|
### Database Connection Issues
|
|
|
|
1. **Connection refused**: Ensure PostgreSQL container is running
|
|
2. **Authentication failed**: Check credentials in `.env` file
|
|
3. **Database doesn't exist**: Run the setup script again
|
|
4. **Data loss**: Check if volume is mounted correctly
|
|
|
|
### Dependency Issues
|
|
|
|
1. **Import errors**: Run `uv sync --dev` to install dependencies
|
|
2. **Version conflicts**: Check `pyproject.toml` for compatibility
|
|
3. **Hot reload not working**: Ensure `watchdog` is installed
|
|
|
|
### Hot Reload Issues
|
|
|
|
1. **Changes not detected**: Check if files are in watched directories
|
|
2. **Rapid restarts**: Built-in 1-second debouncing should prevent this
|
|
3. **Process not stopping**: Use Ctrl+C to gracefully shutdown
|
|
|
|
## Performance Tips
|
|
|
|
1. **Use SSD**: Store Docker volumes on SSD for better database performance
|
|
2. **Increase Docker memory**: Allocate more RAM to Docker Desktop
|
|
3. **Hot reload**: Use `dev-server` for faster development cycles
|
|
4. **Dependency caching**: UV caches dependencies for faster installs
|
|
|
|
## Next Steps
|
|
|
|
After successful setup:
|
|
|
|
1. **Phase 1.0**: Database Infrastructure Setup
|
|
2. **Phase 2.0**: Bot Management System Development
|
|
3. **Phase 3.0**: OKX Integration and Data Pipeline
|
|
4. **Phase 4.0**: Dashboard UI and Visualization
|
|
5. **Phase 5.0**: Backtesting System Implementation
|
|
|
|
See `tasks/tasks-prd-crypto-bot-dashboard.md` for detailed task list. |