init setup
This commit is contained in:
parent
3e4a965895
commit
692611d3ae
2
.gitignore
vendored
2
.gitignore
vendored
@ -1 +1,3 @@
|
||||
*.pyc
|
||||
.env
|
||||
.env.local
|
||||
|
||||
@ -29,6 +29,10 @@ This platform enables rapid strategy testing within 1-2 weeks of development. Bu
|
||||
- Python 3.10+, Docker, UV package manager
|
||||
|
||||
### Setup
|
||||
|
||||
**📖 For detailed setup instructions, see [docs/setup.md](docs/setup.md)**
|
||||
|
||||
Quick setup:
|
||||
```bash
|
||||
python scripts/dev.py setup # Setup environment
|
||||
python scripts/dev.py start # Start services
|
||||
@ -50,6 +54,7 @@ python scripts/dev.py dev-server # Start with hot reload
|
||||
|
||||
## Documentation
|
||||
|
||||
- **[Setup Guide](docs/setup.md)** - Complete setup instructions for new machines
|
||||
- **[Product Requirements](docs/crypto-bot-prd.md)** - Complete system specifications and requirements
|
||||
- **[Technical Architecture](docs/architecture.md)** - Implementation details and component design
|
||||
- **[Platform Overview](docs/specification.md)** - Human-readable system overview
|
||||
|
||||
@ -5,10 +5,10 @@ services:
|
||||
environment:
|
||||
POSTGRES_DB: ${POSTGRES_DB:-dashboard}
|
||||
POSTGRES_USER: ${POSTGRES_USER:-dashboard}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-dashboard123}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-sdkjfh534^jh}
|
||||
POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C"
|
||||
ports:
|
||||
- "${POSTGRES_PORT:-5432}:5432"
|
||||
- "${POSTGRES_PORT:-5434}:5432"
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
- ./database/init:/docker-entrypoint-initdb.d
|
||||
@ -24,14 +24,14 @@ services:
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
container_name: dashboard_redis
|
||||
command: redis-server --appendonly yes --appendfsync everysec
|
||||
command: redis-server --appendonly yes --appendfsync everysec --requirepass ${REDIS_PASSWORD:-redis987secure}
|
||||
ports:
|
||||
- "${REDIS_PORT:-6379}:6379"
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis987secure}", "ping"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
398
docs/setup.md
Normal file
398
docs/setup.md
Normal file
@ -0,0 +1,398 @@
|
||||
# Crypto Trading Bot Dashboard - Setup Guide
|
||||
|
||||
This guide will help you set up the Crypto Trading Bot Dashboard on a new machine from scratch.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Required Software
|
||||
|
||||
1. **Python 3.12+**
|
||||
- Download from [python.org](https://python.org)
|
||||
- Ensure Python is added to PATH
|
||||
|
||||
2. **UV Package Manager**
|
||||
```powershell
|
||||
# Windows (PowerShell)
|
||||
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
|
||||
|
||||
# macOS/Linux
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
```
|
||||
|
||||
3. **Docker Desktop**
|
||||
- Download from [docker.com](https://docker.com)
|
||||
- Ensure Docker is running before proceeding
|
||||
|
||||
4. **Git**
|
||||
- Download from [git-scm.com](https://git-scm.com)
|
||||
|
||||
### System Requirements
|
||||
|
||||
- **RAM**: Minimum 4GB, Recommended 8GB+
|
||||
- **Storage**: At least 2GB free space
|
||||
- **OS**: Windows 10/11, macOS 10.15+, or Linux
|
||||
|
||||
## Project Setup
|
||||
|
||||
### 1. Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
cd TCPDashboard
|
||||
```
|
||||
|
||||
### 2. Environment Configuration
|
||||
|
||||
Create the environment file from the template:
|
||||
|
||||
```powershell
|
||||
# Windows
|
||||
Copy-Item env.template .env
|
||||
|
||||
# macOS/Linux
|
||||
cp env.template .env
|
||||
```
|
||||
|
||||
**Important**: The `.env` file contains pre-configured secure passwords. **Do not commit this file to version control.**
|
||||
|
||||
### 3. Configure Custom Ports (Optional)
|
||||
|
||||
If you have other PostgreSQL instances running, the default configuration uses port `5434` to avoid conflicts.
|
||||
|
||||
Current configuration in `.env`:
|
||||
```env
|
||||
POSTGRES_PORT=5434
|
||||
POSTGRES_PASSWORD=sdkjfh534^jh
|
||||
REDIS_PASSWORD=redis987secure
|
||||
```
|
||||
|
||||
## Database Setup
|
||||
|
||||
### 1. Start Database Services
|
||||
|
||||
Start PostgreSQL and Redis using Docker Compose:
|
||||
|
||||
```powershell
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
This will:
|
||||
- Create a PostgreSQL database on port `5434`
|
||||
- Create a Redis instance on port `6379`
|
||||
- Set up persistent volumes for data storage
|
||||
- Configure password authentication
|
||||
|
||||
### 2. Verify Services are Running
|
||||
|
||||
Check container status:
|
||||
```powershell
|
||||
docker-compose ps
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||
dashboard_postgres postgres:15-alpine "docker-entrypoint.s…" postgres X minutes ago Up X minutes (healthy) 0.0.0.0:5434->5432/tcp
|
||||
dashboard_redis redis:7-alpine "docker-entrypoint.s…" redis X minutes ago Up X minutes (healthy) 0.0.0.0:6379->6379/tcp
|
||||
```
|
||||
|
||||
### 3. Test Database Connections
|
||||
|
||||
Test PostgreSQL connection:
|
||||
```powershell
|
||||
# Test port accessibility
|
||||
Test-NetConnection -ComputerName localhost -Port 5434
|
||||
|
||||
# Test database connection (from inside container)
|
||||
docker exec dashboard_postgres psql -h localhost -p 5432 -U dashboard -d dashboard -c "SELECT version();"
|
||||
```
|
||||
|
||||
Test Redis connection:
|
||||
```powershell
|
||||
docker exec dashboard_redis redis-cli -a redis987secure ping
|
||||
```
|
||||
|
||||
Expected output: `PONG`
|
||||
|
||||
## Application Setup
|
||||
|
||||
### 1. Install Python Dependencies
|
||||
|
||||
```powershell
|
||||
uv sync
|
||||
```
|
||||
|
||||
This will:
|
||||
- Create a virtual environment in `.venv/`
|
||||
- Install all required dependencies
|
||||
- Set up the project for development
|
||||
|
||||
### 2. Activate Virtual Environment
|
||||
|
||||
```powershell
|
||||
# Windows
|
||||
uv run <command>
|
||||
|
||||
# Or activate manually
|
||||
.venv\Scripts\Activate.ps1
|
||||
|
||||
# macOS/Linux
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
### 3. Initialize Database Schema
|
||||
|
||||
```powershell
|
||||
# Run database migrations (when implemented)
|
||||
uv run python scripts/init_db.py
|
||||
```
|
||||
|
||||
## Running the Application
|
||||
|
||||
### 1. Start the Dashboard
|
||||
|
||||
```powershell
|
||||
uv run python main.py
|
||||
```
|
||||
|
||||
### 2. Access the Application
|
||||
|
||||
Open your browser and navigate to:
|
||||
- **Local**: http://localhost:8050
|
||||
- **Network**: http://0.0.0.0:8050 (if accessible from other machines)
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Key configuration options in `.env`:
|
||||
|
||||
```env
|
||||
# Database Configuration
|
||||
POSTGRES_HOST=localhost
|
||||
POSTGRES_PORT=5434
|
||||
POSTGRES_DB=dashboard
|
||||
POSTGRES_USER=dashboard
|
||||
POSTGRES_PASSWORD=sdkjfh534^jh
|
||||
|
||||
# Redis Configuration
|
||||
REDIS_HOST=localhost
|
||||
REDIS_PORT=6379
|
||||
REDIS_PASSWORD=redis987secure
|
||||
|
||||
# Application Configuration
|
||||
DASH_HOST=0.0.0.0
|
||||
DASH_PORT=8050
|
||||
DEBUG=true
|
||||
|
||||
# OKX API Configuration (for real trading)
|
||||
OKX_API_KEY=your_okx_api_key_here
|
||||
OKX_SECRET_KEY=your_okx_secret_key_here
|
||||
OKX_PASSPHRASE=your_okx_passphrase_here
|
||||
OKX_SANDBOX=true
|
||||
```
|
||||
|
||||
### Port Configuration
|
||||
|
||||
If you need to change ports due to conflicts:
|
||||
|
||||
1. **PostgreSQL Port**: Update `POSTGRES_PORT` in `.env` and the port mapping in `docker-compose.yml`
|
||||
2. **Redis Port**: Update `REDIS_PORT` in `.env` and `docker-compose.yml`
|
||||
3. **Dashboard Port**: Update `DASH_PORT` in `.env`
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### 1. Daily Development Setup
|
||||
|
||||
```powershell
|
||||
# Start databases
|
||||
docker-compose up -d
|
||||
|
||||
# Start development server
|
||||
uv run python main.py
|
||||
```
|
||||
|
||||
### 2. Stop Services
|
||||
|
||||
```powershell
|
||||
# Stop application: Ctrl+C in terminal
|
||||
|
||||
# Stop databases
|
||||
docker-compose down
|
||||
```
|
||||
|
||||
### 3. Reset Database (if needed)
|
||||
|
||||
```powershell
|
||||
# WARNING: This will delete all data
|
||||
docker-compose down -v
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Run Unit Tests
|
||||
|
||||
```powershell
|
||||
# Run all tests
|
||||
uv run pytest
|
||||
|
||||
# Run specific test file
|
||||
uv run pytest tests/test_database.py
|
||||
|
||||
# Run with coverage
|
||||
uv run pytest --cov=. --cov-report=html
|
||||
```
|
||||
|
||||
### Test Database Connection
|
||||
|
||||
Create a quick test script:
|
||||
|
||||
```python
|
||||
# test_connection.py
|
||||
import os
|
||||
import psycopg2
|
||||
import redis
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# Test PostgreSQL
|
||||
try:
|
||||
conn = psycopg2.connect(
|
||||
host=os.getenv('POSTGRES_HOST'),
|
||||
port=os.getenv('POSTGRES_PORT'),
|
||||
database=os.getenv('POSTGRES_DB'),
|
||||
user=os.getenv('POSTGRES_USER'),
|
||||
password=os.getenv('POSTGRES_PASSWORD')
|
||||
)
|
||||
print("✅ PostgreSQL connection successful!")
|
||||
conn.close()
|
||||
except Exception as e:
|
||||
print(f"❌ PostgreSQL connection failed: {e}")
|
||||
|
||||
# Test Redis
|
||||
try:
|
||||
r = redis.Redis(
|
||||
host=os.getenv('REDIS_HOST'),
|
||||
port=int(os.getenv('REDIS_PORT')),
|
||||
password=os.getenv('REDIS_PASSWORD')
|
||||
)
|
||||
r.ping()
|
||||
print("✅ Redis connection successful!")
|
||||
except Exception as e:
|
||||
print(f"❌ Redis connection failed: {e}")
|
||||
```
|
||||
|
||||
Run test:
|
||||
```powershell
|
||||
uv run python test_connection.py
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### 1. Port Already in Use
|
||||
|
||||
**Error**: `Port 5434 is already allocated`
|
||||
|
||||
**Solution**:
|
||||
- Change `POSTGRES_PORT` in `.env` to a different port (e.g., 5435)
|
||||
- Update `docker-compose.yml` port mapping accordingly
|
||||
- Restart containers: `docker-compose down && docker-compose up -d`
|
||||
|
||||
#### 2. Docker Permission Issues
|
||||
|
||||
**Error**: `permission denied while trying to connect to the Docker daemon`
|
||||
|
||||
**Solution**:
|
||||
- Ensure Docker Desktop is running
|
||||
- On Linux: Add user to docker group: `sudo usermod -aG docker $USER`
|
||||
- Restart terminal/session
|
||||
|
||||
#### 3. Database Connection Failed
|
||||
|
||||
**Error**: `password authentication failed`
|
||||
|
||||
**Solution**:
|
||||
- Ensure `.env` password matches `docker-compose.yml`
|
||||
- Reset database: `docker-compose down -v && docker-compose up -d`
|
||||
- Wait for database initialization (30-60 seconds)
|
||||
|
||||
#### 4. Python Dependencies Issues
|
||||
|
||||
**Error**: Package installation failures
|
||||
|
||||
**Solution**:
|
||||
```powershell
|
||||
# Clear UV cache
|
||||
uv cache clean
|
||||
|
||||
# Reinstall dependencies
|
||||
rm -rf .venv
|
||||
uv sync
|
||||
```
|
||||
|
||||
### Log Files
|
||||
|
||||
View service logs:
|
||||
```powershell
|
||||
# All services
|
||||
docker-compose logs
|
||||
|
||||
# Specific service
|
||||
docker-compose logs postgres
|
||||
docker-compose logs redis
|
||||
|
||||
# Follow logs in real-time
|
||||
docker-compose logs -f
|
||||
```
|
||||
|
||||
### Database Management
|
||||
|
||||
#### Backup Database
|
||||
|
||||
```powershell
|
||||
docker exec dashboard_postgres pg_dump -U dashboard dashboard > backup.sql
|
||||
```
|
||||
|
||||
#### Restore Database
|
||||
|
||||
```powershell
|
||||
docker exec -i dashboard_postgres psql -U dashboard dashboard < backup.sql
|
||||
```
|
||||
|
||||
#### Access Database CLI
|
||||
|
||||
```powershell
|
||||
docker exec -it dashboard_postgres psql -U dashboard -d dashboard
|
||||
```
|
||||
|
||||
#### Access Redis CLI
|
||||
|
||||
```powershell
|
||||
docker exec -it dashboard_redis redis-cli -a redis987secure
|
||||
```
|
||||
|
||||
## Security Notes
|
||||
|
||||
1. **Never commit `.env` file** to version control
|
||||
2. **Change default passwords** in production environments
|
||||
3. **Use strong passwords** for production deployments
|
||||
4. **Enable SSL/TLS** for production database connections
|
||||
5. **Restrict network access** in production environments
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues not covered in this guide:
|
||||
|
||||
1. Check the [project documentation](../README.md)
|
||||
2. Review [GitHub issues](link-to-issues)
|
||||
3. Contact the development team
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2024-05-30
|
||||
**Version**: 1.0
|
||||
**Tested On**: Windows 11, Docker Desktop 4.x
|
||||
@ -20,6 +20,7 @@
|
||||
- `tests/test_strategies.py` - Unit tests for strategy implementations
|
||||
- `tests/test_bot_manager.py` - Unit tests for bot management functionality
|
||||
- `tests/test_data_collection.py` - Unit tests for data collection and aggregation
|
||||
- `docs/setup.md` - Comprehensive setup guide for new machines and environments
|
||||
|
||||
### Notes
|
||||
|
||||
@ -31,7 +32,7 @@
|
||||
## Tasks
|
||||
|
||||
- [ ] 1.0 Database Foundation and Schema Setup
|
||||
- [ ] 1.1 Install and configure PostgreSQL with Docker
|
||||
- [x] 1.1 Install and configure PostgreSQL with Docker
|
||||
- [ ] 1.2 Create database schema following the PRD specifications (market_data, bots, signals, trades, bot_performance tables)
|
||||
- [ ] 1.3 Implement database connection utility with connection pooling
|
||||
- [ ] 1.4 Create database models using SQLAlchemy or similar ORM
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user