init setup
This commit is contained in:
parent
3e4a965895
commit
692611d3ae
2
.gitignore
vendored
2
.gitignore
vendored
@ -1 +1,3 @@
|
|||||||
*.pyc
|
*.pyc
|
||||||
|
.env
|
||||||
|
.env.local
|
||||||
|
|||||||
@ -29,6 +29,10 @@ This platform enables rapid strategy testing within 1-2 weeks of development. Bu
|
|||||||
- Python 3.10+, Docker, UV package manager
|
- Python 3.10+, Docker, UV package manager
|
||||||
|
|
||||||
### Setup
|
### Setup
|
||||||
|
|
||||||
|
**📖 For detailed setup instructions, see [docs/setup.md](docs/setup.md)**
|
||||||
|
|
||||||
|
Quick setup:
|
||||||
```bash
|
```bash
|
||||||
python scripts/dev.py setup # Setup environment
|
python scripts/dev.py setup # Setup environment
|
||||||
python scripts/dev.py start # Start services
|
python scripts/dev.py start # Start services
|
||||||
@ -50,6 +54,7 @@ python scripts/dev.py dev-server # Start with hot reload
|
|||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
|
- **[Setup Guide](docs/setup.md)** - Complete setup instructions for new machines
|
||||||
- **[Product Requirements](docs/crypto-bot-prd.md)** - Complete system specifications and requirements
|
- **[Product Requirements](docs/crypto-bot-prd.md)** - Complete system specifications and requirements
|
||||||
- **[Technical Architecture](docs/architecture.md)** - Implementation details and component design
|
- **[Technical Architecture](docs/architecture.md)** - Implementation details and component design
|
||||||
- **[Platform Overview](docs/specification.md)** - Human-readable system overview
|
- **[Platform Overview](docs/specification.md)** - Human-readable system overview
|
||||||
|
|||||||
@ -5,10 +5,10 @@ services:
|
|||||||
environment:
|
environment:
|
||||||
POSTGRES_DB: ${POSTGRES_DB:-dashboard}
|
POSTGRES_DB: ${POSTGRES_DB:-dashboard}
|
||||||
POSTGRES_USER: ${POSTGRES_USER:-dashboard}
|
POSTGRES_USER: ${POSTGRES_USER:-dashboard}
|
||||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-dashboard123}
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-sdkjfh534^jh}
|
||||||
POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C"
|
POSTGRES_INITDB_ARGS: "--encoding=UTF-8 --lc-collate=C --lc-ctype=C"
|
||||||
ports:
|
ports:
|
||||||
- "${POSTGRES_PORT:-5432}:5432"
|
- "${POSTGRES_PORT:-5434}:5432"
|
||||||
volumes:
|
volumes:
|
||||||
- postgres_data:/var/lib/postgresql/data
|
- postgres_data:/var/lib/postgresql/data
|
||||||
- ./database/init:/docker-entrypoint-initdb.d
|
- ./database/init:/docker-entrypoint-initdb.d
|
||||||
@ -24,14 +24,14 @@ services:
|
|||||||
redis:
|
redis:
|
||||||
image: redis:7-alpine
|
image: redis:7-alpine
|
||||||
container_name: dashboard_redis
|
container_name: dashboard_redis
|
||||||
command: redis-server --appendonly yes --appendfsync everysec
|
command: redis-server --appendonly yes --appendfsync everysec --requirepass ${REDIS_PASSWORD:-redis987secure}
|
||||||
ports:
|
ports:
|
||||||
- "${REDIS_PORT:-6379}:6379"
|
- "${REDIS_PORT:-6379}:6379"
|
||||||
volumes:
|
volumes:
|
||||||
- redis_data:/data
|
- redis_data:/data
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD", "redis-cli", "ping"]
|
test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis987secure}", "ping"]
|
||||||
interval: 30s
|
interval: 30s
|
||||||
timeout: 10s
|
timeout: 10s
|
||||||
retries: 3
|
retries: 3
|
||||||
|
|||||||
398
docs/setup.md
Normal file
398
docs/setup.md
Normal file
@ -0,0 +1,398 @@
|
|||||||
|
# Crypto Trading Bot Dashboard - Setup Guide
|
||||||
|
|
||||||
|
This guide will help you set up the Crypto Trading Bot Dashboard on a new machine from scratch.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
### Required Software
|
||||||
|
|
||||||
|
1. **Python 3.12+**
|
||||||
|
- Download from [python.org](https://python.org)
|
||||||
|
- Ensure Python is added to PATH
|
||||||
|
|
||||||
|
2. **UV Package Manager**
|
||||||
|
```powershell
|
||||||
|
# Windows (PowerShell)
|
||||||
|
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
|
||||||
|
|
||||||
|
# macOS/Linux
|
||||||
|
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Docker Desktop**
|
||||||
|
- Download from [docker.com](https://docker.com)
|
||||||
|
- Ensure Docker is running before proceeding
|
||||||
|
|
||||||
|
4. **Git**
|
||||||
|
- Download from [git-scm.com](https://git-scm.com)
|
||||||
|
|
||||||
|
### System Requirements
|
||||||
|
|
||||||
|
- **RAM**: Minimum 4GB, Recommended 8GB+
|
||||||
|
- **Storage**: At least 2GB free space
|
||||||
|
- **OS**: Windows 10/11, macOS 10.15+, or Linux
|
||||||
|
|
||||||
|
## Project Setup
|
||||||
|
|
||||||
|
### 1. Clone the Repository
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone <repository-url>
|
||||||
|
cd TCPDashboard
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Environment Configuration
|
||||||
|
|
||||||
|
Create the environment file from the template:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Windows
|
||||||
|
Copy-Item env.template .env
|
||||||
|
|
||||||
|
# macOS/Linux
|
||||||
|
cp env.template .env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Important**: The `.env` file contains pre-configured secure passwords. **Do not commit this file to version control.**
|
||||||
|
|
||||||
|
### 3. Configure Custom Ports (Optional)
|
||||||
|
|
||||||
|
If you have other PostgreSQL instances running, the default configuration uses port `5434` to avoid conflicts.
|
||||||
|
|
||||||
|
Current configuration in `.env`:
|
||||||
|
```env
|
||||||
|
POSTGRES_PORT=5434
|
||||||
|
POSTGRES_PASSWORD=sdkjfh534^jh
|
||||||
|
REDIS_PASSWORD=redis987secure
|
||||||
|
```
|
||||||
|
|
||||||
|
## Database Setup
|
||||||
|
|
||||||
|
### 1. Start Database Services
|
||||||
|
|
||||||
|
Start PostgreSQL and Redis using Docker Compose:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker-compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
- Create a PostgreSQL database on port `5434`
|
||||||
|
- Create a Redis instance on port `6379`
|
||||||
|
- Set up persistent volumes for data storage
|
||||||
|
- Configure password authentication
|
||||||
|
|
||||||
|
### 2. Verify Services are Running
|
||||||
|
|
||||||
|
Check container status:
|
||||||
|
```powershell
|
||||||
|
docker-compose ps
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output:
|
||||||
|
```
|
||||||
|
NAME IMAGE COMMAND SERVICE CREATED STATUS PORTS
|
||||||
|
dashboard_postgres postgres:15-alpine "docker-entrypoint.s…" postgres X minutes ago Up X minutes (healthy) 0.0.0.0:5434->5432/tcp
|
||||||
|
dashboard_redis redis:7-alpine "docker-entrypoint.s…" redis X minutes ago Up X minutes (healthy) 0.0.0.0:6379->6379/tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Test Database Connections
|
||||||
|
|
||||||
|
Test PostgreSQL connection:
|
||||||
|
```powershell
|
||||||
|
# Test port accessibility
|
||||||
|
Test-NetConnection -ComputerName localhost -Port 5434
|
||||||
|
|
||||||
|
# Test database connection (from inside container)
|
||||||
|
docker exec dashboard_postgres psql -h localhost -p 5432 -U dashboard -d dashboard -c "SELECT version();"
|
||||||
|
```
|
||||||
|
|
||||||
|
Test Redis connection:
|
||||||
|
```powershell
|
||||||
|
docker exec dashboard_redis redis-cli -a redis987secure ping
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output: `PONG`
|
||||||
|
|
||||||
|
## Application Setup
|
||||||
|
|
||||||
|
### 1. Install Python Dependencies
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv sync
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
- Create a virtual environment in `.venv/`
|
||||||
|
- Install all required dependencies
|
||||||
|
- Set up the project for development
|
||||||
|
|
||||||
|
### 2. Activate Virtual Environment
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Windows
|
||||||
|
uv run <command>
|
||||||
|
|
||||||
|
# Or activate manually
|
||||||
|
.venv\Scripts\Activate.ps1
|
||||||
|
|
||||||
|
# macOS/Linux
|
||||||
|
source .venv/bin/activate
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Initialize Database Schema
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Run database migrations (when implemented)
|
||||||
|
uv run python scripts/init_db.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Running the Application
|
||||||
|
|
||||||
|
### 1. Start the Dashboard
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
uv run python main.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Access the Application
|
||||||
|
|
||||||
|
Open your browser and navigate to:
|
||||||
|
- **Local**: http://localhost:8050
|
||||||
|
- **Network**: http://0.0.0.0:8050 (if accessible from other machines)
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
Key configuration options in `.env`:
|
||||||
|
|
||||||
|
```env
|
||||||
|
# Database Configuration
|
||||||
|
POSTGRES_HOST=localhost
|
||||||
|
POSTGRES_PORT=5434
|
||||||
|
POSTGRES_DB=dashboard
|
||||||
|
POSTGRES_USER=dashboard
|
||||||
|
POSTGRES_PASSWORD=sdkjfh534^jh
|
||||||
|
|
||||||
|
# Redis Configuration
|
||||||
|
REDIS_HOST=localhost
|
||||||
|
REDIS_PORT=6379
|
||||||
|
REDIS_PASSWORD=redis987secure
|
||||||
|
|
||||||
|
# Application Configuration
|
||||||
|
DASH_HOST=0.0.0.0
|
||||||
|
DASH_PORT=8050
|
||||||
|
DEBUG=true
|
||||||
|
|
||||||
|
# OKX API Configuration (for real trading)
|
||||||
|
OKX_API_KEY=your_okx_api_key_here
|
||||||
|
OKX_SECRET_KEY=your_okx_secret_key_here
|
||||||
|
OKX_PASSPHRASE=your_okx_passphrase_here
|
||||||
|
OKX_SANDBOX=true
|
||||||
|
```
|
||||||
|
|
||||||
|
### Port Configuration
|
||||||
|
|
||||||
|
If you need to change ports due to conflicts:
|
||||||
|
|
||||||
|
1. **PostgreSQL Port**: Update `POSTGRES_PORT` in `.env` and the port mapping in `docker-compose.yml`
|
||||||
|
2. **Redis Port**: Update `REDIS_PORT` in `.env` and `docker-compose.yml`
|
||||||
|
3. **Dashboard Port**: Update `DASH_PORT` in `.env`
|
||||||
|
|
||||||
|
## Development Workflow
|
||||||
|
|
||||||
|
### 1. Daily Development Setup
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Start databases
|
||||||
|
docker-compose up -d
|
||||||
|
|
||||||
|
# Start development server
|
||||||
|
uv run python main.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Stop Services
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Stop application: Ctrl+C in terminal
|
||||||
|
|
||||||
|
# Stop databases
|
||||||
|
docker-compose down
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Reset Database (if needed)
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# WARNING: This will delete all data
|
||||||
|
docker-compose down -v
|
||||||
|
docker-compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
### Run Unit Tests
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Run all tests
|
||||||
|
uv run pytest
|
||||||
|
|
||||||
|
# Run specific test file
|
||||||
|
uv run pytest tests/test_database.py
|
||||||
|
|
||||||
|
# Run with coverage
|
||||||
|
uv run pytest --cov=. --cov-report=html
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test Database Connection
|
||||||
|
|
||||||
|
Create a quick test script:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# test_connection.py
|
||||||
|
import os
|
||||||
|
import psycopg2
|
||||||
|
import redis
|
||||||
|
from dotenv import load_dotenv
|
||||||
|
|
||||||
|
load_dotenv()
|
||||||
|
|
||||||
|
# Test PostgreSQL
|
||||||
|
try:
|
||||||
|
conn = psycopg2.connect(
|
||||||
|
host=os.getenv('POSTGRES_HOST'),
|
||||||
|
port=os.getenv('POSTGRES_PORT'),
|
||||||
|
database=os.getenv('POSTGRES_DB'),
|
||||||
|
user=os.getenv('POSTGRES_USER'),
|
||||||
|
password=os.getenv('POSTGRES_PASSWORD')
|
||||||
|
)
|
||||||
|
print("✅ PostgreSQL connection successful!")
|
||||||
|
conn.close()
|
||||||
|
except Exception as e:
|
||||||
|
print(f"❌ PostgreSQL connection failed: {e}")
|
||||||
|
|
||||||
|
# Test Redis
|
||||||
|
try:
|
||||||
|
r = redis.Redis(
|
||||||
|
host=os.getenv('REDIS_HOST'),
|
||||||
|
port=int(os.getenv('REDIS_PORT')),
|
||||||
|
password=os.getenv('REDIS_PASSWORD')
|
||||||
|
)
|
||||||
|
r.ping()
|
||||||
|
print("✅ Redis connection successful!")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"❌ Redis connection failed: {e}")
|
||||||
|
```
|
||||||
|
|
||||||
|
Run test:
|
||||||
|
```powershell
|
||||||
|
uv run python test_connection.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
#### 1. Port Already in Use
|
||||||
|
|
||||||
|
**Error**: `Port 5434 is already allocated`
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
- Change `POSTGRES_PORT` in `.env` to a different port (e.g., 5435)
|
||||||
|
- Update `docker-compose.yml` port mapping accordingly
|
||||||
|
- Restart containers: `docker-compose down && docker-compose up -d`
|
||||||
|
|
||||||
|
#### 2. Docker Permission Issues
|
||||||
|
|
||||||
|
**Error**: `permission denied while trying to connect to the Docker daemon`
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
- Ensure Docker Desktop is running
|
||||||
|
- On Linux: Add user to docker group: `sudo usermod -aG docker $USER`
|
||||||
|
- Restart terminal/session
|
||||||
|
|
||||||
|
#### 3. Database Connection Failed
|
||||||
|
|
||||||
|
**Error**: `password authentication failed`
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
- Ensure `.env` password matches `docker-compose.yml`
|
||||||
|
- Reset database: `docker-compose down -v && docker-compose up -d`
|
||||||
|
- Wait for database initialization (30-60 seconds)
|
||||||
|
|
||||||
|
#### 4. Python Dependencies Issues
|
||||||
|
|
||||||
|
**Error**: Package installation failures
|
||||||
|
|
||||||
|
**Solution**:
|
||||||
|
```powershell
|
||||||
|
# Clear UV cache
|
||||||
|
uv cache clean
|
||||||
|
|
||||||
|
# Reinstall dependencies
|
||||||
|
rm -rf .venv
|
||||||
|
uv sync
|
||||||
|
```
|
||||||
|
|
||||||
|
### Log Files
|
||||||
|
|
||||||
|
View service logs:
|
||||||
|
```powershell
|
||||||
|
# All services
|
||||||
|
docker-compose logs
|
||||||
|
|
||||||
|
# Specific service
|
||||||
|
docker-compose logs postgres
|
||||||
|
docker-compose logs redis
|
||||||
|
|
||||||
|
# Follow logs in real-time
|
||||||
|
docker-compose logs -f
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Management
|
||||||
|
|
||||||
|
#### Backup Database
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker exec dashboard_postgres pg_dump -U dashboard dashboard > backup.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Restore Database
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker exec -i dashboard_postgres psql -U dashboard dashboard < backup.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Access Database CLI
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker exec -it dashboard_postgres psql -U dashboard -d dashboard
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Access Redis CLI
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker exec -it dashboard_redis redis-cli -a redis987secure
|
||||||
|
```
|
||||||
|
|
||||||
|
## Security Notes
|
||||||
|
|
||||||
|
1. **Never commit `.env` file** to version control
|
||||||
|
2. **Change default passwords** in production environments
|
||||||
|
3. **Use strong passwords** for production deployments
|
||||||
|
4. **Enable SSL/TLS** for production database connections
|
||||||
|
5. **Restrict network access** in production environments
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
If you encounter issues not covered in this guide:
|
||||||
|
|
||||||
|
1. Check the [project documentation](../README.md)
|
||||||
|
2. Review [GitHub issues](link-to-issues)
|
||||||
|
3. Contact the development team
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Last Updated**: 2024-05-30
|
||||||
|
**Version**: 1.0
|
||||||
|
**Tested On**: Windows 11, Docker Desktop 4.x
|
||||||
@ -20,6 +20,7 @@
|
|||||||
- `tests/test_strategies.py` - Unit tests for strategy implementations
|
- `tests/test_strategies.py` - Unit tests for strategy implementations
|
||||||
- `tests/test_bot_manager.py` - Unit tests for bot management functionality
|
- `tests/test_bot_manager.py` - Unit tests for bot management functionality
|
||||||
- `tests/test_data_collection.py` - Unit tests for data collection and aggregation
|
- `tests/test_data_collection.py` - Unit tests for data collection and aggregation
|
||||||
|
- `docs/setup.md` - Comprehensive setup guide for new machines and environments
|
||||||
|
|
||||||
### Notes
|
### Notes
|
||||||
|
|
||||||
@ -31,7 +32,7 @@
|
|||||||
## Tasks
|
## Tasks
|
||||||
|
|
||||||
- [ ] 1.0 Database Foundation and Schema Setup
|
- [ ] 1.0 Database Foundation and Schema Setup
|
||||||
- [ ] 1.1 Install and configure PostgreSQL with Docker
|
- [x] 1.1 Install and configure PostgreSQL with Docker
|
||||||
- [ ] 1.2 Create database schema following the PRD specifications (market_data, bots, signals, trades, bot_performance tables)
|
- [ ] 1.2 Create database schema following the PRD specifications (market_data, bots, signals, trades, bot_performance tables)
|
||||||
- [ ] 1.3 Implement database connection utility with connection pooling
|
- [ ] 1.3 Implement database connection utility with connection pooling
|
||||||
- [ ] 1.4 Create database models using SQLAlchemy or similar ORM
|
- [ ] 1.4 Create database models using SQLAlchemy or similar ORM
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user