From 8aa47731f2dafebd77a9b09a5a614cb29ac4877e Mon Sep 17 00:00:00 2001 From: "Vasily.onl" Date: Wed, 4 Jun 2025 17:05:39 +0800 Subject: [PATCH] docs --- README.md | 17 + docs/components/charts/README.md | 13 +- docs/components/charts/bot-integration.md | 626 ++++++++++++++++++ .../dashboard-modular-structure.md | 0 tasks/3.4. Chart layers.md | 12 +- 5 files changed, 662 insertions(+), 6 deletions(-) create mode 100644 docs/components/charts/bot-integration.md rename docs/{ => components}/dashboard-modular-structure.md (100%) diff --git a/README.md b/README.md index 9587030..420136f 100644 --- a/README.md +++ b/README.md @@ -10,11 +10,26 @@ This platform enables rapid strategy testing within 1-2 weeks of development. Bu - **Multi-Bot Management**: Run 5-10 trading bots simultaneously with different strategies - **Real-time Monitoring**: Live OHLCV charts with bot trading signals overlay +- **📊 Modular Chart Layers**: Advanced technical analysis with 26+ indicators and strategy presets +- **🤖 Bot Signal Integration**: Real-time bot signal visualization with performance analytics - **Virtual Trading**: Simulation-first approach with realistic fee modeling - **JSON Configuration**: Easy strategy parameter testing without code changes - **Backtesting Engine**: Test strategies on historical market data - **Crash Recovery**: Automatic bot restart and state restoration +## Chart System Features + +The platform includes a sophisticated modular chart system with: + +- **Technical Indicators**: 26+ professionally configured indicators (SMA, EMA, Bollinger Bands, RSI, MACD) +- **Strategy Presets**: 5 real-world trading strategy templates (EMA crossover, momentum, mean reversion) +- **Bot Integration**: Real-time visualization of bot signals, trades, and performance +- **Custom Indicators**: User-defined indicators with JSON persistence +- **Validation System**: 10+ validation rules with detailed error reporting +- **Modular Architecture**: Independently testable chart layers and components + +📊 **[Complete Chart Documentation](docs/components/charts/README.md)** + ## Tech Stack - **Framework**: Python 3.10+ with Dash (unified frontend/backend) @@ -58,6 +73,8 @@ python scripts/dev.py dev-server # Start with hot reload - **[Product Requirements](docs/crypto-bot-prd.md)** - Complete system specifications and requirements - **[Technical Architecture](docs/architecture.md)** - Implementation details and component design - **[Platform Overview](docs/specification.md)** - Human-readable system overview +- **📊 [Chart Layers System](docs/components/charts/README.md)** - Modular chart system with technical indicators +- **🤖 [Bot Integration Guide](docs/components/charts/bot-integration.md)** - Real-time bot signal visualization ## Configuration Example diff --git a/docs/components/charts/README.md b/docs/components/charts/README.md index 30bc27d..327a7e2 100644 --- a/docs/components/charts/README.md +++ b/docs/components/charts/README.md @@ -670,12 +670,23 @@ uv run pytest tests/test_defaults.py -v ## Future Enhancements -- **Signal Layer Integration**: Bot trade signals and alerts +- **✅ Signal Layer Integration**: Bot trade signals and alerts - **IMPLEMENTED** - See [Bot Integration Guide](./bot-integration.md) - **Custom Indicators**: User-defined technical indicators - **Advanced Layouts**: Multi-chart and grid layouts - **Real-time Updates**: Live chart updates with indicator toggling - **Performance Monitoring**: Advanced resource usage tracking +## Bot Integration + +The chart system now includes comprehensive bot integration capabilities: + +- **Real-time Signal Visualization**: Live bot signals on charts +- **Trade Execution Tracking**: P&L and trade entry/exit points +- **Multi-Bot Support**: Compare strategies across multiple bots +- **Performance Analytics**: Built-in bot performance metrics + +📊 **[Complete Bot Integration Guide](./bot-integration.md)** - Comprehensive documentation for integrating bot signals with charts + ## Support For issues, questions, or contributions: diff --git a/docs/components/charts/bot-integration.md b/docs/components/charts/bot-integration.md new file mode 100644 index 0000000..b2a5245 --- /dev/null +++ b/docs/components/charts/bot-integration.md @@ -0,0 +1,626 @@ +# Bot Integration with Chart Signal Layers + +The Chart Layers System provides seamless integration with the bot management system, allowing real-time visualization of bot signals, trades, and performance data directly on charts. + +## Table of Contents + +- [Overview](#overview) +- [Architecture](#architecture) +- [Quick Start](#quick-start) +- [Bot Data Service](#bot-data-service) +- [Signal Layer Integration](#signal-layer-integration) +- [Enhanced Bot Layers](#enhanced-bot-layers) +- [Multi-Bot Visualization](#multi-bot-visualization) +- [Configuration Options](#configuration-options) +- [Examples](#examples) +- [Best Practices](#best-practices) +- [Troubleshooting](#troubleshooting) + +## Overview + +The bot integration system provides automatic data fetching and visualization of: + +- **Trading Signals**: Buy/sell/hold signals from active bots +- **Trade Executions**: Entry/exit points with P&L information +- **Bot Performance**: Real-time performance metrics and analytics +- **Strategy Comparison**: Side-by-side strategy analysis +- **Multi-Bot Views**: Aggregate views across multiple bots + +### Key Features + +- **Automatic Data Fetching**: No manual data queries required +- **Real-time Updates**: Charts update with live bot data +- **Database Integration**: Direct connection to bot management system +- **Advanced Filtering**: Filter by bot, strategy, symbol, timeframe +- **Performance Analytics**: Built-in performance calculation +- **Error Handling**: Graceful handling of database errors + +## Architecture + +``` +components/charts/layers/ +├── bot_integration.py # Core bot data services +├── bot_enhanced_layers.py # Enhanced chart layers with bot integration +└── signals.py # Base signal layers + +Bot Integration Components: +├── BotFilterConfig # Configuration for bot filtering +├── BotDataService # Database operations for bot data +├── BotSignalLayerIntegration # Chart-specific integration utilities +├── BotIntegratedSignalLayer # Auto-fetching signal layer +├── BotIntegratedTradeLayer # Auto-fetching trade layer +└── BotMultiLayerIntegration # Multi-bot layer management +``` + +## Quick Start + +### Basic Bot Signal Visualization + +```python +from components.charts.layers import create_bot_signal_layer + +# Create a bot-integrated signal layer for BTCUSDT +signal_layer = create_bot_signal_layer( + symbol='BTCUSDT', + active_only=True, + confidence_threshold=0.5, + time_window_days=7 +) + +# Add to chart +fig = go.Figure() +fig = signal_layer.render(fig, market_data, symbol='BTCUSDT') +``` + +### Complete Bot Visualization Setup + +```python +from components.charts.layers import create_complete_bot_layers + +# Create complete bot layer set for a symbol +result = create_complete_bot_layers( + symbol='BTCUSDT', + timeframe='1h', + active_only=True, + time_window_days=7 +) + +if result['success']: + signal_layer = result['layers']['signals'] + trade_layer = result['layers']['trades'] + + # Add to chart + fig = signal_layer.render(fig, market_data, symbol='BTCUSDT') + fig = trade_layer.render(fig, market_data, symbol='BTCUSDT') +``` + +## Bot Data Service + +The `BotDataService` provides the core interface for fetching bot-related data from the database. + +### Basic Usage + +```python +from components.charts.layers.bot_integration import BotDataService, BotFilterConfig + +# Initialize service +service = BotDataService() + +# Create filter configuration +bot_filter = BotFilterConfig( + symbols=['BTCUSDT'], + strategies=['momentum', 'ema_crossover'], + active_only=True +) + +# Fetch bot data +bots_df = service.get_bots(bot_filter) +signals_df = service.get_signals_for_bots( + bot_ids=bots_df['id'].tolist(), + start_time=datetime.now() - timedelta(days=7), + end_time=datetime.now(), + min_confidence=0.3 +) +``` + +### Available Methods + +| Method | Description | Parameters | +|--------|-------------|------------| +| `get_bots()` | Fetch bot information | `filter_config: BotFilterConfig` | +| `get_signals_for_bots()` | Fetch signals from bots | `bot_ids, start_time, end_time, signal_types, min_confidence` | +| `get_trades_for_bots()` | Fetch trades from bots | `bot_ids, start_time, end_time, sides` | +| `get_bot_performance()` | Fetch performance data | `bot_ids, start_time, end_time` | + +### BotFilterConfig Options + +```python +@dataclass +class BotFilterConfig: + bot_ids: Optional[List[int]] = None # Specific bot IDs + bot_names: Optional[List[str]] = None # Specific bot names + strategies: Optional[List[str]] = None # Strategy filter + symbols: Optional[List[str]] = None # Symbol filter + statuses: Optional[List[str]] = None # Bot status filter + date_range: Optional[Tuple[datetime, datetime]] = None + active_only: bool = False # Only active bots +``` + +## Signal Layer Integration + +The `BotSignalLayerIntegration` provides chart-specific utilities for integrating bot data with chart layers. + +### Chart-Specific Signal Fetching + +```python +from components.charts.layers.bot_integration import BotSignalLayerIntegration + +integration = BotSignalLayerIntegration() + +# Get signals for specific chart context +signals_df = integration.get_signals_for_chart( + symbol='BTCUSDT', + timeframe='1h', + bot_filter=BotFilterConfig(active_only=True), + time_range=(start_time, end_time), + signal_types=['buy', 'sell'], + min_confidence=0.5 +) + +# Get trades for chart context +trades_df = integration.get_trades_for_chart( + symbol='BTCUSDT', + timeframe='1h', + bot_filter=BotFilterConfig(strategies=['momentum']), + time_range=(start_time, end_time) +) + +# Get bot summary statistics +stats = integration.get_bot_summary_stats(bot_ids=[1, 2, 3]) +``` + +### Performance Analytics + +```python +# Get comprehensive performance summary +performance = get_bot_performance_summary( + bot_id=1, # Specific bot or None for all + days_back=30 +) + +print(f"Total trades: {performance['trade_count']}") +print(f"Win rate: {performance['win_rate']:.1f}%") +print(f"Total P&L: ${performance['bot_stats']['total_pnl']:.2f}") +``` + +## Enhanced Bot Layers + +Enhanced layers provide automatic data fetching and bot-specific visualization features. + +### BotIntegratedSignalLayer + +```python +from components.charts.layers import BotIntegratedSignalLayer, BotSignalLayerConfig + +# Configure bot-integrated signal layer +config = BotSignalLayerConfig( + name="BTCUSDT Bot Signals", + auto_fetch_data=True, # Automatically fetch from database + time_window_days=7, # Look back 7 days + active_bots_only=True, # Only active bots + include_bot_info=True, # Include bot info in hover + group_by_strategy=True, # Group signals by strategy + confidence_threshold=0.3, # Minimum confidence + signal_types=['buy', 'sell'] # Signal types to show +) + +layer = BotIntegratedSignalLayer(config) + +# Render automatically fetches data +fig = layer.render(fig, market_data, symbol='BTCUSDT') +``` + +### BotIntegratedTradeLayer + +```python +from components.charts.layers import BotIntegratedTradeLayer, BotTradeLayerConfig + +config = BotTradeLayerConfig( + name="BTCUSDT Bot Trades", + auto_fetch_data=True, + time_window_days=7, + show_pnl=True, # Show profit/loss + show_trade_lines=True, # Connect entry/exit + include_bot_info=True, # Bot info in hover + group_by_strategy=False +) + +layer = BotIntegratedTradeLayer(config) +fig = layer.render(fig, market_data, symbol='BTCUSDT') +``` + +## Multi-Bot Visualization + +### Strategy Comparison + +```python +from components.charts.layers import bot_multi_layer + +# Compare multiple strategies on the same symbol +result = bot_multi_layer.create_strategy_comparison_layers( + symbol='BTCUSDT', + strategies=['momentum', 'ema_crossover', 'mean_reversion'], + timeframe='1h', + time_window_days=14 +) + +if result['success']: + for strategy in result['strategies']: + signal_layer = result['layers'][f"{strategy}_signals"] + trade_layer = result['layers'][f"{strategy}_trades"] + + fig = signal_layer.render(fig, market_data, symbol='BTCUSDT') + fig = trade_layer.render(fig, market_data, symbol='BTCUSDT') +``` + +### Multi-Symbol Bot View + +```python +# Create bot layers for multiple symbols +symbols = ['BTCUSDT', 'ETHUSDT', 'ADAUSDT'] + +for symbol in symbols: + bot_layers = create_complete_bot_layers( + symbol=symbol, + active_only=True, + time_window_days=7 + ) + + if bot_layers['success']: + # Add layers to respective charts + signal_layer = bot_layers['layers']['signals'] + # ... render on symbol-specific chart +``` + +## Configuration Options + +### Auto-Fetch Configuration + +```python +# Disable auto-fetch for manual data control +config = BotSignalLayerConfig( + name="Manual Bot Signals", + auto_fetch_data=False, # Disable auto-fetch + active_bots_only=True +) + +layer = BotIntegratedSignalLayer(config) + +# Manually provide signal data +manual_signals = get_signals_from_api() +fig = layer.render(fig, market_data, signals=manual_signals) +``` + +### Time Window Management + +```python +# Custom time window +config = BotSignalLayerConfig( + name="Short Term Signals", + time_window_days=1, # Last 24 hours only + active_bots_only=True, + confidence_threshold=0.7 # High confidence only +) +``` + +### Bot-Specific Filtering + +```python +# Filter for specific bots +bot_filter = BotFilterConfig( + bot_ids=[1, 2, 5], # Specific bot IDs + symbols=['BTCUSDT'], + active_only=True +) + +config = BotSignalLayerConfig( + name="Selected Bots", + bot_filter=bot_filter, + include_bot_info=True +) +``` + +## Examples + +### Dashboard Integration Example + +```python +# dashboard/callbacks/charts.py +from components.charts.layers import ( + create_bot_signal_layer, + create_bot_trade_layer, + get_active_bot_signals +) + +@app.callback( + Output('chart', 'figure'), + [Input('symbol-dropdown', 'value'), + Input('show-bot-signals', 'value')] +) +def update_chart_with_bots(symbol, show_bot_signals): + fig = create_base_chart(symbol) + + if 'bot-signals' in show_bot_signals: + # Add bot signals + signal_layer = create_bot_signal_layer( + symbol=symbol, + active_only=True, + confidence_threshold=0.3 + ) + fig = signal_layer.render(fig, market_data, symbol=symbol) + + # Add bot trades + trade_layer = create_bot_trade_layer( + symbol=symbol, + active_only=True, + show_pnl=True + ) + fig = trade_layer.render(fig, market_data, symbol=symbol) + + return fig +``` + +### Custom Bot Analysis + +```python +# Custom analysis for specific strategy +def analyze_momentum_strategy(symbol: str, days_back: int = 30): + """Analyze momentum strategy performance for a symbol.""" + + # Get momentum bot signals + signals = get_bot_signals_by_strategy( + strategy_name='momentum', + symbol=symbol, + days_back=days_back + ) + + # Get performance summary + performance = get_bot_performance_summary(days_back=days_back) + + # Create visualizations + signal_layer = create_bot_signal_layer( + symbol=symbol, + active_only=False, # Include all momentum bots + time_window_days=days_back + ) + + return { + 'signals': signals, + 'performance': performance, + 'layer': signal_layer + } + +# Usage +analysis = analyze_momentum_strategy('BTCUSDT', days_back=14) +``` + +### Real-time Monitoring Setup + +```python +# Real-time bot monitoring dashboard component +def create_realtime_bot_monitor(symbols: List[str]): + """Create real-time bot monitoring charts.""" + + charts = {} + + for symbol in symbols: + # Get latest bot data + active_signals = get_active_bot_signals( + symbol=symbol, + days_back=1, # Last 24 hours + min_confidence=0.5 + ) + + # Create monitoring layers + signal_layer = create_bot_signal_layer( + symbol=symbol, + active_only=True, + time_window_days=1 + ) + + trade_layer = create_bot_trade_layer( + symbol=symbol, + active_only=True, + show_pnl=True, + time_window_days=1 + ) + + charts[symbol] = { + 'signal_layer': signal_layer, + 'trade_layer': trade_layer, + 'active_signals': len(active_signals) + } + + return charts +``` + +## Best Practices + +### Performance Optimization + +```python +# 1. Use appropriate time windows +config = BotSignalLayerConfig( + time_window_days=7, # Don't fetch more data than needed + confidence_threshold=0.3 # Filter low-confidence signals +) + +# 2. Filter by active bots only when possible +bot_filter = BotFilterConfig( + active_only=True, # Reduces database queries + symbols=['BTCUSDT'] # Specific symbols only +) + +# 3. Reuse integration instances +integration = BotSignalLayerIntegration() # Create once +# Use multiple times for different symbols +``` + +### Error Handling + +```python +try: + bot_layers = create_complete_bot_layers('BTCUSDT') + + if not bot_layers['success']: + logger.warning(f"Bot layer creation failed: {bot_layers.get('error')}") + # Fallback to manual signal layer + signal_layer = TradingSignalLayer() + else: + signal_layer = bot_layers['layers']['signals'] + +except Exception as e: + logger.error(f"Bot integration error: {e}") + # Graceful degradation + signal_layer = TradingSignalLayer() +``` + +### Database Connection Management + +```python +# The bot integration handles database connections automatically +# But for custom queries, follow these patterns: + +from database.connection import get_session + +def custom_bot_query(): + try: + with get_session() as session: + # Your database operations + result = session.query(Bot).filter(...).all() + return result + except Exception as e: + logger.error(f"Database query failed: {e}") + return [] +``` + +## Troubleshooting + +### Common Issues + +1. **No signals showing on chart** + ```python + # Check if bots exist for symbol + service = BotDataService() + bots = service.get_bots(BotFilterConfig(symbols=['BTCUSDT'])) + print(f"Found {len(bots)} bots for BTCUSDT") + + # Check signal count + signals = get_active_bot_signals('BTCUSDT', days_back=7) + print(f"Found {len(signals)} signals in last 7 days") + ``` + +2. **Database connection errors** + ```python + # Test database connection + try: + from database.connection import get_session + with get_session() as session: + print("Database connection successful") + except Exception as e: + print(f"Database connection failed: {e}") + ``` + +3. **Performance issues with large datasets** + ```python + # Reduce time window + config = BotSignalLayerConfig( + time_window_days=3, # Reduced from 7 + confidence_threshold=0.5 # Higher threshold + ) + + # Filter by specific strategies + bot_filter = BotFilterConfig( + strategies=['momentum'], # Specific strategy only + active_only=True + ) + ``` + +### Debug Mode + +```python +import logging + +# Enable debug logging for bot integration +logging.getLogger('bot_integration').setLevel(logging.DEBUG) +logging.getLogger('bot_enhanced_layers').setLevel(logging.DEBUG) + +# This will show detailed information about: +# - Database queries +# - Data fetching operations +# - Filter applications +# - Performance metrics +``` + +### Testing Bot Integration + +```python +# Test bot integration components +from tests.test_signal_layers import TestBotIntegration + +# Run specific bot integration tests +pytest.main(['-v', 'tests/test_signal_layers.py::TestBotIntegration']) + +# Test with mock data +def test_bot_integration(): + config = BotSignalLayerConfig( + name="Test Bot Signals", + auto_fetch_data=False # Use manual data for testing + ) + + layer = BotIntegratedSignalLayer(config) + + # Provide test data + test_signals = pd.DataFrame({ + 'timestamp': [datetime.now()], + 'signal_type': ['buy'], + 'price': [50000], + 'confidence': [0.8], + 'bot_name': ['Test Bot'] + }) + + fig = go.Figure() + result = layer.render(fig, market_data, signals=test_signals) + + assert len(result.data) > 0 +``` + +## API Reference + +### Core Classes + +- **`BotDataService`** - Main service for database operations +- **`BotSignalLayerIntegration`** - Chart-specific integration utilities +- **`BotIntegratedSignalLayer`** - Auto-fetching signal layer +- **`BotIntegratedTradeLayer`** - Auto-fetching trade layer +- **`BotMultiLayerIntegration`** - Multi-bot layer management + +### Configuration Classes + +- **`BotFilterConfig`** - Bot filtering configuration +- **`BotSignalLayerConfig`** - Signal layer configuration with bot options +- **`BotTradeLayerConfig`** - Trade layer configuration with bot options + +### Convenience Functions + +- **`create_bot_signal_layer()`** - Quick bot signal layer creation +- **`create_bot_trade_layer()`** - Quick bot trade layer creation +- **`create_complete_bot_layers()`** - Complete bot layer set +- **`get_active_bot_signals()`** - Get signals from active bots +- **`get_active_bot_trades()`** - Get trades from active bots +- **`get_bot_signals_by_strategy()`** - Get signals by strategy +- **`get_bot_performance_summary()`** - Get performance analytics + +For complete API documentation, see the module docstrings in: +- `components/charts/layers/bot_integration.py` +- `components/charts/layers/bot_enhanced_layers.py` \ No newline at end of file diff --git a/docs/dashboard-modular-structure.md b/docs/components/dashboard-modular-structure.md similarity index 100% rename from docs/dashboard-modular-structure.md rename to docs/components/dashboard-modular-structure.md diff --git a/tasks/3.4. Chart layers.md b/tasks/3.4. Chart layers.md index f9fc923..feee735 100644 --- a/tasks/3.4. Chart layers.md +++ b/tasks/3.4. Chart layers.md @@ -94,15 +94,15 @@ Implementation of a flexible, strategy-driven chart system that supports technic - [x] 5.6 Prepare integration points for bot management system - [x] 5.7 Create foundation tests for signal layer functionality -- [ ] 6.0 Documentation **⏳ IN PROGRESS** +- [x] 6.0 Documentation **✅ COMPLETED** - [x] 6.1 Create documentation for the chart layers system - - [ ] 6.2 Add documentation to the README + - [x] 6.2 Add documentation to the README - [x] 6.3 Create documentation for the ChartBuilder class - [x] 6.4 Create documentation for the ChartUtils class - [x] 6.5 Create documentation for the ChartConfig package - [x] 6.6 Create documentation how to add new indicators - [x] 6.7 Create documentation how to add new strategies - - [ ] 6.8 Create documentation how to add new bot integration + - [x] 6.8 Create documentation how to add new bot integration ## Current Status @@ -112,6 +112,7 @@ Implementation of a flexible, strategy-driven chart system that supports technic - **3.0 Strategy Configuration**: Comprehensive strategy system with validation - **4.0 Dashboard Integration**: Including modular dashboard structure - **5.0 Signal Layer Foundation**: Complete implementation with bot integration ready +- **6.0 Documentation**: Complete documentation suite with bot integration guide ### 🎯 **KEY ACHIEVEMENTS** - **Strategy dropdown**: Fully functional with auto-loading of strategy indicators @@ -122,11 +123,12 @@ Implementation of a flexible, strategy-driven chart system that supports technic - **Signal layer architecture**: Complete foundation for bot signal visualization - **Bot integration**: Ready-to-use integration points for bot management system - **Foundation tests**: Comprehensive test suite for signal layer functionality +- **Complete documentation**: Comprehensive documentation suite with bot integration guide ### 📋 **NEXT PHASES** -- **6.0 Documentation**: Complete README and final documentation updates +- **Chart Layers System**: ✅ **FULLY COMPLETED** - Ready for production use! -The signal layer foundation is now **COMPLETED and fully ready** for bot integration! 🚀 +The entire Chart Layers System is now **FULLY COMPLETED and production-ready**! 🚀 **Latest Completion:** - **Task 5.6**: Bot integration points created with: