TCPDashboard/docs/modules/charts/bot-integration.md

# Bot Integration with Chart Signal Layers

> **⚠️ Feature Not Yet Implemented**
>
> The functionality described in this document for bot integration with chart layers is **planned for a future release**. It depends on the **Strategy Engine** and **Bot Manager**, which are not yet implemented. This document outlines the intended architecture and usage once these components are available.

The Chart Layers System provides seamless integration with the bot management system, allowing real-time visualization of bot signals, trades, and performance data directly on charts.

## Table of Contents

- [Overview](#overview)
- [Architecture](#architecture)
- [Quick Start](#quick-start)
- [Bot Data Service](#bot-data-service)
- [Signal Layer Integration](#signal-layer-integration)
- [Enhanced Bot Layers](#enhanced-bot-layers)
- [Multi-Bot Visualization](#multi-bot-visualization)
- [Configuration Options](#configuration-options)
- [Examples](#examples)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)

## Overview

The bot integration system provides automatic data fetching and visualization of:

- **Trading Signals**: Buy/sell/hold signals from active bots
- **Trade Executions**: Entry/exit points with P&L information
- **Bot Performance**: Real-time performance metrics and analytics
- **Strategy Comparison**: Side-by-side strategy analysis
- **Multi-Bot Views**: Aggregate views across multiple bots

### Key Features

- **Automatic Data Fetching**: No manual data queries required
- **Real-time Updates**: Charts update with live bot data
- **Database Integration**: Direct connection to bot management system
- **Advanced Filtering**: Filter by bot, strategy, symbol, timeframe
- **Performance Analytics**: Built-in performance calculation
- **Error Handling**: Graceful handling of database errors

## Architecture

```
components/charts/layers/
├── bot_integration.py         # Core bot data services
├── bot_enhanced_layers.py     # Enhanced chart layers with bot integration
└── signals.py                 # Base signal layers

Bot Integration Components:
├── BotFilterConfig           # Configuration for bot filtering
├── BotDataService           # Database operations for bot data
├── BotSignalLayerIntegration # Chart-specific integration utilities
├── BotIntegratedSignalLayer # Auto-fetching signal layer
├── BotIntegratedTradeLayer  # Auto-fetching trade layer
└── BotMultiLayerIntegration # Multi-bot layer management
```

## Quick Start

### Basic Bot Signal Visualization

```python
from components.charts.layers import create_bot_signal_layer

# Create a bot-integrated signal layer for BTCUSDT
signal_layer = create_bot_signal_layer(
    symbol='BTCUSDT',
    active_only=True,
    confidence_threshold=0.5,
    time_window_days=7
)

# Add to chart
fig = go.Figure()
fig = signal_layer.render(fig, market_data, symbol='BTCUSDT')
```

### Complete Bot Visualization Setup

```python
from components.charts.layers import create_complete_bot_layers

# Create complete bot layer set for a symbol
result = create_complete_bot_layers(
    symbol='BTCUSDT',
    timeframe='1h',
    active_only=True,
    time_window_days=7
)

if result['success']:
    signal_layer = result['layers']['signals']
    trade_layer = result['layers']['trades']
    
    # Add to chart
    fig = signal_layer.render(fig, market_data, symbol='BTCUSDT')
    fig = trade_layer.render(fig, market_data, symbol='BTCUSDT')
```

## Bot Data Service

The `BotDataService` provides the core interface for fetching bot-related data from the database.

### Basic Usage

```python
from components.charts.layers.bot_integration import BotDataService, BotFilterConfig

# Initialize service
service = BotDataService()

# Create filter configuration
bot_filter = BotFilterConfig(
    symbols=['BTCUSDT'],
    strategies=['momentum', 'ema_crossover'],
    active_only=True
)

# Fetch bot data
bots_df = service.get_bots(bot_filter)
signals_df = service.get_signals_for_bots(
    bot_ids=bots_df['id'].tolist(),
    start_time=datetime.now() - timedelta(days=7),
    end_time=datetime.now(),
    min_confidence=0.3
)
```

### Available Methods

| Method | Description | Parameters |
|--------|-------------|------------|
| `get_bots()` | Fetch bot information | `filter_config: BotFilterConfig` |
| `get_signals_for_bots()` | Fetch signals from bots | `bot_ids, start_time, end_time, signal_types, min_confidence` |
| `get_trades_for_bots()` | Fetch trades from bots | `bot_ids, start_time, end_time, sides` |
| `get_bot_performance()` | Fetch performance data | `bot_ids, start_time, end_time` |

### BotFilterConfig Options

```python
@dataclass
class BotFilterConfig:
    bot_ids: Optional[List[int]] = None        # Specific bot IDs
    bot_names: Optional[List[str]] = None      # Specific bot names
    strategies: Optional[List[str]] = None     # Strategy filter
    symbols: Optional[List[str]] = None        # Symbol filter
    statuses: Optional[List[str]] = None       # Bot status filter
    date_range: Optional[Tuple[datetime, datetime]] = None
    active_only: bool = False                  # Only active bots
```

## Signal Layer Integration

The `BotSignalLayerIntegration` provides chart-specific utilities for integrating bot data with chart layers.

### Chart-Specific Signal Fetching

```python
from components.charts.layers.bot_integration import BotSignalLayerIntegration

integration = BotSignalLayerIntegration()

# Get signals for specific chart context
signals_df = integration.get_signals_for_chart(
    symbol='BTCUSDT',
    timeframe='1h',
    bot_filter=BotFilterConfig(active_only=True),
    time_range=(start_time, end_time),
    signal_types=['buy', 'sell'],
    min_confidence=0.5
)

# Get trades for chart context
trades_df = integration.get_trades_for_chart(
    symbol='BTCUSDT',
    timeframe='1h',
    bot_filter=BotFilterConfig(strategies=['momentum']),
    time_range=(start_time, end_time)
)

# Get bot summary statistics
stats = integration.get_bot_summary_stats(bot_ids=[1, 2, 3])
```

### Performance Analytics

```python
# Get comprehensive performance summary
performance = get_bot_performance_summary(
    bot_id=1,  # Specific bot or None for all
    days_back=30
)

print(f"Total trades: {performance['trade_count']}")
print(f"Win rate: {performance['win_rate']:.1f}%")
print(f"Total P&L: ${performance['bot_stats']['total_pnl']:.2f}")
```

## Enhanced Bot Layers

Enhanced layers provide automatic data fetching and bot-specific visualization features.

### BotIntegratedSignalLayer

```python
from components.charts.layers import BotIntegratedSignalLayer, BotSignalLayerConfig

# Configure bot-integrated signal layer
config = BotSignalLayerConfig(
    name="BTCUSDT Bot Signals",
    auto_fetch_data=True,              # Automatically fetch from database
    time_window_days=7,                # Look back 7 days
    active_bots_only=True,             # Only active bots
    include_bot_info=True,             # Include bot info in hover
    group_by_strategy=True,            # Group signals by strategy
    confidence_threshold=0.3,          # Minimum confidence
    signal_types=['buy', 'sell']       # Signal types to show
)

layer = BotIntegratedSignalLayer(config)

# Render automatically fetches data
fig = layer.render(fig, market_data, symbol='BTCUSDT')
```

### BotIntegratedTradeLayer

```python
from components.charts.layers import BotIntegratedTradeLayer, BotTradeLayerConfig

config = BotTradeLayerConfig(
    name="BTCUSDT Bot Trades",
    auto_fetch_data=True,
    time_window_days=7,
    show_pnl=True,                     # Show profit/loss
    show_trade_lines=True,             # Connect entry/exit
    include_bot_info=True,             # Bot info in hover
    group_by_strategy=False
)

layer = BotIntegratedTradeLayer(config)
fig = layer.render(fig, market_data, symbol='BTCUSDT')
```

## Multi-Bot Visualization

### Strategy Comparison

```python
from components.charts.layers import bot_multi_layer

# Compare multiple strategies on the same symbol
result = bot_multi_layer.create_strategy_comparison_layers(
    symbol='BTCUSDT',
    strategies=['momentum', 'ema_crossover', 'mean_reversion'],
    timeframe='1h',
    time_window_days=14
)

if result['success']:
    for strategy in result['strategies']:
        signal_layer = result['layers'][f"{strategy}_signals"]
        trade_layer = result['layers'][f"{strategy}_trades"]
        
        fig = signal_layer.render(fig, market_data, symbol='BTCUSDT')
        fig = trade_layer.render(fig, market_data, symbol='BTCUSDT')
```

### Multi-Symbol Bot View

```python
# Create bot layers for multiple symbols
symbols = ['BTCUSDT', 'ETHUSDT', 'ADAUSDT']

for symbol in symbols:
    bot_layers = create_complete_bot_layers(
        symbol=symbol,
        active_only=True,
        time_window_days=7
    )
    
    if bot_layers['success']:
        # Add layers to respective charts
        signal_layer = bot_layers['layers']['signals']
        # ... render on symbol-specific chart
```

## Configuration Options

### Auto-Fetch Configuration

```python
# Disable auto-fetch for manual data control
config = BotSignalLayerConfig(
    name="Manual Bot Signals",
    auto_fetch_data=False,             # Disable auto-fetch
    active_bots_only=True
)

layer = BotIntegratedSignalLayer(config)

# Manually provide signal data
manual_signals = get_signals_from_api()
fig = layer.render(fig, market_data, signals=manual_signals)
```

### Time Window Management

```python
# Custom time window
config = BotSignalLayerConfig(
    name="Short Term Signals",
    time_window_days=1,                # Last 24 hours only
    active_bots_only=True,
    confidence_threshold=0.7           # High confidence only
)
```

### Bot-Specific Filtering

```python
# Filter for specific bots
bot_filter = BotFilterConfig(
    bot_ids=[1, 2, 5],                # Specific bot IDs
    symbols=['BTCUSDT'],
    active_only=True
)

config = BotSignalLayerConfig(
    name="Selected Bots",
    bot_filter=bot_filter,
    include_bot_info=True
)
```

## Examples

### Dashboard Integration Example

```python
# dashboard/callbacks/charts.py
from components.charts.layers import (
    create_bot_signal_layer, 
    create_bot_trade_layer,
    get_active_bot_signals
)

@app.callback(
    Output('chart', 'figure'),
    [Input('symbol-dropdown', 'value'),
     Input('show-bot-signals', 'value')]
)
def update_chart_with_bots(symbol, show_bot_signals):
    fig = create_base_chart(symbol)
    
    if 'bot-signals' in show_bot_signals:
        # Add bot signals
        signal_layer = create_bot_signal_layer(
            symbol=symbol,
            active_only=True,
            confidence_threshold=0.3
        )
        fig = signal_layer.render(fig, market_data, symbol=symbol)
        
        # Add bot trades
        trade_layer = create_bot_trade_layer(
            symbol=symbol,
            active_only=True,
            show_pnl=True
        )
        fig = trade_layer.render(fig, market_data, symbol=symbol)
    
    return fig
```

### Custom Bot Analysis

```python
# Custom analysis for specific strategy
def analyze_momentum_strategy(symbol: str, days_back: int = 30):
    """Analyze momentum strategy performance for a symbol."""
    
    # Get momentum bot signals
    signals = get_bot_signals_by_strategy(
        strategy_name='momentum',
        symbol=symbol,
        days_back=days_back
    )
    
    # Get performance summary
    performance = get_bot_performance_summary(days_back=days_back)
    
    # Create visualizations
    signal_layer = create_bot_signal_layer(
        symbol=symbol,
        active_only=False,  # Include all momentum bots
        time_window_days=days_back
    )
    
    return {
        'signals': signals,
        'performance': performance,
        'layer': signal_layer
    }

# Usage
analysis = analyze_momentum_strategy('BTCUSDT', days_back=14)
```

### Real-time Monitoring Setup

```python
# Real-time bot monitoring dashboard component
def create_realtime_bot_monitor(symbols: List[str]):
    """Create real-time bot monitoring charts."""
    
    charts = {}
    
    for symbol in symbols:
        # Get latest bot data
        active_signals = get_active_bot_signals(
            symbol=symbol,
            days_back=1,  # Last 24 hours
            min_confidence=0.5
        )
        
        # Create monitoring layers
        signal_layer = create_bot_signal_layer(
            symbol=symbol,
            active_only=True,
            time_window_days=1
        )
        
        trade_layer = create_bot_trade_layer(
            symbol=symbol,
            active_only=True,
            show_pnl=True,
            time_window_days=1
        )
        
        charts[symbol] = {
            'signal_layer': signal_layer,
            'trade_layer': trade_layer,
            'active_signals': len(active_signals)
        }
    
    return charts
```

## Best Practices

### Performance Optimization

```python
# 1. Use appropriate time windows
config = BotSignalLayerConfig(
    time_window_days=7,  # Don't fetch more data than needed
    confidence_threshold=0.3  # Filter low-confidence signals
)

# 2. Filter by active bots only when possible
bot_filter = BotFilterConfig(
    active_only=True,    # Reduces database queries
    symbols=['BTCUSDT']  # Specific symbols only
)

# 3. Reuse integration instances
integration = BotSignalLayerIntegration()  # Create once
# Use multiple times for different symbols
```

### Error Handling

```python
try:
    bot_layers = create_complete_bot_layers('BTCUSDT')
    
    if not bot_layers['success']:
        logger.warning(f"Bot layer creation failed: {bot_layers.get('error')}")
        # Fallback to manual signal layer
        signal_layer = TradingSignalLayer()
    else:
        signal_layer = bot_layers['layers']['signals']
        
except Exception as e:
    logger.error(f"Bot integration error: {e}")
    # Graceful degradation
    signal_layer = TradingSignalLayer()
```

### Database Connection Management

```python
# The bot integration handles database connections automatically
# But for custom queries, follow these patterns:

from database.connection import get_session

def custom_bot_query():
    try:
        with get_session() as session:
            # Your database operations
            result = session.query(Bot).filter(...).all()
            return result
    except Exception as e:
        logger.error(f"Database query failed: {e}")
        return []
```

## Troubleshooting

### Common Issues

1. **No signals showing on chart**
   ```python
   # Check if bots exist for symbol
   service = BotDataService()
   bots = service.get_bots(BotFilterConfig(symbols=['BTCUSDT']))
   print(f"Found {len(bots)} bots for BTCUSDT")
   
   # Check signal count
   signals = get_active_bot_signals('BTCUSDT', days_back=7)
   print(f"Found {len(signals)} signals in last 7 days")
   ```

2. **Database connection errors**
   ```python
   # Test database connection
   try:
       from database.connection import get_session
       with get_session() as session:
           print("Database connection successful")
   except Exception as e:
       print(f"Database connection failed: {e}")
   ```

3. **Performance issues with large datasets**
   ```python
   # Reduce time window
   config = BotSignalLayerConfig(
       time_window_days=3,  # Reduced from 7
       confidence_threshold=0.5  # Higher threshold
   )
   
   # Filter by specific strategies
   bot_filter = BotFilterConfig(
       strategies=['momentum'],  # Specific strategy only
       active_only=True
   )
   ```

### Debug Mode

```python
import logging

# Enable debug logging for bot integration
logging.getLogger('bot_integration').setLevel(logging.DEBUG)
logging.getLogger('bot_enhanced_layers').setLevel(logging.DEBUG)

# This will show detailed information about:
# - Database queries
# - Data fetching operations
# - Filter applications
# - Performance metrics
```

### Testing Bot Integration

```python
# Test bot integration components
from tests.test_signal_layers import TestBotIntegration

# Run specific bot integration tests
pytest.main(['-v', 'tests/test_signal_layers.py::TestBotIntegration'])

# Test with mock data
def test_bot_integration():
    config = BotSignalLayerConfig(
        name="Test Bot Signals",
        auto_fetch_data=False  # Use manual data for testing
    )
    
    layer = BotIntegratedSignalLayer(config)
    
    # Provide test data
    test_signals = pd.DataFrame({
        'timestamp': [datetime.now()],
        'signal_type': ['buy'],
        'price': [50000],
        'confidence': [0.8],
        'bot_name': ['Test Bot']
    })
    
    fig = go.Figure()
    result = layer.render(fig, market_data, signals=test_signals)
    
    assert len(result.data) > 0
```

## API Reference

### Core Classes

- **`BotDataService`** - Main service for database operations
- **`BotSignalLayerIntegration`** - Chart-specific integration utilities
- **`BotIntegratedSignalLayer`** - Auto-fetching signal layer
- **`BotIntegratedTradeLayer`** - Auto-fetching trade layer
- **`BotMultiLayerIntegration`** - Multi-bot layer management

### Configuration Classes

- **`BotFilterConfig`** - Bot filtering configuration
- **`BotSignalLayerConfig`** - Signal layer configuration with bot options
- **`BotTradeLayerConfig`** - Trade layer configuration with bot options

### Convenience Functions

- **`create_bot_signal_layer()`** - Quick bot signal layer creation
- **`create_bot_trade_layer()`** - Quick bot trade layer creation
- **`create_complete_bot_layers()`** - Complete bot layer set
- **`get_active_bot_signals()`** - Get signals from active bots
- **`get_active_bot_trades()`** - Get trades from active bots
- **`get_bot_signals_by_strategy()`** - Get signals by strategy
- **`get_bot_performance_summary()`** - Get performance analytics

For complete API documentation, see the module docstrings in:
- `components/charts/layers/bot_integration.py`
- `components/charts/layers/bot_enhanced_layers.py`