TCPDashboard/docs/modules/charts/indicators.md

# Indicator System Documentation

## Overview

The Crypto Trading Bot Dashboard features a comprehensive modular indicator system that allows users to create, customize, and manage technical indicators for chart analysis. The system supports both overlay indicators (displayed on the main price chart) and subplot indicators (displayed in separate panels below the main chart).

**Key Features:**
- **Vectorized Calculations**: High-performance pandas-based indicator calculations
- **Clean DataFrame Output**: Returns only relevant indicator columns with timestamp index
- **Safe Trading**: Proper warm-up periods ensure no early/invalid values
- **Gap Handling**: Maintains timestamp alignment without interpolation
- **Real-time Integration**: Seamless integration with chart visualization

## Table of Contents

1. [System Architecture](#system-architecture)
2. [Current Indicators](#current-indicators)
3. [User Interface](#user-interface)
4. [File Structure](#file-structure)
5. [Adding New Indicators](#adding-new-indicators)
6. [Configuration Format](#configuration-format)
7. [API Reference](#api-reference)
8. [Troubleshooting](#troubleshooting)

## System Architecture

### Core Components

```
components/charts/
├── indicator_manager.py          # Core indicator CRUD operations
├── indicator_defaults.py         # Default indicator templates
├── data_integration.py          # DataFrame-based indicator calculations
├── layers/
│   ├── indicators.py            # Overlay indicator rendering
│   └── subplots.py             # Subplot indicator rendering
└── config/
    └── indicator_defs.py        # Indicator definitions and schemas

config/indicators/
└── user_indicators/             # User-created indicators (JSON files)
    ├── sma_abc123.json
    ├── ema_def456.json
    └── ...

data/common/indicators/
├── technical.py                 # Vectorized indicator calculations
├── implementations/             # Individual indicator implementations
    ├── sma.py                  # Simple Moving Average
    ├── ema.py                  # Exponential Moving Average
    ├── rsi.py                  # Relative Strength Index
    ├── macd.py                 # MACD
    └── bollinger.py            # Bollinger Bands
```

### Key Classes

- **`IndicatorManager`**: Handles CRUD operations for user indicators
- **`UserIndicator`**: Data structure for indicator configuration
- **`IndicatorStyling`**: Appearance and styling configuration
- **`TechnicalIndicators`**: Vectorized calculation engine
- **Indicator Layers**: Rendering classes for different indicator types

## Current Indicators

### Overlay Indicators
These indicators are displayed directly on the price chart:

| Indicator | Type | Parameters | Description | Output Columns |
|-----------|------|------------|-------------|----------------|
| **Simple Moving Average (SMA)** | `sma` | `period` (1-200) | Average price over N periods | `['sma']` |
| **Exponential Moving Average (EMA)** | `ema` | `period` (1-200) | Weighted average giving more weight to recent prices | `['ema']` |
| **Bollinger Bands** | `bollinger_bands` | `period` (5-100), `std_dev` (0.5-5.0) | Price channels based on standard deviation | `['upper_band', 'middle_band', 'lower_band']` |

### Subplot Indicators
These indicators are displayed in separate panels:

| Indicator | Type | Parameters | Description | Output Columns |
|-----------|------|------------|-------------|----------------|
| **Relative Strength Index (RSI)** | `rsi` | `period` (2-50) | Momentum oscillator (0-100 scale) | `['rsi']` |
| **MACD** | `macd` | `fast_period` (2-50), `slow_period` (5-100), `signal_period` (2-30) | Moving average convergence divergence | `['macd', 'signal', 'histogram']` |

## User Interface

### Adding Indicators

1. **Click "➕ Add New Indicator"** button
2. **Configure Basic Settings**:
   - Name: Custom name for the indicator
   - Type: Select from available indicator types
   - Description: Optional description
3. **Set Parameters**: Type-specific parameters appear dynamically
4. **Customize Styling**:
   - Color: Hex color code
   - Line Width: 1-5 pixels
5. **Save**: Creates a new JSON file and updates the UI

### Managing Indicators

- **✅ Checkboxes**: Toggle indicator visibility on chart
- **✏️ Edit Button**: Modify existing indicator settings
- **🗑️ Delete Button**: Remove indicator permanently

### Real-time Updates

- Chart updates automatically when indicators are toggled
- Changes are saved immediately to JSON files
- No page refresh required

## File Structure

### Indicator JSON Format

```json
{
  "id": "ema_ca5fd53d",
  "name": "EMA 10",
  "description": "10-period Exponential Moving Average for fast signals",
  "type": "ema",
  "display_type": "overlay",
  "parameters": {
    "period": 10
  },
  "styling": {
    "color": "#ff6b35",
    "line_width": 2,
    "opacity": 1.0,
    "line_style": "solid"
  },
  "visible": true,
  "created_date": "2025-06-04T04:16:35.455729+00:00",
  "modified_date": "2025-06-04T04:54:49.608549+00:00"
}
```

### Directory Structure

```
config/indicators/
└── user_indicators/
    ├── sma_abc123.json          # Individual indicator files
    ├── ema_def456.json
    ├── rsi_ghi789.json
    └── macd_jkl012.json
```

## Adding New Indicators

For developers who want to add new indicator types to the system, please refer to the comprehensive step-by-step guide:

**📋 [Quick Guide: Adding New Indicators (`adding-new-indicators.md`)](./adding-new-indicators.md)**

This guide covers:
- ✅ Complete 11-step implementation checklist
- ✅ Full code examples (Stochastic Oscillator implementation)
- ✅ File modification requirements
- ✅ Testing checklist and common patterns
- ✅ Tips and best practices

## Configuration Format

### User Indicator Structure

```python
@dataclass
class UserIndicator:
    id: str                    # Unique identifier
    name: str                  # Display name
    description: str           # User description
    type: str                  # Indicator type (sma, ema, etc.)
    display_type: str          # "overlay" or "subplot"
    parameters: Dict[str, Any] # Type-specific parameters
    styling: IndicatorStyling  # Appearance settings
    visible: bool = True       # Default visibility
    created_date: datetime     # Creation timestamp
    modified_date: datetime    # Last modification timestamp
```

### Styling Options

```python
@dataclass
class IndicatorStyling:
    color: str = "#007bff"        # Hex color code
    line_width: int = 2           # Line thickness (1-5)
    opacity: float = 1.0          # Transparency (0.0-1.0)
    line_style: str = "solid"     # Line style
```

### Parameter Examples

```python
# SMA/EMA Parameters
{"period": 20}

# RSI Parameters
{"period": 14}

# MACD Parameters
{
    "fast_period": 12,
    "slow_period": 26,
    "signal_period": 9
}

# Bollinger Bands Parameters
{
    "period": 20,
    "std_dev": 2.0
}
```

## API Reference

### IndicatorManager Class

```python
class IndicatorManager:
    def create_indicator(self, name: str, indicator_type: str, 
                        parameters: Dict[str, Any], **kwargs) -> Optional[UserIndicator]
    
    def load_indicator(self, indicator_id: str) -> Optional[UserIndicator]
    
    def update_indicator(self, indicator_id: str, **kwargs) -> bool
    
    def delete_indicator(self, indicator_id: str) -> bool
    
    def list_indicators(self) -> List[UserIndicator]
    
    def get_indicators_by_type(self, display_type: str) -> List[UserIndicator]
```

### TechnicalIndicators Class (Vectorized Calculations)

```python
class TechnicalIndicators:
    def sma(self, df: pd.DataFrame, period: int, price_column: str = 'close') -> pd.DataFrame
    def ema(self, df: pd.DataFrame, period: int, price_column: str = 'close') -> pd.DataFrame
    def rsi(self, df: pd.DataFrame, period: int = 14, price_column: str = 'close') -> pd.DataFrame
    def macd(self, df: pd.DataFrame, fast_period: int = 12, slow_period: int = 26, 
             signal_period: int = 9, price_column: str = 'close') -> pd.DataFrame
    def bollinger_bands(self, df: pd.DataFrame, period: int = 20, std_dev: float = 2.0, 
                       price_column: str = 'close') -> pd.DataFrame
    def calculate(self, indicator_type: str, df: pd.DataFrame, **kwargs) -> Optional[pd.DataFrame]
```

### Usage Examples

```python
# Get indicator manager
manager = get_indicator_manager()

# Create new indicator
indicator = manager.create_indicator(
    name="My SMA 50",
    indicator_type="sma",
    parameters={"period": 50},
    description="50-period Simple Moving Average",
    color="#ff0000"
)

# Load indicator
loaded = manager.load_indicator("sma_abc123")

# Update indicator
success = manager.update_indicator(
    "sma_abc123",
    name="Updated SMA",
    parameters={"period": 30}
)

# Delete indicator
deleted = manager.delete_indicator("sma_abc123")

# List all indicators
all_indicators = manager.list_indicators()

# Get by type
overlay_indicators = manager.get_indicators_by_type("overlay")
subplot_indicators = manager.get_indicators_by_type("subplot")

# Calculate indicators (vectorized)
from data.common.indicators import TechnicalIndicators
indicators = TechnicalIndicators()

# Calculate SMA - returns DataFrame with 'sma' column
sma_df = indicators.sma(df, period=20)
print(f"SMA values: {sma_df['sma'].tolist()}")

# Calculate MACD - returns DataFrame with 'macd', 'signal', 'histogram' columns
macd_df = indicators.macd(df, fast_period=12, slow_period=26, signal_period=9)
print(f"MACD signal: {macd_df['signal'].iloc[-1]}")
```

## Data Integration and Chart Rendering

### DataFrame-Based Calculations

The system now uses vectorized DataFrame calculations for high performance:

```python
# In data_integration.py
def get_indicator_data(self, main_df: pd.DataFrame, indicator_configs: List[UserIndicator]) -> Dict[str, pd.DataFrame]:
    """
    Calculate indicator data using vectorized operations.
    
    Returns:
        Dict mapping indicator_id to DataFrame with indicator values
    """
    indicator_data_map = {}
    
    for config in indicator_configs:
        indicator = self.indicator_manager.load_indicator(config.id)
        if not indicator:
            continue
            
        # Calculate using vectorized methods
        result_df = self.indicators.calculate(
            indicator.type, 
            indicator_df, 
            **indicator.parameters
        )
        
        if result_df is not None and not result_df.empty:
            indicator_data_map[config.id] = result_df
    
    return indicator_data_map
```

### Chart Integration

Indicators are rendered using their clean DataFrame output:

```python
# In layers/indicators.py
def create_traces(self, indicator_data: pd.DataFrame, styling: IndicatorStyling) -> List[go.Scatter]:
    """
    Create plotly traces from indicator DataFrame.
    """
    traces = []
    
    # For SMA/EMA - single line
    if 'sma' in indicator_data.columns:
        traces.append(go.Scatter(
            x=indicator_data.index,
            y=indicator_data['sma'],
            name=self.indicator.name,
            line=dict(color=styling.color, width=styling.line_width),
            opacity=styling.opacity
        ))
    
    # For MACD - multiple lines
    elif 'macd' in indicator_data.columns:
        traces.extend([
            go.Scatter(x=indicator_data.index, y=indicator_data['macd'], name='MACD'),
            go.Scatter(x=indicator_data.index, y=indicator_data['signal'], name='Signal'),
            go.Bar(x=indicator_data.index, y=indicator_data['histogram'], name='Histogram')
        ])
    
    return traces
```

## Troubleshooting

### Common Issues

1. **Indicator not appearing in dropdown**
   - Check if registered in `INDICATOR_REGISTRY`
   - Verify the indicator type matches the class name

2. **Parameters not saving**
   - Ensure parameter fields are added to save callback
   - Check parameter collection logic in `save_new_indicator`

3. **Chart not updating**
   - Verify the indicator layer implements `calculate_values` and `create_traces`
   - Check if indicator is registered in the correct registry
   - Ensure DataFrame output is not empty

4. **Empty indicator results**
   - Check if DataFrame has sufficient data (warm-up periods)
   - Verify timestamp column is present and properly formatted
   - Check for gaps in data that might affect calculations

5. **File permission errors**
   - Ensure `config/indicators/user_indicators/` directory is writable
   - Check file permissions on existing JSON files

### Debug Information

- Check browser console for JavaScript errors
- Look at application logs for Python exceptions
- Verify JSON file structure with a validator
- Test indicator calculations with sample data
- Check DataFrame structure and column names

### Performance Considerations

- **Vectorized calculations** provide significant performance improvements
- Indicators with large periods may take longer to calculate
- Consider data availability when setting parameter limits
- Subplot indicators require additional chart space
- Real-time updates may impact performance with many indicators

### Warm-up Periods

All indicators implement proper warm-up periods for safe trading:

- **SMA/EMA/RSI/BB**: First `period-1` values are excluded
- **MACD**: First `max(slow_period, signal_period)-1` values are excluded
- **Result**: Only reliable, fully-calculated values are returned

This ensures that no early/invalid values are used for trading decisions.

## Best Practices

1. **Naming Conventions**
   - Use descriptive names for indicators
   - Include parameter values in names (e.g., "SMA 20")
   - Use consistent naming patterns

2. **Parameter Validation**
   - Set appropriate min/max values for parameters
   - Provide helpful descriptions for parameters
   - Use sensible default values

3. **Error Handling**
   - Handle insufficient data gracefully
   - Provide meaningful error messages
   - Log errors for debugging

4. **Performance**
   - Leverage vectorized calculations for speed
   - Cache calculated values when possible
   - Optimize calculation algorithms
   - Limit the number of active indicators

5. **User Experience**
   - Provide immediate visual feedback
   - Use intuitive color schemes
   - Group related indicators logically

6. **Data Handling**
   - Respect warm-up periods for safe trading
   - Handle data gaps without interpolation
   - Maintain timestamp alignment
   - Use clean DataFrame output for plotting

---

*Back to [Chart System Documentation (`README.md`)]*