Refactor technical indicators to return DataFrames and enhance documentation

- Updated all technical indicators to return pandas DataFrames instead of lists, improving consistency and usability. - Modified the `calculate` method in `TechnicalIndicators` to directly return DataFrames with relevant indicator values. - Enhanced the `data_integration.py` to utilize the new DataFrame outputs for better integration with charting. - Updated documentation to reflect the new DataFrame-centric approach, including usage examples and output structures. - Improved error handling to ensure empty DataFrames are returned when insufficient data is available. These changes streamline the indicator calculations and improve the overall architecture, aligning with project standards for maintainability and performance.
2025-06-09 16:28:16 +08:00
parent fc3cac24bd
commit ec8f5514bb
14 changed files with 542 additions and 542 deletions
--- a/docs/modules/charts/indicators.md
+++ b/docs/modules/charts/indicators.md
@@ -4,6 +4,13 @@

 The Crypto Trading Bot Dashboard features a comprehensive modular indicator system that allows users to create, customize, and manage technical indicators for chart analysis. The system supports both overlay indicators (displayed on the main price chart) and subplot indicators (displayed in separate panels below the main chart).

+**Key Features:**
+- **Vectorized Calculations**: High-performance pandas-based indicator calculations
+- **Clean DataFrame Output**: Returns only relevant indicator columns with timestamp index
+- **Safe Trading**: Proper warm-up periods ensure no early/invalid values
+- **Gap Handling**: Maintains timestamp alignment without interpolation
+- **Real-time Integration**: Seamless integration with chart visualization
+
 ## Table of Contents

 1. [System Architecture](#system-architecture)
@@ -23,6 +30,7 @@ The Crypto Trading Bot Dashboard features a comprehensive modular indicator syst
 components/charts/
 ├── indicator_manager.py          # Core indicator CRUD operations
 ├── indicator_defaults.py         # Default indicator templates
+├── data_integration.py          # DataFrame-based indicator calculations
 ├── layers/
 │   ├── indicators.py            # Overlay indicator rendering
 │   └── subplots.py             # Subplot indicator rendering
@@ -34,6 +42,15 @@ config/indicators/
    ├── sma_abc123.json
    ├── ema_def456.json
    └── ...
+
+data/common/indicators/
+├── technical.py                 # Vectorized indicator calculations
+├── implementations/             # Individual indicator implementations
+    ├── sma.py                  # Simple Moving Average
+    ├── ema.py                  # Exponential Moving Average
+    ├── rsi.py                  # Relative Strength Index
+    ├── macd.py                 # MACD
+    └── bollinger.py            # Bollinger Bands
 ```

 ### Key Classes
@@ -41,6 +58,7 @@ config/indicators/
 - **`IndicatorManager`**: Handles CRUD operations for user indicators
 - **`UserIndicator`**: Data structure for indicator configuration
 - **`IndicatorStyling`**: Appearance and styling configuration
+- **`TechnicalIndicators`**: Vectorized calculation engine
 - **Indicator Layers**: Rendering classes for different indicator types

 ## Current Indicators
@@ -48,19 +66,19 @@ config/indicators/
 ### Overlay Indicators
 These indicators are displayed directly on the price chart:

-| Indicator | Type | Parameters | Description |
-|-----------|------|------------|-------------|
-| **Simple Moving Average (SMA)** | `sma` | `period` (1-200) | Average price over N periods |
-| **Exponential Moving Average (EMA)** | `ema` | `period` (1-200) | Weighted average giving more weight to recent prices |
-| **Bollinger Bands** | `bollinger_bands` | `period` (5-100), `std_dev` (0.5-5.0) | Price channels based on standard deviation |
+| Indicator | Type | Parameters | Description | Output Columns |
+|-----------|------|------------|-------------|----------------|
+| **Simple Moving Average (SMA)** | `sma` | `period` (1-200) | Average price over N periods | `['sma']` |
+| **Exponential Moving Average (EMA)** | `ema` | `period` (1-200) | Weighted average giving more weight to recent prices | `['ema']` |
+| **Bollinger Bands** | `bollinger_bands` | `period` (5-100), `std_dev` (0.5-5.0) | Price channels based on standard deviation | `['upper_band', 'middle_band', 'lower_band']` |

 ### Subplot Indicators
 These indicators are displayed in separate panels:

-| Indicator | Type | Parameters | Description |
-|-----------|------|------------|-------------|
-| **Relative Strength Index (RSI)** | `rsi` | `period` (2-50) | Momentum oscillator (0-100 scale) |
-| **MACD** | `macd` | `fast_period` (2-50), `slow_period` (5-100), `signal_period` (2-30) | Moving average convergence divergence |
+| Indicator | Type | Parameters | Description | Output Columns |
+|-----------|------|------------|-------------|----------------|
+| **Relative Strength Index (RSI)** | `rsi` | `period` (2-50) | Momentum oscillator (0-100 scale) | `['rsi']` |
+| **MACD** | `macd` | `fast_period` (2-50), `slow_period` (5-100), `signal_period` (2-30) | Moving average convergence divergence | `['macd', 'signal', 'histogram']` |

 ## User Interface

@@ -212,6 +230,20 @@ class IndicatorManager:
    def get_indicators_by_type(self, display_type: str) -> List[UserIndicator]
 ```

+### TechnicalIndicators Class (Vectorized Calculations)
+
+```python
+class TechnicalIndicators:
+    def sma(self, df: pd.DataFrame, period: int, price_column: str = 'close') -> pd.DataFrame
+    def ema(self, df: pd.DataFrame, period: int, price_column: str = 'close') -> pd.DataFrame
+    def rsi(self, df: pd.DataFrame, period: int = 14, price_column: str = 'close') -> pd.DataFrame
+    def macd(self, df: pd.DataFrame, fast_period: int = 12, slow_period: int = 26, 
+             signal_period: int = 9, price_column: str = 'close') -> pd.DataFrame
+    def bollinger_bands(self, df: pd.DataFrame, period: int = 20, std_dev: float = 2.0, 
+                       price_column: str = 'close') -> pd.DataFrame
+    def calculate(self, indicator_type: str, df: pd.DataFrame, **kwargs) -> Optional[pd.DataFrame]
+```
+
 ### Usage Examples

 ```python
@@ -246,6 +278,86 @@ all_indicators = manager.list_indicators()
 # Get by type
 overlay_indicators = manager.get_indicators_by_type("overlay")
 subplot_indicators = manager.get_indicators_by_type("subplot")
+
+# Calculate indicators (vectorized)
+from data.common.indicators import TechnicalIndicators
+indicators = TechnicalIndicators()
+
+# Calculate SMA - returns DataFrame with 'sma' column
+sma_df = indicators.sma(df, period=20)
+print(f"SMA values: {sma_df['sma'].tolist()}")
+
+# Calculate MACD - returns DataFrame with 'macd', 'signal', 'histogram' columns
+macd_df = indicators.macd(df, fast_period=12, slow_period=26, signal_period=9)
+print(f"MACD signal: {macd_df['signal'].iloc[-1]}")
+```
+
+## Data Integration and Chart Rendering
+
+### DataFrame-Based Calculations
+
+The system now uses vectorized DataFrame calculations for high performance:
+
+```python
+# In data_integration.py
+def get_indicator_data(self, main_df: pd.DataFrame, indicator_configs: List[UserIndicator]) -> Dict[str, pd.DataFrame]:
+    """
+    Calculate indicator data using vectorized operations.
+    
+    Returns:
+        Dict mapping indicator_id to DataFrame with indicator values
+    """
+    indicator_data_map = {}
+    
+    for config in indicator_configs:
+        indicator = self.indicator_manager.load_indicator(config.id)
+        if not indicator:
+            continue
+            
+        # Calculate using vectorized methods
+        result_df = self.indicators.calculate(
+            indicator.type, 
+            indicator_df, 
+            **indicator.parameters
+        )
+        
+        if result_df is not None and not result_df.empty:
+            indicator_data_map[config.id] = result_df
+    
+    return indicator_data_map
+```
+
+### Chart Integration
+
+Indicators are rendered using their clean DataFrame output:
+
+```python
+# In layers/indicators.py
+def create_traces(self, indicator_data: pd.DataFrame, styling: IndicatorStyling) -> List[go.Scatter]:
+    """
+    Create plotly traces from indicator DataFrame.
+    """
+    traces = []
+    
+    # For SMA/EMA - single line
+    if 'sma' in indicator_data.columns:
+        traces.append(go.Scatter(
+            x=indicator_data.index,
+            y=indicator_data['sma'],
+            name=self.indicator.name,
+            line=dict(color=styling.color, width=styling.line_width),
+            opacity=styling.opacity
+        ))
+    
+    # For MACD - multiple lines
+    elif 'macd' in indicator_data.columns:
+        traces.extend([
+            go.Scatter(x=indicator_data.index, y=indicator_data['macd'], name='MACD'),
+            go.Scatter(x=indicator_data.index, y=indicator_data['signal'], name='Signal'),
+            go.Bar(x=indicator_data.index, y=indicator_data['histogram'], name='Histogram')
+        ])
+    
+    return traces
 ```

 ## Troubleshooting
@@ -263,8 +375,14 @@ subplot_indicators = manager.get_indicators_by_type("subplot")
 3. **Chart not updating**
   - Verify the indicator layer implements `calculate_values` and `create_traces`
   - Check if indicator is registered in the correct registry
+   - Ensure DataFrame output is not empty

-4. **File permission errors**
+4. **Empty indicator results**
+   - Check if DataFrame has sufficient data (warm-up periods)
+   - Verify timestamp column is present and properly formatted
+   - Check for gaps in data that might affect calculations
+
+5. **File permission errors**
   - Ensure `config/indicators/user_indicators/` directory is writable
   - Check file permissions on existing JSON files

@@ -274,14 +392,26 @@ subplot_indicators = manager.get_indicators_by_type("subplot")
 - Look at application logs for Python exceptions
 - Verify JSON file structure with a validator
 - Test indicator calculations with sample data
+- Check DataFrame structure and column names

 ### Performance Considerations

+- **Vectorized calculations** provide significant performance improvements
 - Indicators with large periods may take longer to calculate
 - Consider data availability when setting parameter limits
 - Subplot indicators require additional chart space
 - Real-time updates may impact performance with many indicators

+### Warm-up Periods
+
+All indicators implement proper warm-up periods for safe trading:
+
+- **SMA/EMA/RSI/BB**: First `period-1` values are excluded
+- **MACD**: First `max(slow_period, signal_period)-1` values are excluded
+- **Result**: Only reliable, fully-calculated values are returned
+
+This ensures that no early/invalid values are used for trading decisions.
+
 ## Best Practices

 1. **Naming Conventions**
@@ -300,6 +430,7 @@ subplot_indicators = manager.get_indicators_by_type("subplot")
   - Log errors for debugging

 4. **Performance**
+   - Leverage vectorized calculations for speed
   - Cache calculated values when possible
   - Optimize calculation algorithms
   - Limit the number of active indicators
@@ -307,7 +438,14 @@ subplot_indicators = manager.get_indicators_by_type("subplot")
 5. **User Experience**
   - Provide immediate visual feedback
   - Use intuitive color schemes
-   - Group related indicators logically 
+   - Group related indicators logically
+
+6. **Data Handling**
+   - Respect warm-up periods for safe trading
+   - Handle data gaps without interpolation
+   - Maintain timestamp alignment
+   - Use clean DataFrame output for plotting
+
 ---

 *Back to [Chart System Documentation (`README.md`)]*