-`tests/database/test_strategy_repository.py` - Unit tests for strategy repository
### Notes
- **Strict Adherence to Indicator Patterns**: The strategy engine components (BaseStrategy, StrategyFactory, StrategyManager, Strategy implementations, and configurations) MUST strictly mirror the existing `data/common/indicators/` module's structure, factory approach, and configuration management. This ensures consistency and simplifies development.
- **Database Segregation for Signals**: The newly created `strategy_signals` table is exclusively for strategy analysis and backtesting results, distinct from the existing `signals` table which is for live bot trading operations. Maintain this clear separation.
- **Initial Full Recalculation**: For real-time strategy execution, strategies will initially recalculate completely on each new candle, similar to how technical indicators currently operate. Optimizations for incremental updates can be considered in a later phase.
- **Multi-timeframe Support**: Strategies should be designed to support and utilize market data from multiple timeframes, following the pattern established by indicators that can consume data from different timeframes.
- **Exclusive Use of Repository Pattern**: All database interactions, including storing and retrieving strategy signals and run data, must be performed exclusively through the `StrategyRepository` and other existing repositories. Avoid raw SQL queries.
- **JSON-based Configuration**: Strategy parameters and configurations are to be managed via JSON files within `config/strategies/`, aligning with the existing configuration system for indicators and other components.
- **Layered Chart Integration**: Strategy signals and performance visualizations will be integrated into the dashboard as a new chart layer, utilizing the existing modular chart system.
- **Comprehensive Testing**: Ensure that all new classes, functions, and modules within the strategy engine have corresponding unit tests placed in the `tests/strategies/` directory, following established testing conventions.
- **Decision**: Refactored strategy signal detection to use vectorized Pandas operations (e.g., `shift()`, boolean indexing) instead of iterative Python loops.
- **Reasoning**: Significantly improves performance for signal generation, especially with large datasets, while maintaining identical results as verified by dedicated tests.
- **Impact**: All core strategy implementations (EMA Crossover, RSI, MACD) now leverage vectorized functions for their primary signal detection logic.
### 2. Indicator Key Generation Consistency
- **Decision**: Centralized the `_create_indicator_key` logic into a shared utility function `create_indicator_key()` in `strategies/utils.py`.
- **Reasoning**: Eliminates code duplication in `StrategyFactory` and individual strategy implementations, ensuring consistent key generation and easier maintenance if indicator naming conventions change.
- **Impact**: `StrategyFactory` and all strategy implementations now use this shared utility for generating unique indicator keys.
- **Decision**: The `calculate_multiple_strategies` method was removed from `strategies/factory.py`.
- **Reasoning**: This functionality is not immediately required for the current phase of development and can be re-introduced later when needed, to simplify the codebase and testing efforts.
- **Impact**: The `StrategyFactory` now focuses on calculating signals for individual strategies, simplifying its interface and reducing initial complexity.
### 4. `strategy_name` in Concrete Strategy `__init__`
- **Decision**: Updated the `__init__` methods of concrete strategy implementations (e.g., `EMAStrategy`, `RSIStrategy`, `MACDStrategy`) to accept and pass `strategy_name` to `BaseStrategy.__init__`.
- **Reasoning**: Ensures consistency with the `BaseStrategy` abstract class, which now requires `strategy_name` during initialization, providing a clear identifier for each strategy instance.
- **Impact**: All strategy implementations now correctly initialize their `strategy_name` via the base class, standardizing strategy identification across the engine.
### 5. Database Schema Design for Strategy Analysis
- **Decision**: Created separate `strategy_signals` and `strategy_runs` tables distinct from the existing `signals` table used for bot operations.
- **Reasoning**: Maintains clear separation between live bot trading signals and strategy analysis/backtesting data, avoiding conflicts and ensuring data integrity for different use cases.
- **Impact**: Strategy analysis can be performed independently without affecting live trading operations, with dedicated tables optimized for analytical queries and data retention policies.
### 6. Repository Pattern Integration
- **Decision**: Implemented `StrategyRepository` following the established `BaseRepository` pattern and integrated it into the centralized `DatabaseOperations` class.
- **Reasoning**: Maintains consistency with existing database access patterns, ensures proper session management, and provides a clean API for strategy data operations.
- **Impact**: All strategy database operations follow the same patterns as other modules, with proper error handling, logging, and transaction management.
- **Decision**: Implement vectorized approaches in `StrategyDataIntegrator` for DataFrame construction, indicator batching, and multi-strategy processing while maintaining iterative interfaces for backward compatibility.
- **Reasoning**: Significant performance improvements for backtesting and bulk analysis scenarios, better memory efficiency with pandas operations, and preparation for multi-strategy batch processing capabilities.
- **Impact**: Enhanced performance for large datasets while maintaining existing single-strategy interfaces. Sets foundation for efficient multi-strategy and multi-timeframe processing in future phases.
### 8. Single-Strategy Orchestration Focus
- **Decision**: Implement strategy calculation orchestration focused on single-strategy optimization with indicator dependency resolution, avoiding premature multi-strategy complexity.
- **Reasoning**: Multi-strategy coordination is better handled at the backtesting layer or through parallelization. Single-strategy optimization provides immediate benefits while keeping code maintainable and focused.
- **Impact**: Cleaner, more maintainable code with optimized single-strategy performance. Provides foundation for future backtester-level parallelization without architectural complexity.
### 9. Indicator Warm-up Handling for Streaming Batch Processing
- **Decision**: Implemented dynamic warm-up period calculation and overlapping windows with result trimming for streaming batch processing.
- **Reasoning**: To ensure accurate indicator calculations and prevent false signals when processing large datasets in chunks, as indicators require a certain amount of historical data to 'warm up'.
- **Impact**: Guarantees correct backtest results for strategies relying on indicators with warm-up periods, even when using memory-efficient streaming. Automatically adjusts chunk processing to include necessary historical context and removes duplicate/invalid initial signals.
### 10. Real-time Strategy Execution Architecture
- **Decision**: Implemented event-driven real-time strategy execution pipeline with signal broadcasting, chart integration, and concurrent processing capabilities.
- **Reasoning**: Real-time strategy execution requires different architecture than batch processing - event-driven triggers, background signal processing, throttled chart updates, and integration with existing dashboard refresh cycles.
- **Impact**: Enables live strategy signal generation that integrates seamlessly with the existing chart system. Provides concurrent strategy execution, real-time signal storage, error handling with automatic strategy disabling, and performance monitoring for production use.
### 11. Exclusion of "HOLD" Signals from All Signal Storage
- **Decision**: Modified signal storage mechanisms across both real-time and batch processing to exclude "HOLD" signals, only persisting explicit "BUY" or "SELL" signals in the database.
- **Reasoning**: To significantly reduce data volume and improve storage/query performance across all execution types, as the absence of a BUY/SELL signal implicitly means "HOLD" and can be inferred during analysis or visualization.
- **Impact**: Leads to more efficient database usage and faster data retrieval for all stored signals. Requires careful handling in visualization and backtesting components to correctly interpret gaps as "HOLD" periods.
