vasily/TCPDashboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

4.0 strategies docs

Browse Source
This commit is contained in:
Vasily.onl 2025-06-12 18:30:48 +08:00
parent 8c23489ff0
commit 0d6bfea48a
2 changed files with 169 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
CONTEXT.md
View File

@ -18,7 +18,9 @@ The platform is a **monolithic application** built with Python, designed for pro
- **Real-time Messaging Infrastructure**: Redis-based pub/sub system for component communication, with organized channel structures and both synchronous and asynchronous support.
- **Strategy Engine & Bot Manager**: (⚠️ **NOT YET IMPLEMENTED**) Core trading logic components for strategy execution, bot lifecycle management, and virtual portfolio tracking.
- **Strategy Engine**: (✅ **IMPLEMENTED**) Core trading logic components for strategy execution, including base strategy interface, dynamic loading, signal generation, batch processing, and real-time execution.
- **Bot Manager**: (⚠️ **NOT YET IMPLEMENTED**) Core trading logic components for bot lifecycle management, and virtual portfolio tracking.
- **Backtesting Engine**: (⚠️ **NOT YET IMPLEMENTED**) Historical strategy testing capabilities.
@ -29,7 +31,7 @@ The platform is a **monolithic application** built with Python, designed for pro
4. **Storage**: Both raw trade data and processed OHLCV stored in PostgreSQL
5. **Distribution**: Real-time data distributed via Redis pub/sub channels
6. **Visualization**: Dashboard reads from database for charts and system monitoring
7. **Strategy Processing**: (Future) Strategy engine will consume OHLCV data for signal generation
7. **Strategy Processing**: Strategy engine now consumes OHLCV data for signal generation and execution.
### System Health & Monitoring
The platform includes comprehensive system health monitoring with:
@ -55,7 +57,7 @@ The platform includes comprehensive system health monitoring with:
## 3. Coding Conventions
- **Modular Design**: Code is organized into modules with a clear purpose (e.g., `data`, `database`, `dashboard`). See `architecture.md` for more details.
- **Modular Design**: Code is organized into modules with a clear purpose (e.g., `data`, `database`, `dashboard`, `strategies`). See `architecture.md` for more details.
- **Naming Conventions**:
- **Classes**: `PascalCase` (e.g., `MarketData`, `BaseDataCollector`, `CollectorManager`)
- **Functions & Methods**: `snake_case` (e.g., `get_system_health_layout`, `connect`)
@ -66,7 +68,7 @@ The platform includes comprehensive system health monitoring with:
- **File Organization & Code Structure**:
- **Repository Pattern**: Database operations centralized through repository classes
- **Factory Pattern**: Used for collector creation and management
- **Factory Pattern**: Used for collector creation, strategy loading and management
- **Abstract Base Classes**: Well-defined interfaces for extensibility
- **Dependency Injection**: Configuration and dependencies injected rather than hardcoded
- **Error Handling**: Custom exception hierarchies with proper error context
@ -121,6 +123,14 @@ The platform includes comprehensive system health monitoring with:
- **Chart Integration**: Sophisticated indicator overlay system with configuration options
- **Strategy Configurations**: Advanced chart strategy templates and examples
**🤖 Strategy Engine (95% Complete)**
- **BaseStrategy Interface**: Implemented and serving as a foundation for strategies
- **Strategy Implementations**: EMA Crossover, MACD, and RSI strategies implemented
- **Strategy Factory**: Dynamic strategy loading and registration system in place
- **Signal Generation**: Core trading signal logic implemented with vectorized operations
- **Batch Processing**: Capabilities for backtesting large datasets
- **Real-time Execution**: Event-driven real-time signal generation and broadcasting
**🔧 Supporting Systems (90% Complete)**
- **Logging System**: Unified, component-specific logging with automatic cleanup
- **Configuration Management**: Type-safe settings with environment variable support
@ -129,12 +139,6 @@ The platform includes comprehensive system health monitoring with:
### ⚠️ **Critical Gaps - Core Business Logic Missing**
**🤖 Strategy Engine (0% Complete)**
- **BaseStrategy Interface**: Not implemented
- **Strategy Implementations**: No EMA crossover, MACD, or RSI strategies exist
- **Strategy Factory**: No dynamic strategy loading system
- **Signal Generation**: No actual trading signal logic
**🎮 Bot Management System (10% Complete)**
- **Bot Database Models**: Exist but no business logic implementation
- **BotManager Class**: Does not exist (`bot_manager.py` missing)
@ -154,22 +158,19 @@ The platform includes comprehensive system health monitoring with:
### 🔧 **Technical Debt & Issues**
**📋 Testing Issues**
- Import errors in test suite due to file structure changes
- Some tests reference non-existent modules
- Import errors in test suite due to file structure changes (mostly resolved with recent updates)
- Test coverage needs update for new components
**📄 Documentation Gaps**
- Context documentation was significantly outdated
- Some complex functions lack comprehensive documentation
- API documentation needs expansion
### 🎯 **Next Phase Priority**
The project has **excellent infrastructure** but needs **core business logic implementation**:
The project has **excellent infrastructure** and a nearly complete **Strategy Engine**. The next phases will focus on:
1. **Strategy Engine Foundation** (Task Group 4.0) - Implement base strategy classes and signal generation
1. **Chart Integration and Visualization** (Task Group 5.0) - Implement live strategy signal visualization layers and performance metrics display.
2. **Bot Management System** (Task Group 6.0) - Create bot lifecycle management and virtual portfolios
3. **Backtesting Engine** (Task Group 5.0) - Build historical strategy testing capabilities
4. **Real-Time Simulation** (Task Group 7.0) - Implement live strategy execution with virtual trading
3. **Backtesting Engine** (Task Group 7.0) - Build historical strategy testing capabilities
4. **Real-Time Simulation** (Task Group 8.0) - Implement live strategy execution with virtual trading
The sophisticated infrastructure provides a solid foundation for rapid development of the trading logic components.

150
docs/modules/strategies.md Normal file
View File

@ -0,0 +1,150 @@
# Module: Strategies
## Purpose
This module encompasses the core strategy engine for the TCP Trading Platform. It provides a flexible and extensible framework for defining, managing, and executing trading strategies. It includes components for real-time signal generation, batch backtesting, and data integration with market data and technical indicators.
## Public Interface
### Classes
- `strategies.base.BaseStrategy`: Abstract base class for all trading strategies, defining the common interface for `calculate()` and `get_required_indicators()`.
- `strategies.factory.StrategyFactory`: Manages dynamic loading and registration of strategy implementations based on configuration.
- `strategies.manager.StrategyManager`: Handles user-defined strategy configurations, enabling loading, saving, and validation of strategy parameters.
- `strategies.data_integration.StrategyDataIntegrator`: Orchestrates the integration of market data and pre-calculated technical indicators for strategy execution. It handles data fetching, caching, and dependency resolution.
- `strategies.batch_processing.BacktestingBatchProcessor`: Provides capabilities for efficient batch execution of multiple strategies across large historical datasets, including memory management and performance optimization.
- `strategies.realtime_execution.RealTimeStrategyProcessor`: Manages real-time strategy execution, generating signals incrementally as new market data becomes available. It integrates with the dashboard's chart refresh cycle and handles signal broadcasting.
- `strategies.realtime_execution.StrategySignalBroadcaster`: A component of the real-time execution pipeline responsible for distributing generated signals, storing them in the database, and triggering chart updates.
### Functions
- `strategies.realtime_execution.get_realtime_strategy_processor()`: Returns the singleton instance of the `RealTimeStrategyProcessor`.
- `strategies.realtime_execution.initialize_realtime_strategy_system()`: Initializes and starts the real-time strategy execution system.
- `strategies.realtime_execution.shutdown_realtime_strategy_system()`: Shuts down the real-time strategy execution system.
## Usage Examples
### 1. Registering and Executing a Real-time Strategy
```python
from strategies.realtime_execution import initialize_realtime_strategy_system
from config.strategies.config_utils import StrategyConfigurationManager
# Initialize the real-time system (usually done once at application startup)
processor = initialize_realtime_strategy_system()
# Load a strategy configuration (e.g., from a user-defined config or template)
config_manager = StrategyConfigurationManager()
strategy_name = "ema_crossover"
strategy_config = config_manager.load_user_strategy_config(strategy_name) or \
config_manager.load_strategy_template(strategy_name)
if strategy_config:
# Register the strategy for real-time execution on a specific symbol and timeframe
context_id = processor.register_strategy(
strategy_name=strategy_name,
strategy_config=strategy_config,
symbol="BTC-USDT",
timeframe="1h"
)
print(f"Strategy '{strategy_name}' registered for real-time updates with ID: {context_id}")
# In a real application, new data would arrive (e.g., via a WebSocket or API poll)
# and `processor.execute_realtime_update()` would be called from a data refresh callback.
# Example of manual trigger (for illustration):
# signals = processor.execute_realtime_update(symbol="BTC-USDT", timeframe="1h")
# print(f"Generated {len(signals)} signals for {strategy_name}")
# To stop the system (usually done at application shutdown)
# shutdown_realtime_strategy_system()
```
### 2. Running a Backtesting Batch Process
```python
from strategies.batch_processing import BacktestingBatchProcessor, BatchProcessingConfig
from config.strategies.config_utils import StrategyConfigurationManager
import pandas as pd
from datetime import datetime, timedelta, timezone
# Example historical data (in a real scenario, this would come from a database)
# This is a placeholder for demonstration purposes
# Ensure you have actual OHLCVCandle data or a DataFrame that mimics it
example_data = [
# ... populate with enough OHLCVCandle objects for indicator warm-up
# For instance, 100 OHLCVCandle objects
]
# Create a mock DataFrame for demonstration, ensure it has 'open', 'high', 'low', 'close', 'volume', 'timestamp'
# For testing, you'd load actual data or generate a synthetic one.
ohlcv_data = []
start_ts = datetime.now(timezone.utc) - timedelta(days=30)
for i in range(300):
ohlcv_data.append({
"timestamp": (start_ts + timedelta(hours=i)).isoformat(),
"open": 100.0 + i * 0.1,
"high": 101.0 + i * 0.1,
"low": 99.0 + i * 0.1,
"close": 100.5 + i * 0.1,
"volume": 1000.0 + i * 10
})
market_df = pd.DataFrame(ohlcv_data).set_index("timestamp")
market_df.index = pd.to_datetime(market_df.index).to_pydatetime() # Convert to datetime objects if needed
# Initialize batch processor config
batch_config = BatchProcessingConfig(
chunk_size=100, # Process in chunks for memory efficiency
max_concurrent_tasks=2
)
batch_processor = BacktestingBatchProcessor(batch_config)
# Load strategies to backtest
config_manager = StrategyConfigurationManager()
strategies_to_backtest = {
"ema_crossover": config_manager.load_strategy_template("ema_crossover"),
"macd": config_manager.load_strategy_template("macd")
}
# Run batch processing
if all(strategies_to_backtest.values()): # Ensure all configs are loaded
print("Starting batch backtesting...")
results = batch_processor.process_strategies_batch(
market_df=market_df,
strategies_config=strategies_to_backtest,
symbols=["BTC-USDT", "ETH-USDT"],
timeframe="1h"
)
print("Batch backtesting complete. Results:")
for strategy_name, symbol_results in results.items():
for symbol, strategy_results_list in symbol_results.items():
print(f" Strategy: {strategy_name}, Symbol: {symbol}, Signals: {len(strategy_results_list)}")
else:
print("Failed to load all strategy configurations. Skipping batch processing.")
```
## Dependencies
### Internal Dependencies
- `database.operations`: For database interactions (storing signals).
- `data.common.data_types`: Defines common data structures like `OHLCVCandle`.
- `data.common.indicators.TechnicalIndicators`: Provides access to pre-calculated technical indicators.
- `components.charts.data_integration.MarketDataIntegrator`: Used by `StrategyDataIntegrator` to fetch raw market data.
- `utils.logger`: For logging within the module.
- `config.strategies.config_utils`: For loading and managing strategy configurations.
### External Dependencies
- `pandas`: For data manipulation and vectorized operations.
- `datetime`: For handling timestamps and time-related calculations.
- `typing`: For type hints.
- `dataclasses`: For defining data classes like `RealTimeConfig`, `StrategyExecutionContext`, etc.
- `threading`, `queue`, `concurrent.futures`: For managing concurrency and background processing in real-time execution.
## Testing
To run tests for the `strategies` module, navigate to the project root and execute:
```bash
python -m pytest tests/strategies/test_realtime_execution.py -v
python -m pytest tests/strategies/test_batch_processing.py -v
# Add other relevant strategy tests here, e.g., for data_integration, manager, factory
```
## Known Issues
- **Incremental Calculation Optimization**: The `_calculate_incremental_signals` method in `RealTimeStrategyProcessor` currently falls back to full recalculation. True incremental calculation (only processing new candles and updating existing indicator series) is a future optimization.
- **Chart Layer Updates**: While the `StrategySignalBroadcaster` triggers a chart update callback, the actual chart layer integration (`components/charts/layers/strategy_signals.py`) and dynamic signal display on the chart are part of Task 5.0 and not fully implemented within this module.