Add complete time series aggregation example and refactor OKXCollector for repository pattern

- Introduced `example_complete_series_aggregation.py` to demonstrate time series aggregation, emitting candles even when no trades occur.
- Implemented `CompleteSeriesProcessor` extending `RealTimeCandleProcessor` to handle time-based candle emission and empty candle creation.
- Refactored `OKXCollector` to utilize the new repository pattern for database operations, enhancing modularity and maintainability.
- Updated database operations to centralize data handling through `DatabaseOperations`, improving error handling and logging.
- Enhanced documentation to include details on the new aggregation example and repository pattern implementation, ensuring clarity for users.
This commit is contained in:
Vasily.onl
2025-06-02 13:27:01 +08:00
parent 5b4547edd5
commit cffc54b648
11 changed files with 1460 additions and 149 deletions

View File

@@ -194,6 +194,70 @@ def calculate_performance_metrics(portfolio_values: List[float]) -> dict:
6. **Virtual Trading**: Simulation-first approach with fee modeling
7. **Simplified Architecture**: Monolithic design with clear component boundaries for future scaling
## Repository Pattern for Database Operations
### Database Abstraction Layer
The system uses the **Repository Pattern** to abstract database operations from business logic, providing a clean, maintainable, and testable interface for all data access.
```python
# Centralized database operations
from database.operations import get_database_operations
class DataCollector:
def __init__(self):
# Use repository pattern instead of direct SQL
self.db = get_database_operations()
def store_candle(self, candle: OHLCVCandle):
"""Store candle using repository pattern"""
success = self.db.market_data.upsert_candle(candle, force_update=False)
def store_raw_trade(self, data_point: MarketDataPoint):
"""Store raw trade data using repository pattern"""
success = self.db.raw_trades.insert_market_data_point(data_point)
```
### Repository Structure
```python
# Clean API for database operations
class DatabaseOperations:
def __init__(self):
self.market_data = MarketDataRepository() # Candle operations
self.raw_trades = RawTradeRepository() # Raw data operations
def health_check(self) -> bool:
"""Check database connection health"""
def get_stats(self) -> dict:
"""Get database statistics and metrics"""
class MarketDataRepository:
def upsert_candle(self, candle: OHLCVCandle, force_update: bool = False) -> bool:
"""Store or update candle with duplicate handling"""
def get_candles(self, symbol: str, timeframe: str, start: datetime, end: datetime) -> List[dict]:
"""Retrieve historical candle data"""
def get_latest_candle(self, symbol: str, timeframe: str) -> Optional[dict]:
"""Get most recent candle for symbol/timeframe"""
class RawTradeRepository:
def insert_market_data_point(self, data_point: MarketDataPoint) -> bool:
"""Store raw WebSocket data"""
def get_raw_trades(self, symbol: str, data_type: str, start: datetime, end: datetime) -> List[dict]:
"""Retrieve raw trade data for analysis"""
```
### Benefits of Repository Pattern
- **No Raw SQL**: Business logic never contains direct SQL queries
- **Centralized Operations**: All database interactions go through well-defined APIs
- **Easy Testing**: Repository methods can be easily mocked for unit tests
- **Database Agnostic**: Can change database implementations without affecting business logic
- **Automatic Transaction Management**: Sessions, commits, and rollbacks handled automatically
- **Consistent Error Handling**: Custom exceptions with proper context
- **Type Safety**: Full type hints for better IDE support and error detection
## Database Architecture
### Core Tables