Add complete time series aggregation example and refactor OKXCollector for repository pattern
- Introduced `example_complete_series_aggregation.py` to demonstrate time series aggregation, emitting candles even when no trades occur. - Implemented `CompleteSeriesProcessor` extending `RealTimeCandleProcessor` to handle time-based candle emission and empty candle creation. - Refactored `OKXCollector` to utilize the new repository pattern for database operations, enhancing modularity and maintainability. - Updated database operations to centralize data handling through `DatabaseOperations`, improving error handling and logging. - Enhanced documentation to include details on the new aggregation example and repository pattern implementation, ensuring clarity for users.
This commit is contained in:
@@ -194,6 +194,70 @@ def calculate_performance_metrics(portfolio_values: List[float]) -> dict:
|
||||
6. **Virtual Trading**: Simulation-first approach with fee modeling
|
||||
7. **Simplified Architecture**: Monolithic design with clear component boundaries for future scaling
|
||||
|
||||
## Repository Pattern for Database Operations
|
||||
|
||||
### Database Abstraction Layer
|
||||
The system uses the **Repository Pattern** to abstract database operations from business logic, providing a clean, maintainable, and testable interface for all data access.
|
||||
|
||||
```python
|
||||
# Centralized database operations
|
||||
from database.operations import get_database_operations
|
||||
|
||||
class DataCollector:
|
||||
def __init__(self):
|
||||
# Use repository pattern instead of direct SQL
|
||||
self.db = get_database_operations()
|
||||
|
||||
def store_candle(self, candle: OHLCVCandle):
|
||||
"""Store candle using repository pattern"""
|
||||
success = self.db.market_data.upsert_candle(candle, force_update=False)
|
||||
|
||||
def store_raw_trade(self, data_point: MarketDataPoint):
|
||||
"""Store raw trade data using repository pattern"""
|
||||
success = self.db.raw_trades.insert_market_data_point(data_point)
|
||||
```
|
||||
|
||||
### Repository Structure
|
||||
```python
|
||||
# Clean API for database operations
|
||||
class DatabaseOperations:
|
||||
def __init__(self):
|
||||
self.market_data = MarketDataRepository() # Candle operations
|
||||
self.raw_trades = RawTradeRepository() # Raw data operations
|
||||
|
||||
def health_check(self) -> bool:
|
||||
"""Check database connection health"""
|
||||
|
||||
def get_stats(self) -> dict:
|
||||
"""Get database statistics and metrics"""
|
||||
|
||||
class MarketDataRepository:
|
||||
def upsert_candle(self, candle: OHLCVCandle, force_update: bool = False) -> bool:
|
||||
"""Store or update candle with duplicate handling"""
|
||||
|
||||
def get_candles(self, symbol: str, timeframe: str, start: datetime, end: datetime) -> List[dict]:
|
||||
"""Retrieve historical candle data"""
|
||||
|
||||
def get_latest_candle(self, symbol: str, timeframe: str) -> Optional[dict]:
|
||||
"""Get most recent candle for symbol/timeframe"""
|
||||
|
||||
class RawTradeRepository:
|
||||
def insert_market_data_point(self, data_point: MarketDataPoint) -> bool:
|
||||
"""Store raw WebSocket data"""
|
||||
|
||||
def get_raw_trades(self, symbol: str, data_type: str, start: datetime, end: datetime) -> List[dict]:
|
||||
"""Retrieve raw trade data for analysis"""
|
||||
```
|
||||
|
||||
### Benefits of Repository Pattern
|
||||
- **No Raw SQL**: Business logic never contains direct SQL queries
|
||||
- **Centralized Operations**: All database interactions go through well-defined APIs
|
||||
- **Easy Testing**: Repository methods can be easily mocked for unit tests
|
||||
- **Database Agnostic**: Can change database implementations without affecting business logic
|
||||
- **Automatic Transaction Management**: Sessions, commits, and rollbacks handled automatically
|
||||
- **Consistent Error Handling**: Custom exceptions with proper context
|
||||
- **Type Safety**: Full type hints for better IDE support and error detection
|
||||
|
||||
## Database Architecture
|
||||
|
||||
### Core Tables
|
||||
|
||||
Reference in New Issue
Block a user