Refactor BaseDataCollector to integrate ConnectionManager for connection handling

- Extracted connection management logic into a new `ConnectionManager` class, promoting separation of concerns and enhancing modularity.
- Updated `BaseDataCollector` to utilize the `ConnectionManager` for connection, disconnection, and reconnection processes, improving code clarity and maintainability.
- Refactored connection-related methods and attributes, ensuring consistent error handling and logging practices.
- Enhanced the `OKXCollector` to implement the new connection management approach, streamlining its connection logic.
- Added unit tests for the `ConnectionManager` to validate its functionality and ensure robust error handling.

These changes improve the architecture of the data collector, aligning with project standards for maintainability and performance.
This commit is contained in:
Vasily.onl
2025-06-09 17:42:06 +08:00
parent 60434afd5d
commit 41f0e8e6b6
8 changed files with 434 additions and 77 deletions

View File

@@ -10,7 +10,8 @@ from .data_types import (
OHLCVCandle,
MarketDataPoint,
DataValidationResult,
CandleProcessingConfig
CandleProcessingConfig,
DataType
)
from .transformation.trade import (
@@ -44,6 +45,7 @@ __all__ = [
'MarketDataPoint',
'DataValidationResult',
'CandleProcessingConfig',
'DataType',
# Trade transformation
'TradeTransformer',

View File

@@ -10,8 +10,33 @@ from decimal import Decimal
from typing import Dict, List, Optional, Any
from dataclasses import dataclass, field
from enum import Enum
import asyncio
from ..base_collector import DataType, MarketDataPoint # Import from base
# from ..base_collector import DataType, MarketDataPoint # Import from base
class DataType(Enum):
"""Types of data that can be collected."""
TICKER = "ticker"
TRADE = "trade"
ORDERBOOK = "orderbook"
CANDLE = "candle"
BALANCE = "balance"
@dataclass
class MarketDataPoint:
"""Standardized market data structure."""
exchange: str
symbol: str
timestamp: datetime
data_type: DataType
data: Dict[str, Any]
def __post_init__(self):
"""Validate data after initialization."""
if not self.timestamp.tzinfo:
self.timestamp = self.timestamp.replace(tzinfo=timezone.utc)
@dataclass