Implement data collection architecture with modular components
- Introduced a comprehensive data collection framework, including `CollectorServiceConfig`, `BaseDataCollector`, and `CollectorManager`, enhancing modularity and maintainability. - Developed `CollectorFactory` for streamlined collector creation, promoting separation of concerns and improved configuration handling. - Enhanced `DataCollectionService` to utilize the new architecture, ensuring robust error handling and logging practices. - Added `TaskManager` for efficient management of asynchronous tasks, improving performance and resource management. - Implemented health monitoring and auto-recovery features in `CollectorManager`, ensuring reliable operation of data collectors. - Updated imports across the codebase to reflect the new structure, ensuring consistent access to components. These changes significantly improve the architecture and maintainability of the data collection service, aligning with project standards for modularity, performance, and error handling.
This commit is contained in:
@@ -10,7 +10,8 @@ from typing import Dict, List, Optional, Any, Type, Tuple
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from utils.logger import get_logger
|
||||
from ..base_collector import BaseDataCollector, DataType
|
||||
from ..collector.base_collector import BaseDataCollector
|
||||
from ..common.data_types import DataType
|
||||
from ..common import CandleProcessingConfig
|
||||
from .registry import EXCHANGE_REGISTRY, get_supported_exchanges, get_exchange_info
|
||||
from .exceptions import (
|
||||
|
||||
@@ -11,7 +11,7 @@ from datetime import datetime, timezone
|
||||
from typing import Dict, List, Optional, Any
|
||||
from dataclasses import dataclass
|
||||
|
||||
from ...base_collector import (
|
||||
from ...collector.base_collector import (
|
||||
BaseDataCollector, DataType, CollectorStatus, MarketDataPoint,
|
||||
OHLCVData, DataValidationError, ConnectionError
|
||||
)
|
||||
|
||||
@@ -11,7 +11,7 @@ from decimal import Decimal
|
||||
from typing import Dict, List, Optional, Any, Union, Tuple
|
||||
from enum import Enum
|
||||
|
||||
from ...base_collector import DataType, MarketDataPoint
|
||||
from ...collector.base_collector import DataType, MarketDataPoint
|
||||
from ...common import (
|
||||
DataValidationResult,
|
||||
StandardizedTrade,
|
||||
|
||||
Reference in New Issue
Block a user