Implement Service Configuration Manager for data collection service

- Introduced `service_config.py` to manage configuration loading, validation, and schema management, enhancing modularity and security.
- Created a `ServiceConfig` class for handling configuration with robust error handling and default values.
- Refactored `DataCollectionService` to utilize the new `ServiceConfig`, streamlining configuration management and improving readability.
- Added a `CollectorFactory` to encapsulate collector creation logic, promoting separation of concerns.
- Updated `CollectorManager` and related components to align with the new architecture, ensuring better maintainability.
- Enhanced logging practices across the service for improved monitoring and debugging.

These changes significantly improve the architecture and maintainability of the data collection service, aligning with project standards for modularity and performance.
This commit is contained in:
Vasily.onl
2025-06-10 12:55:27 +08:00
parent 33f2110f19
commit 2890ba2efa
15 changed files with 1939 additions and 979 deletions

View File

@@ -8,7 +8,8 @@ from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock
from utils.logger import get_logger
from data.collector_manager import CollectorManager, ManagerStatus, CollectorConfig
from data.collector_manager import CollectorManager
from data.collector_types import ManagerStatus, CollectorConfig
from data.base_collector import BaseDataCollector, DataType, CollectorStatus

View File

@@ -24,7 +24,8 @@ from collections import defaultdict
# Import modules under test
from data.base_collector import BaseDataCollector, DataType, MarketDataPoint, CollectorStatus
from data.collector_manager import CollectorManager, CollectorConfig
from data.collector_manager import CollectorManager
from data.collector_types import CollectorConfig
from data.collection_service import DataCollectionService
from data.exchanges.okx.collector import OKXCollector
from data.exchanges.okx.data_processor import OKXDataProcessor, OKXDataValidator, OKXDataTransformer