Vasily.onl 33f2110f19 Refactor data module to enhance modularity and maintainability

- Extracted `OHLCVData` and validation logic into a new `common/ohlcv_data.py` module, promoting better organization and reusability.
- Updated `BaseDataCollector` to utilize the new `validate_ohlcv_data` function for improved data validation, enhancing code clarity and maintainability.
- Refactored imports in `data/__init__.py` to reflect the new structure, ensuring consistent access to common data types and exceptions.
- Removed redundant data validation logic from `BaseDataCollector`, streamlining its responsibilities.
- Added unit tests for `OHLCVData` and validation functions to ensure correctness and reliability.

These changes improve the architecture of the data module, aligning with project standards for maintainability and performance.

2025-06-10 12:04:58 +08:00

3.2 KiB

Raw Permalink Blame History

ADR-002: BaseDataCollector Refactoring and Component Extraction

Status

Accepted

Context

The BaseDataCollector class was initially monolithic, handling connection management, state and telemetry, and callback dispatching directly. This led to a less modular, harder-to-test, and less maintainable codebase. Additionally, OHLCVData and its associated validation, although broadly applicable, were tightly coupled within the data module, leading to potential import complexities and naming conflicts.

Decision

To improve modularity, maintainability, testability, and reusability, we decided to refactor BaseDataCollector by extracting its core responsibilities into dedicated, smaller, and focused components. We also decided to relocate OHLCVData to a more common and accessible location.

Extracted Components:

CollectorStateAndTelemetry: Responsible for managing collector status, health, statistics, and logging.
ConnectionManager: Responsible for handling WebSocket connection lifecycle (connect, disconnect, reconnect) and related error management.
CallbackDispatcher: Responsible for managing and dispatching data callbacks to registered listeners.

OHLCVData Relocation:

The OHLCVData class and the validate_ohlcv_data function, along with the DataValidationError exception, were moved from data/ohlcv_data.py to data/common/ohlcv_data.py.

Consequences

Positive:

Improved Modularity: BaseDataCollector is now leaner and focuses solely on orchestrating the new components.
Enhanced Testability: Each extracted component can be unit-tested in isolation, reducing test complexity and improving test coverage.
Increased Maintainability: Changes to connection logic, state management, or callback handling are isolated to their respective components, minimizing impact on other parts of the system.
Greater Reusability: CollectorStateAndTelemetry, ConnectionManager, and CallbackDispatcher can potentially be reused in other contexts or for different types of collectors.
Clearer Separation of Concerns: Each component has a single, well-defined responsibility.
Centralized OHLCVData: Moving OHLCVData to data/common provides a more intuitive and accessible location for a common data structure, resolving potential import conflicts and improving code organization.

Negative:

Increased File Count: More files are introduced, potentially increasing initial navigation overhead (mitigated by clear naming and directory structure).
Refactoring Overhead: Required updating existing code to use the new components and adjusting imports across multiple files.

Alternatives Considered

Keeping Monolithic BaseDataCollector: Rejected due to the drawbacks of tightly coupled code (poor testability, maintainability).
Partial Extraction: Considered extracting only one or two components, but decided against it to achieve maximum modularity benefits.
Different OHLCVData Location: Considered utils/data_types.py or data/models.py, but data/common/ohlcv_data.py was deemed most appropriate given its nature as a common data structure within the data module.

3.2 KiB Raw Permalink Blame History