96ee25bd01
- Removed the existing `validation.py` file and replaced it with a modular structure, introducing separate files for validation results, field validators, and the base validator class. - Implemented comprehensive validation functions for common data types, enhancing reusability and maintainability. - Added a new `__init__.py` to expose the validation utilities, ensuring a clean public interface. - Created detailed documentation for the validation module, including usage examples and architectural details. - Introduced extensive unit tests to cover the new validation framework, ensuring reliability and preventing regressions. These changes enhance the overall architecture of the data validation module, making it more scalable and easier to manage.
63 lines
3.6 KiB
Markdown
63 lines
3.6 KiB
Markdown
## Relevant Files
|
|
|
|
- `data/common/aggregation.py` - To be broken into a sub-package.
|
|
- `data/common/indicators.py` - To be broken into a sub-package and have a bug fixed.
|
|
- `data/common/validation.py` - To be refactored for better modularity.
|
|
- `data/common/transformation.py` - To be refactored for better modularity.
|
|
- `data/common/data_types.py` - To be updated with new types from other modules.
|
|
- `data/common/__init__.py` - To be updated to reflect the new package structure.
|
|
- `tests/` - Existing tests will need to be run after each step to ensure no regressions.
|
|
|
|
### Notes
|
|
|
|
- This refactoring focuses on improving modularity by splitting large files into smaller, more focused modules, as outlined in the `refactoring.mdc` guide.
|
|
- Each major step will be followed by a verification phase to ensure the application remains stable.
|
|
|
|
## Tasks
|
|
|
|
- [x] 1.0 Refactor `aggregation.py` into a dedicated sub-package.
|
|
- [x] 1.1 Create safety net tests to ensure the aggregation logic still works as expected.
|
|
- [x] 1.2 Create a new directory `data/common/aggregation`.
|
|
- [x] 1.3 Create `data/common/aggregation/__init__.py` to mark it as a package.
|
|
- [x] 1.4 Move the `TimeframeBucket` class to `data/common/aggregation/bucket.py`.
|
|
- [x] 1.5 Move the `RealTimeCandleProcessor` class to `data/common/aggregation/realtime.py`.
|
|
- [x] 1.6 Move the `BatchCandleProcessor` class to `data/common/aggregation/batch.py`.
|
|
- [x] 1.7 Move the utility functions to `data/common/aggregation/utils.py`.
|
|
- [x] 1.8 Update `data/common/aggregation/__init__.py` to expose all public classes and functions.
|
|
- [x] 1.9 Delete the original `data/common/aggregation.py` file.
|
|
- [x] 1.10 Run tests to verify the aggregation logic still works as expected.
|
|
|
|
- [x] 2.0 Refactor `indicators.py` into a dedicated sub-package.
|
|
- [x] 2.1 Create safety net tests for indicators module.
|
|
- [x] 2.2 Create a new directory `data/common/indicators`.
|
|
- [x] 2.3 Create `data/common/indicators/__init__.py` to mark it as a package.
|
|
- [x] 2.4 Move the `TechnicalIndicators` class to `data/common/indicators/technical.py`.
|
|
- [x] 2.5 Move the `IndicatorResult` class to `data/common/indicators/result.py`.
|
|
- [x] 2.6 Move the utility functions to `data/common/indicators/utils.py`.
|
|
- [x] 2.7 Update `data/common/indicators/__init__.py` to expose all public classes and functions.
|
|
- [x] 2.8 Delete the original `data/common/indicators.py` file.
|
|
- [x] 2.9 Run tests to verify the indicators logic still works as expected.
|
|
|
|
- [x] 3.0 Refactor `validation.py` for better modularity.
|
|
- [x] 3.1 Create safety net tests for validation module.
|
|
- [x] 3.2 Extract common validation logic into separate functions.
|
|
- [x] 3.3 Improve error handling and validation messages.
|
|
- [x] 3.4 Run tests to verify validation still works as expected.
|
|
|
|
- [ ] 4.0 Refactor `transformation.py` for better modularity.
|
|
- [ ] 4.1 Create safety net tests for transformation module.
|
|
- [ ] 4.2 Extract common transformation logic into separate functions.
|
|
- [ ] 4.3 Improve error handling and transformation messages.
|
|
- [ ] 4.4 Run tests to verify transformation still works as expected.
|
|
|
|
- [ ] 5.0 Update `data_types.py` with new types.
|
|
- [ ] 5.1 Review and document all data types.
|
|
- [ ] 5.2 Add any missing type hints.
|
|
- [ ] 5.3 Add validation for data types.
|
|
- [ ] 5.4 Run tests to verify data types still work as expected.
|
|
|
|
- [ ] 6.0 Final verification and cleanup.
|
|
- [ ] 6.1 Run all tests to ensure no regressions.
|
|
- [ ] 6.2 Update documentation to reflect new structure.
|
|
- [ ] 6.3 Review and clean up any remaining TODOs.
|
|
- [ ] 6.4 Create PR with changes. |