TCPDashboard/tasks/refactor-common-package.md

Relevant Files

• data/common/aggregation.py - To be broken into a sub-package.
• data/common/indicators.py - To be broken into a sub-package and have a bug fixed.
• data/common/validation.py - To be refactored for better modularity.
• data/common/transformation.py - ✅ Refactored into transformation package with safety limits.
• data/common/data_types.py - To be updated with new types from other modules.
• data/common/__init__.py - ✅ Updated to reflect the new package structure.
• tests/ - Existing tests will need to be run after each step to ensure no regressions.

Notes

• This refactoring focuses on improving modularity by splitting large files into smaller, more focused modules, as outlined in the refactoring.mdc guide.
• Each major step will be followed by a verification phase to ensure the application remains stable.

Tasks

• 1.0 Refactor aggregation.py into a dedicated sub-package.

  • 1.1 Create safety net tests to ensure the aggregation logic still works as expected.
  • 1.2 Create a new directory data/common/aggregation.
  • 1.3 Create data/common/aggregation/__init__.py to mark it as a package.
  • 1.4 Move the TimeframeBucket class to data/common/aggregation/bucket.py.
  • 1.5 Move the RealTimeCandleProcessor class to data/common/aggregation/realtime.py.
  • 1.6 Move the BatchCandleProcessor class to data/common/aggregation/batch.py.
  • 1.7 Move the utility functions to data/common/aggregation/utils.py.
  • 1.8 Update data/common/aggregation/__init__.py to expose all public classes and functions.
  • 1.9 Delete the original data/common/aggregation.py file.
  • 1.10 Run tests to verify the aggregation logic still works as expected.

• 2.0 Refactor indicators.py into a dedicated sub-package.

  • 2.1 Create safety net tests for indicators module.
  • 2.2 Create a new directory data/common/indicators.
  • 2.3 Create data/common/indicators/__init__.py to mark it as a package.
  • 2.4 Move the TechnicalIndicators class to data/common/indicators/technical.py.
  • 2.5 Move the IndicatorResult class to data/common/indicators/result.py.
  • 2.6 Move the utility functions to data/common/indicators/utils.py.
  • 2.7 Update data/common/indicators/__init__.py to expose all public classes and functions.
  • 2.8 Delete the original data/common/indicators.py file.
  • 2.9 Run tests to verify the indicators logic still works as expected.

• 3.0 Refactor validation.py for better modularity.

  • 3.1 Create safety net tests for validation module.
  • 3.2 Extract common validation logic into separate functions.
  • 3.3 Improve error handling and validation messages.
  • 3.4 Run tests to verify validation still works as expected.

• 4.0 Refactor transformation.py for better modularity.

  • 4.1 Create safety net tests for transformation module.
  • 4.2 Extract common transformation logic into separate functions.
  • 4.3 Improve error handling and transformation messages.
  • 4.4 Run tests to verify transformation still works as expected.
  • 4.5 Create comprehensive safety limits system.
  • 4.6 Add documentation for the transformation module.
  • 4.7 Delete redundant transformation.py file.

• 5.0 Update data_types.py with new types.

  • 5.1 Review and document all data types.
  • 5.2 Add any missing type hints.
  • 5.3 Add validation for data types.
  • 5.4 Run tests to verify data types still work as expected.

• 6.0 Final verification and cleanup.

  • 6.1 Run all tests to ensure no regressions.
  • 6.2 Update documentation to reflect new structure.
  • 6.3 Review and clean up any remaining TODOs.
  • 6.4 Create PR with changes.