12 Commits

Author SHA1 Message Date
Vasily.onl
622fda9d2e data ingestion 2025-06-13 18:01:10 +08:00
Vasily.onl
fd5a59fc39 4.0 - 1.0 Implement strategy engine foundation with modular components
- Introduced a new `strategies` package containing the core structure for trading strategies, including `BaseStrategy`, `StrategyFactory`, and various strategy implementations (EMA, RSI, MACD).
- Added utility functions for signal detection and validation in `strategies/utils.py`, enhancing modularity and maintainability.
- Updated `pyproject.toml` to include the new `strategies` package in the build configuration.
- Implemented comprehensive unit tests for the strategy foundation components, ensuring reliability and adherence to project standards.

These changes establish a solid foundation for the strategy engine, aligning with project goals for modularity, performance, and maintainability.
2025-06-12 14:41:16 +08:00
Vasily.onl
0a7e444206 Refactor data collection architecture for modularity and maintainability
- Updated `pyproject.toml` to include the new `data` package in the build configuration, ensuring all components are properly included.
- Introduced `ADR-004` documentation outlining the rationale for refactoring the data collection system into a modular architecture, addressing complexity and maintainability issues.
- Enhanced `data_collectors.md` to reflect the new component structure, detailing responsibilities of `CollectorLifecycleManager`, `ManagerHealthMonitor`, `ManagerStatsTracker`, and `ManagerLogger`.
- Refactored `DataCollectionService` to utilize the new modular components, improving orchestration and error handling.
- Removed the obsolete `collector-service-tasks-optimization.md` and `refactor-common-package.md` files, streamlining the tasks documentation.

These changes significantly improve the architecture and maintainability of the data collection service, aligning with project standards for modularity, performance, and documentation clarity.
2025-06-10 14:32:00 +08:00
Ajasra
96ee25bd01 Refactor data validation module for improved modularity and functionality
- Removed the existing `validation.py` file and replaced it with a modular structure, introducing separate files for validation results, field validators, and the base validator class.
- Implemented comprehensive validation functions for common data types, enhancing reusability and maintainability.
- Added a new `__init__.py` to expose the validation utilities, ensuring a clean public interface.
- Created detailed documentation for the validation module, including usage examples and architectural details.
- Introduced extensive unit tests to cover the new validation framework, ensuring reliability and preventing regressions.

These changes enhance the overall architecture of the data validation module, making it more scalable and easier to manage.
2025-06-07 12:31:47 +08:00
Vasily.onl
58a754414a Removed Mantine for UI, and used bootstrap for simplicity 2025-06-06 13:33:59 +08:00
Vasily.onl
82f4e0ef48 3.5 Enhance system health monitoring dashboard with comprehensive market data tracking
- Added `psutil` dependency for system performance metrics.
- Implemented a new layout in `dashboard/layouts/system_health.py` using Mantine components for real-time monitoring of data collection services, database health, Redis status, and system performance.
- Enhanced callbacks in `dashboard/callbacks/system_health.py` for detailed status updates and error handling.
- Introduced quick status indicators for data collection, database, Redis, and performance metrics with auto-refresh functionality.
- Created modals for viewing detailed data collection information and service logs.
- Updated documentation to reflect the new features and usage guidelines.
2025-06-04 17:46:50 +08:00
Vasily.onl
cdee9f04d6 Remove main application file app.py and update dependencies for modular dashboard architecture
- Deleted `app.py`, consolidating the main application logic into a modular structure for improved maintainability.
- Added `dash-mantine-components` dependency to enhance UI component capabilities.
- Updated `pyproject.toml` and `uv.lock` to reflect the new dependency.
- Adjusted imports in `components/__init__.py` and `chart_controls.py` to align with the new modular design.
- Cleaned up unused parameter controls in the market data layout to streamline the user interface.
2025-06-04 15:30:50 +08:00
Vasily.onl
4510181b39 Add OKX data collector implementation and modular exchange architecture
- Introduced the `OKXCollector` and `OKXWebSocketClient` classes for real-time market data collection from the OKX exchange.
- Implemented a factory pattern for creating exchange-specific collectors, enhancing modularity and scalability.
- Added configuration support for the OKX collector in `config/okx_config.json`.
- Updated documentation to reflect the new modular architecture and provide guidance on using the OKX collector.
- Created unit tests for the OKX collector and exchange factory to ensure functionality and reliability.
- Enhanced logging and error handling throughout the new implementation for improved monitoring and debugging.
2025-05-31 20:49:31 +08:00
Vasily.onl
4936e5cd73 Implement enhanced data collection system with health monitoring and management
- Introduced `BaseDataCollector` and `CollectorManager` classes for standardized data collection and centralized management.
- Added health monitoring features, including auto-restart capabilities and detailed status reporting for collectors.
- Updated `env.template` to include new logging and health check configurations.
- Enhanced documentation in `docs/data_collectors.md` to provide comprehensive guidance on the new data collection system.
- Added unit tests for `BaseDataCollector` and `CollectorManager` to ensure reliability and functionality.
2025-05-30 20:33:56 +08:00
Vasily.onl
b7263b023f Enhance logging system and update dependencies
- Updated `.gitignore` to exclude log files from version control.
- Added `pytest` as a dependency in `pyproject.toml` for testing purposes.
- Included `pytest` in `uv.lock` to ensure consistent dependency management.
- Introduced comprehensive documentation for the new unified logging system in `docs/logging.md`, detailing features, usage, and configuration options.
2025-05-30 19:54:56 +08:00
Ajasra
f5b8bd3c96 dev setup 2025-05-29 23:50:41 +08:00
Ajasra
d90681b788 init 2025-05-29 23:04:08 +08:00