Vasily.onl cffc54b648 Add complete time series aggregation example and refactor OKXCollector for repository pattern

- Introduced `example_complete_series_aggregation.py` to demonstrate time series aggregation, emitting candles even when no trades occur.
- Implemented `CompleteSeriesProcessor` extending `RealTimeCandleProcessor` to handle time-based candle emission and empty candle creation.
- Refactored `OKXCollector` to utilize the new repository pattern for database operations, enhancing modularity and maintainability.
- Updated database operations to centralize data handling through `DatabaseOperations`, improving error handling and logging.
- Enhanced documentation to include details on the new aggregation example and repository pattern implementation, ensuring clarity for users.

2025-06-02 13:27:01 +08:00

6.0 KiB

Raw Blame History

Components Documentation

This section contains detailed technical documentation for all system components in the TCP Dashboard platform.

📋 Contents

Data Collection System

Data Collectors - Comprehensive guide to the enhanced data collector system
- BaseDataCollector abstract class with health monitoring
- CollectorManager for centralized management
- Exchange Factory Pattern for standardized collector creation
- Modular Exchange Architecture for scalable implementation
- Auto-restart and failure recovery mechanisms
- Health monitoring and alerting systems
- Performance optimization techniques
- Integration examples and patterns
- Comprehensive troubleshooting guide

Database Operations

Database Operations - Repository pattern for clean database interactions
- Repository Pattern implementation for data access abstraction
- MarketDataRepository for candle/OHLCV operations
- RawTradeRepository for WebSocket data storage
- Automatic transaction management and session cleanup
- Configurable duplicate handling with force update options
- Custom error handling with DatabaseOperationError
- Database health monitoring and performance statistics
- Migration guide from direct SQL to repository pattern

Logging & Monitoring

Enhanced Logging System - Unified logging framework
- Multi-level logging with automatic cleanup
- Console and file output with formatting
- Performance monitoring integration
- Cross-component logging standards
- Log aggregation and analysis

🔧 Component Architecture

Core Components

┌─────────────────────────────────────────────────────────────┐
│                   CollectorManager                          │
│  ┌─────────────────────────────────────────────────────┐    │
│  │              Global Health Monitor                  │    │
│  │  • System-wide health checks                       │    │
│  │  • Auto-restart coordination                       │    │
│  │  • Performance analytics                           │    │
│  └─────────────────────────────────────────────────────┘    │
│                           │                                 │
│  ┌─────────────────┐  ┌─────────────────┐  ┌──────────────┐ │
│  │  OKX Collector  │  │Binance Collector│  │   Custom     │ │
│  │                 │  │                 │  │  Collector   │ │
│  │ • Health Monitor│  │ • Health Monitor│  │ • Health Mon │ │
│  │ • Auto-restart  │  │ • Auto-restart  │  │ • Auto-resta │ │
│  │ • Data Validate │  │ • Data Validate │  │ • Data Valid │ │
│  └─────────────────┘  └─────────────────┘  └──────────────┘ │
└─────────────────────────────────────────────────────────────┘

Design Patterns

Factory Pattern: Standardized component creation across exchanges
Observer Pattern: Event-driven data processing and callbacks
Strategy Pattern: Pluggable data processing strategies
Singleton Pattern: Centralized logging and configuration management

🚀 Quick Start

Using Components

# Data Collector usage
from data.exchanges import create_okx_collector
from data.base_collector import DataType

collector = create_okx_collector(
    symbol='BTC-USDT',
    data_types=[DataType.TRADE, DataType.ORDERBOOK]
)

# Logging usage
from utils.logger import get_logger

logger = get_logger("my_component")
logger.info("Component initialized")

Component Integration

# Integrating multiple components
from data.collector_manager import CollectorManager
from utils.logger import get_logger

manager = CollectorManager("production_system")
logger = get_logger("system_manager")

# Components work together seamlessly
await manager.start()
logger.info("System started successfully")

📊 Performance & Monitoring

Health Monitoring

All components include built-in health monitoring:

Real-time Status: Component state and performance metrics
Auto-Recovery: Automatic restart on failures
Performance Tracking: Message rates, uptime, error rates
Alerting: Configurable alerts for component health

Logging Integration

Unified logging across all components:

Structured Logging: JSON-formatted logs for analysis
Multiple Levels: Debug, Info, Warning, Error levels
Automatic Cleanup: Log rotation and old file cleanup
Performance Metrics: Built-in performance tracking

Exchange Documentation - Exchange-specific implementations
Architecture Overview - System design and patterns
Setup Guide - Component configuration and deployment
API Reference - Technical specifications

📈 Future Components

Planned component additions:

Strategy Engine: Trading strategy execution framework
Portfolio Manager: Position and risk management
Dashboard UI: Web-based monitoring and control interface
Alert Manager: Advanced alerting and notification system
Data Analytics: Historical data analysis and reporting

For the complete documentation index, see the main documentation README.

6.0 KiB Raw Blame History