Remove OKX configuration file and enhance data collector with timeframes support

- Deleted the `okx_config.json` file as part of the configuration refactor.
- Updated `BaseDataCollector` to include an optional `timeframes` parameter for more flexible data collection.
- Modified `DataCollectionService` and `OKXCollector` to pass and utilize the new `timeframes` parameter.
- Enhanced `ExchangeCollectorConfig` to validate timeframes, ensuring they are provided and correctly formatted.
- Updated documentation to reflect the new configurable timeframes feature, improving clarity for users.

These changes streamline the configuration process and improve the flexibility of data collection, aligning with project standards for maintainability and usability.
This commit is contained in:
Ajasra
2025-06-07 15:46:24 +08:00
parent 24394d7b92
commit 90cb450640
8 changed files with 93 additions and 81 deletions

View File

@@ -19,10 +19,26 @@ This module provides a standardized interface for collecting real-time data from
| Exchange | Status | Features | Documentation |
|----------|---------|-----------|---------------|
| OKX | ✅ Production | Trades, Order Book, Ticker, Candles | [Guide](okx_collector.md) |
| OKX | ✅ Production | Trades, Order Book, Ticker, Configurable Timeframes (1s+) | [Guide](okx_collector.md) |
| Binance | 🔄 Planned | TBD | - |
| Coinbase | 🔄 Planned | TBD | - |
## Features
### Core Features
- Real-time data collection
- Robust error handling
- Automatic reconnection
- Health monitoring
- Configurable timeframes
- Support for 1-second intervals
- Flexible timeframe configuration
- Custom timeframe aggregation
### Exchange-Specific Features
- OKX: Full WebSocket support with configurable timeframes (1s+)
- More exchanges coming soon
## Adding New Exchanges
See [Technical Documentation](exchanges.md) for detailed implementation guide.

View File

@@ -30,6 +30,7 @@ class ExchangeCollectorConfig:
exchange: str
symbol: str
data_types: List[DataType]
timeframes: Optional[List[str]] = None # Timeframes for candle collection
auto_restart: bool = True
health_check_interval: float = 30.0
store_raw_data: bool = True
@@ -43,6 +44,11 @@ class ExchangeCollectorConfig:
raise InvalidConfigurationError("Symbol cannot be empty")
if not self.data_types:
raise InvalidConfigurationError("At least one data type must be specified")
if self.timeframes is not None:
if not all(isinstance(tf, str) for tf in self.timeframes):
raise InvalidConfigurationError("All timeframes must be strings")
if not self.timeframes:
raise InvalidConfigurationError("Timeframes list cannot be empty if provided")
```
### Registry Configuration
@@ -56,11 +62,26 @@ EXCHANGE_REGISTRY = {
'websocket': 'data.exchanges.okx.websocket.OKXWebSocketClient',
'name': 'OKX',
'supported_pairs': ['BTC-USDT', 'ETH-USDT', 'SOL-USDT', 'DOGE-USDT', 'TON-USDT'],
'supported_data_types': ['trade', 'orderbook', 'ticker', 'candles']
'supported_data_types': ['trade', 'orderbook', 'ticker', 'candles'],
'supported_timeframes': ['1s', '5s', '1m', '5m', '15m', '1h', '4h', '1d'] # Available timeframes
}
}
```
### Example Usage with Timeframes
```python
# Create collector with specific timeframes
config = ExchangeCollectorConfig(
exchange="okx",
symbol="BTC-USDT",
data_types=[DataType.TRADE, DataType.CANDLE],
timeframes=['1s', '5s', '1m', '5m'] # Specify desired timeframes
)
collector = ExchangeFactory.create_collector(config)
```
### Error Handling
Custom exceptions hierarchy for precise error handling:

View File

@@ -17,7 +17,10 @@ The OKX Data Collector provides real-time market data collection from OKX exchan
- **Trades**: Real-time trade executions (`trades` channel)
- **Orderbook**: 5-level order book depth (`books5` channel)
- **Ticker**: 24h ticker statistics (`tickers` channel)
- **Candles**: Real-time OHLCV aggregation (1s, 5s, 10s, 15s, 30s, 1m, 5m, 15m, 1h, 4h, 1d)
- **Candles**: Real-time OHLCV aggregation with configurable timeframes
- Supports any timeframe from 1s upwards
- Common timeframes: 1s, 5s, 1m, 5m, 15m, 1h, 4h, 1d
- Custom timeframes can be configured in data_collection.json
### 🔧 **Configuration Options**
- Auto-restart on failures
@@ -25,7 +28,8 @@ The OKX Data Collector provides real-time market data collection from OKX exchan
- Raw data storage toggle
- Custom ping/pong timing
- Reconnection attempts configuration
- Multi-timeframe candle aggregation
- Flexible timeframe configuration (1s, 5s, 1m, 5m, 15m, 1h, etc.)
- Configurable candle aggregation settings
## Quick Start
@@ -173,9 +177,9 @@ from data.base_collector import DataType
from data.common import CandleProcessingConfig
async def main():
# Configure multi-timeframe candle processing
# Configure multi-timeframe candle processing with 1s support
candle_config = CandleProcessingConfig(
timeframes=['1s', '5s', '10s', '15s', '30s', '1m', '5m', '15m', '1h'],
timeframes=['1s', '5s', '1m', '5m', '15m', '1h'], # Including 1s timeframe
auto_save_candles=True,
emit_incomplete_candles=False
)
@@ -184,6 +188,7 @@ async def main():
collector = OKXCollector(
symbol='BTC-USDT',
data_types=[DataType.TRADE], # Trades needed for candle aggregation
timeframes=['1s', '5s', '1m', '5m', '15m', '1h'], # Specify desired timeframes
candle_config=candle_config,
auto_restart=True,
store_raw_data=False # Disable raw storage for production