Remove OKX configuration file and enhance data collector with timeframes support
- Deleted the `okx_config.json` file as part of the configuration refactor. - Updated `BaseDataCollector` to include an optional `timeframes` parameter for more flexible data collection. - Modified `DataCollectionService` and `OKXCollector` to pass and utilize the new `timeframes` parameter. - Enhanced `ExchangeCollectorConfig` to validate timeframes, ensuring they are provided and correctly formatted. - Updated documentation to reflect the new configurable timeframes feature, improving clarity for users. These changes streamline the configuration process and improve the flexibility of data collection, aligning with project standards for maintainability and usability.
This commit is contained in:
@@ -19,10 +19,26 @@ This module provides a standardized interface for collecting real-time data from
|
||||
|
||||
| Exchange | Status | Features | Documentation |
|
||||
|----------|---------|-----------|---------------|
|
||||
| OKX | ✅ Production | Trades, Order Book, Ticker, Candles | [Guide](okx_collector.md) |
|
||||
| OKX | ✅ Production | Trades, Order Book, Ticker, Configurable Timeframes (1s+) | [Guide](okx_collector.md) |
|
||||
| Binance | 🔄 Planned | TBD | - |
|
||||
| Coinbase | 🔄 Planned | TBD | - |
|
||||
|
||||
## Features
|
||||
|
||||
### Core Features
|
||||
- Real-time data collection
|
||||
- Robust error handling
|
||||
- Automatic reconnection
|
||||
- Health monitoring
|
||||
- Configurable timeframes
|
||||
- Support for 1-second intervals
|
||||
- Flexible timeframe configuration
|
||||
- Custom timeframe aggregation
|
||||
|
||||
### Exchange-Specific Features
|
||||
- OKX: Full WebSocket support with configurable timeframes (1s+)
|
||||
- More exchanges coming soon
|
||||
|
||||
## Adding New Exchanges
|
||||
|
||||
See [Technical Documentation](exchanges.md) for detailed implementation guide.
|
||||
|
||||
@@ -30,6 +30,7 @@ class ExchangeCollectorConfig:
|
||||
exchange: str
|
||||
symbol: str
|
||||
data_types: List[DataType]
|
||||
timeframes: Optional[List[str]] = None # Timeframes for candle collection
|
||||
auto_restart: bool = True
|
||||
health_check_interval: float = 30.0
|
||||
store_raw_data: bool = True
|
||||
@@ -43,6 +44,11 @@ class ExchangeCollectorConfig:
|
||||
raise InvalidConfigurationError("Symbol cannot be empty")
|
||||
if not self.data_types:
|
||||
raise InvalidConfigurationError("At least one data type must be specified")
|
||||
if self.timeframes is not None:
|
||||
if not all(isinstance(tf, str) for tf in self.timeframes):
|
||||
raise InvalidConfigurationError("All timeframes must be strings")
|
||||
if not self.timeframes:
|
||||
raise InvalidConfigurationError("Timeframes list cannot be empty if provided")
|
||||
```
|
||||
|
||||
### Registry Configuration
|
||||
@@ -56,11 +62,26 @@ EXCHANGE_REGISTRY = {
|
||||
'websocket': 'data.exchanges.okx.websocket.OKXWebSocketClient',
|
||||
'name': 'OKX',
|
||||
'supported_pairs': ['BTC-USDT', 'ETH-USDT', 'SOL-USDT', 'DOGE-USDT', 'TON-USDT'],
|
||||
'supported_data_types': ['trade', 'orderbook', 'ticker', 'candles']
|
||||
'supported_data_types': ['trade', 'orderbook', 'ticker', 'candles'],
|
||||
'supported_timeframes': ['1s', '5s', '1m', '5m', '15m', '1h', '4h', '1d'] # Available timeframes
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Example Usage with Timeframes
|
||||
|
||||
```python
|
||||
# Create collector with specific timeframes
|
||||
config = ExchangeCollectorConfig(
|
||||
exchange="okx",
|
||||
symbol="BTC-USDT",
|
||||
data_types=[DataType.TRADE, DataType.CANDLE],
|
||||
timeframes=['1s', '5s', '1m', '5m'] # Specify desired timeframes
|
||||
)
|
||||
|
||||
collector = ExchangeFactory.create_collector(config)
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
Custom exceptions hierarchy for precise error handling:
|
||||
|
||||
@@ -17,7 +17,10 @@ The OKX Data Collector provides real-time market data collection from OKX exchan
|
||||
- **Trades**: Real-time trade executions (`trades` channel)
|
||||
- **Orderbook**: 5-level order book depth (`books5` channel)
|
||||
- **Ticker**: 24h ticker statistics (`tickers` channel)
|
||||
- **Candles**: Real-time OHLCV aggregation (1s, 5s, 10s, 15s, 30s, 1m, 5m, 15m, 1h, 4h, 1d)
|
||||
- **Candles**: Real-time OHLCV aggregation with configurable timeframes
|
||||
- Supports any timeframe from 1s upwards
|
||||
- Common timeframes: 1s, 5s, 1m, 5m, 15m, 1h, 4h, 1d
|
||||
- Custom timeframes can be configured in data_collection.json
|
||||
|
||||
### 🔧 **Configuration Options**
|
||||
- Auto-restart on failures
|
||||
@@ -25,7 +28,8 @@ The OKX Data Collector provides real-time market data collection from OKX exchan
|
||||
- Raw data storage toggle
|
||||
- Custom ping/pong timing
|
||||
- Reconnection attempts configuration
|
||||
- Multi-timeframe candle aggregation
|
||||
- Flexible timeframe configuration (1s, 5s, 1m, 5m, 15m, 1h, etc.)
|
||||
- Configurable candle aggregation settings
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -173,9 +177,9 @@ from data.base_collector import DataType
|
||||
from data.common import CandleProcessingConfig
|
||||
|
||||
async def main():
|
||||
# Configure multi-timeframe candle processing
|
||||
# Configure multi-timeframe candle processing with 1s support
|
||||
candle_config = CandleProcessingConfig(
|
||||
timeframes=['1s', '5s', '10s', '15s', '30s', '1m', '5m', '15m', '1h'],
|
||||
timeframes=['1s', '5s', '1m', '5m', '15m', '1h'], # Including 1s timeframe
|
||||
auto_save_candles=True,
|
||||
emit_incomplete_candles=False
|
||||
)
|
||||
@@ -184,6 +188,7 @@ async def main():
|
||||
collector = OKXCollector(
|
||||
symbol='BTC-USDT',
|
||||
data_types=[DataType.TRADE], # Trades needed for candle aggregation
|
||||
timeframes=['1s', '5s', '1m', '5m', '15m', '1h'], # Specify desired timeframes
|
||||
candle_config=candle_config,
|
||||
auto_restart=True,
|
||||
store_raw_data=False # Disable raw storage for production
|
||||
|
||||
Reference in New Issue
Block a user