Enhance DataLoader and MinuteDataBuffer for improved data handling

- Added error handling in DataLoader to attempt reading CSV files with a fallback to the Python engine if the default engine fails.
- Converted numpy float32 columns to Python float for compatibility in DataLoader.
- Updated MinuteDataBuffer to accept both Python and numpy numeric types, ensuring consistent data validation and conversion.
This commit is contained in:
Ajasra
2025-05-29 14:21:16 +08:00
parent 790bd9ccdd
commit b0ea701020
2 changed files with 25 additions and 2 deletions

View File

@@ -319,8 +319,13 @@ class MinuteDataBuffer:
for field in required_fields:
if field not in ohlcv_data:
raise ValueError(f"Missing required field: {field}")
if not isinstance(ohlcv_data[field], (int, float)):
# Accept both Python numeric types and numpy numeric types
if not isinstance(ohlcv_data[field], (int, float, np.number)):
raise ValueError(f"Field {field} must be numeric, got {type(ohlcv_data[field])}")
# Convert numpy types to Python types to ensure compatibility
if isinstance(ohlcv_data[field], np.number):
ohlcv_data[field] = float(ohlcv_data[field])
# Check timestamp ordering (allow equal timestamps for updates)
if self._last_timestamp is not None and timestamp < self._last_timestamp: