Enhance DataLoader and MinuteDataBuffer for improved data handling
- Added error handling in DataLoader to attempt reading CSV files with a fallback to the Python engine if the default engine fails. - Converted numpy float32 columns to Python float for compatibility in DataLoader. - Updated MinuteDataBuffer to accept both Python and numpy numeric types, ensuring consistent data validation and conversion.
This commit is contained in:
@@ -319,8 +319,13 @@ class MinuteDataBuffer:
|
||||
for field in required_fields:
|
||||
if field not in ohlcv_data:
|
||||
raise ValueError(f"Missing required field: {field}")
|
||||
if not isinstance(ohlcv_data[field], (int, float)):
|
||||
# Accept both Python numeric types and numpy numeric types
|
||||
if not isinstance(ohlcv_data[field], (int, float, np.number)):
|
||||
raise ValueError(f"Field {field} must be numeric, got {type(ohlcv_data[field])}")
|
||||
|
||||
# Convert numpy types to Python types to ensure compatibility
|
||||
if isinstance(ohlcv_data[field], np.number):
|
||||
ohlcv_data[field] = float(ohlcv_data[field])
|
||||
|
||||
# Check timestamp ordering (allow equal timestamps for updates)
|
||||
if self._last_timestamp is not None and timestamp < self._last_timestamp:
|
||||
|
||||
Reference in New Issue
Block a user