68030730e9
- Introduced a new transformation module that includes safety limits for trade operations, enhancing data integrity and preventing errors. - Refactored existing transformation logic into dedicated classes and functions, improving modularity and maintainability. - Added detailed validation for trade sizes, prices, and symbol formats, ensuring compliance with trading rules. - Implemented logging for significant operations and validation checks, aiding in monitoring and debugging. - Created a changelog to document the new features and changes, providing clarity for future development. - Developed extensive unit tests to cover the new functionality, ensuring reliability and preventing regressions. These changes significantly enhance the architecture of the transformation module, making it more robust and easier to manage.
165 lines
3.9 KiB
Markdown
165 lines
3.9 KiB
Markdown
# Transformation Module
|
|
|
|
## Purpose
|
|
The transformation module provides safe and standardized data transformation utilities for crypto trading operations, with built-in safety limits and validations to prevent errors and protect against edge cases.
|
|
|
|
## Architecture
|
|
The module is organized into several submodules:
|
|
|
|
### safety.py
|
|
Provides safety limits and validations for trading operations:
|
|
- Trade size limits (min/max)
|
|
- Price deviation checks
|
|
- Symbol format validation
|
|
- Stablecoin-specific rules
|
|
|
|
### trade.py
|
|
Handles trade data transformation with comprehensive safety checks:
|
|
- Trade side normalization
|
|
- Size and price validation
|
|
- Symbol validation
|
|
- Market price deviation checks
|
|
|
|
## Safety Limits
|
|
|
|
### Default Limits
|
|
```python
|
|
DEFAULT_LIMITS = TradeLimits(
|
|
min_size=Decimal('0.00000001'), # 1 satoshi
|
|
max_size=Decimal('10000.0'), # 10K units
|
|
min_notional=Decimal('1.0'), # Min $1
|
|
max_notional=Decimal('10000000.0'), # Max $10M
|
|
price_precision=8,
|
|
size_precision=8,
|
|
max_price_deviation=Decimal('30.0') # 30%
|
|
)
|
|
```
|
|
|
|
### Stablecoin Pairs
|
|
```python
|
|
STABLECOIN_LIMITS = DEFAULT_LIMITS._replace(
|
|
max_size=Decimal('1000000.0'), # 1M units
|
|
max_notional=Decimal('50000000.0'), # $50M
|
|
max_price_deviation=Decimal('5.0') # 5%
|
|
)
|
|
```
|
|
|
|
### Volatile Pairs
|
|
```python
|
|
VOLATILE_LIMITS = DEFAULT_LIMITS._replace(
|
|
max_size=Decimal('1000.0'), # 1K units
|
|
max_notional=Decimal('1000000.0'), # $1M
|
|
max_price_deviation=Decimal('50.0') # 50%
|
|
)
|
|
```
|
|
|
|
## Usage Examples
|
|
|
|
### Basic Trade Transformation
|
|
```python
|
|
from data.common.transformation.trade import TradeTransformer
|
|
|
|
# Initialize transformer
|
|
transformer = TradeTransformer()
|
|
|
|
# Transform trade data
|
|
trade_data = {
|
|
'symbol': 'BTC-USDT',
|
|
'side': 'buy',
|
|
'size': '1.5',
|
|
'price': '50000'
|
|
}
|
|
|
|
try:
|
|
transformed = transformer.transform_trade(trade_data)
|
|
print(f"Transformed trade: {transformed}")
|
|
except ValueError as e:
|
|
print(f"Validation error: {e}")
|
|
```
|
|
|
|
### With Market Price Validation
|
|
```python
|
|
from data.common.transformation.trade import TradeTransformer
|
|
from your_market_data_provider import MarketDataProvider
|
|
|
|
# Initialize with market data for price deviation checks
|
|
transformer = TradeTransformer(
|
|
market_data_provider=MarketDataProvider()
|
|
)
|
|
|
|
# Transform with price validation
|
|
try:
|
|
transformed = transformer.transform_trade({
|
|
'symbol': 'ETH-USDT',
|
|
'side': 'sell',
|
|
'size': '10',
|
|
'price': '2000'
|
|
})
|
|
print(f"Transformed trade: {transformed}")
|
|
except ValueError as e:
|
|
print(f"Validation error: {e}")
|
|
```
|
|
|
|
## Error Handling
|
|
|
|
The module uses explicit error handling with descriptive messages:
|
|
|
|
```python
|
|
try:
|
|
transformed = transformer.transform_trade(trade_data)
|
|
except ValueError as e:
|
|
if "below minimum" in str(e):
|
|
# Handle size too small
|
|
pass
|
|
elif "exceeds maximum" in str(e):
|
|
# Handle size too large
|
|
pass
|
|
elif "deviation" in str(e):
|
|
# Handle price deviation too large
|
|
pass
|
|
else:
|
|
# Handle other validation errors
|
|
pass
|
|
```
|
|
|
|
## Logging
|
|
|
|
The module integrates with Python's logging system for monitoring and debugging:
|
|
|
|
```python
|
|
import logging
|
|
|
|
# Configure logging
|
|
logging.basicConfig(level=logging.INFO)
|
|
logger = logging.getLogger(__name__)
|
|
|
|
# Transformer will log warnings when approaching limits
|
|
transformer = TradeTransformer()
|
|
```
|
|
|
|
## Testing
|
|
|
|
Run the test suite:
|
|
```bash
|
|
uv run pytest tests/common/transformation/test_safety.py -v
|
|
```
|
|
|
|
Key test areas:
|
|
- Trade size validation
|
|
- Price deviation checks
|
|
- Symbol format validation
|
|
- Stablecoin detection
|
|
- Edge case handling
|
|
|
|
## Dependencies
|
|
- Internal:
|
|
- `data.common.types`
|
|
- `data.common.validation`
|
|
- External:
|
|
- Python's decimal module
|
|
- Python's logging module
|
|
|
|
## Known Limitations
|
|
- Market price validation requires a market data provider
|
|
- Stablecoin detection is based on a predefined list
|
|
- Price deviation checks are percentage-based only |