Cycles/IncrementalTrader/docs/architecture.md

Architecture Overview

Design Philosophy

IncrementalTrader is built around the principle of incremental computation - processing new data points efficiently without recalculating the entire history. This approach provides significant performance benefits for real-time trading applications.

Core Principles

1. Modularity: Clear separation of concerns between strategies, execution, and testing
2. Efficiency: Constant memory usage and minimal computational overhead
3. Extensibility: Easy to add new strategies, indicators, and features
4. Reliability: Robust error handling and comprehensive testing
5. Simplicity: Clean APIs that are easy to understand and use

System Architecture

┌─────────────────────────────────────────────────────────────┐
│                    IncrementalTrader                        │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │ Strategies  │  │   Trader    │  │    Backtester       │  │
│  │             │  │             │  │                     │  │
│  │ • Base      │  │ • Execution │  │ • Configuration     │  │
│  │ • MetaTrend │  │ • Position  │  │ • Results           │  │
│  │ • Random    │  │ • Tracking  │  │ • Optimization      │  │
│  │ • BBRS      │  │             │  │                     │  │
│  │             │  │             │  │                     │  │
│  │ Indicators  │  │             │  │                     │  │
│  │ • Supertrend│  │             │  │                     │  │
│  │ • Bollinger │  │             │  │                     │  │
│  │ • RSI       │  │             │  │                     │  │
│  └─────────────┘  └─────────────┘  └─────────────────────┘  │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Component Details

Strategies Module

The strategies module contains all trading logic and signal generation:

• Base Classes: IncStrategyBase provides the foundation for all strategies
• Timeframe Aggregation: Built-in support for multiple timeframes
• Signal Generation: Standardized signal types (BUY, SELL, HOLD)
• Incremental Indicators: Memory-efficient technical indicators

Strategy Lifecycle

# 1. Initialize strategy with parameters
strategy = MetaTrendStrategy("metatrend", params={"timeframe": "15min"})

# 2. Process data points sequentially
for timestamp, ohlcv in data_stream:
    signal = strategy.process_data_point(timestamp, ohlcv)
    
# 3. Get current state and signals
current_signal = strategy.get_current_signal()

Trader Module

The trader module handles trade execution and position management:

• Trade Execution: Converts strategy signals into trades
• Position Management: Tracks USD/coin balances and position state
• Risk Management: Stop-loss and take-profit handling
• Performance Tracking: Real-time performance metrics

Trading Workflow

# 1. Create trader with strategy
trader = IncTrader(strategy, initial_usd=10000)

# 2. Process data and execute trades
for timestamp, ohlcv in data_stream:
    trader.process_data_point(timestamp, ohlcv)
    
# 3. Get final results
results = trader.get_results()

Backtester Module

The backtester module provides comprehensive testing capabilities:

• Single Strategy Testing: Test individual strategies
• Parameter Optimization: Systematic parameter sweeps
• Multiprocessing: Parallel execution for faster testing
• Results Analysis: Comprehensive performance metrics

Backtesting Process

# 1. Configure backtest
config = BacktestConfig(
    initial_usd=10000,
    stop_loss_pct=0.03,
    start_date="2024-01-01",
    end_date="2024-12-31"
)

# 2. Run backtest
backtester = IncBacktester()
results = backtester.run_single_strategy(strategy, config)

# 3. Analyze results
performance = results['performance_metrics']

Data Flow

Real-time Processing

Market Data → Strategy → Signal → Trader → Trade Execution
     ↓           ↓        ↓        ↓           ↓
  OHLCV      Indicators  BUY/SELL Position   Portfolio
   Data      Updates     Signals   Updates    Updates

Backtesting Flow

Historical Data → Backtester → Multiple Traders → Results Aggregation
       ↓             ↓             ↓                    ↓
   Time Series   Strategy      Trade Records      Performance
    OHLCV       Instances      Collections         Metrics

Memory Management

Incremental Computation

Traditional batch processing recalculates everything for each new data point:

# Batch approach - O(n) memory, O(n) computation
def calculate_sma(prices, period):
    return [sum(prices[i:i+period])/period for i in range(len(prices)-period+1)]

Incremental approach maintains only necessary state:

# Incremental approach - O(1) memory, O(1) computation
class IncrementalSMA:
    def __init__(self, period):
        self.period = period
        self.values = deque(maxlen=period)
        self.sum = 0
        
    def update(self, value):
        if len(self.values) == self.period:
            self.sum -= self.values[0]
        self.values.append(value)
        self.sum += value
        
    def get_value(self):
        return self.sum / len(self.values) if self.values else 0

Benefits

• Constant Memory: Memory usage doesn't grow with data history
• Fast Updates: New data points processed in constant time
• Real-time Capable: Suitable for live trading applications
• Scalable: Performance independent of history length

Error Handling

Strategy Level

• Input validation for all parameters
• Graceful handling of missing or invalid data
• Fallback mechanisms for indicator failures

Trader Level

• Position state validation
• Trade execution error handling
• Balance consistency checks

System Level

• Comprehensive logging at all levels
• Exception propagation with context
• Recovery mechanisms for transient failures

Performance Characteristics

Computational Complexity

Operation	Batch Approach	Incremental Approach
Memory Usage	O(n)	O(1)
Update Time	O(n)	O(1)
Initialization	O(1)	O(k) where k = warmup period

Benchmarks

• Processing Speed: ~10x faster than batch recalculation
• Memory Usage: ~100x less memory for long histories
• Latency: Sub-millisecond processing for new data points

Extensibility

Adding New Strategies

1. Inherit from IncStrategyBase
2. Implement process_data_point() method
3. Return appropriate IncStrategySignal objects
4. Register in strategy module

Adding New Indicators

1. Implement incremental update logic
2. Maintain minimal state for calculations
3. Provide consistent API (update/get_value)
4. Add comprehensive tests

Integration Points

• Data Sources: Easy to connect different data feeds
• Execution Engines: Pluggable trade execution backends
• Risk Management: Configurable risk management rules
• Reporting: Extensible results and analytics framework

Testing Strategy

Unit Tests

• Individual component testing
• Mock data for isolated testing
• Edge case validation

Integration Tests

• End-to-end workflow testing
• Real data validation
• Performance benchmarking

Accuracy Validation

• Comparison with batch implementations
• Historical data validation
• Signal timing verification

---

This architecture provides a solid foundation for building efficient, scalable, and maintainable trading systems while keeping the complexity manageable and the interfaces clean.