191 lines
10 KiB
Markdown
191 lines
10 KiB
Markdown
# Product Requirements Document: Migration from Dash/Plotly to PySide6/PyQtGraph
|
|
|
|
## Introduction/Overview
|
|
|
|
This PRD outlines the complete migration of the orderflow backtest visualization system from the current Dash/Plotly web-based implementation to a native desktop application using PySide6 and PyQtGraph. The migration addresses critical issues with the current implementation including async problems, debugging difficulties, performance bottlenecks, and data handling inefficiencies.
|
|
|
|
The goal is to create a robust, high-performance desktop application that provides better control over the codebase, eliminates current visualization bugs (particularly the CVD graph display issue), and enables future real-time trading strategy monitoring capabilities.
|
|
|
|
## Goals
|
|
|
|
1. **Eliminate Current Technical Issues**
|
|
- Resolve async-related problems causing visualization failures
|
|
- Fix CVD graph display issues that persist despite correct-looking code
|
|
- Enable proper debugging capabilities with breakpoint support
|
|
- Improve overall application performance and responsiveness
|
|
|
|
2. **Improve Development Experience**
|
|
- Gain better control over the codebase through native Python implementation
|
|
- Reduce dependency on intermediate file-based data exchange
|
|
- Simplify the development and debugging workflow
|
|
- Establish a foundation for future real-time capabilities
|
|
|
|
3. **Maintain and Enhance Visualization Capabilities**
|
|
- Preserve all existing chart types and interactions
|
|
- Improve performance for granular dataset handling
|
|
- Prepare infrastructure for real-time data streaming
|
|
- Enhance user experience through native desktop interface
|
|
|
|
## User Stories
|
|
|
|
1. **As a trading strategy developer**, I want to visualize OHLC data with volume, OBI, and CVD indicators in a single, synchronized view so that I can analyze market behavior patterns effectively.
|
|
|
|
2. **As a data analyst**, I want to zoom, pan, and select specific time ranges on charts so that I can focus on relevant market periods for detailed analysis.
|
|
|
|
3. **As a system developer**, I want to debug visualization issues with breakpoints and proper debugging tools so that I can identify and fix problems efficiently.
|
|
|
|
4. **As a performance-conscious user**, I want smooth chart rendering and interactions even with large, granular datasets so that my analysis workflow is not interrupted by lag or freezing.
|
|
|
|
5. **As a future trading system operator**, I want a foundation that can handle real-time data updates so that I can monitor live trading strategies effectively.
|
|
|
|
## Functional Requirements
|
|
|
|
### Core Visualization Components
|
|
|
|
1. **Main Chart Window**
|
|
- The system must display OHLC candlestick charts in a primary plot area
|
|
- The system must allow customizable time window selection for OHLC display
|
|
- The system must synchronize all chart components to the same time axis
|
|
|
|
2. **Integrated Indicator Charts**
|
|
- The system must display Volume bars below the OHLC chart
|
|
- The system must display Order Book Imbalance (OBI) indicator
|
|
- The system must display Cumulative Volume Delta (CVD) indicator
|
|
- All indicators must share the same X-axis as the OHLC chart
|
|
|
|
3. **Depth Chart Visualization**
|
|
- The system must display order book depth at selected time snapshots
|
|
- The system must update depth visualization based on time selection
|
|
- The system must provide clear bid/ask visualization
|
|
|
|
### User Interaction Features
|
|
|
|
4. **Chart Navigation**
|
|
- The system must support zoom in/out functionality across all charts
|
|
- The system must allow panning across time ranges
|
|
- The system must provide time range selection capabilities
|
|
- The system must support rectangle selection for detailed analysis
|
|
|
|
5. **Data Inspection**
|
|
- The system must display mouseover information for all chart elements
|
|
- The system must show precise values for OHLC, volume, OBI, and CVD data points
|
|
- The system must provide crosshair functionality for precise data reading
|
|
|
|
### Technical Architecture
|
|
|
|
6. **Application Framework**
|
|
- The system must be built using PySide6 for the GUI framework
|
|
- The system must use PyQtGraph for all chart rendering and interactions
|
|
- The system must implement a native desktop application architecture
|
|
|
|
7. **Data Integration**
|
|
- The system must integrate with existing data processing modules (metrics_calculator, ohlc_processor, orderbook_manager)
|
|
- The system must eliminate dependency on intermediate JSON files for data display
|
|
- The system must support direct in-memory data transfer between processing and visualization
|
|
|
|
8. **Performance Requirements**
|
|
- The system must handle granular datasets efficiently without UI blocking
|
|
- The system must provide smooth chart interactions (zoom, pan, selection)
|
|
- The system must render updates in less than 100ms for typical dataset sizes
|
|
|
|
### Development and Debugging
|
|
|
|
9. **Code Quality**
|
|
- The system must be fully debuggable with standard Python debugging tools
|
|
- The system must follow the existing project architecture patterns
|
|
- The system must maintain clean separation between data processing and visualization
|
|
|
|
## Non-Goals (Out of Scope)
|
|
|
|
1. **Web Interface Maintenance** - The existing Dash/Plotly implementation will be completely replaced, not maintained in parallel
|
|
|
|
2. **Backward Compatibility** - No requirement to maintain compatibility with existing Dash/Plotly components or web-based deployment
|
|
|
|
3. **Multi-Platform Distribution** - Initial focus on development environment only, not packaging for distribution
|
|
|
|
4. **Real-Time Implementation** - While the architecture should support future real-time capabilities, the initial migration will focus on historical data visualization
|
|
|
|
5. **Advanced Chart Types** - Only migrate existing chart types; new visualization features are out of scope for this migration
|
|
|
|
## Design Considerations
|
|
|
|
### User Interface Layout
|
|
- **Main Window Structure**: Primary chart area with integrated indicators below
|
|
- **Control Panel**: Side panel or toolbar for time range selection and chart configuration
|
|
- **Status Bar**: Display current data range, loading status, and performance metrics
|
|
- **Menu System**: File operations, view options, and application settings
|
|
|
|
### PyQtGraph Integration
|
|
- **Plot Organization**: Use PyQtGraph's PlotWidget for main charts with linked axes
|
|
- **Custom Plot Items**: Implement custom plot items for OHLC candlesticks and depth visualization
|
|
- **Performance Optimization**: Utilize PyQtGraph's fast plotting capabilities for large datasets
|
|
|
|
### Data Flow Architecture
|
|
- **Direct Memory Access**: Replace JSON file intermediates with direct Python object passing
|
|
- **Lazy Loading**: Implement efficient data loading strategies for large time ranges
|
|
- **Caching Strategy**: Cache processed data to improve navigation performance
|
|
|
|
## Technical Considerations
|
|
|
|
### Dependencies and Integration
|
|
- **PySide6**: Main GUI framework, provides native desktop capabilities
|
|
- **PyQtGraph**: High-performance plotting library, optimized for real-time data
|
|
- **Existing Modules**: Maintain integration with metrics_calculator.py, ohlc_processor.py, orderbook_manager.py
|
|
- **Database Integration**: Continue using existing SQLite database through db_interpreter.py
|
|
|
|
### Migration Strategy (Iterative Implementation)
|
|
- **Phase 1**: Basic PySide6 window with single PyQtGraph plot
|
|
- **Phase 2**: OHLC candlestick chart implementation
|
|
- **Phase 3**: Volume, OBI, and CVD indicator integration
|
|
- **Phase 4**: Depth chart implementation
|
|
- **Phase 5**: User interaction features (zoom, pan, selection)
|
|
- **Phase 6**: Data integration and performance optimization
|
|
|
|
### Performance Considerations
|
|
- **Memory Management**: Efficient data structure handling for large datasets
|
|
- **Rendering Optimization**: Use PyQtGraph's ViewBox and plotting optimizations
|
|
- **Thread Safety**: Proper handling of data processing in background threads
|
|
- **Resource Cleanup**: Proper cleanup of chart objects and data structures
|
|
|
|
## Success Metrics
|
|
|
|
### Technical Success Criteria
|
|
1. **Bug Resolution**: CVD graph displays correctly and all existing visualization bugs are resolved
|
|
2. **Performance Improvement**: Chart interactions respond within 100ms for typical datasets
|
|
3. **Debugging Capability**: Developers can set breakpoints and debug visualization code effectively
|
|
4. **Data Handling**: Elimination of intermediate JSON files reduces data transfer overhead by 50%
|
|
|
|
### User Experience Success Criteria
|
|
1. **Feature Parity**: All existing chart types and interactions are preserved and functional
|
|
2. **Responsiveness**: Application feels more responsive than the current Dash implementation
|
|
3. **Stability**: No crashes or freezing during normal chart operations
|
|
4. **Visual Quality**: Charts render clearly with proper scaling and anti-aliasing
|
|
|
|
### Development Success Criteria
|
|
1. **Code Maintainability**: New codebase follows established project patterns and is easier to maintain
|
|
2. **Development Velocity**: Future visualization features can be implemented more quickly
|
|
3. **Testing Capability**: Comprehensive testing can be performed with proper debugging tools
|
|
4. **Architecture Foundation**: System is ready for future real-time data integration
|
|
|
|
## Open Questions
|
|
|
|
1. **Data Loading Strategy**: Should we implement progressive loading for very large datasets, or rely on existing data chunking mechanisms?
|
|
|
|
2. **Configuration Management**: How should chart configuration and user preferences be stored and managed in the desktop application?
|
|
|
|
3. **Error Handling**: What specific error handling and user feedback mechanisms should be implemented for data loading and processing failures?
|
|
|
|
4. **Performance Monitoring**: Should we include built-in performance monitoring and profiling tools in the application?
|
|
|
|
5. **Future Real-Time Integration**: What specific interface patterns should be established now to facilitate future real-time data streaming integration?
|
|
|
|
## Implementation Approach
|
|
|
|
This migration will follow the iterative development workflow with explicit approval checkpoints between each phase. Each implementation phase will be:
|
|
- Limited to manageable scope (≤250 lines per module)
|
|
- Tested immediately after implementation
|
|
- Integrated with existing data processing modules
|
|
- Validated for performance and functionality before proceeding to the next phase
|
|
|
|
The implementation will begin with basic PySide6 application structure and progressively add PyQtGraph visualization capabilities while maintaining integration with the existing data processing pipeline.
|