vasily/TCPDashboard
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

4.0 - 3.0 Implement strategy analysis tables and repository for backtesting

Browse Source 
- Added `StrategyRun` and `StrategySignal` models to track strategy execution sessions and generated signals, respectively, ensuring a clear separation from live trading data.
- Introduced `StrategyRepository` for managing database operations related to strategy runs and signals, including methods for creating, updating, and retrieving strategy data.
- Updated `DatabaseOperations` to integrate the new repository, enhancing the overall architecture and maintaining consistency with existing database access patterns.
- Enhanced documentation to reflect the new database schema and repository functionalities, ensuring clarity for future development and usage.

These changes establish a robust foundation for strategy analysis and backtesting, aligning with project goals for modularity, performance, and maintainability.
This commit is contained in:
Vasily.onl
2025-06-12 15:29:14 +08:00
parent d34da789ec
commit f09864d61b
7 changed files with 529 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/architecture.md
View File

@@ -65,12 +65,13 @@ class BotManager:
```
### 4. Database
**Responsibility**: Data persistence and storage
**Responsibility**: Data persistence and storage for market data, bot configurations, trading signals, and strategy analysis results.
**Implementation**: `database/`
**Key Features**:
- PostgreSQL with TimescaleDB extension for time-series data
- SQLAlchemy for ORM and schema management
- Alembic for database migrations
- Dedicated tables for strategy backtesting (e.g., `strategy_signals`, `strategy_runs`) separate from live trading `signals`.
- See [Database Operations Documentation (`modules/database_operations.md`)](./modules/database_operations.md) for details.
### 5. Backtesting Engine

196
docs/modules/database_operations.md
View File

@@ -27,23 +27,23 @@ The Database Operations module (`database/operations.py`) provides a clean, cent
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ DatabaseOperations │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Health Check & Stats │ │
│ │ • Connection health monitoring │ │
│ │ • Database statistics │ │
│ │ • Performance metrics │ │
│ └─────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐ │
│ │MarketDataRepo │ │RawTradeRepo │ │ BotRepo │ │
│ │ │ │ │ │ │ │
│ │ • upsert_candle │ │ • insert_data │ │ • add │ │
│ │ • get_candles │ │ • get_trades │ │ • get_by_id │ │
│ │ • get_latest │ │ • raw_websocket │ │ • update/delete│ │
│ └─────────────────┘ └─────────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌───────────────────────────────────────────────────────────────────┐
│ DatabaseOperations │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ Health Check & Stats │ │
│ │ • Connection health monitoring │ │
│ │ • Database statistics │ │
│ │ • Performance metrics │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐ ┌───────────────────┐ │
│ │MarketDataRepo │ │RawTradeRepo │ │ BotRepo │ │ StrategyRepo │ │
│ │ │ │ │ │ │ │ │ │
│ │ • upsert_candle │ │ • insert_data │ │ • add │ │ • create_run │ │
│ │ • get_candles │ │ • get_trades │ │ • get_by_id │ │ • store_signals │ │
│ │ • get_latest │ │ • raw_websocket │ │ • update/delete│ │ • get_signals/stats │ │
│ └─────────────────┘ └─────────────────┘ └──────────────┘ └───────────────────┘ │
└───────────────────────────────────────────────────────────────────┘
│
┌─────────────────┐
│ BaseRepository │
@@ -152,6 +152,8 @@ stats = db.get_stats()
print(f"Bots: {stats['bot_count']:,}")
print(f"Candles: {stats['candle_count']:,}")
print(f"Raw trades: {stats['raw_trade_count']:,}")
print(f"Strategy runs: {stats['strategy_runs_count']:,}")
print(f"Strategy signals: {stats['strategy_signals_count']:,}")
print(f"Health: {stats['healthy']}")
```
@@ -369,6 +371,166 @@ print(f"Raw Trades Table Size: {raw_stats.get('table_size')}")
print(f"Total Raw Records: {raw_stats.get('total_records')}")
```
### StrategyRepository
Repository for `strategy_signals` and `strategy_runs` table operations.
#### Methods
##### `create_strategy_run(run_data: Dict[str, Any]) -> StrategyRun`
Create a new strategy run session.
**Parameters:**
- `run_data`: Dictionary containing run information (`strategy_name`, `symbol`, `timeframe`, `start_time`, `status`, `config`, `run_metadata`)
**Returns:** The newly created `StrategyRun` object.
```python
from datetime import datetime, timezone
run_data = {
"strategy_name": "EMA Crossover Backtest",
"symbol": "BTC-USDT",
"timeframe": "1h",
"start_time": datetime(2023, 1, 1, tzinfo=timezone.utc),
"status": "running",
"config": {"fast_ema": 12, "slow_ema": 26},
"run_metadata": {"backtest_period": "2023-Q1"}
}
new_run = db.strategies.create_strategy_run(run_data)
print(f"Created strategy run with ID: {new_run.id}")
```
##### `get_strategy_run_by_id(run_id: int) -> Optional[StrategyRun]`
Get a strategy run by its ID.
```python
run = db.strategies.get_strategy_run_by_id(1)
if run:
print(f"Found strategy run: {run.strategy_name} ({run.status})")
```
##### `update_strategy_run(run_id: int, update_data: Dict[str, Any]) -> Optional[StrategyRun]`
Update a strategy run's information.
```python
update_payload = {"status": "completed", "total_signals": 150}
updated_run = db.strategies.update_strategy_run(1, update_payload)
if updated_run:
print(f"Strategy run status updated to: {updated_run.status}")
```
##### `complete_strategy_run(run_id: int, total_signals: int) -> bool`
Mark a strategy run as completed.
**Parameters:**
- `run_id`: The ID of the strategy run to complete.
- `total_signals`: The total number of signals generated during the run.
**Returns:** `True` if successful, `False` otherwise.
```python
success = db.strategies.complete_strategy_run(1, 150)
if success:
print("Strategy run marked as completed.")
```
##### `store_strategy_signals(run_id: int, strategy_results: List[StrategyResult]) -> int`
Store multiple strategy signals from strategy results.
**Parameters:**
- `run_id`: The strategy run ID these signals belong to.
- `strategy_results`: List of `StrategyResult` objects containing signals.
**Returns:** Number of signals stored.
```python
from strategies.data_types import StrategyResult, StrategySignal, SignalType
from datetime import datetime, timezone
from decimal import Decimal
signal1 = StrategySignal(
timestamp=datetime.now(timezone.utc),
symbol="BTC-USDT",
timeframe="1h",
signal_type=SignalType.BUY,
price=Decimal("50000.0"),
confidence=Decimal("0.9")
)
result1 = StrategyResult(
timestamp=datetime.now(timezone.utc),
symbol="BTC-USDT",
timeframe="1h",
strategy_name="EMA Crossover",
signals=[signal1],
indicators_used={"ema_fast": 49900.0, "ema_slow": 50100.0}
)
signals_stored = db.strategies.store_strategy_signals(new_run.id, [result1])
print(f"Stored {signals_stored} signals.")
```
##### `get_strategy_signals(run_id: Optional[int] = None, strategy_name: Optional[str] = None, symbol: Optional[str] = None, timeframe: Optional[str] = None, start_time: Optional[datetime] = None, end_time: Optional[datetime] = None, signal_type: Optional[str] = None, limit: Optional[int] = None) -> List[StrategySignal]`
Retrieve strategy signals with flexible filtering.
**Parameters:**
- `run_id`: Filter by strategy run ID.
- `strategy_name`: Filter by strategy name.
- `symbol`: Filter by trading symbol.
- `timeframe`: Filter by timeframe.
- `start_time`: Filter signals after this time.
- `end_time`: Filter signals before this time.
- `signal_type`: Filter by signal type (e.g., "buy", "sell", "entry_long").
- `limit`: Maximum number of signals to return.
**Returns:** List of `StrategySignal` objects.
```python
signals = db.strategies.get_strategy_signals(
run_id=new_run.id,
signal_type="buy",
limit=10
)
for signal in signals:
print(f"Signal: {signal.signal_type} at {signal.price} on {signal.timestamp}")
```
##### `get_strategy_signal_stats(run_id: Optional[int] = None) -> Dict[str, Any]`
Get statistics about strategy signals.
**Parameters:**
- `run_id`: Optional run ID to get stats for a specific run.
**Returns:** Dictionary with statistics like total signals, counts by type, average confidence.
```python
signal_stats = db.strategies.get_strategy_signal_stats(run_id=new_run.id)
print(f"Total Signals: {signal_stats['total_signals']}")
print(f"Signal Counts: {signal_stats['signal_counts']}")
print(f"Average Confidence: {signal_stats['average_confidence']:.2f}")
```
##### `cleanup_old_strategy_data(days_to_keep: int = 30) -> Dict[str, int]`
Clean up old strategy signals and runs.
**Parameters:**
- `days_to_keep`: Number of days to retain completed strategy runs and their signals.
**Returns:** Dictionary with counts of deleted runs and signals.
```python
deleted_info = db.strategies.cleanup_old_strategy_data(days_to_keep=60)
print(f"Deleted {deleted_info['deleted_runs']} old strategy runs.")
```
## Error Handling
The database operations module includes comprehensive error handling with custom exceptions.