74 lines
4.4 KiB
Markdown
74 lines
4.4 KiB
Markdown
# Storage Utilities
|
|
|
|
This document describes the storage utility functions found in `cycles/utils/storage.py`.
|
|
|
|
## Overview
|
|
|
|
The `storage.py` module provides a `Storage` class designed for handling the loading and saving of data and results. It supports operations with CSV and JSON files and integrates with pandas DataFrames for data manipulation. The class also manages the creation of necessary `results` and `data` directories.
|
|
|
|
## Constants
|
|
|
|
- `RESULTS_DIR`: Defines the default directory name for storing results (default: "results").
|
|
- `DATA_DIR`: Defines the default directory name for storing input data (default: "data").
|
|
|
|
## Class: `Storage`
|
|
|
|
Handles storage operations for data and results.
|
|
|
|
### `__init__(self, logging=None, results_dir=RESULTS_DIR, data_dir=DATA_DIR)`
|
|
|
|
- **Description**: Initializes the `Storage` class. It creates the results and data directories if they don't already exist.
|
|
- **Parameters**:
|
|
- `logging` (optional): A logging instance for outputting information. Defaults to `None`.
|
|
- `results_dir` (str, optional): Path to the directory for storing results. Defaults to `RESULTS_DIR`.
|
|
- `data_dir` (str, optional): Path to the directory for storing data. Defaults to `DATA_DIR`.
|
|
|
|
### `load_data(self, file_path, start_date, stop_date)`
|
|
|
|
- **Description**: Loads data from a specified file (CSV or JSON), performs type optimization, filters by date range, and converts column names to lowercase. The timestamp column is set as the DataFrame index.
|
|
- **Parameters**:
|
|
- `file_path` (str): Path to the data file (relative to `data_dir`).
|
|
- `start_date` (datetime-like): The start date for filtering data.
|
|
- `stop_date` (datetime-like): The end date for filtering data.
|
|
- **Returns**: `pandas.DataFrame` - The loaded and processed data, with a `timestamp` index. Returns an empty DataFrame on error.
|
|
|
|
### `save_data(self, data: pd.DataFrame, file_path: str)`
|
|
|
|
- **Description**: Saves a pandas DataFrame to a CSV file within the `data_dir`. If the DataFrame has a DatetimeIndex, it's converted to a Unix timestamp (seconds since epoch) and stored in a column named 'timestamp', which becomes the first column in the CSV. The DataFrame's active index is not saved if a 'timestamp' column is created.
|
|
- **Parameters**:
|
|
- `data` (pd.DataFrame): The DataFrame to save.
|
|
- `file_path` (str): Path to the data file (relative to `data_dir`).
|
|
|
|
### `format_row(self, row)`
|
|
|
|
- **Description**: Formats a dictionary row for output to a combined results CSV file, applying specific string formatting for percentages and float values.
|
|
- **Parameters**:
|
|
- `row` (dict): The row of data to format.
|
|
- **Returns**: `dict` - The formatted row.
|
|
|
|
### `write_results_chunk(self, filename, fieldnames, rows, write_header=False, initial_usd=None)`
|
|
|
|
- **Description**: Writes a chunk of results (list of dictionaries) to a CSV file. Can append to an existing file or write a new one with a header. An optional `initial_usd` can be written as a comment in the header.
|
|
- **Parameters**:
|
|
- `filename` (str): The name of the file to write to (path is absolute or relative to current working dir).
|
|
- `fieldnames` (list): A list of strings representing the CSV header/column names.
|
|
- `rows` (list): A list of dictionaries, where each dictionary is a row.
|
|
- `write_header` (bool, optional): If `True`, writes the header. Defaults to `False`.
|
|
- `initial_usd` (numeric, optional): If provided and `write_header` is `True`, this value is written as a comment in the CSV header. Defaults to `None`.
|
|
|
|
### `write_results_combined(self, filename, fieldnames, rows)`
|
|
|
|
- **Description**: Writes combined results to a CSV file in the `results_dir`. Uses tab as a delimiter and formats rows using `format_row`.
|
|
- **Parameters**:
|
|
- `filename` (str): The name of the file to write to (relative to `results_dir`).
|
|
- `fieldnames` (list): A list of strings representing the CSV header/column names.
|
|
- `rows` (list): A list of dictionaries, where each dictionary is a row.
|
|
|
|
### `write_trades(self, all_trade_rows, trades_fieldnames)`
|
|
|
|
- **Description**: Writes trade data to separate CSV files based on timeframe and stop-loss percentage. Files are named `trades_{tf}_ST{sl_percent}pct.csv` and stored in `results_dir`.
|
|
- **Parameters**:
|
|
- `all_trade_rows` (list): A list of dictionaries, where each dictionary represents a trade.
|
|
- `trades_fieldnames` (list): A list of strings for the CSV header of trade files.
|
|
|