Cycles/docs/utils_storage.md

Storage Utilities

This document describes the storage utility functions found in cycles/utils/storage.py.

Overview

The storage.py module provides a Storage class designed for handling the loading and saving of data and results. It supports operations with CSV and JSON files and integrates with pandas DataFrames for data manipulation. The class also manages the creation of necessary results and data directories.

Constants

• RESULTS_DIR: Defines the default directory name for storing results (default: "results").
• DATA_DIR: Defines the default directory name for storing input data (default: "data").

Class: Storage

Handles storage operations for data and results.

__init__(self, logging=None, results_dir=RESULTS_DIR, data_dir=DATA_DIR)

• Description: Initializes the Storage class. It creates the results and data directories if they don't already exist.
• Parameters:
  • logging (optional): A logging instance for outputting information. Defaults to None.
  • results_dir (str, optional): Path to the directory for storing results. Defaults to RESULTS_DIR.
  • data_dir (str, optional): Path to the directory for storing data. Defaults to DATA_DIR.

load_data(self, file_path, start_date, stop_date)

• Description: Loads data from a specified file (CSV or JSON), performs type optimization, filters by date range, and converts column names to lowercase. The timestamp column is set as the DataFrame index.
• Parameters:
  • file_path (str): Path to the data file (relative to data_dir).
  • start_date (datetime-like): The start date for filtering data.
  • stop_date (datetime-like): The end date for filtering data.
• Returns: pandas.DataFrame - The loaded and processed data, with a timestamp index. Returns an empty DataFrame on error.

save_data(self, data: pd.DataFrame, file_path: str)

• Description: Saves a pandas DataFrame to a CSV file within the data_dir. If the DataFrame has a DatetimeIndex, it's converted to a Unix timestamp (seconds since epoch) and stored in a column named 'timestamp', which becomes the first column in the CSV. The DataFrame's active index is not saved if a 'timestamp' column is created.
• Parameters:
  • data (pd.DataFrame): The DataFrame to save.
  • file_path (str): Path to the data file (relative to data_dir).

format_row(self, row)

• Description: Formats a dictionary row for output to a combined results CSV file, applying specific string formatting for percentages and float values.
• Parameters:
  • row (dict): The row of data to format.
• Returns: dict - The formatted row.

write_results_chunk(self, filename, fieldnames, rows, write_header=False, initial_usd=None)

• Description: Writes a chunk of results (list of dictionaries) to a CSV file. Can append to an existing file or write a new one with a header. An optional initial_usd can be written as a comment in the header.
• Parameters:
  • filename (str): The name of the file to write to (path is absolute or relative to current working dir).
  • fieldnames (list): A list of strings representing the CSV header/column names.
  • rows (list): A list of dictionaries, where each dictionary is a row.
  • write_header (bool, optional): If True, writes the header. Defaults to False.
  • initial_usd (numeric, optional): If provided and write_header is True, this value is written as a comment in the CSV header. Defaults to None.

write_results_combined(self, filename, fieldnames, rows)

• Description: Writes combined results to a CSV file in the results_dir. Uses tab as a delimiter and formats rows using format_row.
• Parameters:
  • filename (str): The name of the file to write to (relative to results_dir).
  • fieldnames (list): A list of strings representing the CSV header/column names.
  • rows (list): A list of dictionaries, where each dictionary is a row.

write_trades(self, all_trade_rows, trades_fieldnames)

• Description: Writes trade data to separate CSV files based on timeframe and stop-loss percentage. Files are named trades_{tf}_ST{sl_percent}pct.csv and stored in results_dir.
• Parameters:
  • all_trade_rows (list): A list of dictionaries, where each dictionary represents a trade.
  • trades_fieldnames (list): A list of strings for the CSV header of trade files.