How to Load Historical Data in freqtrade (and Where to Get It)

freqtrade is one of the most popular open-source crypto trading bot frameworks. Its built-in data downloader is slow, rate-limited, and frequently breaks for assets with long histories. If you have ever tried to download 5 years of 1m BTC data through freqtrade and given up, this article is for you.

How freqtrade Stores Historical Data

user_data/data/binance/BTC_USDT-1m.json
user_data/data/binance/ETH_USDT-1h.json

The JSON format is a list of arrays: [timestamp_ms, open, high, low, close, volume].

Converting External Parquet Data to freqtrade Format

import polars as pl
import json
from pathlib import Path

df = pl.read_parquet("BTC_OHLCV_1h.parquet")

records = df.select([
    (pl.col("candle_time").cast(pl.Int64) // 1_000_000).alias("ts"),
    "open", "high", "low", "close", "volume"
]).rows()

out = Path("user_data/data/binance/BTC_USDT-1h.json")
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(records))
print(f"Written {len(records):,} candles")

Verifying the Data in freqtrade

freqtrade list-data --exchange binance --pairs BTC/USDT

Why Funding Rate Data Breaks freqtrade Backtests

A recurring issue (GitHub issues #11680, #12174) is funding rate data stopping updates. If your strategy uses funding rates as a signal, gaps in that data silently corrupt your backtest. Source funding rate data from a complete historical archive rather than live API pulls.

Quick checklist for external data

1. Columns named date, open, high, low, close, volume.
2. date as timezone-aware UTC, sorted ascending, no duplicates.
3. File name BTC_USDT-1h.feather (pair with underscore, then timeframe) inside user_data/data/binance/.
4. Verify with freqtrade list-data --exchange binance.

Convert a CSV in a few lines

import pandas as pd   # needs pyarrow for feather
df = pd.read_csv("btcusdt_1h.csv", parse_dates=["timestamp"])
df = df.rename(columns={"timestamp": "date"})
df["date"] = pd.to_datetime(df["date"], utc=True)
df = df[["date", "open", "high", "low", "close", "volume"]].sort_values("date")
df.to_feather("user_data/data/binance/BTC_USDT-1h.feather")

Check for gaps before you backtest

A converted file can load fine and still hide missing candles. Compare the candle count with the expected grid and investigate every gap, as explained in what is OHLCV data. For the structure of a good backtest, see the OHLCV backtesting guide, and audit the results with the Backtest CSV Auditor.

Skip the slow downloader

The Glitch List publishes a free OHLCV Bundle of ready-made packs in six timeframes, each with an integrity declaration that lists its known gaps, so you start from files you can verify instead of a multi-day download.

Browse the full dataset catalog

FAQ

Where does freqtrade store historical data?

In user_data/data//, one file per pair and timeframe, in the format set by the dataformat_ohlcv option (feather by default in current versions).

How do I check which data freqtrade has?

Run freqtrade list-data –exchange binance. It lists the pairs, timeframes and date ranges it finds on disk.

Can I use my own CSV or Parquet data in freqtrade?

Yes. Convert it to freqtrade’s candle format (columns date, open, high, low, close, volume, with timezone-aware UTC dates), save it with the pair-timeframe file name in the data folder, and verify it with list-data.

Why does the built-in downloader fail for long histories?

It pulls candles through the exchange API, which is rate-limited and paginated, so multi-year 1-minute downloads are slow and often get interrupted. Downloading ready-made files avoids the API limits.

Summary

freqtrade’s built-in downloader is convenient for short histories but painful for multi-year backtests. Converting external Parquet datasets to freqtrade’s JSON format is a one-time script. Once the data is in user_data/data/, freqtrade’s backtesting engine works exactly as expected.