Downloading Data¶
ProgridPy provides unified clients for downloading raw market data from MISO, SPP, and ERCOT. Each client exposes the same download_raw_data() interface defined by ISOBase, with ISO-specific data type enums and source options.
Core Concepts¶
Data Type Enums¶
Each ISO defines a StrEnum of available raw data types. Pass one or more values to data_types to select what to download.
from progridpy.iso import MISORawDataType
MISORawDataType.DAY_AHEAD_EXANTE_PRICE
MISORawDataType.DAY_AHEAD_EXPOST_PRICE
MISORawDataType.REAL_TIME_PRELIM_PRICE
MISORawDataType.REAL_TIME_FINAL_PRICE
MISORawDataType.DAILY_FORECAST_AND_ACTUAL_LOAD_BY_LOCAL_RESOURCE_ZONE
MISORawDataType.DAILY_REGIONAL_FORECAST_AND_ACTUAL_LOAD
MISORawDataType.CLEARED_BIDS
MISORawDataType.DEAD_NODES
MISORawDataType.MULTIDAY_OPERATING_MARGIN_FORECAST
MISORawDataType.SOLAR_FORECAST
MISORawDataType.WIND_FORECAST
MISORawDataType.LMP_ROLLING
# Enverus API data types
MISORawDataType.WIND_FORECAST_HRRR_ENVERUS
MISORawDataType.WIND_FORECAST_ECMWF_ENVERUS
MISORawDataType.SOLAR_FORECAST_HRRR_ENVERUS
MISORawDataType.SOLAR_FORECAST_ECMWF_ENVERUS
MISORawDataType.EXTREME_WIND_SPEED_ENVERUS
MISORawDataType.SOLAR_SMOKE_ENVERUS
MISORawDataType.SOLAR_SNOW_ENVERUS
MISORawDataType.TURBINE_ICING_ENVERUS
from progridpy.iso import SPPRawDataType
SPPRawDataType.DAY_AHEAD_LMP
SPPRawDataType.REAL_TIME_PRELIM_LMP
SPPRawDataType.REAL_TIME_FINAL_LMP
SPPRawDataType.REAL_TIME_ROLLING_LMP
SPPRawDataType.LMP_FORECAST
SPPRawDataType.LOAD_ACTUAL
SPPRawDataType.LOAD_FORECAST
SPPRawDataType.RESOURCE_ACTUAL
SPPRawDataType.RESOURCE_FORECAST
from progridpy.iso import ERCOTRawDataType
ERCOTRawDataType.AS_PRICES
ERCOTRawDataType.HOURLY_SOLAR_POWER_PRODUCTION
ERCOTRawDataType.HOURLY_WIND_POWER_PRODUCTION
ERCOTRawDataType.LF_BY_MODEL_WEATHER_ZONE
ERCOTRawDataType.ACTUAL_SYSTEM_LOAD_BY_WEATHER_ZONE
ERCOTRawDataType.SPP_REAL_TIME_15_MIN
ERCOTRawDataType.SPP_DAY_AHEAD_HOURLY
FileLocation¶
The download_src parameter controls where data is fetched from:
from progridpy.common.types import FileLocation
FileLocation.ISO # Download directly from the ISO's public API/website
FileLocation.S3 # Download from the ProgridPy S3 data lake
FileLocation.LOCAL # Read from a local directory (no network request)
The default is FileLocation.ISO.
Method Signature¶
All ISO clients share this signature from ISOBase:
def download_raw_data(
self,
start_date: str | datetime | None = None,
end_date: str | datetime | None = None,
data_types: RawT | list[RawT] | None = None,
download_src: str | FileLocation = FileLocation.ISO,
output_dir: str | Path | None = None,
overwrite: bool = False,
verbose: bool = False,
) -> list[Path]:
| Parameter | Description |
|---|---|
start_date |
Inclusive start date. Accepts "YYYY-MM-DD" strings or datetime objects. |
end_date |
Inclusive end date. |
data_types |
One enum value, a list of values, or None for all types. |
download_src |
Where to fetch data from (ISO, S3, or LOCAL). |
output_dir |
Local directory for downloaded files. Uses a default if None. |
overwrite |
Re-download files that already exist locally. |
verbose |
Enable detailed logging output. |
Returns a list of Path objects for every file written to disk.
Date range is inclusive
Both start_date and end_date are inclusive. Requesting "2026-01-01" to "2026-01-03" downloads data for January 1, 2, and 3.
Download Examples¶
MISO¶
from progridpy.iso import MISO, MISORawDataType
miso = MISO()
# Download day-ahead and real-time LMP for January 2026
files = miso.download_raw_data(
start_date="2026-01-01",
end_date="2026-01-31",
data_types=[
MISORawDataType.DAY_AHEAD_EXANTE_PRICE,
MISORawDataType.REAL_TIME_FINAL_PRICE,
],
)
# Download all available data types for a single day
files = miso.download_raw_data(
start_date="2026-03-15",
end_date="2026-03-15",
)
# Download from S3 instead of the ISO website
files = miso.download_raw_data(
start_date="2026-01-01",
end_date="2026-01-31",
data_types=[MISORawDataType.DAY_AHEAD_EXANTE_PRICE],
download_src="s3",
)
SPP¶
from progridpy.iso import SPP, SPPRawDataType
spp = SPP()
# Download day-ahead LMP and load data
files = spp.download_raw_data(
start_date="2026-01-01",
end_date="2026-01-31",
data_types=[
SPPRawDataType.DAY_AHEAD_LMP,
SPPRawDataType.LOAD_ACTUAL,
SPPRawDataType.RESOURCE_ACTUAL,
],
)
# Download to a specific directory, overwriting existing files
files = spp.download_raw_data(
start_date="2026-02-01",
end_date="2026-02-28",
data_types=[SPPRawDataType.DAY_AHEAD_LMP],
output_dir="./data/spp/raw",
overwrite=True,
verbose=True,
)
ERCOT¶
from progridpy.iso import ERCOT, ERCOTRawDataType
ercot = ERCOT()
# Download day-ahead and real-time settlement point prices
files = ercot.download_raw_data(
start_date="2026-01-01",
end_date="2026-01-31",
data_types=[
ERCOTRawDataType.SPP_DAY_AHEAD_HOURLY,
ERCOTRawDataType.SPP_REAL_TIME_15_MIN,
],
)
# Download ancillary services and load data
files = ercot.download_raw_data(
start_date="2026-01-01",
end_date="2026-01-07",
data_types=[
ERCOTRawDataType.AS_PRICES,
ERCOTRawDataType.ACTUAL_SYSTEM_LOAD_BY_WEATHER_ZONE,
],
)
Downloading Processed Data¶
Processed Hive-partitioned Parquet files can be downloaded from S3 using download_processed_data():
from progridpy.iso import MISO, MISOProcessedDataType
miso = MISO()
files = miso.download_processed_data(
start_date="2026-01-01",
end_date="2026-01-31",
data_types=[MISOProcessedDataType.NODAL],
output_dir="./data/miso/processed",
)
The processed data types available per ISO are:
| ISO | Processed Types |
|---|---|
| MISO | NODAL, SYSTEM, REGIONAL |
| SPP | NODAL, SYSTEM, ZONAL |
| ERCOT | NODAL, SYSTEM, WEATHER_ZONE, LOAD_ZONE |
Uploading Data to S3¶
Upload local raw or processed files to S3:
from progridpy.iso import SPP, SPPRawDataType
spp = SPP()
# Upload raw files
keys = spp.upload_raw_data(
start_date="2026-01-01",
end_date="2026-01-31",
data_types=[SPPRawDataType.DAY_AHEAD_LMP],
input_dir="./data/spp/raw",
)
# Upload processed files
keys = spp.upload_processed_data(
start_date="2026-01-01",
end_date="2026-01-31",
input_dir="./data/spp/processed",
)
Listing All Data Types¶
Each enum provides a get_all() class method: