Skip to content

Miso trading sop

CURRENTLY, THE PROCESS OF GENERATING AND SUBMITTING OFFERS AND BIDS TO MISO IS AUTOMATED USING CRON JOBS.

IMPORTANT: MISO’s Day Ahead market for OD (Operating Day) closes at 10:30 AM Eastern Prevailing Time (does NOT change with daylight saving) on OD-1. Although the protocol outlined below takes less than 10 minutes to complete, it is recommended to submit it as early as possible to allow for any unexpected issues.

  1. [OPTIONAL] Check the forecast for impending natural disasters. Check the latest hurricane map from NOAA, which is reproduced at the top of our electricity demand dashboard (use the password Electrum). You can also view the latest hurricane map directly on the NOAA website.

If a hurricane or tropical storm is forecasted to make landfall in the continental US on OD, then it is not advisable to place large volumes of offers or bids. If the dashboard is unavailable for some reason, then check NOAA’s website here.

NOAA Hurricane Map

  1. [OPTIONAL] Check forecasted demand for OD. If not running a High Demand Day (HDD) algorithm skip to step 3. If running a High Demand Day (HDD) algorithm like Algorithm 2, check the demand forecast for MISO on the dashboard.

2.1. If the demand dashboard is unavailable for whatever reason, download the demand forecast for OD from MISO here. Sum the MISO total forecasts for OD over the hours of the day to obtain the total forecasted demand for OD.

2.2. If the latest demand forecast for OD cannot be obtained, then do not submit any offers or bids (this has never happened).

2.3. If the forecasted demand for OD is below 2.2 million MWH, it is not a high demand day. Hence, do not submit any offers or bids.

MISO Demand Dashboard

2.4. If the forecasted demand for OD is equal to or greater than 2.2 million MWH, it is a high demand day.

Note: 2.2 million MWH is the 80% of the daily forecasted loads from 2019 onwards in MISO calculated in 2023. This may need to be updated in the future. If updated, all the retrospective simulations must be rerun.

  1. [OPTIONAL] Check credit exposure report.

    DO NOT SKIP WHEN INCREASING TRADING VOLUME

    3.1. Navigate to https://markets.midwestiso.org/, which prompts me to select a digital certificate. Choose the correct digital certificate (this should be configured uniquely for anyone performing the trade - a copy of Jake’s digital certificate is available here).

    3.2. Select the credit exposure report tab and view the report for OD, to make sure that the total financial security and total potential exposure are close to what is expected, given our collateral and the gains from previous days’ that have not yet been settled. Also check for the daily virtual MWH limit - this is the maximum volume that can be traded. (This cannot be changed on the same day, ensure this is updated at least a few weeks in advance via the LSA account.)

MISO portal and DA LMP files

  1. Login to the EC2 Instance. All trading scripts are currently hosted on an EC2 instance which can be accessed via ssh. ssh -i ~/.ssh/progrid-admmin.pem ubuntu@prod.progrid.xyz.

  2. Download latest price and deadnodes data. The data required for the current set of strategies include DA LMP (da_expost), RT LMP (rt_final as long as possible or rt_prelim) and deadnodes. Currently, these files are automatically downloaded to the EC2 instance.

    5.1. All the required files should be in the folder ~/MISO-operational-data. THIS CAN BE A POINT OF FAILURE. Sometime the cronjobs may not run correctly or there might be a delay on when MISO uploads the data on their website.

    5.2. Ensure the files are available for up to OD-2.

    5.2.1 Deadnodes files have been observed to be available only at 8 AM EST. Hence, time the offer generation accordingly

    5.3. If for some reason, the required files are not already present in the folder, you can do one of the following:

    5.3.1. Manually run the script to download files:

    python ~/progrid-utils/scripts/miso_trading/miso_daily_fetch.py

OR,

5.3.2. Another way is to download the files manually and upload it to the EC2 instance using scp command. MISO shares these prices in separate files, one for each day.

  3. Generate and Submit offers.

    6.1. The script to generate trades and submit the jobs are put in a basch script and scheduled as a cron job.

    You can manually run these as follows:

```ISO=MISO python /home/ubuntu/progrid-utils/scripts/miso_trading/generate_trades.py -y -od "${operating_day}"```

```ISO=MISO PROGRID_ENV=PROD python /home/ubuntu/progrid-utils/scripts/miso_trading/submit_job.py -i "/home/ubuntu/MISO-jobs/MISO-${operating_day}.csv"```

Terminal running generate_trades.py script 

The generate_trades.py script by default runs for the next day as Operating Day because we currently only trade on the Day Ahead market. The script takes the parameter-od (would need to run as python generate_trades.py -od 20250105) if needed to generate for offers and bids for a different day as the Operating Day. The script is interactive and will prompt the user with some statistics before writing the offers and bids to file.

The script outputs a single file with all the offers and bids in the <folder> with the filename format <format>. Any strategy specific parameters are hardcoded in the script.

The `submit_job.py` script submits the offers and bids in the file and then queries MISO's api to confirm the submssions. The summary of submitted jobs is logged.

  1. Logging and DataDog Metrics

    7.1. All the outputs of the scripts are logged to the logs in ~/.logs/ directory. The logs are also viewable on datadog.

    7.2. Dashboards have been created on the DataDog website to track the number of offers and bids submitted and confirmed.

    7.3. A couple of alerts have been created based on the number of submissions. There should be an alert everyday. If no alert or unexpected alerts are received, it may be required to rerun the entire process manually.

    7.3.1. DataDog Alert 1: Submission Failure --> This looks at the total number of submission mades, and triggers if the total number of submissions is not 480. This needs to be updated if the number of bids and or offers change.

7.3.2. Data Alert 2: Submssion Success --> This looks at the total number of submissions made, and triggers when the total number is as expected. This also needs to be updated if the nuber of bids and or offers change.

Expected Behavior: If everything goes right, DataDog Alert 2 will be alerted and recovery alert will follow in an hour. If DataDog Alert 1 is alerted or DataDog Alert 2 is not alerted, then inspection is required.