Skip to main content

Phase 6 · Model lab

Baseline forecast evaluation

Ensemble forecasts power production dashboards. These baselines establish the comparison bar on weekly weighted activity index for Illinois and Cook County. The Neural ODE is documented separately as a learned dynamics research layer and it is calibrated and useful at short horizons, but not promoted at 4 weeks.

What this page is

A behind-the-scenes look at the forecasting models and how accurate they are. The ensemble baseline powers the public dashboards. The Neural ODE is a research experiment that is not used on the dashboards. If you just want the current situation, the region dashboards are the place to start.

Model runs

ensemble_v1
production
ensemble · v1.0.0

Evaluations scored: 546

Updated Aug 8

moving_average_v1
production
moving_average · v1.0.0

Evaluations scored: 546

Updated Aug 8

neural_ode_17031
candidate
neural_ode · v1.4.0

Evaluations scored: 48

Updated May 21

neural_ode_17031
candidate
neural_ode · v1.7.2-hardbound

Evaluations scored: 230

Updated Jun 1

neural_ode_17031
candidate
neural_ode · v1.2.1

Evaluations scored: 48

Updated May 20

neural_ode_17031
candidate
neural_ode · v1.7.3-hardbound-linear

Evaluations scored: 230

Updated Jun 2

neural_ode_17031
candidate
neural_ode · v1.2.0

Evaluations scored: 48

Updated May 20

neural_ode_17031
candidate
neural_ode · v1.5.0

Evaluations scored: 48

Updated May 21

neural_ode_17031
candidate
neural_ode · v1.3.0

Evaluations scored: 48

Updated May 21

neural_ode_17031
candidate
neural_ode · v1.7.4-shrinkage

Evaluations scored: 230

Updated Jun 3

neural_ode_17031
candidate
neural_ode · v1.7.5-shrinkage-conservative

Evaluations scored: 230

Updated Aug 8

neural_ode_17031
candidate
neural_ode · v1.7.6-shrinkage-h4-abstain

Evaluations scored: 230

Updated Jun 4

neural_ode_17031
archived
neural_ode · v1.0.0

Evaluations scored: 237

Updated Jun 5

neural_ode_17031
candidate
neural_ode · v1.6.0

Evaluations scored: 48

Updated May 21

neural_ode_17031
candidate
neural_ode · v1.1.0

Evaluations scored: 48

Updated May 20

neural_ode_17031
candidate
neural_ode · v1.8.0-reliability

Evaluations scored: 226

Updated Jun 1

neural_ode_17031
candidate
neural_ode · v1.7.1

Evaluations scored: 230

Updated Jun 1

neural_ode_IL
candidate
neural_ode · v1.1.0

Evaluations scored: 48

Updated May 20

neural_ode_IL
candidate
neural_ode · v1.2.0

Evaluations scored: 48

Updated May 20

neural_ode_IL
candidate
neural_ode · v1.2.1

Evaluations scored: 48

Updated May 20

neural_ode_IL
candidate
neural_ode · v1.3.0

Evaluations scored: 48

Updated May 21

neural_ode_IL
candidate
neural_ode · v1.4.0

Evaluations scored: 48

Updated May 21

neural_ode_IL
archived
neural_ode · v1.0.0

Evaluations scored: 237

Updated Jun 5

neural_ode_IL
candidate
neural_ode · v1.5.0

Evaluations scored: 48

Updated May 21

neural_ode_IL
candidate
neural_ode · v1.6.0

Evaluations scored: 48

Updated May 21

neural_ode_IL
candidate
neural_ode · v1.8.0-reliability

Evaluations scored: 226

Updated Jun 1

neural_ode_IL
candidate
neural_ode · v1.7.1

Evaluations scored: 230

Updated Jun 1

neural_ode_IL
candidate
neural_ode · v1.7.2-hardbound

Evaluations scored: 230

Updated Jun 1

neural_ode_IL
candidate
neural_ode · v1.7.3-hardbound-linear

Evaluations scored: 230

Updated Jun 2

neural_ode_IL
candidate
neural_ode · v1.7.4-shrinkage

Evaluations scored: 230

Updated Jun 3

neural_ode_IL
candidate
neural_ode · v1.7.6-shrinkage-h4-abstain

Evaluations scored: 230

Updated Jun 4

neural_ode_IL
candidate
neural_ode · v1.7.5-shrinkage-conservative

Evaluations scored: 230

Updated Aug 8

persistence_v1
production
persistence · v1.0.0

Evaluations scored: 546

Updated Aug 8

seasonal_naive_v1
production
seasonal_naive · v1.0.0

Evaluations scored: 546

Updated Aug 8

trend_v1
production
trend · v1.0.0

Evaluations scored: 546

Updated Aug 8

Performance metrics

Baseline performance
Rolling-origin backtest metrics by forecast horizon. Lower MAE/RMSE is better; trend accuracy measures direction (rising/falling/stable).
Model1-week2-week3-week4-weekN
MAERMSETrendMAERMSETrendMAERMSETrendMAERMSETrend
ensemble_v10.3420.42235.5%0.4050.50126.8%0.4710.57934.6%0.5290.64633.6%546
moving_average_v10.3930.47733.3%0.4720.58033.3%0.5500.67636.0%0.6150.74632.8%546
persistence_v10.3040.38848.5%0.4000.49136.2%0.4900.60031.6%0.5530.68331.3%546
seasonal_naive_v10.8140.97640.6%0.8260.98739.9%0.8220.98342.6%0.8330.99541.8%546
trend_v10.3600.47644.9%0.4880.62738.4%0.6020.78042.6%0.7050.92447.0%546
MAE by horizon
Mean absolute error on held-out weekly activity index. Compare models before trusting more complex approaches.

How to read these results

  • Persistence assumes next week equals this week, which is often hard to beat at 1-week horizon.
  • Moving average smooths recent weeks; useful when signal is noisy.
  • Trend extrapolates recent direction; can overshoot at longer horizons.
  • Seasonal naive compares to the same week last year (52-week lag).
  • Ensemble averages component baselines for dashboard display; Neural ODE (Phase 7) must beat these on held-out weeks to be promoted.