In which I introduce "The Forecasting Paradox" on.ft.com/3BGgLMM (1st 300 clicks free to read - enjoy!) - ThreadSky

timharford.ft.com • 60 days ago

In which I introduce "The Forecasting Paradox"
https://on.ft.com/3BGgLMM (1st 300 clicks free to read - enjoy!)

Comments

For many years I have built forecast models in the electricity industry.

Ofgem insist that the appropriate measure for usefulness is accuracy.

However, consider a forecast for a cold winter's day. I forecast very high demand, so small expensive generators are run because ...

andrewwr235.bsky.social•59 days ago

they are cheaper than the most expensive large generators. This lowers the measured demand (which doesn't include small generation), and my forecast is wrong.

Had I accounted for this, the demand would have been lower, and that extra small generation would not have run. And my forcast is too low.

andrewwr235.bsky.social•59 days ago

Whichever I do, the forecast is inaccurate, because the forecast influences the behaviour I'm trying to predict.

In my opinion then the best forecast is the one which leads to the best outcomes, and that's when I forecast the demand that would occur if no small generation runs.

andrewwr235.bsky.social•59 days ago

Because it ensures there is enough electricity to meet demand. This is more important than accuracy, in this case.

andrewwr235.bsky.social•59 days ago

Similiar considerations applied to forecasts of covid admissions and deaths made before the vaccine was available.

The forecasts were made on the basis that people didn't alter their behaviour, and the forecast outcomes were horrific.

andrewwr235.bsky.social•59 days ago

These forecasts caused people to alter their behaviour, so the forecasts over-estimated admissions and deaths. They were inaccurate.

atmbnl.bsky.social•60 days ago

“Scenarios are not forecasts because they are not aiming to be accurate, but to be useful”

My one foray into the epidemiological literature ended up in our paper being rejected precisely because the reviewers (and the editor) did not understand the distinction between a scenario and a forecast!

tallbeanpole.bsky.social•60 days ago

Really interesting read, thank you for sharing. It certainly made me think and reflect on the scenario planning and (what you called pre-mortem) thinking we did.

pwc50.bsky.social•60 days ago

A useful piece of work from around the dawn of modern day technology time was identifying single points of failure in a “system” which helped in disaster recovery planning scenarios

Sadly scenario planning tends to be a victim of its own success as “nothing happened” 🤷

apcrft.bsky.social•60 days ago

📌

markescott.bsky.social•60 days ago

Reminds me of the adage attributed to Eisenhower, that plans are useless, but planning is essential.

rudydotca.bsky.social•60 days ago

"Never make predictions, especially about the future." -- Casey Stengel

ocultado-real.bsky.social•60 days ago

I plan IT projects by getting the team to think about what happens if a meteor hits the data centre? If godzilla stands on Office Y? If Fred spills a sugary cappuccino on his laptop?

johnlytham.bsky.social•60 days ago

Very interesting Tim - as I used to tell my retail merchandising clients - the only thing you can be sure of is that your forecast will be wrong.

Comments

Posting Rules

Reply