What you’re interested in is actuarial science. It’s an interesting and complex field. Pretty much every insurance company is built on the backs of a few talented actuaries.

https://www.kobo.com/us/en/ebook/superforecasting

_ /\ _

I've heard podcasts about super forecasters, but haven't come across a systemic way to learn about this. But knowing a name used for this might help in your search

https://en.wikipedia.org/wiki/Superforecaster

Yes, maths and statistics courses in school, college, university would be the tried and tested route for learning the analytical and practical tools and techniques.

Forecasting rare events with any precision is almost a contradiction in terms though. When you're down to the 1/10000 type events you need such a large dataset/sample, that there are almost always unobservable sub-populations, or unobservable historical / environmental factors that your data is likely to be missing; something important that could materially change the forecast if you were to have had complete, unbiassed data on it for you whole sample.

Practical forecasting though , i think, should be tied into the decision making, and trying to reduce the risk of choosing the course of action to take. The set of possible / feasible actions shapes the forecasting approach - you can't really learn that pragmatic tradeoff in academic institutions - i think it's just experience. Make some predictions, get them wrong, do a forensic analysis. Or collaborate with people who have done this for a living.

In respect of the AI, you need to check it had a reasonable concept of the population of events you want to know about. Understands its sample of observations, how that sample was drawn (i.e. it wasn't random), and the biases in that sample or sampling method. Then it should be easy to recalculate its output, then you come up with some scenarios of the bias, or adjustments see what changes those scenarios have. A competent forecaster should have addressed major/obvious sources of bias, with ranges / scenarios etc. " how wrong might this forecast be if we assumed, X, Y, Z instead?" I don't trust anyone who asserts they have a representative sample, it's impossible to prove that 100% - otherwise you'd not need a sample in the first place.

I can recommend the book by Douglas W. Hubbard:

How to Measure Anything: Finding the Value of Intangibles in Business

For one off events like 9/11, you can't. There isn't an accepted definition of "probability" for things like that. Either The question is completely nonsensical (Frequentist view) or the numbers are more or less all arbitrary (Bayesian view).

Look to the past. People are constantly doing the same dumb shit.

Generally yes there is TONS of theories and methods relating to forecasting and prediction. Its a very interesting field.

What was the link to? Cant seem to work for me.

Oh and just fyi AI models as most know them only run through outout as highest likely to be following a given chain, all based on trained data. The closest you can get to real "math" would be training a model to context strip metadata from an input, to very specific output, use that to input data into a database, have an actual program do the math or call relevant statistics, peform the calc then give the output. Buuuuut that is pretty much not done at all by anyone beyond custom written systems for specific use cases, and any forward facing companies having AI models arent much doing that, the output is a fluke.

Prediction markets are a thing...

When in doubt, the AI is gaslighting you.

When you're sure, the AI is still probably gaslighting you.

It kind of depends on if there is anything specific you want to try and forecast. That said, a lot to data scientists would probably point to Bayesian statistics

50/50

It either happens, or it doesn't. So easy, next question pls.

This is not an ability the Jedi of Lemmy will ever teach you.

Predictions are hard, especially about the future.

You don't say exactly what you want to predict so it's hard to say exactly what you need to learn. The link returns an error to me.

If you want to predict the next economic crash you'll have to study a lot of economics. If you want to predict the next pandemic you'll have to study medicine etc. Years of study no matter what.

Once you know a lot about a topic you have to put a ton of effort into study current events and still be wrong a lot of the time.

Seeing as 9-11 only happened once in the past few thousand years, the AI odds check out.