Home > data science > Continuously forecasting

Continuously forecasting

April 9, 2012

So I’ve been thinking about how to forecast a continuous event. In other words, I don’t want to forecast a “yes” or a “no”, which is something you might do in face recognition using logistic regression, and I don’t want to split the samples into multiple but finite bins, which is something you may want to do in handwriting recognition using neural networks or decision trees or recommender systems.

I want essentially a score, in fact an expected value of something, where the answers could range over the real numbers (but will probably just range over a pretty small subset but I don’t know exactly what smallish subset).

What happens when you look around is that you realize machine learning algorithms pretty much all do the former, except for various types of regression (adding weights, adding prior, nonlinear terms), which I already know about from my finance quant days. So I’m using various types of regression, but it would be fun to also use a new kind of machine learning algorithm to compare the two. But it looks like there’s nothing out there to compare with.

It’s something I hadn’t noticed til now, and I’d love to be wrong about it, so tell me if I’m wrong.



Categories: data science
  1. April 9, 2012 at 7:20 am

    Your description is a bit vague, but have you tried Kalman filters?


  2. April 9, 2012 at 7:46 am

    When I did forecasting for a living 2 rules all forecasts are wrong and the trend is your friend.

    Boss had her model to compare all output (next days’ natural gas demand to) with – she would ask what yesterdays or todays was and why is it different to whatever the forecast was. Ie f(n+1)=f(n) it captures the seasonal trend 🙂

    Expected value as is in expectation value?
    The way I see statistical mechanics differ from statistics is that the former is harder to calculate but easier to interpret.


  3. jonathan
    April 9, 2012 at 11:14 am

    What about k nearest neighbors?


  4. Matt
    April 9, 2012 at 11:29 am

    What about neural networks?


  5. Ivan
    April 10, 2012 at 10:25 am

    Ensembles of regression trees. Bagged ensembles a.k.a. Random Forests and gradient boosting models for trees.


  6. Johannes
    April 10, 2012 at 11:04 am

    There are two main classes of (supervised) machine learning methods: classifiers (the output is discrete, e.g. yes/no, or a greater number of classes), and regressors (the output is continuous). The latter contains many many different methods (e.g. ordinary linear regression, neural networks). This book goes covers a LOT (pdf): http://www.stanford.edu/~hastie/local.ftp/Springer/OLD//ESLII_print4.pdf

    If you want to describe your problem a bit more, I could say what techniques I’d be likely to try.


  7. April 10, 2012 at 5:18 pm

    See Mandelbrot. He points out that most factors that effect outcomes are recent events, but some events have very long tails. So your forecast has to weight recent events heavily, but also include events from the distant past, albeit with very low weights. And estimation of the tail probabilities gets very tricky.


  8. rocambolesq
    April 12, 2012 at 3:14 am

    How about a a top-down simulation of the underlying factors and interconnections and their evolution over time? I used to do that for predicting clinical variables in virtual patients given various hypothesized interventions when I worked with Entelos:

    and Archimedes does something similar:

    These approaches have provided a lot of insights, and help point out the next most important questions. They do take a long time to build well though.


  1. No trackbacks yet.
Comments are closed.
%d bloggers like this: