Before entering the methods, we have to discuss how to assess the results. The
most obvious quantity for the quantification of predictability is the average
forecast error, i.e. the root of the mean squared (rms) deviation of the
individual prediction from the actual future value. If it is computed on those
values which were also used to construct the model (or to perform the
predictions), it is called the *in-sample error*. It is always advisable
to save some data for an out-of-sample test. If the out-of-sample error is
considerably larger than the in-sample error, data are either non-stationary
or one has overfitted the data, i.e. the fit extracted structure from random
fluctuations. A model with less parameters will then serve better. In cases
where the data base is poor, on can apply *complete cross-validation* or
*take-one-out statistics*, i.e. one constructs as many models as one
performs forecasts, and in each case ignores the point one wants to predict.
By construction, this method is realized in the local approaches, but not in
the global ones.

The most significant, but least quantitative way of model validation is to iterate the model and to compare this synthetic time series to the experimental data. If they are compatible (e.g. in a delay plot), then the model is likely to be reasonable. Quantitatively, it is not easy to define the compatibility. One starts form an observed delay vector as intial condition, performs the first forecast, combines the forecast with all but the last components of the initial vector to a new delay vector, performs the next forecast, and so on. The resulting time series should then be compared to the measured data, most easily the attractor in a delay representation.

Wed Jan 6 15:38:27 CET 1999