Multivariate time series prediction of marine zooplankton by artificial neural networks

Most applications of Neural Networks are based on their high adaptivity to almost any set of given input-output relations ("patterns"). An example is pattern recognition: here, a usually complex input, e.g. a picture of a number, has to be mapped on a much simpler output, e.g. the binary representation of that number. Networks trained to reproduce such input-output relations can be used to identify a complex input from the much simpler output. In applications to time series prediction these input-output relations relate past to future data. But here the task is not to reproduce a previously known inputoutput relation. Instead one wants the Neural Network to produce correctly from known past data presently unknown future data. Here another feature of Neural Networks comes into play, namely their ability to generalize. Neural Networks cannot only be trained to learn particular input-output relations. During training they also seem to develop a more general representation of these relations. On the basis of these relations predictions can be performed. The nature of this more general representation is not very clear and may depend on the chosen network structure, but one can understand it as a kind of inter- or extrapolation of the trained input-output relations. Operationally the ability of a Neural Network to generalize is usually defined as its ability to produce correct outputs - i.e. predictions in the present context - for inputs that have not been used during training (see e.g. Hertz et al. (1991)). It is clear that an appropriate generalization may fail, but there are striking examples where this generalization works extremely well, e.g. for the prediction of the (secondary) protein structure from its sequence of amino acids (Rost and Sander 1993) or the prediction of chaotic dynamics (Wan 1994). Especially when low quality data are used for prediction or if the number of simultaneous variables is high, it is often hard to judge, whether a particular Neural Net is able to generalize. This situation is often encountered when working with environmental data sets, as everyone knows, who tried to work e.g. with biological time series (see e.g. Reick and Page (2000)). The reasons for this low quality are simple: First, environmental data are typically taken under nonlaboratory conditions, i.e. external disturbances cannot be controlled and the data get noisy. Second, one has usually to measure what one can get, and not what one would like to measure. So one cannot be sure, whether the data are representative for some hidden deterministic dynamics. And finally, long measurement campaigns are expensive so that environmental data sets are typically quite short compared to their noise level. This problem is only insufficiently compensated by measuring simultaneously several variables (the extra costs are typically low), because hereby one can only improve the information on particular system states, but cannot gain additional information on the diversity of system states; this could only be obtained from sufficiently long time series. But this information on the diversity of states is indispensable for predictions of high quality. Moreover, the information of additional variables is often redundant and also noisy, to the consequence that by using these data as additional inputs in Neural Nets their performance can get even worse. The prediction quality is usually measured by computing the average prediction error for a number of prediction instants. But the question is, how far this prediction error can be trusted, when a Neural Network is used to predict unknown data. When working with Neural Networks one experiences that the lower the data quality, the less reliable are the computed prediction errors. As already discussed, this situation is especially encountered, when working with environmental data so that here one should always carefully analyze their reliability. This means one has to inquire the ability of a Neural Network to generalize. How to do this by crossvalidation techniques will be discussed in section II. Unfortunately, crossvalidation is very laborous, because the same Neural Network has to be trained over and over again with different parts of the available data. The solution can only be a complete automatization of the training process. Standard Neural Network software, like e.g. SNNS (Zell 1994), supports mainly the visual supervision of the training at the computer monitor. But this is much too laborous when performing crossvalidation studies. Alternatively, one could use the programming interfaces, that are part of many Neural Network products. But besides the uncomfortability of such a solution, there is a more fundamental problem with automatization: Many training algorithms have been developed in the past and many of them are available in Neural Network packages. But when using them for automatized training the main problem is how to stop the training, such that the network is neither under- and nor overadapted in order to guarantee optimal generalization. For visual supervision at the screen, there is a widely accepted stopping technique by Weigend et al. (1991). As a first step to automatization we show in section III how this technique can be cast into an algorithm. Finally we show in section IV how crossvalidation and automatized training can be applied to an environmental data set, namely to plankton time series from the North Sea. In contrast to most other prediction studies, we will not show how well Neural Nets predict these time series, but instead show how the failure of their ability to generalize can be substantiated.