Artificial neural network based universal time series

[0002] This application is based on a provisional patent application No. 60/150,790 filed Aug. 26, 1999.

[0003] The invention relates to a time series prediction system for financial securities utilizing Artificial Neural Network (ANN). More particularly, the invention relates to a processing system based on the recurrent ANN architecture capable of outputting upper and lower prediction bounds at any given confidence, which is based on the validation errors of the ANN. Specifically, the invention relates a prediction system that can be applied to any financial time series which can be called by any computer language and Web applications supporting the system.

[0004] In the recent past, Recurrent Artificial Neural Networks have successfully improved the quality of forecasting of share movements in relation to its statistically based counterparts. The known recurrent neural networks (RNN) make a prediction of the appreciation potential of each stock based on the available historical data. The training process continues until at least one stopping criterion is met. Such criteria include the determination that the connections between the nodes of the net have reached a steady state, that the error between the predicted output and the actual target values is less than a certain threshold, or that a predefined time period has elapsed without any improvement in the net's performance. Once the neural nets for each stock of the capital market have been trained and tested on the available historical data the neural nets are tied to surpass the underlying market benchmark by predicting, the task becomes one of holding a smaller subset of all stocks of the market, such that this subset has a higher expected return and about the same level of risk as the market index. Such a task requires one to focus on individual stocks and their performance in relation to the index that serves as the underlying performance benchmark.

[0005] Individual stocks usually have their own unique performance characteristics some of which can be quantified. Clearly, however, the relationships among such data are complicated and frequently non-linear, making them difficult to analyze. In summary, an investment decision in the modern capital markets requires processing of large volumes of data and taking into account a number of factors which may exhibit significant non-linear relationships among different components of the data.

[0006] Computers, in general, are very adept at dealing with large amounts of numerical information. However, sophisticated techniques are required to analyze and combine disparate information that can potentially impact security prices. Several expert computer systems have been deployed in the domain of finance, including some in the area of investment management.

[0007] In the past several years, recurrent neural networks (RNN) have become popular in solving a variety of problems. Neural nets mimic the ability of the human brain to recognize recurring patterns that are not just identical but similar. A neural net can predict the value of one variable based on input from several other variables that can impact it. The prediction is made on the basis of an inventory of patterns previously learned, seeking the most relevant in a particular situation. In summary, RNNs can “learn” by example and modify themselves by adjusting and adapting to changing conditions. Several applications of neural nets to the domain of finance are already known in the art. Typically, the RNN prediction systems are “self” trained by adjusting weights and biases as a result of numerous repetitions. What the known systems typically do not do is to calculate an error function so the system's output can be adjusted or controlled in accordance with the determined error.

[0008] U.S. Pat. No. 5,109,475 to Kosaka et al. discloses a neural network for selection of time series data. This process is illustrated in a particular application to the problem of stock portfolio selection. In the first step of the proposed process, certain characteristics for each security are calculated from time series data related to the security. The characteristics to be computed include the historical risk (variance and co-variance) and the return. The following step involves the establishment of a performance function based on the calculated characteristics and, in the third step of the process, a Hopfield neural network is used to select a subset of securities from a predefined universe. Due to the fact that the Kosaka system only considers historical risk and return data, and implicitly assumes that the relationship between risk and return factors will remain stable in the future, in a typical rapidly changing market environment, it is unlikely to predict accurately price variations which are subject to complicated non-linear relationships.

[0009] U.K Pat. application 2 253 081 A to Hatano et al. discloses a neural net for stock selection using only price data as input. The main idea of the proposed system is to calculate runs (sequences) of price trends, increases and decreases, using a point-and-figure chart and using the maximum and minimum values from the chart to make a time-series prediction using a neural network. As in the previous case, the Hatano system only uses historic price data which places limitation on the type and quality of predictions that may be achieved. Additionally, the use of only the external points of the price chart obscures even further information about any time dependencies that might be present in the original data.

[0010] The above-described financial systems do not fully utilize the potential of the neural nets for stock selection. Notably missing is the possibility to develop the standard adaptive training procedure of the RNN to determine a prediction error or function in accordance to which the RNN output can be controlled. Further, many of the known investment management systems have not been able to effectively output the upper and lower error bounds at a given confidence level. Further, the movements of the stock prices, as well as price movements of other financial instruments, generally present a deterministic trend superimposed with some “noise” signals, which are, in turn, composed of truly random and chaotic signals, as illustrated in FIG. 1. Deterministic trends can be detected and assessed by some maximum-likelihood processes. Although a truly random signal, often represented by a Brownian motion, is unpredictable, it can be estimated by its mean and standard deviation. The chaotic signal, seemingly random but with deterministic nature, proves predictable to some degree by means of several analysis techniques, among which the ANN techniques have proven most effective over the widest range of predictive variables. However, this trend is largely ignored by the above-discussed references. As a result, at least some of the known systems are fed with data including this deterministic trend that influences the training stage of the known systems. Overall, many of the known systems are limited for the prediction of specific types of securities and data, such as the price of a single stock and, thus, cannot be universally applied to any financial time series, price series and volatility series.

[0011] It is, therefore, desirable to provide a prediction system based on the recurrent Artificial Neural Network (ANN) architecture which is able to output upper and lower predictions bounds at any given confidence level. Also, an ANN prediction that can be applied to financial time series, price series and volatility series, for single securities and for portfolios of securities is desirable. A universal prediction system employing a pipeline recurrent neural network (PRNN), which provides the satisfactory accuracy of the nonlinear and adaptive prediction of nonstationary signal and time series processes is also desirable. Further, a universal ANN prediction system having high computation efficiency and multi-stage adaptive supervised training process is also desirable.

[0012] Accordingly, an inventive universal ANN prediction system motivated in its design by the human nervous system is capable of learning by training to generalize from special cases and outputting a three-line band to forecast shortterm movements of stock prices.

[0013] Referring to one aspect of the invention, a supervised training and prediction system is so trained that the online investors are presented with the forecast of short-term movements of stock prices in a scientific way. Particularly, a three-line presentation that defines a confidence range or level, within which a price stock is predicted to fluctuate, enables the online investors to obtain a good understanding of the possible stock prices in a probability sense.

[0014] Such presentation becomes possible due a training procedure of a ANN prediction system in accordance with the invention. Particularly, a training set, which includes the input data for the ANN to “see” and the known target data for ANN to learn to output, is first collected. For stock price predictions, for example, the training set and target data would naturally be historical stock prices. A vector of 100 consecutive historical stock prices, for instance, can constitute a training data and the 101st stock price can be a target datum. The inventive process further is characterized by feeding the input data to ANN; compare ANN output with the known target, and adjust ANN's internal parameters (weights and biases) so that ANN output and the known target are close to one another—more precisely, so that a certain error function is minimized. Further, the process is characterized by feeding ANN some future input data (not seen by ANN); if ANN is well trained and if the input data is predictable, then ANN will give accurate predictions.

[0015] The inventive system is essentially an Artificial Neural Network trained for adaptive prediction of stock prices. During the prediction process, the inventive system (TradetrekTm Neuro-Predictorm) determines whether a particular stock is predictable with the accuracy required for a statistically significant prediction. This is accomplished, essentially, by comparing the ANN validation error against stock price fluctuations. We know that stocks with larger chaotic components and smaller truly random components tend to be more predictable than others.

[0016] In accordance with still another aspect of the invention, the deterministic or expected trend of the chaotic component of a signal representing the evaluated time-series data is determined in accordance with log-linear chisquared linear least squares based on the Black-Scholes stock price formula. The Black-Scholes formula, or other option pricing formula, is used to determine expected option costs in determining necessary hedging and pricing. The Black-Scholes formula provides an option cost based upon index price, exercise price, option term and assumptions of risk free rates of return, average dividend yield, and volatility of returns (standard deviation of returns). The trend is removed before feeding the data to the ANN engine and added back to the data in the post processing stage of the inventive process.

[0017] Overall, the simplified ANN (supervised) training and prediction process can be illustrated by the following steps.

[0018] Stage One:

[0019] Collect the training set, which includes the input data for the ANN to “see” and the known target data for ANN to learn to output. For stock price predictions, for example, the training set and target data would naturally be historical stock prices. A vector of 100 consecutive historical stock prices, for instance, can constitute training data and with the 101st stock price as a target datum.

[0020] Stage Two:

[0021] Feed the input data to ANN; compare ANN output with the known target, and adjust ANN's internal parameters (weights and biases) so that ANN output and the known target are close to one another—more precisely, so that a certain error function is determined and further minimized.

[0022] Step Three:

[0023] Feed ANN some future input data (not seen by ANN); if ANN is well trained and if the input data are predictable, then ANN will give accurate predictions.

[0024] The inventive system has managed to yield prediction refinements well beyond those of other systems by employing a pipelined recurrent ANN architecture (best for time-series prediction) and an adaptive supervised training procedure.

[0025] It is, accordingly, an object of the present invention to provide an artificial neural network system operating with a determined error function for data processing and predicting stock prices at a given confidence level.

[0026] It is yet another object of the present invention to develop a process for stock prediction on the basis of evaluation of the collected data using a neural network system.

[0027] Yet another object of the present invention is to provide a data processing system based on an artificial neural network employing a pipelined recurrent ANN architecture to provide the satisfactory accuracy of the nonlinear and adaptive prediction of time series process.

[0028] A further object of the invention relates to a component object module (COP) technique allowing any COM support computer languages and applications to call the inventive prediction system.

[0029] The above and other objects, features and advantages will become more readily apparent from the following detailed description, which is accompanied by the following drawings, in which:

[0030] FIG. 1 is a graph representing a truly random signal and a chaotic or oscillating component of a stock.

[0031] FIG. 2 is a graphical presentation of a three-line band forecast illustrating a time-series financial security exhibiting a predicted trend within upper and lower boundaries which define a confidence level in accordance with the invention.

[0032] FIG. 3 is a basic flow chart generally illustrating a universal time series prediction system for financial securities in accordance with the invention.

[0033] FIG. 4 is a flow chart illustrating an ANN engine of the universal time series prediction system shown in FIG. 3 and including a pre- and post-processing system for removing the deterministic trend from the data fed to the ANN engine and adding it to the processed data, respectively.

[0034] FIG. 5 is a flow chart illustrating the universal time series prediction system for financial securities shown in FIGS. 2-4.

[0035] FIG. 6 is a flow chart illustrating a step-by-step process of training the universal time series prediction system for financial securities shown in FIGS. 2-5 in accordance with the present invention.

[0036] Referring to FIGS. 2-6 and particularly to FIG. 2, a three-line band, which graphically presents a universal time series prediction system for financial securities stock price system and enables the online investors to obtain an illustrative understanding of the possible stock price movements. Particularly, price fluctuation of a stock 20 is traced for a period of time until a real time point at which financial time series prediction is outputted at a given confidence level. Specifically, the prediction system is capable of outputting upper 22 and lower 24 prediction bounds of the most likely price movement of a stock 20 which could be predicted within a confidence range W delimited by the upper and lower bounds. The confidence range depends on how well a prediction system is trained, and, in accordance with the invention, can be as high as 80%. This probability indicates that the stock 20 will remain between the lower and upper boundaries within the next few days. In order to obtain such a high probability, an artificial neural network (ANN) prediction system is trained in accordance with the inventive method better illustrated in FIGS. 3-7.

[0037] The Neural Network Stock Prediction Engine 30 basically includes a three-step process, comprised of a pre-processing stage 32, training or learning stage 34 and a post-processing stage 36. The pre-processing stage includes software executing on a computer 38 for determination of the deterministic or expected trend of a financial time-series signal 10, which, as mentioned before and illustrated in FIG. 1 is comprised of a truly random component 12 and a chaotic component 14 which has the expected trend. The trend determination relies on certain chi-squared analysis based on Black-Scholes formula well known in the art. Particularly, for predicting prices of a single stock, log-linear chi-squared linear least squares analysis is used. For the prediction of the prices of a portfolio of stocks, linear chi-squared linear least squares analysis is used. For the prediction of other financial quantities, such as volatilities and the prices of other financial securities, the corresponding financial models (e.g., GARCH) are used in the pre-processing analysis. Different pre-processing processes can be selected dynamically at run time.

[0038] The basic idea of these processes is to minimize the following error function:

[0039] E=Σw(f(x)−y)²

[0040] where: x is time, y is the quantities of the particular item, such as stock prices to be predicted, f is the expected trend to be determined, w is the trading volume divided by the volatility or spread of the quantities y at the time x.

[0041] After this trend has been determined it is removed from the further training process, and after the Neural Network Stock Prediction engine (ANN) is trained at 34, it is added back to the prediction result at 36.

[0042] The Neural network prediction system shown in FIGS. 4-6 includes a computer receiving an input 40 from a powerful database 60 (FIG. 5), which in case of stock price predictions, will naturally be a plurality of historical stock prices and a known target. For instance, a vector of 100 consecutive stock prices constituting a training data and a price of the 101st stock price selected as a target data constitute a first training set.

[0043] The training set is fed into ANN 42 which processes this set in accordance with software executing on the computer in order to enable ANN to adjust its weights and biases after the ANN's output 58 has been repeatedly compared to the target data. After the input data is adjusted by the ANN 42 in such a manner that the ANN output 44 and the target data are close, a new set of historical data is fed in ANN 42 and, as the process is repeated, an error function of the ANN is determined and stored at 46 as an estimated prediction error which is applied to any future results predicted by ANN at 48.

[0044] Once the ANN is trained, any prediction of a short term price fluctuation of a stock which is presented with a probability range, as shown in FIG. 2, can be transmitted to a trader 50 through a communication link 62.

[0045] As shown in greater detail in FIG. 5, a set of M consecutive stock prices is selected and pre-processed at 54 in accordance with the pre-processed process described in reference to FIG. 4 and fed to ANN which is comprised of a plurality of modules 56, 58 and 62. Each of modules is basically a cell processing a uniform number N<M of the consecutive stock prices and a target data which is a stock price N+1. Thus, a first module 56 is adapted to process a first 1 through N stock prices, generating an output and compare it to the target price, which typically necessitates a certain correction in order to bring the output closer to the target data. Once the result is satisfactory, a next set of 2 through N+1 stock prices is fed and processed in order to be compared with the target value N+2, after which internal parameters once again are adjusted to minimize a function error of ANN. This process continues until the target data is the last M stock price of the first training set.

[0046] Subsequently, a second training set previously unseen by ANN is processed at 64 analogously to the first set, but in addition to adjusting the internal ANN parameters, an average prediction error is determined at 66, as will be explained in detail hereinbelow, and any future result transmitted through the communication link as explained in reference to FIG. 4.

[0047] FIG. 6 illustrates a step-by-step training process of ANN and includes a powerful computer distinguished for its high computational efficiency which allows the ANN to process any financial time series, price series and volatility series, for single securities and for portfolios of securities.

[0048] An adaptive supervised training and prediction process in accordance with the invention includes collecting the training set selected from a stored time series at 70. As disclosed above, the training set includes an input data for the ANN to “see” and the known supervised target data for the ANN to learn to output. After inputting subsequent groups of samples at 72 and 76, the ANN performs adjustment steps 74 and 78, respectively, to adjust its internal parameters so as to bring the ANN's output and the respective known target close to one another. As a consequence, an error function of the ANN is continuously minimized due to a great number of repetitions at 80 and 82. A new set selected from the same data but unknown to the ANN is collected at 84 to be processed in a one-step adaptive training stage. This procedure is similar to the previous one, but in addition to the adjustment of internal parameters, the difference between each output of the ANN and a respective target data is first determined and stored at 86 in order to determine an average prediction error at 88. Finally, all further predictions will be determined in accordance with the one-step stage 90 and automatically adjusted to incorporate the estimated prediction error.

[0049] The ANN engine uses a Component Object Module technique (COM) which provides the interfaces between modules and allows any COM support computer language and applications to call the inventive prediction system.

[0050] Although the invention has been described with reference to a particular arrangements of parts, features and the like, these are not intended to exhaust all possible arrangements or features, and indeed many other modifications and variations will be ascertainable to those of skill in the art.