Method of numerical times series prediction based on non-numerical time series

[0002] The present invention relates to a method for determining a future value of a numerical time series and, in particular, to a method for predicting a future value or direction of a numerical time series based on a non-numerical time series.

[0003] Prior art methods exist for predicting a future value or direction of a numerical time series. Generally, these prior art methods apply statistical analysis techniques to the numerical time series data as a basis for predicting future occurrences in the numerical time series. An application of these prior art methods can be found in financial trading systems. Typically, financial trading systems use indicators and predictors for determining a future value or direction of a financial time series, such as the price of a commodity or a market index, and whether the market for the instrument underlying the financial time series should be entered or exited.

[0004] An indicator can be a financial time series, for example the historical daily closing prices of the Nasdaq composite, or functions, such as fast stochastic or moving averages that generate a numerical time series. A predictor is a statistical technique, for example simplex prediction, simplex difference prediction, polynomial prediction and fractional prediction, that when applied to a financial time series predict, with a given probability, a future value or direction of the financial time series. Based on a selection of indicators and predictors, the trading system predicts a future value or direction of the financial time series and whether a trade should be made.

[0005] Predictive systems are also used to analyze numerical time series to predict a future value or direction in a variety of other contexts as well, such as to project sales, perform climate forecasts (e.g. El Niño) and risk analysis.

[0006] Although prior art techniques for predicting a future value or direction of a numerical time series work for their intended purpose, the accuracy of the predictions made using such techniques is necessarily limited to information embedded within the numerical time series and the effectiveness of the statistical techniques used in mining such information. Accordingly, it is desirable to provide a more accurate method of predicting the future value or direction of a numerical time series.

[0007] The present invention is directed to a method for using a non-numerical time series to predict a future value or direction of a numerical time series. In accordance with methods of the present invention, a future value or direction of a first numerical time series is predicted by performing the steps of forming a non-numerical time series; optimizing the non-numerical time series; converting at least a portion of the optimized non-numerical time series into a second numerical time series; and using the second numerical time series to predict the future value of the first numerical time series.

[0008] The methods of the present invention are adaptable to predict the future value or direction of a broad range of numerical time series. For example, in one application of the present invention, a text-based time series, such as a series of newspaper articles, is used to predict a future value or direction of a financial time series thereby increasing the profitability of trading the financial instrument underlying the financial time series. Similarly, the methods of the present invention may applied in a variety of contexts including, but not limited to, predicting future sales trends, political trends and reactions and media responses.

[0009] Accordingly, it is an object of the present invention to provide a more accurate method for predicting the future value or direction of a numerical or non-numerical time series.

[0010] Another object of the present invention is to use a non-numerical time series to predict the future value or direction of a numerical time series or another non-numerical time series.

[0011] It is yet another object of the present invention to use a numerical time series to predict the future value of non-numerical time series.

[0012] Still other objects and advantages of the invention will in part be obvious and will in part be apparent from the specification.

[0013] The invention accordingly comprises the several steps and the relation of one or more of such steps with respect to each of the others thereof, which will be exemplified in the method hereinafter disclosed, and the scope of the invention will be indicated in the claims.

[0014] For a fuller understanding of the invention, reference is had to the following description taken in conjunction with the accompanying drawings, in which:

[0015] FIG. 1 is a flowchart of the method of the present invention applied to a financial trading system; and

[0016] FIG. 2 is a flowchart of a genetic algorithm optimization process according to an exemplary embodiment of the present invention.

[0017] Referring now to FIG. 1, there is shown a flowchart of the method of the present invention as applied to a financial trading system. The first step of the method, Step 1, is to formulate a text-based time series, X[t] which in the context of a financial trading system, may consist of financial articles such as articles from the Wall Street Journal or any other source which consistently reports financial information within a short time period of occurrence. The length of X[t], or its training size, is selected to be of sufficient length to provide statistically significant results but not too long so that any statistical significance embedded in X[t] is not averaged (diluted) over time. In an exemplary embodiment, the length of X[t] may be in the range of 6 months to several years. Also, the resolution of time series X[t], and thus the maximum rate at which predictions based on X[t] can be generated, may be varied depending on the nature of text-based time series selected. For example, if the text-based time series consists of the daily Wall Street Journal articles, then the resolution of X[t] is one day. On the other hand, if the text-based time series consists of real-time news source, then X[t] will have a smaller resolution, perhaps one minute.

[0018] Once the text-based time series X[t] is selected, X[t] is parsed and all functional words, for example words such as the, and, or, etc., are removed from X[t] because such words provide little predictive value on their own. In an alternative embodiment, the functional words are not removed from X[t] and a parser, such as a link grammar parser (by Sleator and Temperley) is applied to X[t] to disambiguate words and phrases of X[t]. For example, such a parser can distinguish between different uses of a word, for example between a train versus to train.

[0019] The next step in the method, Step 2, is to optimize the non-numerical time series X[t] so that any predictive information embedded in X[t] can be identified. Non-numerical time series X[t] may be optimized using any optimization techniques known in the art, including, but not limited to, simulated annealing, conjugate gradients, Newton's method and Monte Carlo method, steepest descent and multi-start.

[0020] Referring now to FIG. 2, there is shown a flowchart showing the optimization of X[t] using a genetic algorithm according to an exemplary embodiment of the present invention. The first step in optimizing X[t] using a genetic algorithm is to form a plurality of qsets based on the text-based time series X[t], as in Step 1. A qset is a number of words selected pseudo-randomly from X[t] and can be of any size, i.e. contain any number of words. In an exemplary embodiment, the qset size will be in the range of one to ten words and will be selected based on how well the text series X[t] correlates with a numerical time series Y[t], the numerical time series for which a prediction of a future value or direction is desired. For example, if X[t] contains Wall Street Journal articles and Y[t] is the historical pricing of the Dow Jones Industrial Average, a qset size of 4 or 5 words may be acceptable because Y[t] and X[t] are likely highly correlated. If, however, Y[t] is widget sales data and X[t] contains articles relating generally to sales, a larger qset may be required if X[t] and Y[t] are not highly correlated.

[0021] Although the number of qsets selected from X[t], i.e. the qset population size, can be of any size, in an exemplary embodiment the qset population size is in the range of 100 to 100,000. While it is most preferred to select a number of qsets towards the upper end of this range thereby assuring a highly optimized X[t], the precise number of qsets selected will depend on the size of X[t] and will be limited by the available computer processing power. Also, if the qset population is too small, “inbreeding” of the qsets may result during the genetic algorithm optimization process in which case some qsets may get stuck at local minima and the most optimal qsets may not be identified.

[0022] Once the initial population of qsets have been selected, the fitness of the qset population is optimized by recursively applying a genetic algorithm to the qset population. Under the genetic algorithm, an optimizing technique known in the art, (see, for e.g., D. Goldberg, Genetic Algorithms, Optimization and Machine learning, 1989), the qsets in the initial qset population reproduce to form an offspring qset generation. The less fit qsets in the offspring generation are identified based on a selected fitness measurement applied to each qset. The less fit qsets are weeded out from the population so that only the most fit qsets survive.

[0023] In particular, the qset offspring generation is derived from the initial qset population by applying the genetic algorithm processes of mutation and crossover to the initial qset population, as in Step 2. In the mutation process, an element is randomly removed from a particular qset and replaced with a randomly selected element from X[t]. The resulting qset is included in the offspring qset population. In the crossover process, two qsets are selected and mated to produce an offspring qset. Although the mating process used may be any mating techniques known in the art, an exemplary mating process includes forming the offspring qset by randomly selecting from the elements of the two parent qsets. For example, if the first parent qset contains Asia, yen, Greenspan, Clinton and ruble, and the second qset parent contains Syquest, Nasdaq, Russia, correction and rebound, then the offspring q-set may contain Asia, Syquest, Nasdaq, Clinton and rebound, or any other permutation of the two parents.

[0024] The manner in which the genetic algorithm produces an offspring qset generation may be varied by changing the appropriate genetic algorithm parameters. These parameters control which qsets are selected for mutation and crossover, the mutation rate, whether patricide occurs, the percent of the offspring qset generation that survive, the percent that are reaped based on a particular fitness measurement and the maximum number of generations allowed. The genetic algorithm parameters that will provide optimal results are selected in accordance with the ordinary skill in the art. In an exemplary embodiment, the mutation rate is set to be in the range of 0.1% and 3%, the population size is set to 500, and the maximum number of generations is set to 500.

[0025] The next step in evolving the qset population under the genetic algorithm is to determine the fitness of each of the offspring qsets which is used as a basis for reaping out the less fit qsets from the offspring generation, as in Step 3. To determine the fitness of each qset, we first apply a P function to X[t] for all qsets in the qset population so that

[0026] P(X[t], q_iset)→x_i(t)  (1)

[0027] where i is the population size and x_i(t) is a numerical time series having at each time t a number equal to the times the words in qset_iappear in the text-based time series X[t]. Next, a fitness function m is applied to x_i(t) and Y[t+k] (where k can be zero or greater; if k=0 it indicates that X[t] includes articles from a morning newspaper published before the market in the security underlying Y[t] opens and Y[t] is that day's closing price in that market and, if k>0, then Y[t+k] is the closing price in that market k days later) yielding a series of numbers n_ifor each numerical time series x_i(t) number according to the formula

[0028] m(x_i(t), Y[t+k])=n_i, −1≦n_i≦1.  (2)

[0029] Fitness function m may be any function known in the art that measures the relationship between two numerical time series, such as, for example, Discrete Fourier analysis, dynamic programming string matching, subsequence matching, or wavelets. In an exemplary embodiment, fitness function m is a mean-square correlation of numerical time series of x_i(t) and Y[t]. The number n_iis a measure of the correlation between numerical time series x_i(t) and Y[t]. For example, if n=1 for a particular i, then time series x_i(t) is highly correlated to time series Y[t] i.e. x_i(t) is a positive indicator of the future value or direction of Y[t]. If n=−1 for a particular i, then x_i(t) is highly anti-correlated to time series Y[t], i.e. x_i(t) is a contrary indicator of the future value or direction of Y[t]. If n=0 for a particular i, then there is no correlation between x_i(t) and Y[t].

[0030] So, for example, if the application of the P function according to equation (1) yields for a particular qset_ia numerical time series x_i(t)={7, 8, 0, 15}, which represents the number times the words contained in qset_iappear in X[t] for each time t, then applying the fitness function m according to equation (2) will produce a number, for example 0.3, which indicates the level of correlation between the particular x_i(t) and Y[t].

[0031] In an exemplary embodiment, a delta function F is applied to the numerical time series Y[t] to convert the numerical time series data from absolute values to the incremental changes in value of Y[t] for each time t. For example, if the Dow Jones Industrial Average for four previous days is 7980, 7985, 7983 and 7990, applying delta function F would yield a Y[t]={0, +5, −2, +7} which represent the changes in the Dow Jones Industrial Average over the course of the financial time series. The purpose of filtering Y[t] in this manner is so that x_i(t) is correlated to the changes in Y[t], and not to the absolute value of Y[t]. In addition to applying delta function F to Y[t], other functions may be used to precondition numerical time series Y[t] or x_i[t] to reduce “noise” inherent in Y[t] and x_i[t], i.e. data in Y[t] and x_i[t] that does not provide any predictive value or that may dilute the fitness measure of a particular qset.

[0032] Thus, fitness function m is applied to x_i(t) for all i thereby producing i fitness measurements, one for each qset in the population. Based on the qset fitness measurements, the least fit members of the qset population are weeded out of the qset population leaving only some portion of the fittest qsets to survive, as in Step 4. As described above, the size of the qset population after reaping is a function of the genetic algorithm parameters which are selected to yield a population size that is sufficiently large to avoid inbreeding and not too large so as to be computationally unwieldy. After reaping, the average fitness of all qsets in the population is calculated, as in Step 7. Also, the qset that is found to be the most fit is saved.

[0033] Next, in Step 8, it is determined whether the average fitness of the most recently evolved qset population converges, i.e. the population has ceased to improve substantially. If the qset population fitness did not yet converge, the genetic algorithm optimization process is again applied to the latest qset offspring generation thereby evolving more fit qset populations. After each iteration, it is determined whether the most fit qset of the latest generation is the most fit yet found, as in Step 5, and if so, that qset is saved, as in Step 6. The genetic algorithm optimization process is stopped, as in Step 9, when the average fitness of the most recently evolved qset population converges. At this point, it is likely that the most optimally fit qset, referred to as q_best, has already been identified. This optimal qset q_bestis used as a basis for predicting a future value or direction of Y[t]. For example, in context of financial trading systems, the method of the present invention may be used to improve the performance such financial trading systems, as will now be described.

[0034] The first step in improving a financial trading system using the present invention is to generate a population of rule sets, rset as in Step 3 of FIG. 1. An rset is a Boolean function that includes one or more indicators I_j(t), a predictor D, threshold values and x_best[t] which is generated by applying the P function to X[t] using the best qset, q_best, as follows:

[0035] P_qbest(X[t])→x_best(t).

[0036] For example, an rset may be as follows: enter LONG iff simplex difference predicts>0.0014% rise (a predictor) AND slow stochastic (FS window size=34, MA window size=11) is less than 50.5 (an indicator) AND X_best[t]>0.86 (a text indicator compared to threshold), then enter the market in the financial instrument underlying the financial time series Y[t] and exit after one session. The population of rsets are generated by choosing random initial values, using all available indicators I_j(t) and predictor D and x_best(t). The rset population size is chosen using the same criteria used in selecting the qset population size—it should be large enough to avoid inbreeding but not too large so as not to be computationally unwieldy. In an exemplary embodiment, the initial rset population size is 100.

[0037] Once the initial rset population is formed, a genetic algorithm, similar to the genetic algorithm described above with respect to qset optimization, is applied to optimize the initial rset population. The fitness of each rset is measured by applying a fitness function

[0038] E_rset(D,I_j(t), x_best(t))  (3)

[0039] for each rset in the population which measures the performance of the trading system. The fitness measure used may be any measure by which the performance of the trading system is evaluated, such as, for example, profits or profits/maximum loss. The fitness of a particular rset as measured by E_rsetis determined by selecting a training period consisting of financial time series data selected from Y[t] and performing a trade in the financial instrument underlying financial time series Y[t] for each time t based on the parameters D, I_j(t) and x_best(t) of the particular rset. The profitability (if that is the selected fitness measure) of all the trades performed by the rset over the training period is calculated and used as the fitness of the rset. This process is repeated for all rsets in the population resulting in a fitness measure for each rset.

[0040] After the fitness of the initial rset population is determined, the genetic algorithm is repeatedly applied to evolve the rset population to identify the optimal rule set rsetbest, as in Step 4. This optimal rset rset_bestrepresents the optimal trading rule set based on indicators I_j(t), predictor D and x_best(t).

[0041] Optimal rset rset_bestand qset_best, which embodies predictive information gleaned from text series X[t], are used to produce a signal to either enter or exit the market for the financial instrument that underlies Y[t] in the following manner. First, by applying the P function of equation (1), as in Step 5, it is determined the number of times the words contained in qset_bestappears in a newly-released text series information, X[t+1], for example today's Wall Street Journal articles published prior to the opening of the market of the financial instrument that underlies Y[t]. This generates a numerical time series X_best(t+1) as follows:

[0042] P_qbest(X(t+1))→x_best(t+1).

[0043] Then we apply a trading function

[0044] T_rbest(x_best(t+1))

[0045] which uses the best rule set rset_bestto produce a market enter/exit signal based on x_best(t+1)), the number of times the words contained in q_bestappear in the newly released articles, as in Step 6. As a result, the predictive information embedded in the text-based time series X[t] is used to predict a future value or direction of the financial time series Y[t] thereby improving the ability of the trading system to time a market entry/exit.

[0046] Applying the methods of the present invention to a financial trading system yielded a significant increase in trading profits. To measure the benefits provided by the invention, a prior art trading system was used to generate trades in U.S Yen futures over a particular time range. Of the 61 trades made by the system, 36 were profitable and 25 were unprofitable. Then, the trading system was modified to include the method of present invention and was used to generate trades in the U.S. Yen futures over the same time range. Of the 12 trades made by the system, 10 were profitable and 2 were not. This indicates that although the improved trading system invention generated a lesser number of profitable trades, the trading system avoided a significant number of unprofitable trades that were made by prior art trading system. Thus, the profitability of trades made by a trading system incorporating the methods of the present invention was significantly higher than trades made by the prior art trading system alone.

[0047] Although the methods of the present invention were described in the context of using a text-based time series to predict a future value or direction of a financial time series to time market entry/exit points, it will be obvious to one of ordinary skill in the art that the present invention may be used to predict the future value and direction of other numerical and non-numerical time series. For example, a text-based time series consisting of trade articles relating to a particular market may be used to predict future sales of a product in that market.

[0048] Also, according to the methods of the present invention, a first text-based time series X1[t] can be used to predict the occurrence of a word, words or phrase in a second text-based time series X2[t]. In this case, a qset q is selected from X1[t] and includes the words or phrases that one desires to predict in X2[t]. A numeric series x₁[t] is then formed by applying

[0049] P(X1[t], q)→x₁[t].

[0050] Next, a qset population is formed based on text-based time series X2[t] and a series of numerical time series x_2i[t] is formed by

[0051] P(X2[t], q_iset)→x_2i[t]

[0052] as described above. Then, the fitness of the qset population based on X2[t] is measured by

[0053] m(x_2i[t], x₁[t]).

[0054] In an alternative embodiment, two qset populations Q1 and Q2 are formed from text-based time series X1[t] and X2[t], respectively. Then, a genetic algorithm is applied to each of qset populations Q1 and Q1 independently. For each qset in each qset population Q1, Q2, a time series is generated as follows:

[0055] P(X1[t], Q_i1)→x_1i[t]

[0056] P(X2[t], Q_i2)→x_1i[t].

[0057] Then, pairs of qsets are formed by selecting one qset from each qset population Q1 and Q2, and the fitness (correlation) of the qset pairs is measured by

[0058] m(x_1i[t], x_2i[t]).

[0059] In addition, the methods of the present invention can be used to predict the future occurrence of a word, words or phrase in a text-based time series based X[t] on a numerical timer series Y[t]. This approach is identical to that one used to predict a numerical time series based on a text-based timer series, described above, the only difference being that the m function is applied as follows:

[0060] m(Y[t], x_i[t+k]).

[0061] It will thus be seen that the objects set forth above, among those made apparent from the preceding description, are efficiently attained and, since certain changes may be made in carrying out the above method without departing from the spirit and scope of the invention, it is intended that all matter contained in the above description shall be interpreted as illustrative and not in a limiting sense.

[0062] It is also to be understood that the following claims are intended to cover all of the generic and specific features of the invention herein described, and all statements of the scope of the invention which, as a matter of language, might be said to fall therebetween.