6 Estimation, Smoothing, Interpolation, and Distribution for Structural Time-Series Models in Continuous Time A. C. Harvey and James H. Stock Introduc n Rex Bergstrom’s work has stressed not just the technical aspects but also the philosophical basis for applying continuous time models to time-series data; see Bergstrom (1966, 1976, 1984). Because many economic variables are essentially continuous and decisions are made continuously, it is often more appealing to set up models in continuous time even though observations are made at discrete intervals. The dynamic structure of a model is then not dependent on the observation interval, something which may bear no relation to the underlying data generation process. An application of continuous-time models emphasized in this paper is their use to estimate intermediate values of a discretely sampled time series. Adopting Litterman’s (1983) terminology, estimation at points between observations will be termed interpolation for a stock variable (sampled at a point in time) and distribution for a flow (sampled as an integral over a time interval). Using a continuous-time model in this context is appealing for several reasons. First, as emphasized by the contributors to Bergstrom (1976), the continuous-time framework provides a logically consistent basis for the handling of stocks and flows. Second, it provides a natural conceptual framework, with considerable technical simplifications, for handling irregularly spaced observations. Third, it provides a well-defined framework for interpolation and distribution to arbitrary subintervals. Historically, a key technical hurdle in applying continuous-time models to economic data has been the difficulty of evaluating the exact Gaussian likelihood for flow data and for mixed stock-flow systems. These problems have largely been solved for large classes of models by Bergstrom (1983, 1984, 1985, 1986) and by his students and collaborators. Here, we consider interpolation for stocks and distribution for flows. To simplify the discussion, we restrict attention to univariate series. This article studies continuous-time formulations within the context of structural time-series models in the sense of Harvey (1989). Structural models are formulated directly in terms of components of interest, such as trends, seasonals, and cycles. These components are functions of time and it is natural to regard them as being continuous. The essence of a structural time-series model is that its components are 55A.C. Harvey and James H., Stock 56 stochastic rather than deterministic. A continuous-time model can be set up to parameterize these stochastic movements. It can then be shown that, for the principal structural time-series models, the implied discrete-time model is, apart from some ‘minor differences, of the same form as a discrete-time model which one would set up without reference to the continuous-time formulation. This is true for both stocks and flows. Thus there is a logical consistency in the structural class. Since the components of structural time-series models have a direct interpretation, these models can often be specified without a detailed initial analysis of the data. The appropriateness of a particular specification is then checked by various diagnostics, and the whole model selection exercise is much more akin to what it is in econometrics; see for example Harvey (1985). Thus although data are not available on a continuous basis, the greater emphasis on prior considerations in mode! specification means that it is just as easy to adopt a continuous-time model formulation as a discrete one. The next two sections examine the exact discrete-time models implied by the underlying continuous-time structural models sampled at the observation timing interval. This is straightforward for stock variables, less so for flows. In each case we consider the following statistical problems: time domain estimation of the model parameters by maximum likelihood; prediction of future observations; estimation of the unobserved components at the observation points and at intermediate points; and estimation of what the observations themselves would have been at intermediate points. Structural Time-series Models in Discrete Time A structural time-series model is one which is set up in terms of components which have a direct interpretation. For an economic time series, these components will typically consist of a trend, a seasonal, an irregular, and perhaps even a cycle. Examples of the application of such models can be found in Engle (1978), Harvey (1985), and Kitagawa (1981), Other components can be brought into the model. For example, daily or weekly components can be included if appropriate data are available, A general review can be found in Harvey (1989). In the present article, attention is restricted primarily to trend, cycle, seasonal, and irregular components, defined as follows. Trend The level, 4», and slope, B,, are generated by the multivariate random walk process, He = thar + Broa +o (ia) 8, atte (ib) where , and ¢, are mutually uncorrelated white-noise processes with zero means and variances o2 and o? respectively. Cycle The cycle, y,, is stationary and is centered on a frequency A,, which lies in the range [0, x]. Its statistical formulation is GL tele 8Time-series Models in Continuous Time where x, and x} are uncorrelated white-noise disturbances with a common variance 2, and p is a damping factor which lies in the range 0 < p <1. Seasonal The seasonal component, 7,, is defined as the sum of an appropriate number of trigonometric terms, ,,, each having a specification of the form (2) with p equal to unity and A, equal to a given seasonal frequency, 2, = 2nj/s. Thus 24 = DI2u Yq» Where s is the number of “seasons” (assumed to be even) and where wl af cst sings Wane] fox [%] ~ [S. Aj cos Heal * (%] ©) Var (wf) with Var (@,. 2 for all j. Irregular The irregular term, ¢,, is generally taken to be a white-noise process, with variance 42, unless there are strong a priori grounds to assume otherwise, as in Hausman and Watson (1985). These components — trend, cycle, seasonal, and irregular — combine in various ways to give the principal structural time-series models for an observed series, Y,, t=1,..., 7. These models are: 1. Local Linear Trend The discrete-time process obeys Y= yt by @ where 1, is a stochastic trend of the form (1) and ¢, is a white-noise irregular term. 2. Local Level This is a special case of the local linear trend in which y, is just a random wal Be Bean Fe (5) 3, Basic Structural Model with Cycles Both seasonal and cyclical components may be brought into the model by expanding (4) to give Ya Met it Vet be © Each of these models can be handled statistically by putting them in state space form: &% = Ta. + Rim, Var (ny) (7a) ht (7b) = 20, + &, Var (e,) where a, is an m x 1 state vector and ¢, and n, are respectively scalar and g x 1 zero mean white-noise disturbances which are mutually uncorrelated. The matrices z,, T;, R,, and Q, are mx 1, mx m, mx g and g x g respectively. These matrices may depend on a number of parameters, known as hyperparameters. Thus, for example, 7A.C. Harvey and James H. Stock 58 {in the local linear trend model, (4), the hyperparameters are the variances oy, of and @2.In some circumstances 2, and R, will depend on time, say if there are some missing “observations, More often, z, and R, will be time-invariant and henceforth will be denoted by z and R. The state vector may be estimated by the Kalman filter. Furthermore, if the disturbances are normally distributed, the unknown hyperparameters may be estimated by maximum likelihood via the prediction error decomposition; see Ansley and Kohn (1985), De Jong (1991) and Harvey (1989, chapter 4). General State Form of Continuous Time Models with Stocks and Flows This section summarizes some results for the general continuous-time model when the data are observed at T irregular observation times {t,}, t= 1,..., 7. These observation times are separated by calendar time units ,, so that t, +5. ‘The continuous-time state vector is denoted by a(t); at the observation times, itis denoted by a, = a(t,). Thus, in the notational convention adopted here, a(t), y(t), ete denote continuous-time processes, and @,, y,, ete. denote these processes at the appropriate discrete-time sampling dates. The observable (discrete time) process is ¥, and the data are T observations on the (discrete time) time series pees pe ‘The continuous-time analog of the time-invariant discrete-time transition equation in (Ja) is da(t) = Aa(t) de + R dn(e) ® where the matrices A and R are m x m and m x g respectively and may be functions of hyperparameters and y(t) is a g x 1 continuous-time multivariate Wiener process. Fora discussion ofthe formal interpretation of linear stochastic differential equations, see Bergstrom (1983). The Wiener process has independent increments that are ‘Gaussian with mean zero and covariance matrix lf’ dnt) f ani | =(6-70. Suppose we have a univariate series of observations at time {t,} for t For a stock variable the observations are defined by Ye= a(t.) +64 ® where ¢, is white-noise disturbance term with mean zero and variance a? which is uncorrelated with differences of n(t) in all time periods. For a flow Y= ik za(r) dr + {' de(), 1 =1,.. (10) where e(t) is a continuous-time Gaussian process with uncorrelated increments, meanTime-series Models in Continuous Time zero and variance o2, which is uncorrelated with n(¢) in all time periods in that a[{’ an) [ au] =0 for all r 2, p(t) = 0. Whereas in (4) and (5), p(1) is always negative, lying in the range [—0.5,0], in this case p(t) ef—05,025}. ‘One interesting consequence is that a series with a first-order autocorrelation of 0.25 in first differences can be modeled simply by a time-aggregated Brownian motion; compare Working (1960). A second point is that A ¥, follows a discrete-time random walk when q = 6. This means that a discrete-time random walk can be smoothed to a limited extent since the corresponding continuous-time model contains an additive disturbance term. Of course the same can be done when a discrete-time random- walk-plus-noise model is formulated at a finer timing interval than the observation interval; see Harvey (1989, chapter 6). Smoothing and Distribution Suppose that the observations are evenly spaced at intervals of 5 and that one wishes to estimate certain integrals of linear combinations of the state vector at evenly spaced intervals A time periods apart where 5/A is a positive integer. For example, it might bbe desirable to distribute quarterly observations to a monthly level. The quantitiesTime-series Models in Continuous Time to be estimated may be written in an m* x 1 vector as a(t) FP zt. +985 tial... @/A)T, (5) ° where Z is an m* x m selection matrix. In the continuous-time basic structural model, the components of interest might be (for example) the level of the trend, the slope and the seasonal, in which case Z’a(¢) = {u(), B(@, (0}'. In addition it may be desirable to estimate the values of the series itself, 4 te alt). +8) ds + { yes y(t) = [ de(s). 36) 0 : Smoothed estimates of the quantities of interest may be obtained from an augmented discrete-time state space model. The transition equation is o 0 0 OPany 10 0 yf ZW(A) ¢, 0 0 yha og 1: 1 a=] = rade ot} g] eo af ZW) 0 0 Off at, 0 z' {inl} o yt. zw) 0 0 odLys,. o 7 with 4, = 0, i = (6/A)r — 1) + 1,¢ = 1,..., Trand 6, = 1 otherwise. The covariance matrix of (nj, nf) is defined as in (24) with 6, replaced by A, while Var (ef) = Ac?. The corresponding measurement equation is only defined at observation times and can be written as Y= [0 1 0° Ole} fori=(6/A)r, t=1,..., 7. The distributed values of the series (the y$ terms) could alternatively be estimated by differencing the estimators of the y{ terms. The appearance of y? in the state is really only necessary if the MSE of its estimator is required. Of course if A = 4, it becomes totally superfluous, Predictions In making predictions for a flow it is necessary to distinguish between the total accumulation from time 1, to time t, +1, which might include several unit time intervals, and the amount of the flow in a single time period ending at time t, +1. The latter concept corresponds to the usual idea of prediction in a discrete model. Cumulative predictions are perhaps more natural in a continuous-time model, particularly when the observations are made at irregular intervals. Here, three types of predictions are discussed: cumulative predictions, predictions over the unit interval, and predictions over a variable lead time, Cumulative Predictions Predictions for the cumulative effect of y(t) are obtained by noting that the quantity required is y‘(t, + 1) which in terms of the state space model 23) is yr41 with 5,4, = 1, where it is assumed that ! > 0. The optimal predictor can therefore be obtained directly from the Kalman filter as can its MSE, Written out explicitly, y'(tr + 117) = Yyu slr = 2 Way. Because the quantity to be estimated is 67‘A.C. Harvey and James H. Stock V(t +1) = fo vr +) dr = Dar + znfer + e+. the prediction MSE is MSE [yp + ITV] = 2 ODP, WOY2 + 2 Var (nee + Var r+). G8) Note that if the modified Kalman fiter based on (25) is run, ar = af. x17 (because af, = a;) and soa, isthe optimal estimator of a based on all the observations. ‘AS a simple example, consider the local level model. For I > 0, YGp4 UT) = ly, MSELy Ge + T= P Pr + Po3/3 + loz. G9) Corresponding expressions for the discrete-time models can be obtained; see for example Johnson and Harrison (1986). However, the derivation of (39) is both simpler and more elegant. For the local linear trend, (27) gives Yee +7) = Mery + 3PBrirs (40) MSE [y'(te +117] = Pir + Pian + M*Paar/4 + Pog /3 + 1502/20 + 102, (41) where Pir is the (i,j) element of Pr. Predictions over the Unit Interval Predictions over the unit interval emerge quite naturally from the state space form (23) as the predictions of Yq 1, ! = 1,2... with Br, set equal to unity forall , The forecast function for the state vector, ay +1 = ¢“'8r, has the same form as in the corresponding stock variable model. The presence of the term (I) in (23a) leads to a slight modification when these forecasts are translated into a prediction for the series itself. Specifically, Year = 2 WO)ars-1yr = 20) eA Magy 12. (42) |As a special case, in the local linear trend model apy1—ayr = Orr + = DBrir» Brirls 80 Yrsur = Brin + C—PBrix for = 1,2,.... The one-half arises here because each observation is cumulated over the unit interval. Predictions over a Variable Lead Time In some applications the lead time itself can bbe regarded as a random variable. This happens, for example, in inventory control problems where an order is put in to meet demand, but the delivery time is uncertain. In such situations it may be useful to determine the unconditional distribution of the cumulation of y(?) from the current point in time, T. Assume the random lead oe is independent of {y(9)}. The unconditional PDF of this cumulation from tp to ty + lis Py tr + DIMz) = fore +D|Mr, I) dF, 43) where F(\) is the distribution of lead times and p(y'(¢, i ict here F(| , 1+ + D|Mz, 1) is the predictive distribution of y"(®) at time t; + J, that is the distribution of “(tp + 1) ‘conditional on | and on My = {¥,, ¥a,..., Yr}, ie. the information available at time ty. In a 68Time-series Models in Continuous Time Gaussian model, the mean of y‘(ty +!) (conditional on and M;) is given by y'Grr + 1) = 2 War and its conditional variance is the MSE given in (39). Although it can be difficult to derive the full PDF p(y'(ty + 1)|Mz), expressions for the mean and variance of this distribution may be obtained for the principal structural time series models; see Harvey and Snyder (1990). If the lead time distribution is taken to be discrete, the derivation of such expressions is much more tedious. Of course, in concrete applications the integral (43) can be evaluated numerically. Conclusion The formulas provided here for univariate stocks or flows are readily extended to multivariate mixed stock-flow systems; compare Harvey and Stock (1985) and Zadrozny (1988). Harvey and Stock (1988) develop this extension for a model in which the variables are cointegrated (so that the stochastic trend term is common among several multivariate time series); they also provide an empirical application to the estimation of the common stochastic trend among consumption and income using a mutivariate continuous-time components model. ‘An advantage of the continuous-time framework is that, in multivariate applica- tions, the observational frequency need not be the same for all the series. As a concrete example, weekly observations on interest rates (a “stock” variable), observations on some of the components of investment that are available monthly (a “flow”), and quarterly observations on total investment could be used to distribute total quarterly investment to a monthly level. Some of the components ~ say, trend and cycle - could be modeled as common among these series, and some could be modeled as independent (or perhaps correlated) across series. It should be emphasized, however, that the distributed values (or, for stocks, the interpolated values) resulting from the procedures outlined in this article have unavoidable measurement error. Moreover, this article has not addressed issues of aliasing, which could pose additional difficulties, for interpolation and distribution. Thus care must be taken in using these values in subsequent statistical analysis. Structural time-series models are based on fitting stochastic functions of time to the observations. A continuous-time formulation of structural models both is intuitively appealing and provides a logical consistency for both stock and flow data, The form of the model does not depend on the observation timing interval and hence can be applied to irregular observations. From the technical point of view, estimation, prediction, interpolation, and distribution can all be based on state space algorithms. Note Stock thanks the Sloan Foundation and the National Science Foundation for financial support through grants SES-86-18984 and SES-89-10601. References Anderson, B. D.O, and J.B. Moore (1979). Optimal Filtering. Englewood Cliffs: Prentice-Hall. Ansley, C. F., and R. Kobn (1985). “Estimation, filtering and smoothing in state space models with incompletely specified initial conditions”. Annals of Statistics, 13, 1286-316. 69A.C, Harvey and James H. Stock Bergstrom, A. R (1966). “Non-recursive models as discrete approximations to systems of stochastic differential equations,” Econometrica, 34, 173-82. Bergstrom, A. R. (1976) (ed). Statistical Inference in Continuous Time Economic Models. ‘Amsterdam: North-Holland. ; Bergstrom, A. R. (1983). “Gaussian estimation of structural parameters in higher-order continuous time dynamic models,” Econometrica, 51, 117-52. Bergstrom, A. R. (1984). “Continuous time stochastic models and issues of aggregation over time?” in Z. Griliches and M. D. Intrilligator (eds) Handbook of Econometrics, volume 2, pp. 1145-212. Amsterdam: North-Holland, | Bergstrom, A. R. (1985). “The estimation of parameters in non-stationary higher-order continuous time dynamic models,” Econometric Theory, 1, 369-85. Bergstrom, A. R. (1986). “The estimation of open higher-order continuous time dynamic models with mixed stock and flow data,” Econometric Theory, 2, 350-73, De Jong, P. (1991). “The Diffuse Kalman Filter,” Annals of Statistics, 19, 1073-83, Engle, R. F. (1978). “Estimating structural models of seasonality,” in A. Zeliner (ed.) Seasonal “Analysis of Economic Time Series, pp. 281-308. Washington: Bureau of the Census. Harvey, A. C. (1985). “Trends and cycles in macroeconomic time series,” Journal of Business and Economie Statistics, 3, 216-21. Harvey, A. C. (1989). Forecasting, Structural Time Series Models, and the Kalman Filter. ‘Cambridge: Cambridge University Press, Harvey, A. C, and R. D. Snyder (1990). “Structural time series models in inventory control,” International Journal of Forecasting, 6, 187-98. Harvey, A. C, and J. H. Stock (1985). “The estimation of higher order continuous time autoregressive processes,” Econometric Theory, 1, 97-117. Harvey, A. C, and J. H. Stock (1988). “Continuous time autoregressive models with common stochastic trends”, Journal of Economic Dynamics and Control, 12, 365-84. Hausman, J. A, and M. W. Watson (1985). “Errors in variable and seasonal adjustment procedures,” Journal of the American Statistical Association, 80, 541-52. Jazwinski, A. H. (1970) Stochastic Processes and Filtering Theory. New York: Academic Press. Johnston, F. R, and P. J. Harrison (1986). “The variance of lead time demand,” Journal of the Operational Research Society, 37, 303-8. Jones, R. H. (1981). “Fitting a continuous time autoregression to discrete data,” in D. F. Findley (ed.) Applied Time Series Analysis II. New York: Academic Press. Jones, R. H, (1984). “Fitting multivariate models to unequally spaced data,” in E. Parzen (ed) Time Series Analysis of Irregularly Obseroed Data, pp. 158-88. New York: Springer-Verlag. Kitagawa, G. (1981).““A nonstationary time series model and its fitting by a recursive filter,” Journal of Time Series Analysis, 2, 103-16. Kitagawa, G. (1984). “State space modelling of nonstationary time series and smoothing of ‘unequally spaced data,” in E. Parzen (ed,) Time Series Analysis of Irregularly Observed Data, PP. 189-210. New York: Springer-Verlag. Litterman, R. B. (1983). “A random walk Markov model for the distribution of time series,” Journal of Business and Economic Statistics, 1, 169-73. Phadke, M. S, and S. M. Wu (1974). “Modelling of continuous stochastic processes from discrete observations with application to sunspots data,” Journal of the American Statistical Association, 69, 325-9. Phillips, P. C. B. (1988). “Error correction and long run equilibrium in cont Cowles Foundation Discussion Paper 882, Yale University. Working, H. (1960). “Note on the correlations of first differences of a random chain.” Econometrica, 28, 916-18, Zadrozny, P. (1988). “Gaussian likelihood of continuous-time ARMAX models when data are stocks and flows at different frequencies,” Econometric Theory, 4, 108-24. juous time, 70