Variational Bayes for Regime-Switching Log-Normal Models
Abstract
:1. Introduction
1.1. Variational Bayes
1.2. Regime-Switching Models
2. Variational Bayes and Informational Geometry
3. Applications of Variational Bayes
3.1. Geometric Foundation
3.2. Variational Bayes for the RSLN Model
Initialize
,
,
,
, and
at step 0 while, , , , , , , and do not converge do
t ⇐ t+1 end while |
3.3. Interpretation of Results
4. Numerical Studies
4.1. Simulated Data
4.2. Real Data
5. Conclusions
Author Contributions
Conflicts of Interest
References
- Zhao, H.; Marriott, P. Diagnostics for variational bayes approximations. 2013. arXiv:1309.5117. [Google Scholar]
- Amari, S.-I. Differential-Geometrical Methods in Statistics; Springer: New York, NY, USA, 1990. [Google Scholar]
- Eguchi, S. Second order efficiency of minimum contrast estimators in a curved exponential family. Ann. Stat 1983, 11, 793–803. [Google Scholar]
- Kass, R.; Vos, P. Geometrical Foundations of Asymptotic Inference; Wiley: New York, NY, USA, 1997. [Google Scholar]
- Hinton, G.E.; van Camp, D. Keeping neural networks simple by minimizing the description length of the weights. Proceedings of the 6th ACM Conference on Computational Learning Theory, Santa Cruz, CA, USA, 26–28 July 1993; ACM: New York, NY, USA, 1993. [Google Scholar]
- MacKay, D. Developments in Probabilistic Modelling with Neural Networks—Ensemble Learning. In Neural Networks: Artifical Intelligence and Industrial Applications; Springer: London, UK, 1995; pp. 191–198. [Google Scholar]
- Attias, H. Independent Factor Analysis. Neur. Comput 1999, 11, 803–851. [Google Scholar]
- Lappalainen, H. Ensemble Learning For Independent Component Analysis. Proceedings of the First International Workshop on Independent Component Analysis, Aussois, France, 11–15 January 1999; pp. 7–12.
- Beal, M.; Ghahramani, Z. The variational Bayesian EM algorithm for incomplete data: With application to scoring graphical model structures. Bayesian Stat 2003, 7, 453–463. [Google Scholar]
- Winn, J. Variational Message Passing and its Applications., Department of Physics, University of Cambridge, Cambridge, UK, 2003.
- Blei, D.M.; Ng, A.Y.; Jordan, M.I.; Lafferty, J. Latent Dirichlet allocation. J. Mach. Learn. Res 2003, 3, 993–1022. [Google Scholar]
- Ghahramani, Z.; Beal, M.J. A Variational Inference for Bayesian Mixtures of Factor Analysers. Adv. Neur. Inf. Process. Syst 2000, 12, 449–455. [Google Scholar]
- Haff, L.R. The Variational Form of Certain Bayes Estimators. Ann. Stat 1991, 19, 1163–1190. [Google Scholar]
- Faes, C.; Ormerod, J.T.; Wand, M.P. Variational Bayesian Inference for Parametric and Nonparametric Regression With Missing Data. J. Am. Stat. Assoc 2011, 106, 959–971. [Google Scholar]
- McGrory, C.; Titterington, D.; Reeves, R.; Pettitt, A.N. Variational Bayes for estimating the parameters of a hidden Potts model. Stat. Comput 2009, 19, 329–340. [Google Scholar]
- Ormerod, J.T.; Wand, M.P. Gaussian Variational Approximate Inference for Generalized Linear Mixed Models. J. Comput. Graph. Stat 2011, 21, 1–16. [Google Scholar]
- Hall, P.; Humphreys, K.; Titterington, D.M. On the Adequacy of Variational Lower Bound Functions for Likelihood-Based Inference in Markovian Models with Missing Values. J. R. Stat. Soc. Ser. B 2002, 64, 549–564. [Google Scholar]
- Wang, B.; Titterington, M. Convergence Properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model. Bayesian Anal 2006, 1, 625–650. [Google Scholar]
- Hardy, M.R. A Regime-Switching Model of Long-Term Stock Returns. N. Am. Actuar. J. 2001, 5, 41–53. [Google Scholar]
- Hamilton, J.D. A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle. Econometrica 1989, 57, 357–384. [Google Scholar]
- Hardy, M.R. Bayesian Risk Management for Equity-Linked Insurance. Scand. Actuar. J. 2002, 2002, 185–211. [Google Scholar]
- Schwarz, G. Estimating the dimension of a model. Ann. Stat 1978, 6, 461–464. [Google Scholar]
- Akaike, H. A new look at the statistical model identification. IEEE Trans. Autom. Control 1974, 19, 716–723. [Google Scholar]
- Hartman, B.M.; Heaton, M.J. Accounting for regime and parameter uncertainty in regime-switching models. Insur. Math. Econ 2011, 49, 429–437. [Google Scholar]
- Green, P.J. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 1995, 82, 711–732. [Google Scholar]
- Watanabe, S. Algebraic Geometry and Statistical Learning Theory; Cambridge University Press: Cambridge, UK, 2009. [Google Scholar]
- Brooks, S.P. Markov Chain Monte Carlo Method and Its Application. J. R. Stat. Soc. Ser. D 1998, 47, 69–100. [Google Scholar]
- Ghahramani, Z.; Hinton, G.E. Variational learning for switching state-space models. Neur. Comput 1998, 12, 831–864. [Google Scholar]
- Kullback, S.; Leibler, R.A. On information and sufficiency. Ann. Math. Stat 1951, 22, 79–86. [Google Scholar]
- Baum, L.E.; Petrie, T.; Soules, G.; Weiss, N. A maximization technique occurring in the statistical analysis of probabilistic functions of markov chains. Ann. Math. Stat 1970, 41, 164–171. [Google Scholar]
- Rue, H.; Martino, S.; Chopin, N. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. J. R. Stat. Soc. Ser. B 2009, 71, 319–392. [Google Scholar]
- Bishop, C.M. Pattern Recognition and Machine Learning; Springer: New York, NY, USA, 2006. [Google Scholar]
Case | Regime 1 (μi, σi) | Regime 2 (μi,σi) | Regime 3 (μi, σi) | Transition Probability |
---|---|---|---|---|
1 | (0.012, 0.035) | (−0.016, 0.078) | - | |
2 | (0.014, 0.050) | - | - | - |
3 | (0.000, 0.035) | (0.000, 0.078) | - | |
4 | (0.012, 0.035) | (−0.016, 0.078) | (0.04, 0.01) |
Case 1 | Case 2 | Case 3 | Case 4 | |
---|---|---|---|---|
Iterations to converge | 62 | 182 | 132 | 94 |
Computational time [s] | 27.161 | 80.842 | 58.510 | 45.044 |
Case | No. of Regimes | MLE BIC (Log Likelihood) | RJMCMC Posterior Probability | VB Magnitude Relative Matrix |
---|---|---|---|---|
1 | 1 | 1, 108.875(1, 115.384) | 0.647 | |
2 | 1, 158.227(1, 174.499) | 0.214 | ||
3 | 1, 156.370(1, 182.405) | 0.088 | ||
4 | 1, 153.150(1, 188.948) | <0.052 | ||
2 | 1 | 1, 045.448(1, 051.957) | 0.864 | |
2 | 1, 038.360(1, 054.632) | 0.109 | ||
3 | 1, 030.733(1, 056.768) | 0.020 | ||
4 | 1, 026.882(1, 062.680) | <0.006 | ||
3 | 1 | 1, 110.903(1, 117.411) | 0.629 | |
2 | 1, 139.214(1, 155.486) | 0.221 | ||
3 | 1, 131.904(1, 157.719) | 0.098 | ||
4 | 1, 121.921(1, 157.940) | <0.052 | ||
4 | 1 | 1, 044.819(1, 051.328) | 0.641 | |
2 | 1, 092.610(1, 108.881) | 0.203 | ||
3 | 1, 087.435(1, 113.470) | 0.094 | ||
4 | 1, 080.240(1, 116.038) | <0.06 |
January 1956–December 1999 | |
---|---|
R. M. M. |
Parameter | Distribution | Mean | s.d. | Transition Probability |
---|---|---|---|---|
μ1 | T454.61(0.0123, 370778.19) | 0.0123 | 0.00165 | - |
IG(227.30, 0.28) | 0.00122(0.0349) | 0.00008 | - | |
μ2 | t80.39(−0.0161, 12987.55) | −0.0161 | 0.00889 | - |
IG(40.20, 0.24) | 0.00603(0.0777) | 0.00098 | - | |
p1,2 | Beta(15.21, 434.78) | 0.0338 | 0.00851 | |
p2,1 | Beta(15.00, 61.21) | 0.1969 | 0.04525 |
μ1 | σ1 | p1,2 | μ2 | σ2 | p2,1 | |
---|---|---|---|---|---|---|
VB | 0.0123(0.00165) | 0.0349(0.00008) | 0.0338(0.00851) | −0.0161(0.00889) | 0.0777(0.00098) | 0.1969(0.04525) |
MLE | 0.0123(0.002) | 0.0347(0.001) | 0.0371(0.012) | −0.0157(0.010) | 0.0778(0.009) | 0.2101(0.086) |
MCMC | 0.0122(0.002) | 0.0351(0.002) | 0.0334(0.012) | −0.0164(0.010) | 0.0804(0.009) | 0.2058(0.065) |
© 2014 by the authors; licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).
Share and Cite
Zhao, H.; Marriott, P. Variational Bayes for Regime-Switching Log-Normal Models. Entropy 2014, 16, 3832-3847. https://doi.org/10.3390/e16073832
Zhao H, Marriott P. Variational Bayes for Regime-Switching Log-Normal Models. Entropy. 2014; 16(7):3832-3847. https://doi.org/10.3390/e16073832
Chicago/Turabian StyleZhao, Hui, and Paul Marriott. 2014. "Variational Bayes for Regime-Switching Log-Normal Models" Entropy 16, no. 7: 3832-3847. https://doi.org/10.3390/e16073832
APA StyleZhao, H., & Marriott, P. (2014). Variational Bayes for Regime-Switching Log-Normal Models. Entropy, 16(7), 3832-3847. https://doi.org/10.3390/e16073832