Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
117 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck (2311.04896v2)

Published 8 Nov 2023 in cs.LG, cs.IT, math.IT, and nlin.CD

Abstract: Deterministic chaos permits a precise notion of a "perfect measurement" as one that, when obtained repeatedly, captures all of the information created by the system's evolution with minimal redundancy. Finding an optimal measurement is challenging, and has generally required intimate knowledge of the dynamics in the few cases where it has been done. We establish an equivalence between a perfect measurement and a variant of the information bottleneck. As a consequence, we can employ machine learning to optimize measurement processes that efficiently extract information from trajectory data. We obtain approximately optimal measurements for multiple chaotic maps and lay the necessary groundwork for efficient information extraction from general time series.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. J.-P. Eckmann and D. Ruelle, Ergodic theory of chaos and strange attractors, Reviews of Modern Physics 57, 617 (1985).
  2. J. Slingo and T. Palmer, Uncertainty in weather and climate prediction, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 369, 4751 (2011).
  3. G. Nicolis and C. Nicolis, Foundations of complex systems: emergence, information and predicition (World Scientific, 2012).
  4. E. Bradley and H. Kantz, Nonlinear time-series analysis revisited, Chaos: An Interdisciplinary Journal of Nonlinear Science 25 (2015).
  5. D. J. Wales, Calculating the rate of loss of information from chaotic time series by forecasting, Nature 350, 485 (1991).
  6. P. Amil, M. C. Soriano, and C. Masoller, Machine learning algorithms for predicting the amplitude of chaotic laser pulses, Chaos: An Interdisciplinary Journal of Nonlinear Science 29 (2019).
  7. W. Gilpin, Model scale versus domain knowledge in statistical forecasting of chaotic systems, Physical Review Research 5, 043252 (2023).
  8. S. G. Williams, Introduction to symbolic dynamics, in Proceedings of Symposia in Applied Mathematics, Vol. 60 (2004) pp. 1–12.
  9. M.-P. Béal and D. Perrin, Symbolic dynamics and finite automata, in Handbook of Formal Languages: Volume 2. Linear Modeling: Background and Application (Springer, 2013) pp. 463–506.
  10. D. Lind and B. Marcus, An introduction to symbolic dynamics and coding (Cambridge University Press, 2021).
  11. Y. Hirata and J. M. Amigó, A review of symbolic dynamics and symbolic reconstruction of dynamical systems, Chaos: An Interdisciplinary Journal of Nonlinear Science 33 (2023).
  12. A. Ray, Symbolic dynamic analysis of complex systems for anomaly detection, Signal Processing 84, 1115 (2004).
  13. C. Beck and F. Schögl, Thermodynamics of chaotic systems, Cambridge Nonlinear Science Series (Cambridge University Press, 1993).
  14. T. Schürmann and P. Grassberger, Entropy estimation of symbol sequences, Chaos: An Interdisciplinary Journal of Nonlinear Science 6, 414 (1996).
  15. A. Cohen and I. Procaccia, Computing the Kolmogorov entropy from time signals of dissipative and conservative dynamical systems, Physical Review A 31, 1872 (1985).
  16. R. Badii and A. Politi, Complexity: Hierarchical Structures and Scaling in Physics, Cambridge Nonlinear Science Series (Cambridge University Press, 1997).
  17. M. B. Kennel and M. Buhl, Estimating good discrete partitions from observed data: Symbolic false nearest neighbors, Physical Review Letters 91, 084102 (2003).
  18. K. Ikeda, Multiple-valued stationary state and its instability of the transmitted light by a ring cavity system, Optics Communications 30, 257 (1979).
  19. P. Grassberger and H. Kantz, Generating partitions for the dissipative Hénon map, Physics Letters A 113, 235 (1985).
  20. L. Jaeger and H. Kantz, Structure of generating partitions for two-dimensional maps, Journal of Physics A: Mathematical and General 30, L567 (1997).
  21. J. Plumecoq and M. Lefranc, From template analysis to generating partitions II: Characterization of the symbolic encodings, Physica D: Nonlinear Phenomena 144, 259 (2000).
  22. K. A. Mitchell, Partitioning two-dimensional mixed phase spaces, Physica D: Nonlinear Phenomena 241, 1718 (2012).
  23. M. Chai and Y. Lan, Symbolic partition in chaotic maps, Chaos: An Interdisciplinary Journal of Nonlinear Science 31 (2021).
  24. C. Zhang, H. Li, and Y. Lan, Phase space partition with Koopman analysis, Chaos: An Interdisciplinary Journal of Nonlinear Science 32 (2022).
  25. Y. Hirata, K. Judd, and D. Kilminster, Estimating a generating partition from observed time series: Symbolic shadowing, Physical Review E 70, 016215 (2004).
  26. N. S. Patil and J. P. Cusumano, Empirical generating partitions of driven oscillators using optimized symbolic shadowing, Physical Review E 98, 032211 (2018).
  27. N. F. Ghalyan, D. J. Miller, and A. Ray, A locally optimal algorithm for estimating a generating partition from an observed time series and its application to anomaly detection, Neural Computation 30, 2500 (2018).
  28. N. Rubido, C. Grebogi, and M. S. Baptista, Entropy-based generating markov partitions for complex systems, Chaos: An Interdisciplinary Journal of Nonlinear Science 28 (2018).
  29. I. E. Aguerri and A. Zaidi, Distributed information bottleneck method for discrete and gaussian sources, in International Zurich Seminar on Information and Communication (IZS 2018). Proceedings (ETH Zurich, 2018) pp. 35–39.
  30. K. A. Murphy and D. S. Bassett, Interpretability with full complexity by constraining feature information, in The Eleventh International Conference on Learning Representations (2023).
  31. N. Tishby, F. C. Pereira, and W. Bialek, The information bottleneck method, arXiv preprint physics/0004057  (2000).
  32. K. A. Murphy and D. S. Bassett, Information decomposition to identify relevant variation in complex systems with machine learning, arXiv preprint arXiv:2307.04755  (2023b).
  33. C. E. Shannon, A mathematical theory of communication, The Bell System Technical Journal 27, 379 (1948).
  34. T. M. Cover and J. A. Thomas, Elements of information theory (John Wiley & Sons, 1999).
  35. A. v. d. Oord, Y. Li, and O. Vinyals, Representation learning with contrastive predictive coding, arXiv preprint arXiv:1807.03748  (2018).
  36. I. E. Aguerri and A. Zaidi, Distributed variational representation learning, IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 120 (2021).
  37. E. Jang, S. Gu, and B. Poole, Categorical reparameterization with Gumbel-softmax, in International Conference on Learning Representations (2017).
  38. M. B. Kennel and A. I. Mees, Context-tree modeling of observed symbolic dynamics, Physical Review E 66, 056209 (2002).
  39. S. Martiniani, P. M. Chaikin, and D. Levine, Quantifying hidden order out of equilibrium, Physical Review X 9, 011031 (2019).
  40. F. M. Willems, Y. M. Shtarkov, and T. J. Tjalkens, The context-tree weighting method: Basic properties, IEEE Transactions on Information Theory 41, 653 (1995).
  41. F. Creutzig, A. Globerson, and N. Tishby, Past-future information bottleneck in dynamical systems, Physical Review E 79, 041925 (2009).
  42. S. Still, J. P. Crutchfield, and C. J. Ellison, Optimal causal inference: Estimating stored information and approximating causal architecture, Chaos: An Interdisciplinary Journal of Nonlinear Science 20 (2010).
  43. S. E. Marzen and J. P. Crutchfield, Predictive rate-distortion for infinite-order Markov processes, Journal of Statistical Physics 163, 1312 (2016).
  44. S. Still, Information bottleneck approach to predictive inference, Entropy 16, 968 (2014).
  45. T. Schürmann, A note on entropy estimation, Neural Computation 27, 2097 (2015).
  46. D. P. Kingma and M. Welling, Auto-encoding variational Bayes, in International Conference on Learning Representations (ICLR) (2014).
  47. A. Kolchinsky, B. D. Tracey, and D. H. Wolpert, Nonlinear information bottleneck, Entropy 21, 1181 (2019).
  48. D. Maliniak, R. Powers, and B. F. Walter, The gender citation gap in international relations, International Organization 67, 889 (2013).
  49. N. Caplar, S. Tacchella, and S. Birrer, Quantitative evaluation of gender bias in astronomical publications from citation counts, Nature Astronomy 1, 1 (2017).
  50. M. L. Dion, J. L. Sumner, and S. M. Mitchell, Gendered citation patterns across political science and social science methodology fields, Political Analysis 26, 312 (2018).
  51. J. Dworkin, P. Zurn, and D. S. Bassett, (In)citing action to realize an equitable future, Neuron 106, 890 (2020b).
  52. Z. Budrikis, Growing citation gender gap, Nature Reviews Physics 2, 346 (2020).
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets