Information Theory for Complex Systems: An Information Perspective on Complexity in Dynamical Systems and Statistical Mechanics (Understanding Complex Systems)

✍ Scribed by Kristian Lindgren

• Publisher: Springer
• Year: 2024
• Tongue: English
• Leaves: 161
• Category: Library

✦ Synopsis

This book introduces a comprehensive framework tailored for dissecting complex systems across diverse disciplines. What defines a complex system? How can we harness information to quantify its order, structure, and intricacy? Delving into phenomena from the intricate processes in physical systems to the dynamic behaviours in cellular automata and pattern formation, readers will uncover the profound interplay between physics and information theory. This intricate relationship provides fresh insight into physical phenomena, reimagining them through the lens of information. Notably, the book demystifies how seemingly opposing forces―rising order and increasing disorder―coexist, ultimately shedding light on the second law of thermodynamics as an outcome of deterministic, reversible dynamics beneath the surface. Geared towards graduate students, this book presumes an undergraduate foundation in mathematics and physics, ensuring a deep, engaging exploration for its readers.

✦ Table of Contents

Preface
Contents
1 Introduction
2 Information Theory
2.1 Basic Concepts
2.1.1 Information and Shannon Entropy
2.1.2 Relative Information, Relative Entropy, or Kullback-Leibler Divergence
2.2 Maximum Entropy Formalism
2.2.1 Example: The Bose-Einstein Distribution
2.3 Generalisation of Entropies to a Continuous State-Space
3 Information Theory for Lattice Systems
3.1 Symbol Sequences and Information
3.1.1 Probabilistic Description of a Symbol Sequence
3.1.2 Quantifying Disorder in a Symbol Sequence
3.1.3 Quantifying Order and Correlations in a Symbol Sequence
3.2 Markov Processes and Hidden Markov Models
3.2.1 Markov Processes and Entropy
3.2.2 Hidden Markov Models and Entropy
3.2.3 Some Examples
3.3 Measuring Complexity
3.3.1 Correlation Complexity for Markov Processes and Hidden Markov Models
3.4 Extensions to Higher Dimensions
Problems
4 Cellular Automata
4.1 Cellular Automata—a Class of Discrete Dynamical Systems
4.2 Elementary Cellular Automata
4.3 Information Theory for Cellular Automata
4.3.1 Deterministic Rules
4.3.2 Almost Reversible Rules
4.3.3 Rules with Noise
4.4 Examples of Information-Theoretic Properties in the Evolution of Simple CA
4.5 Analysis of CA Time Evolution Using Hidden Markov Models
4.6 Local Information Detecting Patterns in CA Time Evolution
5 Physics and Information Theory
5.1 Basic Thermodynamics
5.1.1 Intensive and Extensive Variables
5.2 Work and Information—An Extended Example
5.3 From Information Theory to Statistical Mechanics and Thermodynamics
5.3.1 Comparing Two Different Gibbs Distributions
5.3.2 Information and Free Energy in Non-equilibrium Concentrations
5.4 Microscopic and Macroscopic Entropy
5.5 Information Theory for Spin Systems
5.5.1 Example: The One-Dimensional Ising Model
5.6 The Approach Towards Equilibrium in a Reversible …
5.6.1 Q2R—A Microscopically Reversible Ising Dynamics
5.6.2 Analysis of the Time Evolution Starting from a Low Entropy State
5.6.3 Time Evolution of the Magnetisation
5.6.4 Time Evolution of Information-Theoretic Characteristics
6 Geometric Information Theory
6.1 Information Decomposition with Respect to Position and Resolution
6.1.1 Resolution-Dependent Probability Density
6.1.2 Examples on Resolution-Dependent Probability Densities
6.1.3 Connection Between Resolution and Diffusion
6.1.4 Decomposition of Information
6.2 Dimension, Information, and Fractals
6.2.1 Dimensions
6.2.2 Fractal Dimension
6.2.3 Dimension and Information
7 Pattern Formation in Chemical Systems
7.1 The Transition from Micro to Macro
7.2 Information Analysis of Chemical Pattern Formation
7.2.1 Chemical and Spatial Information
7.2.2 Decomposition of Spatial Information in a Chemical Pattern
7.2.3 Reaction-Diffusion Dynamics
7.2.4 Destruction of Information in Closed Chemical Systems
7.2.5 Flows of Information in Closed Chemical Systems
7.2.6 A Continuity Equation for Information in the Case of a Closed System
7.2.7 A Continuity Equation for Information in the Case of an Open System
7.3 Application to the Self-replicating Spots Dynamics
8 Chaos and Information
8.1 Basic Dynamical Systems Concepts
8.1.1 Iterated Maps, Fixed Points, and Periodic Orbits
8.1.2 Probability Densities and Measures on the State Space
8.1.3 Lyapunov Exponent
8.1.4 The Lyapunov Exponent as an Information Flow from Micro'' toMacro''
8.2 Dynamical Systems Entropy and Information Flow
8.2.1 Extended Example of a Generating Partition for a Skew Roof Map
8.2.2 An Example of a Partition that is not Generating
9 Appendix
Appendix References