Hands-On Markov Models with Python

✍ Scribed by Ankur Ankan

• Year: 2018
• Tongue: English
• Leaves: 172
• Category: Library

✦ Table of Contents

Cover
Title Page
Copyright and Credits
Packt Upsell
Contributors
Table of Contents
Preface
Chapter 1: Introduction to the Markov Process
Random variables
Random processes
Markov processes
Installing Python and packages
Installation on Windows
Installation on Linux
Markov chains or discrete-time Markov processes
Parameterization of Markov chains
Properties of Markov chains
Reducibility
Periodicity
Transience and recurrence
Mean recurrence time
Expected number of visits
Absorbing states
Ergodicity
Steady-state analysis and limiting distributions
Continuous-time Markov chains
Exponential distributions
Poisson process
Continuous-time Markov chain example
Continuous-time Markov chain
Summary
Chapter 2: Hidden Markov Models
Markov models
State space models
The HMM
Parameterization of HMM
Generating an observation sequence
Installing Python packages
Evaluation of an HMM
Extensions of HMM
Factorial HMMs
Tree-structured HMM
Summary
Chapter 3: State Inference - Predicting the States
State inference in HMM
Dynamic programming
Forward algorithm
Computing the conditional distribution of the hidden state given the observations
Backward algorithm
Forward-backward algorithm (smoothing)
The Viterbi algorithm
Summary
Chapter 4: Parameter Learning Using Maximum Likelihood
Maximum likelihood learning
MLE in a coin toss
MLE for normal distributions
MLE for HMMs
Supervised learning
Code
Unsupervised learning
Viterbi learning algorithm
The Baum-Welch algorithm (expectation maximization)
Code
Summary
Chapter 5: Parameter Inference Using the Bayesian Approach
Bayesian learning
Selecting the priors
Intractability
Bayesian learning in HMM
Approximating required integrals
Sampling methods
Laplace approximations
Stolke and Omohundro's method
Variational methods
Code
Summary
Chapter 6: Time Series Predicting
Stock price prediction using HMM
Collecting stock price data
Features for stock price prediction
Predicting price using HMM
Summary
Chapter 7: Natural Language Processing
Part-of-speech tagging
Code
Getting data
Exploring the data
Finding the most frequent tag
Evaluating model accuracy
An HMM-based tagger
Speech recognition
Python packages for speech recognition
Basics of SpeechRecognition
Speech recognition from audio files
Speech recognition using the microphone
Summary
Chapter 8: 2D HMM for Image Processing
Recap of 1D HMM
2D HMMs
Algorithm
Assumptions for the 2D HMM model
Parameter estimation using EM
Summary
Chapter 9: Markov Decision Process
Reinforcement learning
Reward hypothesis
State of the environment and the agent
Components of an agent
The Markov reward process
Bellman equation
MDP
Code example
Summary
Other Books You May Enjoy
Index