#### Consortium for Mathematics and its Applications

Product ID: 99107 & 99111
Supplementary Print

# Markov Chains and Applications of Matrix Methods: Fixed Point and Absorbing Markov Chains (UMAP)

### Author: Sister Mary K. Keller

The Markov process pulls together related prior probabilities into chains of events. The description of the process in matrix form allows one to make long-range predictions. Students define a Markov chain, interpret powers of matrices representing Markov chains, recognize certain processes as Markov chains, formulate a matrix of transition probabilties from a tree diagram of a Markov chain.

1. MARKOV CHAINS
1.1 Introduction
1.2 Tree Diagrams
1.3 Calculating Probabilities From a Tree Diagram
1.4 The Matrix Representation of a Markov Chain
1.5 Experiment 1
1.6 Experiment 2
1.7 Model Exam (Unit 107)

2. APPLICATIONS OF MATRIX METHODS: FIXED POINT AND ABSORBING MARKOV CHAINS
2.1 Challenge Problem
2.2 Regular Transition Matrices
2.3 Fixed-Probability Vectors
2.4 Calculating a Fixed-Probability Vector
2.5 Experiment 1
2.6 A Fixed Probability Vector from a System of Linear Functions
2.7 Experiment 2
2.8 Experiment 3
2.9 Absorbing Markov Chains
2.10 A Second Challenge Problem
2.11 Standard Form for an Absorbing Markov Chain
2.12 Partitioning the Standard Form
2.13 Making Decision Based on Probability
2.14 The Probability of Reaching a Given Absorbing State
2.15 Experiment 4
2.16 Model Exam (Unit 111)

3. ANSWERS TO EXERCISES (UNIT 107)

4. ANSWERS TO MODEL EXAM (UNIT 107)

5. ANSWERS TO MODEL EXAM (UNIT 111)

APPENDIX A

UMAP Module
32 pages

#### Mathematics Topics:

Abstract & Linear Algebra , Operations Research

Various

#### Prerequisites:

Elementary notion of probability; general concept of matrices and operations