1. **Problem Statement:** Learn the key concepts of Continuous-Time Markov Chains (CTMC) from Module 4, Chapter 5, covering lessons 14 to 20. 2. **Markovian Property and Homogeneous Transition Probabilities (Lesson 14):** - The Markovian property means the future state depends only on the current state, not on the past. - Homogeneous transition probabilities imply transition rates do not change over time. 3. **Holding Times and Exponential Distribution (Lesson 15):** - Holding (sojourn) times in each state are exponentially distributed. - Memoryless property: the probability of leaving a state is independent of how long it has been there. - Two scenarios: tracking elapsed time vs forgetting it. 4. **Probability Models for CTMC (Lesson 16):** - States represent possible system conditions. - Holding times are exponential with parameters related to transition rates. - Transition rates form the generator matrix $Q$, where $Q_{ij}$ is the rate from state $i$ to $j$. 5. **Poisson Process as CTMC and Pure Birth Process (Lesson 17):** - Poisson process is a CTMC with transitions only to the next state. - Pure birth process (Yule process) models exponential growth with birth rates only. 6. **Birth-Death Processes (Lesson 18):** - States change by births ($\lambda$) and deaths ($\mu$). - Balance equations relate steady-state probabilities $\pi_i$: $$\lambda_{i-1} \pi_{i-1} + \mu_{i+1} \pi_{i+1} = (\lambda_i + \mu_i) \pi_i$$ - Steady-state probabilities solve these equations. 7. **Kolmogorov Forward and Backward Equations (Lesson 19):** - Forward equations describe how probabilities evolve over time. - Backward equations relate to initial conditions. - Limiting behavior studies steady-state distributions as time approaches infinity. 8. **Applications (Lesson 20):** - Queueing models like M/M/1 and M/M/c use CTMC to analyze waiting lines. - Population models use birth-death processes to study growth and decline. This overview provides foundational understanding of CTMC concepts, their mathematical formulation, and applications.