Building on the author’s more than 35 years of teaching experience, Modeling and Analysis of Stochastic Systems, Third Edition, covers the most important classes of stochastic processes used in the modeling of diverse systems. For each class of stochastic process, the text includes its definition, characterization, applications, transient and limiting behavior, first passage times, and cost/reward models.
The third edition has been updated with several new applications, including the Google search algorithm in discrete time Markov chains, several examples from health care and finance in continuous time Markov chains, and square root staffing rule in Queuing models. More than 50 new exercises have been added to enhance its use as a course text or for self-study. The sequence of chapters and exercises has been maintained between editions, to enable those now teaching from the second edition to use the third edition.
Rather than offer special tricks that work in specific problems, this book provides thorough coverage of general tools that enable the solution and analysis of stochastic models. After mastering the material in the text, readers will be well-equipped to build and analyze useful stochastic models for real-life situations.
Table of Contents:
Introduction
What in the World is a Stochastic Process?
How to Characterize a Stochastic Process
What Do We Do with a Stochastic Process?
Discrete-Time Markov Chains: Transient Behaviour
Definition and Characterization
Examples
DTMCs in Other Fields
Marginal Distributions
Occupancy Times
Computation of Matrix Powers
Modeling Exercises
Computational Exercises
Conceptual Exercises
Discrete-Time Markov Chains: First Passage Times
Definitions
Cumulative Distribution Function of T
Absorption Probabilities
Expectation of T
Generating Function and Higher Moments of T
Computational Exercises
Conceptual Exercises
Discrete-Time Markox Chains: Limiting Behaviour
Exploring the Limiting Behaviour by Examples
Classification of States
Determining Recurrence and Transience: Finite DTMCs
Determining Recurrence and Transience: Infinite DTMSc
Limiting Behaviour of Irreducible DTMCs
Examples: Limiting Behaviour of Infinite State-Space Irreducible DTMCs
Limiting Behaviour of Reducible DTMCs
DTMCs with Costs and Rewards
Reversibility
Computational Exercises
Conceptual Exercises
Poisson Processes
Exponential Distributions
Poisson Process: Definitions
Event Times in a Poisson Process
Superposition and Splitting of Poisson Processes
Non-Homogeneous Poisson Process
Compound Poisson Process
Computational Exercises
Conceptual Exercises
Continuous-Time Markov Chains
Definitions and Sample Path Properties
Examples
CTMCs in Other Fields
Transient Behaviour: Marginal Distribution
Transient Behaviour: Occupancy Times
Computation of P(t): Finite State-Space
Computation of P(t): Infinite State-Space
First-Passage Times
Exploring the Limiting Behaviour by Examples
Classification of States
Limiting Behaviour of Irreducible CTMCs
Limiting Behaviour of Reducible CTMCs
CTMCs with Costs and Rewards
Phase Type Distributions
Reversibility
Modeling Exercises
Computational Exercises
Conceptual Exercises
Queueing Models
Introduction
Properties of General Queueing Systems
Birth and Death Queues
Open Queueing Networks
Closed Queueing Networks
Single Server Queues
Retrial Queue
Infinite Server Queue
Modeling Exercises
Computational Exercises
Renewal Processes
Introduction
Properties of N(t)
The Renewal Function
Renewal-Type Equation
Key Renewal Theorem
Recurrence Times
Delayed Renewal Processes
Semi-Markov Processes
Renewal Processes with Costs/Rewards
Regenerative Processes
Computational Exercises
Conceptual Exercises
Markov Regenerative Processes
Definitions and Examples
Markov Renewal Process and Markov Renewal Function
Key Renewal Theorem for MRPs
Semi-Markov Processes: Further Results
Markov Regenerative Processes
Applications to Queues
Modeling Exercises
Computational Exercises
Conceptual Exercises
Diffusion Process
Brownian Motion
Sample Path Properties of BM
Kolmogorov Equations for Standard Brownian Motion
First Passage Times
Reflected SBM
Reflected BM and Limiting Distributions
BM and Martingales
Cost/Reward Models
Stochastic Integration
Stochastic Differential Equations and Ito's Formula
Applications to Finance
Computational Exercises
Conceptual Exercises
Epilogue
Probability of Events
Appendices
Univariate Random Variables
Multivariate Random Variables
Generating Functions
Laplace-Stieltjes Transforms
Laplace Transforms
Modes of Convergence
Results from Analysis
Difference and Differential Equations
Answers to Selected Problems
References
Index
