Non-Homogeneous Markov Chains and Systems: Theory and Applications fulfills two principal goals. It is devoted to the study of non-homogeneous Markov chains in the first part, and to the evolution of the theory and applications of non-homogeneous Markov systems (populations) in the second. The book is self-contained, requiring a moderate background in basic probability theory and linear algebra, common to most undergraduate programs in mathematics, statistics, and applied probability. There are some advanced parts, which need measure theory and other advanced mathematics, but the readers are alerted to these so they may focus on the basic results. Features A broad and accessible overview of non-homogeneous Markov chains and systems Fills a significant gap in the current literature A good balance of theory and applications, with advanced mathematical details separated from the main results Many illustrative examples of potential applications from a variety of fields Suitable for use as a course text for postgraduate students of applied probability, or for self-study Potential applications included could lead to other quantitative areas The book is primarily aimed at postgraduate students, researchers, and practitioners in applied probability and statistics, and the presentation has been planned and structured in a way to provide flexibility in topic selection so that the text can be adapted to meet the demands of different course outlines. The text could be used to teach a course to students studying applied probability at a postgraduate level or for self-study. It includes many illustrative examples of potential applications, in order to be useful to researchers from a variety of fields.

Table of Contents:
1. Foundations of Probability Theory. 1.1. Introductory Notes. 1.2. Some set theory and topology. 1.3. Important family of Sets in Probability theory. 1.4. Measurable spaces. 1.5. Probability spaces. 1.6. Filtration. 1.7. Random Variables. 1.8. Integration with respect to a probability measure. Expectation of a random variable. 1.9. Indicator functions. 1.10. The space L2 is a Hilbert space. 1.11. Independent σ-algebras and random variables. 1.12. Convergence of sequences of random variables. 1.13. The Laws of Large Numbers and the Central Limit Theorem. 1.14. Conditional Distributions and Conditional Expectations. 1.15. Change of measure. 1.16. Existence and Uniqueness of Conditional Expectations. 1.17. Properties of Conditional Expectation. 2. A Small Review of Matrix Analysis. 2.1. Introductory Notes. 2.2. Matrices. 2.3. The minimal polynomial of A. 2.4. The Norm of a vector. 2.5. The matrix norm. 2.6. The Kronecker product of two matrices. 2.7. The Hadamard product of matrices. 2.8. Canonical forms of a matrix. 2.9. Generalized Inverses. 2.10. The Moore-Penrose Generalized Matrix. 2.11. The Drazin Inverse and the Group Inverse. 2.12. The Group Inverse and Markov Chains. 2.13. Sensitivity of Markov Chains. 3. Non-Homogeneous Markov Chains, Weak Ergodicity. 3.1. Introductory notes. 3.2. Stochastic processes. 3.3. Markov chain. 3.4. The life and work of A.A. Markov. 3.5. Probability distribution in the states of a nonhomogeneous Markov chain. 3.6. Examples. 3.7. Weak and Strong Ergodicity. 3.8. Structures for coefficients of ergodicity. 3.9. Conditions for weak ergodicity for general products of stochastic matrices. 3.10. The dominant role of the Dobrushin ergodicity coefficient. 3.11. Transition probability matrices are in chronological order. 3.12. Examples on the use of weak ergodicity theorems. 4. Non-Homogeneous Markov Chains, Strong Ergodicity. 4.1. Strong Ergodicity. 4.2. Characterization of ergodicity and geometric strong ergodicity. 4.3. Criteria for Strong Ergodicity for Non-Homogeneous Markov Chains.4.4 Convergence in the Cesaro Sense. 4.5. Uniform strong ergodicity with the use of mean visit times. 4.6. Strong Ergodicity for General Products of Matrices. 4.7. Sets of Matrices all Non-homogeneous Products of Which Converge. 4.8. A Geometric approach to ergodic Non-Homogeneous Markov chains. 4.9. Asymptotic behavior for Markov Chains with arbitrary stochastic matrices. 5. The Non-Homogeneous Markov System. 5.1. Introductory Notes. 5.2. The Non-Homogeneous Markov System in discrete time and space. 5.3. The expected and the relative expected population structure. 5.4. A range of NHMS’s environment. 5.5. The б--Non-homogeneous Markov system. 5.6. Change of measure in G-non-homogeneous Markov systems. 5.7. The space of random population structures as a Hilbert space. 5.8. Estimation of the transition probabilities of a NHMS. 5.9. Research Notes. 6. Asymptotic Behaviour of a Non-Homogeneous Markov System. 6.1. Introductory notes. 6.2. Asymptotic behavior of the expected population structure. 6.3. Asymptotic behavior of the relative expected population structure. 6.4. Asymptotic behavior of a contracting NHMS without forced wastage. 6.5. NHMS with non-homogeneous Poisson recruitment. 6.6. Asymptotic stability in NHMS’s. 6.7. A G-non-homogeneous Markov system with non-homogeneous Poisson input, as a martingale. 6.8. Research Notes. 7. Asymptotic Variability of NHMS. 7.1. Introductory Notes. 7.2. The V Matrices and their Properties. 7.3. An illustrative probable application. 7.4. Rate of convergence of the variability vector. 7.5. Research notes. 8. Cyclic Behavior of Non-Homogeneous Markov Systems. 8.1. Introductory Notes. 8.2. Cyclic Non-homogeneous Markov systems. 8.3. An illustrative example from a manpower system. 8.4. Rate of convergence of {μ} t=0∞ in a NHMS under cyclic behavior. 8.5. Research notes. 9. Stochastic Control in NHMS’S. 9.1. Introductory notes. 9.2. Maintainability in NHMS by input control. 9.3. Attainability in NHMS by input control. 9.4. Asymptotically Attainable Structures. 9.5. Periodicity of Asymptotically Attainable structures. 9.6. An illustrative example. 9.7. Control of asymptotic variability in non-homogeneous Markov systems. 9.8. The non-homogeneous Markov system in a stochastic environment. 9.9. Maintainability in a stochastic environment. 9.10. Strategies for attaining a structure in a S-NHMS. 9.11. An illustrative application. 9.12. Research notes. 10. Laws of Large Numbers for Non-Homogeneous Markov Systems. 10.1. Introductory notes. 10.2. Basic concepts and useful results. 10.3. Laws of Large Numbers for a NHMS. 10.4. An illustrative application. 10.5. Laws of Large Numbers for a Cy-NHMS. 10.6. An illustrative application. 10.7. Laws of Large Numbers for NHMS with arbitrary transition probability matrices. 10.8. An illustrative example. 11. The Non-Homogeneous Markov System in a Stochastic Environment in Continuous Time. 11.1. Introductory notes. 11.2. Asymptotic behavior of non-homogeneous Markov process in continuous time. 11.3. The S-NHMS in continuous time. 11.4. The expected population structure of the S-NHMSC. 11.5. The asymptotic behavior of the S-NHMSC. 11.6. An illustrative example from manpower planning. 11.7. Research notes. 12. The Perturbed Non-Homogeneous Markov System. 12.1. Introductory notes. 12.2. The Group inverse and the asymptotic behavior of a Markov chain. 12.3. The Perturbed Non-Homogeneous Markov System. 12.4. The oscillation of the matrix Q in manpower systems. 12.5. Asymptotic behavior of the P-NHMS. 12.6. Sensitivity of the asymptotic relative expected population structure of a NHMS in changes in the limiting input probabilities. 12.7. Asymptotic variability of the P-NHMS. 12.8. Research Notes. 13. Non-Homogeneous Markov Set System. 13.1. Introduction. 13.2. Non-Homogeneous Markov Set System. 13.3. The set of the expected relative population structures of a NHMSS. 13.4. Asymptotic behavior of NHMSS. 13.5. Properties of the limiting set Rng∞ (S;R0). 13.6. An illustrative representative example. 14. Markov Systems on a General State Space. 14.1. Introductory notes. 14.2. A Mental image for MSGS. 14.3. The foundation of a MSGS. 14.4. Asymptotic behavior or ergodicity of MSGS. 14.5. Total variability from the invariant measure. A coupling theorem. 14.6. Rate of convergence of MSGS. 14.7. Asymptotic periodicity of a MSGS. 14.8. Total Variability from the Invariant Measures in the periodic case. 15. The G—Non-Homogeneous Markov System of High Order. 15.1. Introductory notes. 15.2. The G-non-homogeneous Markov system of high order. 15.3. The population structure of the G-inhomogeneous Markov System. 15.4. The Inhomogeneous Mixture Transition Distribution Model. 15.5. The Asymptotic Behavior of the Inherent Markov Chain. 15.6. The asymptotic expected population structure. 15.7. An illustrative example from manpower systems. 15.8. Research Notes. References.