This text provides a comprehensive introduction to the mathematical theory of probability, its application to the modeling of random phenomena encountered in electrical and computer engineering, and its uses in making optimal decisions and inferences. Fine meets the needs of engineering students by addressing both highly conceptual mathematical methods and their real-world applications. He offers a sound introduction to the many elements of applied probability – the presentation is thorough, yet does not require a more advanced mathematical background beyond basic integral calculus.

Table of Contents:
Chapter 1: Background, Events, and Data Brief Historical Background to Probability. Important Examples of Random Phenomena. Modeling Random Phenomena I: Random Experiment and Sample Space. Modeling Random Phenomena II: Events and Combinations of Events. Modeling Random Phenomena III: Event Collections and Algebras. Dealing with Chance Data–Statistics. Appendix 1: Sets. Appendix 2: Functions. Appendix 3: Pseudorandom Numbers Generated by Matlab.   Chapter 2: Classical Probability Choosing at Random. Enumeration of Ordered Sequences. Enumeration of Sets: Binomial Coefficients. Application to Entropy and Data Compression. Application to Graphs. Application to Statistical Mechanics. Multinomial Counting. Conditional Classical Probability. Independence in Classical Probability.   Chapter 3: Probability Foundations Hierarchy of Probability Structural Concepts. Interpretations of Probability. Long-run Time Averages and Relative Frequencies. Modeling Random Phenomena IV: Kolmogorov Axioms. Elementary Consequences of the Axioms K0—K4. Boole’s Inequality. Inclusion-Exclusion Principle. Convex Combinations of Probability Measures. Application to Target Detection. Appendix: Proof of Equivalence Theorem.     Chapter 4: Describing Probability I: Countable Probability Mass Functions. Application to Modeling English Text Letter Frequencies. Commonly Encountered PMFs. Poisson as a Rare Events Limit of the Binomial.   Chapter 5: Describing Probability II: Uncountable and Distributions Cumulative Distribution Functions. Properties of Univariate CDFs. General Representation and Decomposition of Univariate CDFs. The Empirical CDF.   Chapter 6: Describing Probability III: Uncountable and Densities Probability Density Functions. Representing PMFs by PDFs. Closure Under Convex Combinations. PDFs over Finite Intervals. PDFs over Semi-infinite Intervals. PDFs over all of R. Appendix: Beta Density Unit Normalization.   Chapter 7: Multivariate Distribution Functions and Densities Multivariate or Joint CDFs. Multivariate PDF Representation of Multivariate CDF. Examples of Multivariate PDFs. Application to Reliability.   Chapter 8: Functions of Random Variables Random Variables. Single Input—Single Output (SISO) Functions. Simulation and the Probability Integral Transformation. Multiple Input-Multiple Output (MIMO): n = m. MIMO Applications. MIMO Transformation: m < n.   Chapter 9: Expectation and Moments The Concept of Expectation. Interpretation 1: Mean. Elementary Properties of Expectation. Interpretation 2: Definite Integral. Interpretation 3: Long-run Average Outcome. Interpretation 4: Fair Price. Expectation of a Function of a Random Variable. Convex Functions and Jensen’s Inequality. Moments, Especially Variance. Correlation, Covariance, and the Schwarz Inequality. Linear Dependence and Least Mean Square Estimation. Extensions of Expectation to Vector, Matrix, and Complex-Valued Variables. Correlation and Covariance for Real Random Vectors. Linear Transformation and Synthesis of (Gaussian) Random Vectors. Appendix: Summary of Basic Properties of Expectation. Appendix: Correlation for Complex Random Vectors.   Chapter 10: Linear Systems, Signals, and Filtering Background to Linear Systems. Random Signals and Autocorrelation Functions. Moments of Outputs of Linear Systems. Application to Wiener Filtering. Application to Kalman Filtering.   Chapter 11: Discrete Conditional Probability Conditional Probability. Properties of Discrete Conditional Probability. Multiplication/Sequence Theorem. Total Probability Theorem and Its Applications. Inverting Cause and Effect: Bayes’ Theorem and Its Applications. Three Problems that Challenge Our Intuition. Appendix: Discrete Conditional Probability Summary.   Chapter 12: Mixed Conditional Probability Conditional Probability: P(A|B) for P(B) = 0. Bivariate Conditional Density. Multivariate Conditional Densities. Applications of Conditional Densities. Useful Families of Conditional Densities. Extensions of the Total Probability Theorem. Applications of the Extended Total Probability Theorem. Extensions of the Bayes’ Theorem. Applications of the Extended Bayes’ Theorem. Appendix: Conditional Probability Summary I. Appendix: Conditional Probability Summary II.   Chapter 13: MAP, MLE, and Neyman-Pearson Rules Binary Decision-Making Setup. Minimum Error Probability Design: MAP Rule. Hypothesis Testing. Additive Measurement Noise.   Chapter 14: Independence Independent Pairs of Events. Mutually Independent Events. Independent Random Variables and Product Factorization. Derivations of Two Probability Models. Example of Maxima and Minima of Random Variables. Expectation and Independence. Application to Estimation of the Mean and Variance.   Chapter 15: Characteristic Functions  Characteristic Function Definition and Examples. Properties of the Characteristic Function. Relationship Between Characteristic Functions and Moments. Extension to Random Vectors. Linear Transformations and the Multivariate Normal Revisited. Characteristic Functions and Independence. Characteristic Functions of Sums. Central Limit Theorem. Generating Function.   Chapter  16: Probability Bounds and Sums Generalized Chebychev Bounds. Chernoff Bounds. Hoeffding Bounds.   Chapter 17: Conditional Expectation Conditional Expectation Basics. Example of the Multivariate Normal. MMSE Estimation. Two Applications of MMSE Estimation. Application to Kalman Filtering Revisited. Random Sums. Appendix: Conditioning on Event Algebras.   Chapter 18: Bayesian Inference Setup and Notation for Bayesian Decision-Making. Bayes Approach to Decision-Making. Calculating Bayes Rules. Particular Bayes Solutions. Appendix: An Informal Primer on Loss Functions.   Chapter 19: Limits and Laws of Large Numbers Jacob Bernoulli on the Laws of Large Numbers. LLN for Mean Square Convergence. Convergence of Sequences of Functions. Convergence of Sequences of Random Variables. Mutual Convergence. Laws of Large Numbers. An Example of Divergence of Time Averages. Appendix: Proof of Theorem 19.5.1 C1, C2.   Chapter 20: Random Processes Definition of a Random Process. Borel-Cantelli Lemma. Independent Random Variables. Stationary and Ergodic Processes. Markov Chains in Discrete Time. Independent Increment Random Processes. Martingales. Calculus. Wide Sense Stationary Processes and Power Spectral Density. Gaussian Random Process.   Bibliography. Index.