An examination of mathematical formulations of ridge-regression-type estimators points to a curious observation: estimators can be derived by both Bayesian and Frequentist methods. In this updated and expanded edition of his 1990 treatise on the subject, Marvin H. J. Gruber presents, compares, and contrasts the development and properties of ridge-type estimators from these two philosophically different points of view. The book is organized into five sections. Part I gives a historical survey of the literature and summarizes basic ideas in matrix theory and statistical decision theory. Part II explores the mathematical relationships between estimators from both Bayesian and Frequentist points of view. Part III considers the efficiency of estimators with and without averaging over a prior distribution. Part IV applies the methods and results discussed in the previous two sections to the Kalman Filter, analysis of variance models, and penalized splines. Part V surveys recent developments in the field. These include efficiencies of ridge-type estimators for loss functions other than squared error loss functions and applications to information geometry. Gruber also includes an updated historical survey and bibliography. With more than 150 exercises, Regression Estimators is a valuable resource for graduate students and professional statisticians.

Table of Contents:
Preface Part I: Introduction and Mathematical Preliminaries 1. Introduction 1.1. The Purpose of This Book 1.2. Least Square Estimators and the Need for Alternatives 1.3. Historical Survey 1.4. The Structure of the Book 2. Mathematical and Statistical Preliminaries 2.0. Introduction 2.1. Matrix Theory Results 2.2. The Bayes Estimator (BE) 2.3. Admissible Estimators 2.4. The Minimax Estimator 2.5. Criterion for Comparing Estimators: Theobald's 1974 Result 2.6. Some Useful Inequalities: Some Miscellaneous Useful Matrix Results 2.7. Summary Part II: The Estimators, Their Derivations, and Their Relationships 3. The Estimators 3.0. The Least Square Estimator and Its Properties 3.1. The Generalized Ridge Regression Estimator 3.2. The Mixed Estimators 3.3. The Linear Minimax Estimator 3.4. The Bayes Estimator 3.6. Summary 4. How the Different Estimators Are Related 4.0. Introduction 4.1. Alternative Forms of the Bayes Estimator Full-Rank Case 4.2. Alternative Forms of the Bayes Estimator Non-Full-Rank Case Estimable Parametric Functions 4.3. Equivalence of the Generalized Ridge Estimator and the BayesEstimator 4.4. Equivalence of the Mixed Estimator and the Bayes Estimator 4.5. Ridge Estimators in the Literature as Special Cases of the BE, Minimax Estimators, or Mixed Estimators 4.6. An Extension of the Gauss-Markov Theorem 4.7. Generalities 4.8. Summary Part III: Comparing the Efficiency of the Estimators 5. Measures of Efficiency of the Estimators 5.0. Introduction 5.1. The Different Kinds of Mean Square Error 5.2. Zellner's Balanced Loss Function 5.3. The LINEX Loss Function 5.4. Linear Admissibility 5.5. Summary 6. The Average Mean Square Error 6.0. Introduction 6.1. The Forms of the MSE for the Minimax, Bayes, and Mixed Estimators 6.2. The Relationship between the Average Variance and the MSE 6.3. The Average MSE of the Bayes Estimator 6.4. Alternative Forms of the MSE of the Mixed Estimator 6.5. Comparison of the MSE of Different BEs 6.6. Comparison of the MSE of the Ridge and Contraction Estimators 6.7. Comparison of the Average MSE of the Two-Parameter Liu Estimator and the Ordinary Ridge Regression Estimator 6.8. Summary 7. The MSE Neglecting the Prior Assumptions 7.0. Introduction 7.1. The MSE of the BE 7.2. The MSE of the Mixed Estimators Neglecting PriorAssumptions 7.3. Comparison of the Conditional MSE of the Bayes and Least Square Estimators and Comparison of the Conditional and Average MSE 7.4. Comparison of the MSE of a Mixed Estimator with That of the LS Estimators 7.5. Comparison of the MSE of Two Bayes Estimators 7.6. Summary 8. The MSE for Incorrect Prior Assumptions 8.0. Introduction 8.1. The Bayes Estimator and Its MSE 8.2. The Minimax Estimator 8.3. The Mixed Estimator 8.4. Contaminated Priors 8.5. Contaminated (Mixed) Bayes Estimators 8.6. Summary Part IV: Applications 9. The Kalman Filter 9.0. Introduction 9.1. The Kalman Filter as a Bayes Estimator 9.2. The Kalman Filter as a Recursive Least Square Estimator,and the Connection with the Mixed Estimator 9.3. The Minimax Estimator 9.4. The Generalized Ridge Estimator 9.5. The Average Mean Square Error 9.6. The MSE for Incorrect Initial Prior Assumptions 9.7. Applications 9.8. Recursive Ridge Regression 9.9. Summary 10. Experimental Design Models 10.0. Introduction 10.1. The One-Way ANOVA Model 10.2. The Bayes and Empirical Bayes Estimators 10.3. The Two- Way Classification 10.4. The Bayes and Empirical Bayes Estimators 10.5. Summary Appendix to Section 10.2. Calculation of the MSE of Section 10.2 11. How Penalized Splines and Ridge- Type EstimatorsAre Related 11.0. Introduction 11.1. Splines as a Special Kind of Regression Model 11.2. Penalized Splines 11.3. The Best Linear Unbiased Predictor (BLUP) 11.4. Two Examples 11.5. Summary Part V: Alternative Measures of Efficiency 12. Estimation Using Zellner's Balanced Loss Function 12.0. Introduction 12.1. Zellner's Balanced Loss Function 12.2. The Estimators from Different Points of View 12.3. The Average Mean Square Error 12.4. The Risk without Averaging over a Prior Distribution 12.5. Some Optimal Ridge Estimators 12.6. Summary 13. The LINEX and Other Asymmetric Loss Functions 13.0. Introduction 13.1. The LINEX Loss Function 13.2. The Bayes Risk for a Regression Estimator 13.3. The Frequentist Risk 13.4. Summary 14. Distances between Ridge-Type Estimators, andInformation Geometry 14.0. Introduction 14.1. The Relevant Differential Geometry 14.2. The Distance between Two Linear Bayes Estimators, Based on the Prior Distributions 14.3. The Distance between Distributions of Ridge-Type Estimators from a Non-Bayesian Point of View 14.4. Distances between the Mixed Estimators 14.5. An Example Using the Kalman Filter 14.6. Summary References Author Index Subject Index