Statistical Methods and the Improvement of Data Quality contains the proceedings of The Small Conference on the Improvement of the Quality of Data Collected by Data Collection Systems, held on November 11-12, 1982, in Oak Ridge, Tennessee. The conference provided a forum for discussing the use of statistical methods to improve data quality, with emphasis on the problems of data collection systems and how to handle them using state-of-the-art techniques. Comprised of 16 chapters, this volume begins with an overview of some of the limitations of surveys, followed by an annotated bibliography on frames from which the probability sample is selected. The reader is then introduced to sample designs and methods for collecting data over space and time; response effects to behavior and attitude questions; and how to develop and use error profiles. Subsequent chapters focus on principles and methods for handling outliers in data sets; influence functions, outlier detection, and data editing; and application of pattern recognition techniques to data analysis. The use of exploratory data analysis as an aid in modeling and statistical forecasting is also described. This monograph is likely to be of primary benefit to students taking a general course in survey sampling techniques, and to individuals and groups who deal with large data collection systems and are constantly seeking ways to improve the overall quality of their data.

Table of Contents:
Contributors Preface Acknowledgments Errors and Other Limitations of Surveys 1. Introduction 2. The Data Collection System Considered: A Survey 3. An Inherent Limitation 4. Errors in Surveys 5. The Formal Basis for Comprehensive Quality Control 6. Quality Control of the Survey Design 7. Quality Control of the Survey Operations References A Frame on Frames: An Annotated Bibliography 1. Introduction and Definition of a Frame 2. Construction of the Sampling Frame 3. Area Frames 4. Imperfect Frames 5. Sampling from Imperfect List Frames 6. Multiple Frame Surveys The Annotated Bibliography Data Collection for Details over Space and Time 1. Introduction 2. Samples and Censuses 3. Samples Connected with Censuses 4. Data from Registers 5. A Brief Historical Overview of Censuses 6. Postcensal Estimates for Domains 7. Design and Estimation for Domains 8. Purposes and Designs for Periodic Samples 9. Rolling Samples 10. Panels with Rolling Samples 11. Summary References Response Effects to Behavior and Attitude Questions 1. Introduction 2. Non-Threatening Behavior Questions 3. Reducing Memory Error 4. The Length of Questions 5. Threatening Behavioral Questions 6. The Magnitude of Errors in Threatening Questions 7. Determining the Perceived Level of Threat 8. Methods for Improving the Quality of Reporting of Threatening Questions 9. The Use of Familiar Words 10. Deliberately Loading the Question 11. Time Frame for Socially Undesirable and Desirable Behavior 12. Non-Interview Methods 13. Attitudinal Questions 14. Summary References Error Profiles: Uses and Abuses 1. Introduction 2. How to Develop Error Profiles 3. How to Use Error Profiles References Principles and Methods for Handling Outliers in Data Sets 1. Introduction 2. Some Basic Considerations for Univariate Samples 3. Multivariate Outliers 4. Outliers in Linear Models 5. Implications of the Data Collecting Mechanism for the Processing of Outliers References Influence Functions, Outlier Detection, and Data Editing 1. Introduction 2. Multivariate Outliers 3. Influence Functions 4. Form 4 Application 5. Inventory Difference Data Application 6. The Use of Influence Functions for Imputation References Using Exploratory Data Analysis to Monitor Socio-Economic Data Quality in Developing Countries 1. Introduction 2. The Problem 3. Exploratory Data Analysis 4. Conclusion References Application of Pattern Recognition Techniques to Data Analysis 1. Introduction 2. Heuristic Approach 3. Decision-Theoretic Approach 4. Syntactic Approach 5. Summary References Can Automatic Data Editing be Justified? One Person's Opinion 1. Introduction 2. The General Editing Procedure 3. Automatic Data Editing 4. Justifiability of Automatic Data Editing 5. Directions for Further Research References Missing Data in Large Data Sets 1. Introduction 2. Common Incomplete Data Problems 3. Methods for Handling Incomplete Data 4. General Data Patterns: The EM Algorithm 5. Multiple Imputation 6. Conclusion References Reducing the Cost of Studying Survey Measurement Error: Is a Laboratory Approach the Answer? 1. Introduction 2. Measurement Error Models 3. The Pearson Laboratory Experiment 4. Considerations for the Laboratory Experiment Approach 5. Summary References The Implication of Sample Design on Survey Data Analysis 1. Introduction 2. Types of Data Collection Systems 3. Sample Design Effects on Finite Population Inference 4. Examples of Survey Data Analysis and Effects of Design References An Approach to an Evaluation of the Quality of Motor Gasoline Prices 1. Introduction 2. Background 3. Internal Assessment 4. External Comparisons 5. Summary of Findings 6. General Conclusions Relative to the Quality of Data Collection Systems References Health and Mortality Study Error Detection, Reporting, and Resolution System 1. Overview 2. Types of Errors Reported 3. System Description 4. Summary References On Using Exploratory Data Analysis as an Aid in Modeling and Statistical Forecasting 1. Introduction 2. Overview 3. The Wharton Assessment of STIFS 4. The GLOBUS Project 5. Concluding Remarks References Index