The most up-to-date version of the gold standard reference in conducting evaluations
The newly revised fourth edition of The ABCs of Evaluation is an easy-to-read explanation of the concepts and methods of the evaluation process. The author explains what the process offers to people who go through it, how to conduct it properly, and how to draw on its conclusions to improve your workplace. The book also walks you through every step of the evaluation process, answering not just the theoretical question of “Why should we evaluate?” but demonstrating the nuts and bolts of an effective and illuminating evaluation.
Author John Boulmetis presents evidence-based and useful strategies for conducting evaluations, transforming them from something seen by your people as something to “get around” or “get through”—or as something being done “to” them as opposed to “for” them—into an invaluable data gathering and processing tool for organizations of all types and sizes.
Inside the book:
- A comprehensive guide to evaluation for program and project managers who are new to the discipline
- Time-tested and proven evaluation principles and techniques you can apply immediately within your own organization
- How to collect data, set standards, implement an evaluation plan, select an evaluation model, and more
Perfect for managers, supervisors, team facilitators, and data analysts, The ABCs of Evaluation is a step-by-step playbook for conducting useful and effective evaluations in any industry and within organizations of any size.
Table of Contents:
List of Figures ix
List of Tables and Exhibits xi
Preface xiii
The Authors xxi
About the Companion Website xxiii
1 What Is Evaluation? 1
Common Denominators 3
Two Definitions of Evaluation 4
Evaluating Efficiency, Effectiveness, and Impact 5
Evaluating Alternatives 8
Identifying Areas to Improve 9
Other Levels 11
Formal Reasons to Evaluate 12
Practical Application 15
Formative and Summative Evaluations 16
Putting It All Together 16
Key Words and Concepts 22
Questions and Exercises 22
Further Reading 28
2 Why Evaluate? 29
Grants and Grantsmanship 31
Benefits 33
Limitations 36
Considerations 37
Putting It All Together 40
Key Words and Concepts 41
Questions and Exercise 41
Further Reading 46
3 Decision- Making: Whom to Involve, How, and Why? 47
Program and Project Cycles 49
Evaluation and Monitoring 52
Program Planning Cycle and Evaluation 59
Putting It All Together 63
Key Words and Concepts 65
Questions and Exercises 65
Further Reading 73
4 Starting Point: The Evaluator’s Program Description 75
What an Evaluator’s Program Description Can Do 77
Developing an Evaluator’s Program Description 79
Putting It All Together 83
Key Words and Concepts 85
Questions and Exercises 85
Further Reading 97
5 Choosing an Evaluation Model 99
More Than One Way to Evaluate 100
Four Evaluation Models 103
Other Models 110
Choosing the Right Model 112
Evaluation Design Format 113
Other Considerations 115
Putting It All Together 116
Key Words and Concepts 117
Questions and Exercise 117
Further Reading 119
6 Data Sources 121
Starting Point 122
Qualitative and Quantitative Data Sources 126
Four Levels of Data 127
Nature of Data Collection 129
Existing Data 130
Newly Collected Data 132
Putting It All Together 143
Key Words and Concepts 143
Questions and Exercise 143
Further Reading 146
7 Data Analysis 147
Data Analysis 148
Data 150
Terms 151
Levels of Data 157
Putting It All Together 163
Key Words and Concepts 164
Questions and Exercise 164
Further Reading 166
Software for Personal Computers 167
8 Is It Evaluation or Is It Research? 169
Determining Evaluation or Research from Intent 170
Determining Evaluation or Research from Design 172
Qualitative and Quantitative Characteristics 174
Sampling 175
Sampling and Surveys 180
Putting It All Together 181
Key Words and Concepts 183
Questions and Exercise 183
Further Reading 184
9 Writing the Evaluation Report 185
Communicating Your Findings 187
Evaluation Reports 188
The Report Outline 189
Putting It All Together 201
Key Words and Concepts 202
Questions and Exercise 202
Further Reading 203
10 Evaluation as a Business 205
Is There a Market for Program Evaluation? 206
How Do You Find the Business? 213
Proactive Resources 215
Reactive Resources 217
Emergency Resources 218
How Do You Negotiate a Contract? 219
Keeping the Business Alive 223
Putting It All Together 225
Key Words and Concepts 226
Further Reading 227
A: More on Data Analysis 229
Analyses for Each Level 229
Significance Testing 233
Statistical and Practical Significance 237
B: Sample Evaluation Report 239
C: Sample Evaluation Contract 271
Evaluation of the XYZ Training Project 271
D: Sample Evaluator Job Description 277
Evaluator— MA, MEd, Education Technology and Instructional Design 277
Disclaimer 280
References 281
Index 285
