The concepts of cause and effect are critical to the field of program evaluation. Experimentally-designed evaluations-those that randomize to treatment and control groups-offer a convincing means for establishing a causal connection between a program and its effects. This book considers a range of impact evaluation questions, particularly those questions that focus on the impact of specific aspects of a program. Laura R. Peck shows how a variety of experimental evaluation design options can provide answers to these questions, and she suggests opportunities for experiments to be applied in more varied settings and focused on program improvement efforts.
Laura R. Peck, Ph.D., is a principal scientist at Abt Associates and has spent her career evaluating social welfare and employment policies and programs, both in research and academic settings. A policy analyst by training, Dr. Peck specializes in innovative ways to estimate program impacts in experimental and quasi-experimental evaluations, and she applies this to many social safety net programs. Dr. Peck is currently the principal investigator, co-PI, or director of analysis for several major national evaluations for the U.S. Departments of Health and Human Services, Labor, and Housing and Urban Development and over her career has been part of more than 35 informative evaluations of nonprofit, local, state, and federal programs and policies. Peck is a co-author of a public policy textbook and is well published on program evaluation topics. Prior to her work at Abt Associates, Dr. Peck was a tenured professor at the Arizona State University School of Public Affairs and also served as the founding associate dean of the Barrett Honors College, Downtown Phoenix campus. She earned her PhD from the Wagner Graduate School at New York University.
List of Boxes, Figures, and Tables Volume Editors' Introduction About the Author Acknowledgments Chapter 1 * Introduction The State of the Field The Ethics of Experimentation What This Book Covers Questions and Exercises Resources for Additional Learning Chapter 2 * Conceptual Framework: From Program Logic Model to Evaluation Logic Model Program Logic Model Evaluation Logic Model Conclusion Questions and Exercises Resources for Additional Learning Chapter 3 * The Basic Experimental Design Defined Random Assignment Explained The Basic (Two-Armed) Experimental Design To Have a Control Group or Not to Have A Control Group? Questions and Exercises Resources for Additional Learning Chapter 4 * Variants of the Experimental Design Multi-Armed Designs Factorial Designs Multistage Designs Staggered Introduction Designs Blended Designs Aligning Evaluation Design Options With Program Characteristics and Research Questions Conclusion Questions and Exercises Chapter 5 * Practical Considerations and Conclusion Some Practical Considerations Road Testing Principles for Conducting High-Quality Evaluation Questions and Exercises Resources for Additional Learning Appendix * Doing the Math and Other Technical Considerations Estimating Treatment Impacts How to Interpret Results Handling Treatment Group No-Shows and Control Group Crossovers Subgroup Analyses Conclusion Questions and Exercises Resources for Additional Learning References Glossary Index
Peck reminds us that the results from experiments only inform us about average effects, but more importantly provides us with the information necessary to look inside the "black box." -- Roger Boothroyd * Post-revision review * Experimental evaluations are feasible under the right conditions. This book is an excellent guide for evaluators that want to apply this underutilized design in their practice. -- Sebastian Galindo * Post-revision review * A sophisticated and well-written treatise of evaluation design to improve policies and programs. -- Katrin Anacker * pre-publication review *
