Working with the Mission Admission team is a great pleasure. The team is a fantastic group of smart, hardworking people who are passionate about education and equity. The team is full of people who are growing professionally within the education sector. In the future many people on this project will work with or as an evaluator, so I thought some ideas on working with evaluators might be helpful.
The role of evaluators is to help projects be successful. We do that by working to describe the project and its impact to the project team and outside audiences. There are three places where evaluators play that role. First, in the development of the project. Throughout the proposal and planning phases of a project, evaluators provide an outsiders perspective on whether project plans are sound and expectations are realistic. Second, throughout the implementation of a project, evaluators can provide information to support modifications to programs and collect a history of project evolution. Finally, evaluators, through impact analysis, provide objective evidence of the project’s impact.
This impact evaluation is becoming more important to garnering and growing funding from both the philanthropic and government sectors. The federal government is in the middle of an evolution toward “evidence-based policy making.” This movement will push project towards including evaluation designs that can create from a convincing evidence of success. In education, evaluation designs are being driven by the What Works Clearinghouse standards. While I will be the first to point out the limitations of the evidence-based policy movement, in the big picture it represents a positive affirmation of the value of data, rigor and research to policymaking.
The need for rigorous research designs is where this push towards evidence-based policy impacts program implementation. The key design challenge is being able to create comparison groups that are very similar to the people who participate in the program. Random assignment is the best, but not the only way to create this comparison group. Random assignment creates big challenges around the ethics of withholding services for students, needs for detailed data, and working a large group of partners that may receive delayed or no new services.
The advantage of evidence-based policy is it creates a way to identify and reward innovative interventions that impact kids’ academic performance. So while it clearly creates hurdles to program implementation this focus on evidence also creates a path for entrepreneurs to take their good ideas to scale.
As an evaluator, my job is to help create the evidence needed to identify effective programs and help communicate that evidence to both public and private funders. I hope I have made the case that evaluators can be helpful to programs, even if we also cause headaches during implementation. I also hope that I have made the case that the work of evaluators can be really fun: helping smart, ambitious people be successful at changing the world. So if you are one of those people whose career growth might include working in evaluation and you might have some questions, please touch base with me, I would love to tell you more about my great job.