What is evaluation and why do we need it?
Have you ever wondered why evaluation is needed?
In this post we will consider what evaluation is, how it differs from research, and what evaluation can help you with.
What is evaluation?
Evaluation is the process of determining the merit, worth, or value of something.(1) The “something” may be a program, project, initiative, program activity, intervention or service that is being evaluated. This definition has wide support, and has been adopted by the American Evaluation Association.(2) But evaluation is broad and diverse, and not all evaluators define it the same way. The table below describes six different types of evaluation definitions, each with different emphases.(3)
What does this mean for practice?
It may well be that no single definition can fully capture the scope of evaluation practice. In our experience of evaluating programs and other initiatives, how we define evaluation varies between different evaluations, to reflect the requirements, scope, and objectives of the organization requiring the evaluation.
What distinguishes evaluation from research?
Our clients sometimes ask about the difference between evaluation and research. Evaluators and researchers hold different opinions. Some see evaluation as a form of applied research, or a sub-set of research. There are certainly overlaps with research – they use similar methods and designs – but evaluation is distinct from research.(4)
The specific context of evaluation
Evaluation applies the methods of social sciences research to a specific context. Whereas research seeks to make generalizations from its results, evaluation seeks to determine the effects of a specific initiative.
Different questions and methods
Evaluation asks different questions to research – it asks questions about quality, value, importance, and worth.(5) Also, there are methods evaluators use that are specific to evaluation. These include needs and values assessment, synthesis methods and criteria development methods(6).
Four distinguishing qualities
We can distinguish between evaluation and research in terms of four qualities: their purpose, where their questions originate from, how their quality and importance is judged, and how their value is appraised.(7)
What questions does evaluation ask?
When thinking about how to write evaluation questions, it can be helpful to group questions under topics (or domains). The emphasis placed on each will vary between evaluations, and not all will be included in every evaluation.
Evaluation questions can be categorized into five general program issues that evaluation questions normally cover: the need for the program, the program’s design, implementation and delivery, outcomes and impact, cost and efficiency.(8) The table below shows some broad questions for each of these categories. The questions we ask will vary with the type of evaluation conducted.
How can evaluation help?
Evaluation can help you improve your program’s implementation and design, and improve effectiveness, outcomes and impact.
(1) Scriven, M. (1991) Evaluation Thesaurus (4th ed.). Newbury Park: Sage Publications.
(2) American Evaluation Association (2014). What is Evaluation? [Internet] Accessed 26-06-2020. Available at: https://www.eval.org/p/bl/et/blogid=2&blogaid=4
(3) Patton, M.Q. (1982) Practical Evaluation. Newbury Park: Sage Publications.
(4) Scriven, M. (2016). Roadblocks to recognition and revolution. American Journal of Evaluation, 37(1), 27–44.
(5) Davidson, E. J. (2014). How “Beauty” Can Bring Truth and Justice to Life. New Directions for Evaluation, 2014(142), 31–43. doi:10.1002/ev.20083
(6) Montrosse-Moorhead, B., Bellara, A. P., & Gambino, A. J. (2017). Communicating about evaluation: A conceptual model and case example. Journal of MultiDisciplinary Evaluation, 13(29), 16–30.
(7) Patton, M.Q. (2014). Evaluation Flash Cards: Embedding Evaluative Thinking in Organizational Culture. St. Paul, MN: Otto Bremer Foundation.
(8) Rossi, P.H., Lipsi, M.W. and Freeman, H.E. (2004) Evaluation: A Systematic Approach (7th ed.) Thousand Oaks, CA: Sage Publications.
Kim Morral is a freelance researcher, Credentialed Evaluator (CE), and Owner of Qualitas Research Inc. Qualitas Research provides evaluation services to non profits and public sector organizations in Canada.