What is evaluation?
Evaluation refers to the systematic process of determining the merit, worth, or value of something (Scriven 1991). Evaluation may seek to determine what works, for whom, to what extent, in what context, and how (Pawson and Tilley, 1997). The “program” in program evaluation may refer to a program, project, sub-project, project component, initiative, activity, intervention or service that is being evaluated.
The definition provided by Scriven (1991), and later adopted by the American Evaluation Association (2014), has wide support in the field. However, “evaluation is methodologically eclectic, pluralistic, and mixed” (Patton, 2008) and subsequently not all evaluators define evaluation the same way. No single definition fully captures the practice of evaluation, and there is no consensus of opinion amongst evaluators on the boundaries.
In practice, the accepted definition of evaluation may be influenced not only by the evaluator’s perspective, but the particular requirements, scope, and objectives of the organization requiring the evaluation. Patton (1982) describes six different types of evaluation definitions, each with different emphases and perspectives on evaluation practice:
What distinguishes evaluation from research?
Evaluation applies the methods of social sciences research to a specific context. While there are overlaps with research, each using similar methods and designs, evaluation is distinct from social research (Scriven, 2016). Evaluation asks different questions – questions about quality, value, importance, and worth (Davidson, 2014). While drawing on social research methods, there are evaluation-specific methods, such as needs and values assessment, synthesis methods and criteria development methods (Montrosse-Moorhead, Bellara and Gambino, 2017).
Patton (2014) distinguishes between evaluation and research in terms of four qualities: their purpose, where their questions originate from, how their quality and importance is judged, and how their value is appraised.
Source: Patton, Michael Quinn (2014). Evaluation Flash Cards: Embedding Evaluative Thinking in Organizational Culture. St. Paul, MN: Otto Bremer Foundation
What questions does evaluation ask?
When thinking about formulating evaluation questions, it can be helpful to group evaluation questions under topics (or domains). The emphasis placed on each will vary between evaluations, and not all will be represented. Rossi, Lipsi and Freeman (2004) categorized evaluation questions into five general program issues that evaluation questions normally cover.
Source: Rossi, P.H., Lipsi, M.W. and Freeman, H.E. (2004) Evaluation: A Systematic Approach (7th ed.) Thousand Oaks, CA: Sage Publications.
In this post we have considered briefly what evaluation is, how it is defined, and the kinds of questions evaluations ask. To wrap things up, we will consider what an evaluation can bring your organization.
What can evaluation help you with?
American Evaluation Association (2014). What is Evaluation? [Internet] Accessed 26-06-2020. Available at: https://www.eval.org/p/bl/et/blogid=2&blogaid=4
Davidson, E. J. (2014). How “Beauty” Can Bring Truth and Justice to Life. New Directions for Evaluation, 2014(142), 31–43. doi:10.1002/ev.20083
Montrosse-Moorhead, B., Bellara, A. P., & Gambino, A. J. (2017). Communicating about evaluation: A conceptual model and case example. Journal of MultiDisciplinary Evaluation, 13(29), 16–30.
Patton, M.Q. (1982) Practical Evaluation. Newbury Park: Sage Publications.
Patton, M. Q. (2008) Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage Publications.
Patton, Michael Quinn (2014). Evaluation Flash Cards: Embedding Evaluative Thinking in Organizational Culture. St. Paul, MN: Otto Bremer Foundation
Pawson, R. and Tilley, N. (1997) Realistic Evaluation. London: Sage Publications.
Rossi, P.H., Lipsi, M.W. and Freeman, H.E. (2004) Evaluation: A Systematic Approach (7th ed.) Thousand Oaks, CA: Sage Publications.
Scriven, M. (1991) Evaluation Thesaurus (4th ed.). Newbury Park: Sage Publications.
Scriven, M. (2016). Roadblocks to recognition and revolution. American Journal of Evaluation, 37(1), 27–44. doi:10.1177/1098214015617847
Kim Morral is a Credentialed Evaluator (CE) and Owner of Qualitas Research Inc. Qualitas Research provides evaluation services to organizations in Canada and internationally.