“There is no science without evaluation, because without evaluation one could not distinguish science from pseudo-science, let alone good science from poor science.”
“Evaluation is the one indispensable activity of all science.”
These quotes from the Evaluation Thesaurus position science as dependent on evaluative thinking. All disciplines rely on evaluation judgments derived from applying evaluative logic.
Scriven believed that evaluative thinking was a basic human capacity manifest in all cultures as part of meeting challenges and solving problems. His emphasis on making values explicit in evaluation included honoring different cultural perspectives. He did not limit scientific evaluation to universities and academic work but rather saw systematic inquiry as a way of making sense of the world and differentiating what worked from what didn’t in any context. That said, his perspective was deeply steeped in Western philosophy and his language, ideas, metaphors, and examples reflect that background. Understanding the significance and nature of evaluation’s development as a more diverse field that includes different ways of knowing is deepened by understanding the thinking and influence of Michael Scriven, and the vision and limitations of his perspective.
In his masterful and enduring Hard-Won Lessons treatise (Scriven, 1993), he makes the case evaluation as a discipline eloquently and vociferously:
Evaluation, the ugly frog, turns into a prince when the spell on us is broken. Evaluation is all that distinguishes astronomy from astrology, good explanations from bad ones, good experimental designs—or bridge designs—from inferior ones, good scientists and engineers and technologists from run-of-the-mill ones. It is a discipline that is part of every discipline because it distinguishes the discipline from pretentious jargon, just as one distinguishes good food from garbage. (Scriven, 1993, pp. 47-8.)
In this way Scriven (2015) forcefully envisions and advocates positioning evaluation as the alpha transdiscipline: As a transdiscipline, evaluation sits atop the disciplinary, scholarly, and academic hierarchy with philosophy, logic, and statistics as bodies of knowledge, theory, and methods that are essential for the scholarship and knowledge creation of all other disciplines.
Evaluation is the alpha transdiscipline because “its domain includes the methodology of the task of validation at any discipline’s claim to legitimacy as a discipline: it is the master of credentials” (Scriven , 2013, p. 175).
Michael Scriven pioneered treating evaluation as the Science of Valuing (Shadish, Cook, & Leviton, 1991, p. 74). He has reflected that “much of my early work in evaluation was devoted to attacking various attempts to … marginalize evaluation into the category of decision-support apparatus, by contrast with truth-seeking efforts of “true science” (Scriven, 2004, p. 188; emphasis in the original). For Scriven, evaluation is without question “scientifically legitimate… The bottom line… is that a competent program evaluator can show that, for example, a program for teaching reading is truly excellent or truly worthless as a matter of scientific fact” (Scriven, 2013, p. 171; emphasis in the original).
Scriven did not aspire to have evaluation become more like science, for he was scathing in his critique of shoddy reasoning and judgment in much scientific practice (Scriven, 1976). Rather, he challenged science to become more evaluative stating that evaluation is “perhaps the most important and pervasive process in which the human mind is capable. (Scriven, 1991, p 10)
Scriven rendered that judgment to assert evaluation’s rightful status as the alpha transdiscipline of all sciences, which means it must, logically, be a science.
…[W]hat science should be is an evaluative decision, and it is hard to argue that there is a more important kind of decision anywhere in science. This is the decision, or a refinement of it, after all, that we call on many of our peer review panels to make. In particular we ask them to review the scientific quality of the proposals in front of them. They are indeed evaluators, and on them and on their peers on editorial boards and appointment committees rest the whole definition and quality of science, and eventually the whole future of science—and of mathematics , and engineering, and technology…. [S]cience is at its core as well as in all of its applications an evaluative enterprise.
Scriven (2016) called for a “revolution” in understanding and action to realize evaluation’s scientific potential to contribute to both knowledge and more effective decisions and actions. Today we speak of transformation rather than revolution, but perhaps revolution more substantially captures and communicates the scope and significance of the changes needed. Michael Scriven was eloquent in articulating the vision and meaning of evaluation as science. The International Evaluation Academy honors his legacy.
References and Resources
Scriven, M. (1976). Reasoning. New York: McGraw-Hill.
Scriven, M. (1991) Evaluation thesaurus. Beverly Hills, CA: Sage.
Scriven, M. (1993). Hard-won lessons in program evaluation. New Directions for Program Evaluation, 58.
Scriven, M. (2003). Differences between evaluation and social science research. The Evaluation Exchange IX (4, Winter), 1. Cambridge, MA: Harvard Family Research Project.
Scriven, M. (2004). Reflections. Pp. 183-195 in M. Alkin (ed.) Evaluation roots: Tracing theorists’ views and influences. Thousand Oaks, CA: Sage.
Scriven, M. (2007) Evaluation Checklist. Microsoft Word – kec_feb07.rtf (wmich.edu)
Scriven, M. (2013) Conceptual resolutions and evaluation: Past, present, and future. Pp. 167-179 in M. Alkin (Ed.), Evaluation roots, 2nd ed. Los Angeles: Sage.
Scriven, M. (2016). Roadblocks to Recognition and Revolution. American Journal of Evaluation, 37(1), 27–44. https://doi.org/10.1177/1098214015617847
