WHY SHOULD EVALUATORS EMBRACE COMPLEXITY THINKING?

Complexity ideas have long been explored by philosophers, social scientists, systems analysts… and evaluators. Multi-disciplinarity, a distinctive feature of evaluation, is an integral part of the paradigm shift underway in all the sciences.  Putting complex adaptive system models to work facilitates a much-needed rapprochement between the physical, natural, and social sciences. To tap this potential, evaluators are adopting a new mindset and teaming up with other disciplines, including data scientists and mathematicians (https://evaluationuncertainty.com/).

The ‘complexity turn’ in evaluation is far more than a fad. The new science of causation pioneered by Pearl and McKenzie (2018), combined with increasingly powerful computers and the exponential proliferation of Big Data, is breaking the monopoly of randomised control trials in evaluation by allowing computer-based experiments that explore ‘why’, ‘what for’, and ‘what if’ questions. This will vastly improve the quality and relevance of evaluative theories of change.  (https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/643)

Evaluation is a system with permeable boundaries: unlike social research, it is not enclosed in an ivory tower. Indeed, evaluation is mandated to ‘speak truth to power’. It does so even in organizations and enabling environments that privilege vested interests and neglect ethical values. In such environments, complexity concepts can help independent evaluators identify the triggers of positive transformation in organizations and policy networks. 

The evaluation process is a complex adaptive system that agent-modelling can help elucidate. Typically, evaluative outputs are recycled to become inputs in ways that either reverse the change of some variable(s) in the sub-system (negative feedback) or enhance it (positive feedback). The higher-level agents whose behaviours are moulded by the overall ideological environment direct the behaviour of the lower levels that in turn react and cause new patterns to emerge so that the evaluation process is coevolutionary. 

Evaluation costs are typically a minute fraction of those incurred by the organization, e.g., 1-2 percent in international development institutions where evaluation is influential and well-funded.  The potential benefits that flow from a well conducted evaluation for a socially pertinent intervention are so high that it seems reasonable to assert that the overall evaluation enterprise is a high return venture even if it is rarely successful, just as the funding of start-up companies, scientific research ventures, or the popular music industry, where a single blockbuster compensates for dozens of false starts to generate profitability.  

The evaluation system is emergent: the evaluation whole is invariably different from the sum of its parts and evaluation outcomes are non-linear and hard if not impossible to predict. In turn the complex systems metaphor of the evaluation process implies a lack of proportionality between inputs and outcomes. Thus, conceiving of evaluation as a complex system operating within a complex system helps to falsify popular myths about the role of evaluation in society and about evaluation utilization as the acid test of evaluation professionalism.    

Reference

Pearl, J. & Mackenzie, D. (2018). The book of why: The new science of cause and effect. New York. Basic Books

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.