Experimentation for transformative change, while desirable, is far from a frequent characteristic at the policy arena according to participants in group 3. Institutional frameworks, policy funding and evaluation requirements are rarely fit for experimentation. It is sometimes not obvious how to create and assess a space that allows for experiments, and how to reintegrate results from that space into other processes. Whether there needs to be an explicit regulatory context that enables experiments, and to what extent is it possible to establish enough flexibility within that context, remains disputable. Another issue concerns who has access to the insights gained from experiments, and whether it may be difficult to accept the experiment results as new information, depending on the process and actors involved.
It is also quite difficult to measure experimental success, because there are different ways of understanding what success means. Social change does not always get immediate results, it takes more time to engage. The fact that an experiment may fail, and that even failure is a type of learning success, may be complicated for those who set up experiment to solve challenges. A clear understanding of the intent, transparency of the stakeholders, and management of expectations, are thus very important to pave the way towards experimentation. Frequently with experiments, as soon as something goes wrong, the whole project falls apart. Therefore, trust in the process is also very important, and it has to do a lot with how expectations are organized.
When setting up experiments, there needs to be a specific strategy for experimentation that incorporates the fact that a number of those experiments will fail, and also provides a way to connect different experiments and their learning. The aim sometimes is to find ways for policies to open up to further experimentation while public resources are best invested and procurement issues are on the table. Demarcating the scope ex ante for innovative change brings about questions about accountability and evaluation to the table.
Other relevant questions discussed were “what is really transformative?”, “what is new from different perspectives?” and “what does change actually mean in real life contexts?” Transformation involves the economic, social and environmental dimensions of the Sustainable Development Goals, new rules linked to concrete challenges, and system level change. Other important attributes identified by the group include the concepts of following a vision, strategic and logical frames to address a particular challenge, and the understanding that transformation needs to have a positive directionality.
Empowerment itself can be an outcome of both experimentation and evaluation, which is why participatory processes are so relevant to actively engage with citizens. Recognizing that relevant dialogues are already happening outside the public administration and academic environment may be a concrete first step towards opening up the space for experimentation and learning. A short term impact may be difficult to assess, but is sometimes possible to track small changes in people and organisations.
To truly engage in transformative experiments, we need to build a basic understanding of systemic approach and a concrete theory of change associated to the system. The framing of the experiment is not always clear beforehand, but systemic transformation has to be the general objective, or the potential solution. Overall, the most important idea to reflect upon is what may be transformative in experimentation and how to evaluate it.
In a nutshell, experiments are very important, but not always explicitly present in policy nor always transformative in their overall objective. To create more experimental spaces, and to be able to link up a wide variety of different efforts, requires the possibility and expectation that some of those experiments are likely to fail, and that such failure is part of the broad learning that transformative change needs. Evaluating transformative experiments is a very challenging task, but it may start with understanding how change occurs within a particular space, and how such change may be triggered and connected to other changes in the system.
