The impact evaluation model for the network of Finnish Cultural and Academic Institutes will be put in use at the beginning of 2025 to evaluate the impact of the Institutes.
The Institutes defined Sustainable Internationalism as the impact goal of the Institute network in their joint development process.
The purpose of the new evaluation framework is to complement the quantitative data already collected since 2017 with information on the impact of the activities.
The theoretical framework for evaluating impact is Theory of Change i.e. the evaluation of network’s activities’ success is based on an analysis of the extent to which they achieve the desired outcomes that the network has identified for their activities. The outcomes are defined in such a way that, when achieved, the network’s activities can be seen to contribute to the longer-term societal change objectives and the impact goal.
Through the collection of impact data, the network is increasingly focusing its evaluation on the impact of the Institutes’ work and activities on their target groups, stakeholders as well as the society at large.
Impact data is collected through four surveys targeted at the Institutes’ target groups: 1) Survey for Artists and Researchers 2) Partner Survey, 3) Public Opinion Survey, 4) Institute Self-Evaluation. The data collected through these surveys will be supplemented by follow-up surveys and interviews.
The preparation of the impact evaluation model has been a lengthy development process
The impact of the Cultural Institutes has previously been assessed in a study ”Vientiä, vaihtoa, vaikuttavuutta: Selvitys kulttuuri- ja tiedeinstituuttien merkityksestä Suomelle” (Cupore, 2012) as well as the 2023 study ”Mitä olisi, jos kulttuuri- instituutteja ei olisi? – Selvitys Suomen kulttuuri-instituuttien vaikuttavuudesta” (Miltton, 2023). The impact of the Academic Institutes has previously been discussed in the Ministry of Education and Culture’s publication ”Tieteelle tarkoitettujen rahapelitoiminnan voittovarojen käytön arviointi” (2018).
Based on the recommendations of these studies, the association Finnish Cultural and Academic Institutes (SKTI) launched a process in 2022 to develop a model for collecting data on the impact of the network.
In 2023, SKTI, in collaboration with National Council for Voluntary Organisations (NCVO) consultant Sini Rinne-Kerridge, conducted a Theory of Change workshop series for the entire Institute network. The workshop series aimed to identify a common impact goal for the network as well as to define the desired outcomes through a process of co-development. Minimum one representative from each institute participated in the workshops.
The topics of the workshops were:
- Systems change ja Theory of Change
- Outcomes and audiences
- Priorisation and assumptions
- Measuring
As a result, the network produced a Theory of Change map and a narrative describing in words the steps of the development process of the impact evaluation model and the objectives defined in it.
The workshop series ended at the end of 2023. This was followed by the creation of the actual evaluation model, i.e. the development of impact indicators, questions and surveys for the institutes to use in order to collect impact data. As a pilot, the surveys have been used in the Autumn of 2024 in the pARTir project, a collaboration between 20 Institutes and the SKTI.
At the beginning of 2025, the impact surveys will be implemented in all Institutes. The impact data collected through the surveys will be analysed both at the individual institute level as part of the institute’s own development work, and at the network level by the SKTI.
Viitteet:
”Mitä olisi, jos kulttuuri-instituutteja ei olisi? Selvitys Suomen kulttuuri-instituuttien vaikuttavuudestas”, Miltton 2023. https://valtioneuvosto.fi/hanke?tunnus=OKM018:00/2023 https://valtioneuvosto.fi/-/1410845/selvitys-suomen-kulttuuri-instituuttien- vaikuttavuudesta-valmistunut
”Vientiä, vaihtoa, vaikuttavuutta: Selvitys kulttuuri- ja tiedeinstituuttien merkityksestä Suomelle”, Cupore 2012, Saukkonen, Pasi; Mitchell, Ritva; Kontkanen, Riitta https://www.cupore.fi/julkaisut/vientia-vaihtoa-vaikuttavuutta-selvitys-kulttuuri-ja- tiedeinstituuttien-merkityksesta-suomelle/
”Tieteelle tarkoitettujen rahapelitoiminnan voittovarojen käytön arviointi ”, Opetus- ja kulttuuriministeriön julkaisuja 2018:9, Raivio, Tuomas; Sepponen, Susanna; Pessala, Piia; Vaahtera, Anu; Hjelt, Mari https://julkaisut.valtioneuvosto.fi/handle/10024/160537
National Council for Voluntary Organisations (NCVO)
https://www.ncvo.org.uk/