Newsletter article November 2020 - Julia Wiedemann

Assessing the impact of the Museum für Naturkunde Berlin in terms of the Sustainable Development goals - Mission (im)possible?

Julia Wiedemann (Museum für Naturkunde Berlin), Susanne Schmitt (Museum für Naturkunde Berlin), Josef Pinter (Görgen & Köller GmbH), Christoph Köller (Görgen & Köller GmbH)

 

  1. Aim of the paper

Research museums in Germany have been following the call for an assessment of their transfer into society (Wissenschaftsrat, 2016). Recently, several indicator sets have been developed to evaluate the various activities of knowledge exchange at research institutions. So far, they have mostly concentrated on assessing the outputs of activities, such as the numbers of innovative products, spin-offs, consultations, visitors, and workshops with external stakeholders (e.g. Finne et al., 2011). Others are concerned with process indicators (European Commission, 2015). Some indicators also try to capture the spontaneous effects on target groups, most of which focus on traditional museum activities only (e.g. ILFA[1]; Jensen/Lister, 2016,). A few have included impact narratives for individual projects, as done e.g. in the Research Excellence Framework of the UK[2]. However, a common method is lacking to show what long-term effects research transfer can have on society (Rossi/Rosli, 2014: 4). This holds true especially for an impact assessment at the institutional level (Neresini/Bucchi, 2011).

The project “Deep Impact” [3]at the Museum für Naturkunde Berlin aims to develop such a concept for the museum. We hope that it can serve as a blueprint for other research museums and institutions as well. Our approach is to demonstrate the museum’s impact by its contributions towards reaching the Sustainable Development Goals of the UN (SDGs). The SDGs serve as a suitable reference point for the impact of the museum as it strives to work towards a sustainable future, especially with regards to biodiversity, resources, and health. With the help of a logical framework model and the theory of change (Anderson, 2009; Weiss, 1995), we can show that the output and outcome effects of the museum’s activities level a path for an impact towards reaching the SDGs. The relevant tool is a plausible impact narrative of the museum that we have developed for prioritized SDGs in our project. 

 

  1. Prioritized impact goals of the Museum für Naturkunde Berlin

Central reference point for the project “Deep Impact” is a newly developed strategic plan of the Museum für Naturkunde Berlin (MfN). It aims to strengthen the museum’s role in society. Among other goals, the museum will strive to address global societal challenges by supporting an informed democratic knowledge society that is capable to make decisions and take actions. Multiple knowledge transfer activities such as guided tours, apps, workshops, and citizen science projects are supposed to contribute to these goals.

First, we identified four SDGs that the museum mainly aims to address. These are SDG 14 “Life Below Water”, SDG 15 “Life on Land”, SDG 3 “Good Health and Well-being” and SDG 12 “Responsible Consumption and Production”.

The two SDGs 14 and 15 aim to preserve biodiversity in all its forms by protecting the diverse coastal-, marine- and land ecosystems of the planet. A major part of the museum’s research deals with a multitude of questions on biodiversity and on ecosystems’ dynamic adaptations to changing environmental conditions. Results are steadily communicated to society. For example, formats such as citizen science projects on the nightingale[4] and on bees[5] offer people the opportunity to engage in biodiversity research and deepen their knowledge on species protection.

SDG 3 focuses on issues of health and wellbeing including those raised by the animal-human-relationship. To do this, the museum uses a holistic “one health” approach and cooperates with scientists from different disciplines, such as medical researchers of the Charité Berlin. Thematic topics include food safety, the control of zoonoses and combatting antibiotic resistance. The current exhibition “Parasites – Life undercover”[6] that e.g. deals with options for a medical use of parasites is one example where research on corresponding topics is transferred into society.

Finally, SDG 12 looks at resource efficiency and strives for sustainable solutions for their use, such as a circular economy. Topics include the reduction of hazardous and food waste, sustainable production and reporting methods of businesses, and sustainable tourism. Transfer projects at the museum that support this SDG include the recent exhibition “Artefakte”[7] that has dealt with topics of sustainable resource use and offered visitors to engage in live discussion formats with scientists. Next to informing them on the detrimental influences of human actions on nature and ecosystems, the exhibition also discussed problems of resource waste and has shown visitors ways to support resource efficiency in their daily lives.

 

  1. The method: How can we demonstrate the impact of the museum on SDGs?

Using the logical framework approach, we differentiate between inputs (resources and planning), outputs (direct results), outcomes (immediate effects on target groups) and impacts (long-term societal effects) of activities. The theory of change (Anderson, 2009; Weiss, 1995) helps us connect the micro level effects in target groups with the macro-level effects on SDGs. This is done using an impact narrative that demonstrates the impact potential of the museum’s transfer activities. Empirical evidence on the outcome level shows the likelihood of behavioural change in target groups towards supporting the SDGs and is hence a crucial element in the narrative.

For this, we have developed a theoretical model. It illustrates how the museum can achieve an impact towards achieving the SDGs by its newly developed strategic plan. There are seven strategic goals of the museum (in light grey) that we included as elements at different levels of the impact logic. Through fulfilling the strategic goals, the museum will enable actions of target groups on nature-related issues, which is the basis for supporting the SDGs.

Figure 2: Impact model of the MfN’s future plan towards reaching the SDGs

 

A. Generate relevant knowledge (Input)

The foundation for supporting the SDGs is achieving academic excellence. Therefore, at the bottom level of the model we see strategic goals related to the academic work of the museum: Support the Science Hub Berlin, Protect and develop the collections, Build an information and research infrastructure and Develop the academic campus.

By fulfilling these goals, the museum will generate relevant evidence-based knowledge on multiple nature-related issues that are the basis for transfer activities of the museum.

B. Enable actions of target groups (Output and Outcome)

The second step towards achieving a societal impact consists in exchanging and discussing the generated scientific knowledge with society, thereby achieving an effect on the audience. Hence, in the middle part of the model, we see strategic goals of the museum related to its transfer activities, namely to Be a centre for research, transfer, dialogue, innovation and participation and to Support an informed democratic knowledge society that is capable to make decisions and take action. We receive data on the impact potential of all the museum’s activities combined, by measuring outputs and outcomes through indicator-driven evaluations:

Output Measurement

The generated evidence-based knowledge on nature-related issues can be delivered, discussed and further developed with society. This is done through the various transfer activities of the MfN. We have categorized the transfer activities broadly in research, dialogue, innovation and participation. Examples include discussion formats with citizens, consultancy of politicians, citizen science, guided exhibition tours with students, workshops with teachers, cooperations with local businesses and co-creation of products with them.

Indicator-driven data on the output of activities will show to what extent the activities have occurred and will make the museum become a centre for transfer. Main indicators on the output level that we have developed are the number of activities that have been conducted, the number of people/ targeted groups that have been reached, the number of multipliers that have attended and the number of activities that have followed from it.

For example, 2964 public engagement formats were conducted by the museum in 2018. They attracted 96.817 participants (Museum für Naturkunde Berlin, 2019). In the future, we will collect more detailed information on the share of formats that work in support of SDGs 14 and 15. Also, we will be able to show what groups have been reached and how many of them can act as multipliers of their gained knowledge, such as teachers and politicians.

Outcome Measurement

All of the activites have measurable effects on the participating groups. They can raise people’s awareness, deepen their knowledge, and strengthen their ability to become active regarding nature-related topics. Here the museum has already achieved a major positive outcome effect on society. It can support facing challenges related to science scepticism and establishing a factual base for democratic debates on nature-related topics.

Indicator-driven data on the outcome of activities will show to what extent the above mentioned effects have occurred in target groups, which will make the museum fulfill its aim of supporting an informed knowledge society. Relevant indicators include the number of participants that have gained interest or awareness, the number of participants that have raised knowledge and the number of participants that have raised capability to change their action on the topic. The data can be collected with the help of e.g. surveys, interviews and observations.

For example, we plan to collect outcome data on at least 10% of all transfer activities that have dealt with each of the four prioritized SDGs each year. In the case of biodiversity relevant formats, this helps us assess the percentage of participants that have gained awareness, knowledge, capability and readiness to act towards species protection.

C. Contributions to reaching the SDGs (Impact)

By achieving academic and transfer excellence we can presume a societal impact of the museum. This is shown at the top part of the model and happens when target groups change their behaviour in line with the SDGs. Here, the museum has actively made Contributions to achieving the SDGs and fulfilled its final strategic goal to Address global societal challenges.

As described above, the impact potential of the transfer activities can be evaluated with the help of data collected on the output and outcome level. It will deliver plausible arguments for the museum’s contribution to the SDGs. Additional long-term data and observations of behavioural changes in target groups that have followed from the activities can also help support these arguments. Using this data, we can build impact narratives which show that changes in people’s attitudes, knowledge and decisions will contribute to reaching the SDGs.

Regarding our biodiveristy relevant museum formats, the collected output and outcome data help us estimate the likelyhood that participants engage in species protection in their daily lives. For several formats, we will collect detailed information on the percentage of people that have gained new knowledge and awareness by having joined the activities. Most likely, this leads to a change in the perceptions and behaviour of these participants. To support our logic, we will also collect long-term data on the behavioural changes that have taken place for a limited number of projects. They could e.g. start planting bee-friendly plants or setting up bird feeders, support local initiatives for species protection, act as multipliers of their knowledge or encourage others to join action. At this point, the museum has enabled actions to actively support the SDGs 14 and 15.

  1. Conclusion

In conclusion, we find that assessing the impact of the Museum für Naturkunde Berlin on Sustainable Development Goals is not a mission impossible. In the project “DeepImpact” we have developed a conceptual model to make the museum’s contribution to prioritized SDGs visible. It includes strategic goals of the museum that help us to build a plausible impact story. Our method is based on the logical framework model and the theory of change.

For evaluating the impact of the museum in terms of SDGs it is essential to raise indicator-driven data on most transfer activities. Data on the output will be gathered on an institutional level, while data on the outcome will be collected for a relevant number of activities. Overall, this empirical basis will demonstrate what groups have been reached and in what ways they have been affected by the transfer activities of the museum. In addition, we will collect long-term data on behavioural changes for a limited number of projects. All data combined make it possible to assess the likelihood of target groups to change their attitudes, perceptions, and behaviour towards SDGs

The project “DeepImpact” will run for another year. In this time, we will e.g. test and further develop our model. More importantly, we will start to implement our method in the museum. We plan to use a digital tool to collect the museum’s output data and combine it with individual evaluations for the outcome data. Through this, we can start to assess the current impact of the museum on SDGs. This goes hand in hand with communicating about the concept and the method in the museum. We hope that this will enable the museum researchers and others to contribute to and benefit profoundly from our approach.

 

  1. References

 

Anderson, Andrea A. (2009): The Community Builder’s Approach to Theory of Change. A practical guide to theory development. Aspen Institute: New York.

 

European Commission (2015): Indicators for promoting and monitoring Responsible Research and Innovation. Report from the Expert Group on Policy Indicators for Responsible Research and Innovation.

 

Finne, Håkon et al. (2011): A Composite Indicator for Knowledge Transfer. Report from the European Commission’s Expert Group on Knowledge Transfer Indicators. European Commission. 

 

Jensen, Eric Allen and Lister, Thomas J.P. (2015): Evaluating Indicator-Based Methods of ‘Measuring Long-Term Impacts of a Science Center on Its Community’. In: Journal of research in science teaching, 53(1), 60–64.

 

Museum für Naturkunde Berlin (2019): Documentation. Evaluation 2019.

 

Neresini, Federico and Bucchi, Massimiano (2011): Which indicators for the new public engagement activities? An exploratory study of European research institutions. In: Public Understanding of Science, 20(1), 64–79.

 

Rossi, Federica and Rosli, Ainurul (2014): Indicators of university–industry knowledge transfer performance and their implication for universities. Evidence from the United Kingdom. In: Studies in Higher Education, 40 (10), 1970-1991.

 

Weiss, Carol H. (1995): Nothing as Practical as Good Theory. Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families. In: Connell, James P. et al. (Eds.): New Approaches to Evaluating Community Initiatives. Aspen Institute: Washington DC.

 

Wissenschaftsrat (2016): Wissens- und Technologietransfer als Gegenstand institutioneller Strategien. Positionspapier.

 

[1] https://www.artscouncil.org.uk/advice-and-guidance/inspiring-learning-all-home-page

[2] https://impact.ref.ac.uk/casestudies/

[3] https://www.museumfuernaturkunde.berlin/en/science/indicators-assessing-impact-museums-knowledge-transfer

[4] https://www.museumfuernaturkunde.berlin/en/science/nightingale-research-case-citizen-science-project-natural-and-cultural-history-nightingales

[5] https://www.museumfuernaturkunde.berlin/en/science/bees-pollination-and-citizen-science-berlins-gardens

[6] https://www.museumfuernaturkunde.berlin/en/museum/exhibitions/parasites-life-undercover

[7] https://www.museumfuernaturkunde.berlin/en/museum/exhibitions/archive-special-exhibition/artefacts

All AESIS newsletters

show less

You can also follow us on twitter where we share news from the field and updates on our activities.