Skip to main content

How to measure the impact of citizen science on environmental attitudes, behaviour and knowledge? A review of state-of-the-art approaches

A Correction to this article was published on 05 April 2022

This article has been updated

Abstract

The effects of citizen science are wide ranging, influencing science, society, the economy, the environment, as well as individual participants. However, in many citizen science projects, impact evaluation is still overly simplistic. This is particularly the case when assessing the impact of participation in citizen science on the environmental attitudes, behaviour and knowledge of citizen scientists. In an attempt to bridge the gap between the state of the art in relevant scientific fields and citizen science, this systematic literature review identified best practices and approaches in the field of environmental psychology for measuring environmental attitudes, behaviour and knowledge. From the literature, five relevant and validated approaches were identified that can be used to measure changes in attitudes, behaviour and knowledge in citizen science projects. This would allow for improved understanding of the impacts of citizen science, as well as for improved project evaluation as a whole.

Introduction

Citizen science is increasingly popular and has been defined and interpreted in various ways. At a basic level, it is a purpose-designed collaboration in which the general public take part in the scientific research process to support knowledge generation [100], or more specifically “the scientific activities in which non-professional scientists volunteer to participate in data collection, analysis and dissemination of a scientific project” ([46], p. 105). Other definitions point to the role of citizen science in the democratisation of science and public engagement [55, 56] and consider it a tool to link science and society, involving citizens more and more in the scientific process [72]. Citizens can play a voluntary, but highly active role in the process, with input ranging from data collection to the co-design of projects [17], and citizen science as a process is increasingly recognised for its multi-stakeholder nature and complexity [122]. Definitions often conflict in various ways, for example regarding the exact extent of public participation, the opt-in or voluntary nature of participation, and the extent to which generated data can be used in science and/or policy [22]. With these subtle nuances in definition, differences in the implementation of citizen science often occur. Nevertheless, with this rise in popularity, the wide ranging impacts and evaluation of citizen science projects are currently of significant interest.

A range of studies have been conducted to assess the impact of citizen science interventions. These studies have covered a variety of impacts, including: feelings of environmental stewardship in participants [92], economic activity and the creation of jobs [60], scientific outcomes and resulting publications [17], and participant learning outcomes [113]. However, the exact impacts of citizen science are still to be fully and comprehensively understood [65], while up to date impact assessment methods and frameworks are not yet fully integrated in practice.

Based on their systematic review of close to 80 impact assessment frameworks and methods for impact assessment of citizen science, Wehn et al. [122] highlighted that the assessment of environmental attitudes, behaviour and knowledge is particularly lacking throughout the citizen science literature. They also noted that when these impacts have been measured, it is often done simplistically, without using approaches and scales considered the state of the art in relevant scientific literature. This can lead to a suboptimal selection of measurement approaches, and an impact assessment procedure lacking in validity. Furthermore, a more soundly grounded impact assessment procedure would allow for enhanced project evaluation, and therefore an improvement in the implementation of citizen science in the long term.

For example, during their assessment of attitude change in citizen science participants, Brossard et al. [20] highlighted the insufficiencies of their measurement methodology, suggesting that more sensitive scales are needed. This deficit in assessment approaches is visible in a range of other impact assessment studies, aiming to identify the influence of participation in citizen science on attitude [24, 49, 110, 126, 102). These studies used a wide variety of approaches (often with little justification for their selection) to measure environmental attitude, displaying the lack of integration of (and consensus on) measurement approaches within citizen science. This is also apparent in the literature focusing on the assessment of knowledge [17] and behaviour [117].

Apart from individual studies and reviews featuring the assessment of environmental attitudes, knowledge and behaviour, various citizen science impact assessment frameworks have also been developed. One such framework, developed by Kieslinger et al. [65] also attempted to capture attitude, behaviour and knowledge in their three-dimensional impact assessment framework, primarily with the ‘citizen scientist’ dimension (impacts on the individual participants). However, no concrete approaches are integrated into the framework to suggest how these concepts should be assessed, highlighting again the lack of integration of state-of-the-art measurement methods in citizen science.

The field of environmental psychology and the literature on the measurement of environmental attitudes, behaviour and knowledge are well developed, with various establish methods and approaches being refined over the past decades. It has long been claimed and documented with anecdotal evidence that experiences with the environment or environmental matters (such as participation in citizen science initiatives) can alter environmental attitudes, behaviour and knowledge over time [7, 37], suggesting that this field of literature should be of keen interest to citizen science. However, despite the availability of such scales, few citizen science projects actually utilised these approaches during impact assessment [122]. Instead, simplistic approaches are often used to assess project impacts on environmental attitudes, behaviour and knowledge. There is therefore a need for an analysis of the state of the art in the assessment of environmental attitudes, knowledge and behaviour with a view to integrating this in the impact assessment of citizen science.

The systematic literature review presented in this paper aimed to assess the state of the art of measuring environmental attitudes, behaviour and knowledge, and analyse to what degree the currently available measurement approaches can be applied to evaluate the impact of citizen science. However, this paper does not intent to be prescriptive in nature, citizen science is broad in nature, as are the needs and requirements of individual projects. This considered, this paper aims to highlight where consistency in the impact of citizen science projects can be developed, and suggest potential ways in which this could be implemented.

Following this introduction, the methodology of this literature review is outlined in the following section, before the results are detailed. In the discussion, the state of the art for measuring environmental attitudes, behaviour and knowledge are presented and the findings applied to the context of citizen science. Finally, conclusions and future avenues for research in the field are highlighted.

Methods

The analysis of the state of the art in the measurement of environmental attitudes, knowledge and behaviour outlined in this paper was developed from a systematic review of relevant academic literature in these fields. As these are three separate, yet closely related concepts, distinct searches were conducted for each. However, due to the degree of overlap and similarity between the concepts, some of the literature identified during this search is relevant for all three.

The selection of relevant literature for this systematic literature review was based on the method outlined by Moher et al. [84]. The purpose of the systematic literature searches was to identify publications that propose or discuss frameworks, approaches and scales for assessing environmental attitudes (also termed environmental concern), behaviour and knowledge.

The literature search was conducted on Web of Science and Wiley’s Online Library (following an initial preliminary search on Google Scholar), from February to April 2021. Keywords were compiled that referred to the concepts of: (1) environmental attitude, (2) environmental behaviour, and (2) (environmental) knowledge (see Tables 1, 2, 3). Similarly, a set of keywords was identified for the second aspect of the search, which related to the measurement or assessment of these concepts. The Boolean operators “AND” and “OR” were used in order to combine the different terms, while the asterisk (*) was used to ensure the inclusion of variations on each of the terms. To specify the exact phrases that should be contained within the search, quotation marks (“”) were used for each of the first column aspect search terms.

Table 1 Parameters used in the Environmental Attitude literature search
Table 2 Parameters used in the Environmental Behaviour literature search
Table 3 Parameters used in the Environmental Knowledge literature search

In order to identify search terms for each of these concepts, an understanding of appropriate definitions, terms and conceptualisations was required. To acquire this understanding, an initial search of the three concepts was done. This was necessary in particular for ‘environmental attitudes’, which is a term that has been defined in a number of conflicting ways, and which has various synonyms (e.g. beliefs, values) partly due to differing or contradictory definitions [93]. This was highlighted by Rosa and Collado [97], who noted that a range of overlapping terms have been used in the literature in place of (or in conjunction with) ‘environmental attitudes’. For example, some researchers [109] conflate ‘environmental willingness’ with ‘environmental attitudes’, using the former as a measure for the latter. Others (e.g. [95]) note the difference of ‘environmental willingness’ and ‘environmental attitudes’, instead investigating the effect of willingness on attitudes. The literature review undertaken for our paper considered these broad definitions, and therefore included terms such as ‘values’ and ‘beliefs’ when searching for literature related to environmental attitudes.

In the literature, the definitions for both ‘environmental behaviour’ and ‘environmental knowledge’ are less varied. The term ‘pro-environmental behaviour’ [114] is a common substitute for ‘environmental behaviour’ in the literature, and is commonly used when discussion determinants of positive behaviour. Additionally, the words ‘actions’ and ‘activities’ are also commonly included in definitions of the concept (e.g. [88, 124]).

The first part of this literature review was conducted by searching the ‘Topic’ section of literature in the core collection of Web of Science. This search includes title, keywords and abstracts of literature. As this literature review was focusing on review articles, the search was further refined to only include reviews.

In the next step, the Wiley Online Library was searched using the same set of keywords. This search was conducted on 3 and 4 March 2021. As it not possible to search the ‘Topic’ of literature on the Wiley Online Library, the keywords within the abstracts of the records were searched. To limit the search to review articles, the term ‘review’ was also included in the search. Several of the same articles seen during Web of Science search were also seen in the Wiley search. These items were ignored and were not double counted.

Firstly, Web of Science was searched. The ‘environmental attitude’ search returned 7558 records (after filtering to only include review articles, the hit number was further narrowed to 330 records). The title, abstract and keywords of these records were then screened, removing articles unrelated to the measurement of environmental attitude. This process resulted in a shortlist of 23 records. The ‘environmental behaviour’ search (see Table 2) returned 2187 records. When filtering for review articles, this was narrowed to 95 records. After screening the title, abstract and keywords of these records based on their relevance to the measurement of environmental behaviour, a shortlist of 14 records remained. The knowledge assessment search was conducted twice, once without the third parameters (“Environment*” and “Sustain*”), in order to capture the wider literature of knowledge assessment, as well as that specifically relating to environmental knowledge. The wider ‘knowledge assessment’ search (see Table 3) returned 1013 records. When filtering for review articles, this was narrowed to 402 records. After screening the title, abstract and keywords of these records based on their relevance to the assessment of knowledge, a shortlist of seven records remained. The more specific ‘environmental knowledge’ search (see Table 3) returned 702 records. When filtering for review articles, this was narrowed to 301 records. After filtering the title, abstract and keywords of these records to ensure relevance to the measurement of environmental knowledge, a shortlist of eight records remained.

Next, Wileys Online Library was searched. The ‘environmental attitude’ search returned 457 records that were similarly screened for relevance and resulted in a shortlist of 13 records. The ‘environmental behaviour’ search returned 360 records that were similarly screened for relevance and resulted in a shortlist of 13 records. The wider ‘knowledge assessment’ search returned 641 records that were similarly screened for relevance and resulted in a shortlist of two relevant records. The ‘environmental knowledge’ search returned 151 records that were similarly screened for relevance and resulted in a shortlist of two relevant records.

In total, the search of both Web of Science and Wileys Online Library resulted in a shortlist of 23 records for the ‘environmental attitudes’ search (see Additional file 1: Table S1), 20 for the ‘environmental behaviour’ search (see Additional file 1: Table S2), and 18 for the ‘knowledge assessment’ search (see Additional file 1: Table S3). Due to the inherent overlap between the environmental attitude and behaviour literatures, these two shortlists shared four of the same records.

Using Google Scholar, a final search was conducted by combining terms across the previous searches. This search aimed to capture scales or approaches that may have been applied to more than one of environmental attitudes, behaviour and knowledge. Additionally, this search attempted to identify tools that have been used to capture attitudes, behaviour and knowledge across other fields.

Results

There is considerable debate in the field of environmental attitude, knowledge and behaviour, particularly concerning the modelling and measurement of these concepts. Naturally, this has significant implications for the evaluation of impact of participation in citizen science.

Environmental attitudes

Defining and conceptualising the terms ‘environment’ and ‘attitude’ (also termed ‘environmental concern’) occupies a large section of the literature in this field. In general, it is agreed that environmental attitudes are comparable to attitudes to other topics. There is now significant consensus that environmental attitudes are multi-dimensional, but reflect a single overall attitude to the environment [34]. Cruz and Mantana ([28], p. 2) therefore term the concept “a hierarchical attitude system that connects and organizes more specific attitudes about a range of environmental topics”. For example, in one of the fields seminal papers, Schultz [98] highlighted three dimensions of environmental attitude: egoistic (concern for self), altruistic (concern for others), and biospheric (concern for the biosphere). Other dimensions have also been postulated by others. Furthermore, despite this general consensus of a hierarchical model of environmental attitude, there are scholars who conceptualise environmental attitudes differently. Dunlap and Jones [34] highlight some papers which suggest that beliefs, intentions, and attitudes are strongly intertwined, and form a key part of (environmental) behaviour.

Difficulty in creating a unified definition and conceptualisation of ‘environmental attitude’ has led to issues with measuring it. These issues are widely reported, with some being directly caused by a poor definition of the term and invalid dimensions [34]. Additionally, the methodology used when measuring environmental attitudes has also been the focus of a large portion of the literature. Self-reporting (in surveys and questionnaires) has been criticised by many, due to the inherent biases caused [69].

Considering this wide ranging conceptual and methodological debate, it is unsurprising that there is a plethora of available approaches, methods and surveys to measure environmental attitude. In a seminal review paper, Cruz and Mantana [28] identified and examined 26 of the most commonly used scales, to identify the most valid. They identified the Ecology Scale from Maloney and Ward [80] and Schultz’s three-dimensional Scale (2001) as the most valid scales for measuring environmental attitudes.

However, arguably the most comprehensive and (currently) widely used scale is based on the New Ecological Paradigm (NEP) model [32, 33]. The NEP scale assesses attitude across a range of environmental topics, in addition to measuring beliefs, intentions and behaviours. This makes scales based on the NEP model highly practical when measuring environmental attitudes, as the model covers a range of topics and concepts relating to attitudes. The NEP model has therefore resulted in a set of scales, which have been widely adapted within the field (e.g. [21, 48, 83]).

Environmental behaviour

Inherently, there is significant overlap between the fields of environmental behaviour and environmental attitude (as seen by the overlap of papers identified during the literature review), and separating the two from each other has proved difficult and contentious. As with environmental attitudes, dimensions of environmental behaviour have often been a source of disagreement in the literature. For example, there is still significant debate as to whether environmental behaviour is uni- or multi-dimensional (Kaiser and Wilson 2004) [74]. For example, from their study of various pro-environmental behaviours, Kaiser and Wilson (2004) developed a six-dimensional model of behaviour (energy conservation, mobility and transportation, waste avoidance, consumerism, recycling, and vicarious conservation behaviours). Furthermore, a range of studies have also attempted to draw parallels between environmental behaviour and various personality traits, such as openness to experience [19], cognitive flexibility [73], and tendency towards abstract thinking []. Regardless, it is generally agreed, however, that while environmental attitudes are linked to behaviour, strong pro-environmental attitudes do not necessarily lead to corresponding behaviours (as other influencing factors are often present). It is therefore necessary to measure the two separately, with separate scales.

One of the most significant recent developments in the field is the increasingly interdisciplinary nature of the research. Previously, psychology and sociology were relatively separated in this area, whereas current literature is now attempting to reconcile these disparate strands of research, and develop coherent frameworks. Batel et al. [13] provide an in-depth review of this literature, highlighting how wider social changes can interact with psychological processes to influence environmental behaviour. In particular, they compare Social Representations Theory with Social Practices Theory and develop a wider theoretical model to understand behaviour change. This literature is also reinforced by research into how environmental behaviours are associated with broader changes in lifestyle and society [99].

The measurement of environmental behaviour (and the scales required to do so) has also generated a significant portion of literature. This can generally be done in three ways: observation in the field; laboratory observation; or self-reporting. Observation in the field has generally taken the shape of retrospectively assessing past behaviours, for example by analysing prior energy usage or transportation choices [4]. Laboratory observations generally refer to situations or choice-making within a controlled environment [26]. However, these two methodologies are rarely used, with self-report questionnaires being the most used approach to measuring environmental behaviour. As with the measurement of environmental attitudes, there is significant debate around the validity of the self-report approach when assessing environmental behaviour. However, some recent studies have suggested that there is a significant correlation between self-reported and directly observed environmental behaviour [69], lending support to the use of such methodology. However, Kormos and Gifford [69] also emphasised that a large portion of variance in the association between self-reported and objective behaviour remains unexplained, meaning that caution is required when utilising this approach.

Nevertheless, self-reporting is necessary for the majority of research in this field. The study from Kormos and Gifford [69] analysed many of the most frequently used methodologies, and highlighted those most highly correlated with directly observed behaviour. Scales utilised by Kaiser et al. [63], Vadez et al. [115] and Corral-Verdugo and Figueredo [27] appeared to provide the most valid results, suggesting that they may form a useful basis for future self-report studies of environmental behaviour. Kormos and Gifford [69] also propose several suggestions for improving self-report scales—these lessons learned could be used to adjust and improve these past scales.

A further issue to consider is that environmental behaviour is not a single, monolithic concept, but is interdimensional (as previously stated). This is revealed when measuring behaviour. Several studies have suggested that individuals can be relatively inconsistent in environmental-related behaviour. For example, an individual may behave in an environment-friendly manner when it comes to dietary choices, but may often select environmentally damaging modes of transport [42]. Therefore, when measuring environmental behaviour (and making claims about the results of studies), one must be wary that a wide range of environmental behavioural dimensions are covered, or the scope of the study should remain context specific. Considering this, most current measures of environmental behaviour take a broad, general approach to measuring environmental behaviour, with a wide range of items.

Knowledge and environmental knowledge

The literature on the assessment of knowledge and learning outcomes is comprehensive, and has often been grounded in broader frameworks of learning. The key goals of learning (and thus assessment) have been captured in Blooms Taxonomy (1956): knowledge, comprehension, application, analysis, synthesis, and evaluation. These goals each represent a different level of learning and understanding; assessments should therefore be designed to identify at which level a person is operating. While there have been new taxonomies developed since Bloom’s Taxonomy (for example, that of Koedinger et al. [68]), these are the key concepts which still ground learning and assessment.

The fundamental goal of the assessment of learning has been conceptualised in a variety of ways, but the definition from Blythe et al. (16, p. 63) is one of the most commonly cited: “Performances of understanding require students to show their understanding in an observable way. They make students’ thinking visible. It is not enough for students to reshape, expand, extrapolate from, and apply their knowledge in the privacy of their own thoughts…Such an understanding would be untried, possibly fragile, and virtually impossible to assess”.

Assessment of knowledge should not simply be seen as a ‘one way street’, however. Although assessment has long been seen as a ‘normative’ process, many argue should be ongoing or formative, providing students with feedback about their work and also allowing both teacher and students to assess progress towards understanding [10]. Watling and Ginsburg [121] highlight how assessment is a learning opportunity for those taking part, as well as for instructors or teachers. It should allow for feedback and an understanding of how to improve learning in the future.

While the broader knowledge assessment literature sheds much needed light on the theoretical background of learning outcome measurement, the literature is largely based on research within academic settings, and is aimed at improving assessment within schools. It is also largely theoretical and does not highlight particular approaches or scales that could be used to measure learning from participation in citizen science. For this reason, a further search was conducted focusing on the measurement of environmental knowledge.

The majority of the literature into environmental knowledge has been written with the aim to assess the influence of knowledge on environmental behaviours. It is generally accepted that environmental knowledge contributes to sustainable or environmentally conscious behaviour [51, 96], but that knowledge alone does not lead to this behaviour [38]. A portion of the literature goes further than this debate, and investigates how environmental knowledge itself can be measured, and how environmental knowledge can change over time.

Traditionally, the educational and psychological fields split knowledge into declarative knowledge (factual knowledge) and procedural knowledge (skills that transform declarative knowledge into action [76]. Frick et al. [38] further developed these dimensions, and specified them for environmental knowledge: system knowledge (e.g. understand the basic structural and functional characteristics of an ecosystem), action-related knowledge (e.g. understand solutions for environmental issues); and effectiveness knowledge (e.g. understand the benefit of sustainable actions). This framework is now commonly used, and lends itself to environmental studies, as it allows the assessment of environmental core knowledge, as well as knowledge relevant for achieving behavioural goals related to sustainability (which is often the desired outcome of a training or intervention).

Braun and Dierkes [18] used these measures to create a framework with which to assess environmental knowledge before and after an intervention. In the study, the framework was used to measure various areas of environmental knowledge (e.g. water, conservation, renewables, etc.) in a group of participants before and after an intervention. This multi-dimensional framework and approach to environmental knowledge measurement has also been successfully implemented in a similar study by Liefländer et al. [77]. Both of these studies have developed similar scales with which to measure environmental knowledge, which can be used and adapted for future studies of environmental knowledge, including in the context of citizen science projects.

A further challenge in (environmental) education is to determine the best way in which knowledge can be self-reported. It is common in environmental research to use confidence or agreement ratings that self-report one’s own knowledge, i.e. “I can explain what the term ecology means,” [31]. It is often suggested that these tests do not measure actual knowledge, and are more just a representation of subjective knowledge [82]. More direct knowledge assessment approaches are now used more frequently.

Knowledge, Attitude and Practice surveys

Knowledge, Attitude and Practice (KAP) surveys measure attitudes, behaviour and knowledge using a single instrument. These surveys have been used across a range of topics and sectors, primarily within health sciences [75], but also in water and sanitation [101], building design [1], and communication sciences [85]. The KAP survey design is a broad conceptual framework, which can be used to develop surveys to assess what is known (knowledge), believed (attitude), and done (practiced) by a target group. As outlined by Andrade et al. [8], the KAP survey framework first requires the implementer to determine what knowledge, attitudes and practices they are expecting from their target group. Based on this, they must then devise several questions (open and/or closed) which they feel examine relevant knowledge, attitudes and practices. For example, one could examine understanding of common misconceptions related to personal hygiene to further understand the knowledge of a participant. Following this, the scoring of the survey needs to be determined according to the type of question, with a separate overall score for each of the three constructs: knowledge, attitude and practice. Implementers are also advised to pilot the survey before widespread use. Using this step-by-step framework, KAP surveys have become widespread to provide a baseline measurement for knowledge, attitudes and practice. They can also be used to measure the impact of an intervention, including of citizen science projects.

Best practice indicators and approaches for assessing attitudes, knowledge and behaviour in citizen science

As highlighted in the previous section, there is a plethora of information and research regarding the measurement of environmental attitudes, behaviour and knowledge. Yet this literature (and the approaches and scales developed within it) has not yet been fully incorporated into the field of citizen science. Data have been collected from individuals that have engaged in citizen science activities in the past [70], and several past studies have even investigated the impact of engagement with citizen science on attitude [17] behaviour and knowledge [113]. However, the scales used within these studies did not represent the current state of the art of the literature (as identified by this literature review).

Future impact evaluation frameworks should endeavour to integrate the findings of this (and similar) literature reviews, to ensure that accessible and high-quality impact assessment approaches are easily available to citizen science practitioners. The analysis within this literature review has already led to the identification of a number of indicators and approaches for measuring environmental attitudes, behaviour and knowledge which have been included in the MICS Conceptual Framework for the evaluation of citizen science impacts on the environment and society [122].

One of the central discussions identified from the literature (and relevant to citizen science) is whether environmental attitudes, behaviour and knowledge should (or even can) be measured with a single, unified scale or approach. Several theories, such as the Theory of Planned Behaviour [5], suggest that attitudes (which are made up of beliefs) are closely related to behaviour, while evidence has also suggested that knowledge and attitudes are linked [125]. However, despite these relatively high-level psychological theories linking attitudes, behaviour and knowledge, the literature focusing on environmental psychology generally separates the concepts. Therefore, the scales and approaches identified in this literature review are generally specific to measuring one of the three concepts: environmental attitudes, environmental behaviour, or environmental knowledge.

In the field of environmental attitude, there is (to a large degree) currently a consensus on the most valid methods to use when measuring attitude. As outlined previously, there are a range of scales that have been used to measure environmental attitudes. Over the history of the literature, the field has been relatively fragmented, with studies often creating new scales with which to measure environmental attitudes. Despite this, the three most commonly used (and adapted) scales are the Ecology Scale from Maloney and Ward [80], Schultz’s three-dimensional Scale (2001) and the New Ecological Paradigm (NEP) scale [32, 33]. These scales (along with other prominent scales in the field) and the attitudinal dimensions that they identified, have been incorporated into the most comprehensive method currently available in the field, the Environmental Attitudes Inventory (EAI) [83]. The EAI offers a 12-dimensional approach to measuring environmental attitudes, has been used across a range of contexts and has been found to be highly consistent and reliable.

Several other scales have been developed recently, and show promise in the measurement of environmental attitudes. One of the most prominent of these is the Sustainability Attitudes Scale (SAS) [125]. This scale used the three-domain definition of sustainability, looking at attitudes to: Ecological Sustainability, Social Sustainability Subscale; and Economic Sustainability. While testing of this scale generally been found it to be valid, it is still relatively new and has not been used as extensively as other scales. While there is a large amount of overlap between the two, this should be considered when selecting a scale. Despite these flaws, this scale could offer a possible method for measuring environmental attitude within citizen science in the future.

Considering the state of the literature, the current use of attitude measurement scales, and the particular needs for citizen science, the Environmental Attitudes Inventory (EAI) [83] was identified as the most applicable for measuring environmental attitudes within citizen science, due to its comprehensive nature and application of the 12 facets of environmental attitudes identified by the authors. However, due to the length of this instrument, citizen science practitioners should consider the added value of such an in-depth scale alongside the additional resources required to implement it. Alternatively, there are several shortened versions of the inventory that can also be utilised. Due to its prevalence across the literature, strong validity, and short, simple nature, Schultz’s Three Dimensional Scale (2001) was identified as an alternative methodology.

A variety of best practices and approaches are also available in the field of measuring environmental behaviour. However, the comprehensive study by Kormos and Gifford [69] listed three approaches that stand out above others. The approaches utilised by Kaiser et al. [63], Vadez et al. [116] and Corral-Verdugo and Figueredo [27] each have benefits for measuring environmental behaviour, and importantly appear to be highly valid when doing so. However, the scales from both Vadez et al. [116] and Corral-Verdugo and Figueredo [27] are highly context specific (measuring behaviour relating to deforestation and recycling, respectively). The scale used by Kaiser et al. [63]—which was adapted from the “General Measure of Ecological Behaviour” [61]—covers a range of different behaviours, and achieved the highest degree of validity in this large study. Variants of the Kaiser et al. [63] scale have been often adopted by following researchers in the field. The most updated of these scales is the ‘General Ecological Behavior Scale—50’ [62]. This scale is generally the most widely used when measuring environmental behaviour, as well as being the most flexible (in terms of the various behaviours and dimensions assessed). For this reason, it provides the most promise to those aiming to measure environmental behaviour within citizen science.

There is also little consensus regarding best practice when measuring environmental knowledge. One of the most commonly used frameworks is the three-dimensional theory of environmental knowledge, separating knowledge into system, action-related and effectiveness dimensions. Assessment should therefore reflect these dimensions. The assessment method used by Braun and Dierkes [18] does this well, and can measure environmental knowledge across a broad range of topics. This scale therefore can be easily adapted to measure environmental knowledge within citizen science. An alternative scale could be the Assessment of Sustainability Knowledge (ASK) [125]. As this scale was developed alongside the Sustainability Attitudes Scale (SAS), use of both scales would allow for the measurement of environmental knowledge and attitude using the same theoretical framework. However, neither have yet been widely tested.

Discussion

This literature review outlined the current state of the art in the measurement of environmental attitudes, behaviour and knowledge. Driven by the need to inform impact evaluation of citizen science, this review built crucial links with the field of environmental psychology, setting up the basis for an improved understanding of the impact of citizen science on environmental attitudes, behaviour and knowledge.

Five scales emerged as relevant for citizen science projects to measure environmental attitudes, behaviour and knowledge, namely: the Environmental Attitudes Inventory (EAI) [83], the Three Dimensional Scale [98], the General Ecological Behavior Scale—50 [62], the three-dimensional theory of environmental knowledge, used by Braun and Dierkes [18], and the Assessment of Sustainability Knowledge (ASK) [125]. These scales were selected based on their positioning as state-of-the-art approaches, as well as their applicability to citizen science projects.

While these selected approaches represent the state of the art in their respective fields, due to the unique context of citizen science projects a variety of factors can influence the requirements of impact assessment tools. For example, with a large number and (often) wide geographical spread of citizens engaging in citizen science projects, face-to-face or workshop style evaluations are often impractical, meaning that questionnaires and surveys can provide an alternative. Additionally, the demographics and expertise of the participants should be considered when selecting an approach. For example, a survey that is appropriate for adult participants in citizen science may not be practical for children. Finally, the (often limited) resources of citizen science projects should also be considered. Assessment approaches should therefore not be resource intensive, usable by non-experts and experts alike, and should be as simple as possible to implement. Considering these various factors, for the purposes of citizen science scales often provide optimum utility when they are flexible—practitioners should be able to adapt these scales, adding or removing questions where required. For example, the environmental knowledge scale provided by Braun and Dierkes [18] covers knowledge of a range of topics, however it does not explicitly measure knowledge of pollution. Naturally for a citizen science initiative with a focus on pollution measurement, this is a key topic that should be measured. Using the theory provided by Braun and Dierkes [18], practitioners can therefore insert questions covering the knowledge of pollution across the three dimensions (system knowledge, action-related knowledge and effectiveness knowledge). In this way, practitioners can keep the generalizable nature of their results, while also generating results meaningful to their specific context.

Being widely used in other fields to assess attitude, knowledge and behaviour, KAP surveys present a further possible option for citizen science practitioners. However, for the purposes of citizen science practitioners, KAP surveys pose several issues. Firstly, various methodological concerns have been raised about KAP surveys since their widespread use, particularly around the relatively simplistic conceptualizations of attitudes, behaviour and knowledge [2], especially when compared with the highly valid models previously highlighted (e.g. [18]). Furthermore, KAP surveys are generally designed to be highly context specific (e.g. [79]). This means that there are very few KAP instruments that practitioners can adopt easily and comparison across survey results is difficult, if not impossible. Additionally, KAP surveys require resources to be designed and implemented [8]. For example, to design a KAP survey a practitioner must have a deep understanding of the field in order to decide what knowledge, attitudes and practices citizen scientists might acquire during the project. However, if an initiative has the resources to develop a KAP survey, this option can provide a generally accepted step-by-step method for developing high quality scales for environmental attitudes, behaviour and knowledge. The exploration of KAP survey approaches by citizen science initiatives, including the construction of generic building blocks for citizen science applications, may therefore offer a future avenue for researchers and practitioners of citizen science.

The findings of this paper provide a clear advancement in the evaluation and assessment of citizen science impact. Due to the cross-disciplinary nature of the topic, citizen science practitioners often need to draw from different fields in order to add structure to outcomes and findings. However, the integration of environmental psychology within citizen science still requires significant work. This review begins this process by highlighting approaches which can significantly improve the impact assessment and understanding of citizen science, and also brings the field of citizen science in line with the state of the art in environmental psychology. Further work is required to fully integrate these approaches into practice in citizen science.

For this integration of new scales and methodologies to properly take place, citizen science practitioners first need to understand and familiarise themselves with these approaches, in order to well implement them. The few prior evaluations that did assess environmental attitude, behaviour or knowledge, did not fully assess the range of assessment approaches available, or simply created a new scale not grounded in the current literature (e.g. [24, 49, 110]). Practitioners should also understand the resources required to implement these assessment approaches, as well as the organizational efforts necessary. Although practitioners benefit from increasingly profound data offered by the highlighted impact assessment methods, it should be noted that the more complex the analysis, the greater the resources and expertise required [65]. The utilization of these methods can be a relatively complex process, some scales may be more advantageous than others in specific contexts (for example, depending on the demographics of the participants), and the analysis of the results can require significant resources.

The findings of this review also emphasise the fact that wider changes to the structure of the citizen science design process (beyond mere resource allocation) are required in order to fully assess the impact of these initiatives. As already implemented by some initiatives (e.g. [90]) impact assessment should first be considered even before the beginning of citizen science projects and from there embedded within the citizen science project management. This is largely because many impact measurement tools (particularly those offering high-quality data) require baseline measurements before participation. This is particularly the case for tools measuring environmental attitudes, behaviour and knowledge, such as those highlighted by this literature review. Therefore, these approaches should not simply be added as an afterthought, but should become a key part of the citizen science process. However, as previously mentioned, the additional resources required by such approaches must also be considered from the beginning of the process.

The integration of these assessment methods within citizen science can be supported by their inclusion within broader impact frameworks, providing a roadmap for the impact assessment of future citizen science initiatives. This is already being done—these tools were highlighted in the impact assessment framework outlined by Wehn et al. [122], allowing for citizen science practitioners to easily select specific approaches for measuring their impact area of interest. Such integration efforts should be continued in the future to further support citizen science practitioners in their evaluation efforts.

By using these approaches to measure citizen science impact, a range of benefits can be expected. With improved approaches to project evaluation, a greater understanding of citizen science could be obtained, supporting relevant policy and research in the future. Furthermore, these approaches can be useful for understanding design principles of citizen science projects, for example the aspects of initiatives that foster environmental attitude, knowledge or behaviour. In future, these aspects could be tailored to ensure that the most benefit is gained.

In the broader picture, citizen science is often highlighted as a route to achieving a range of environmental and social goals, including up to 76 of the 231 SDG indicators [36]. However, to fully assess the influence of citizen science in achieving these goals, the impacts need to be fully understood and accurately measured [90]. By using state-of-the-art evaluation tools of high scientific quality, the real potential of citizen science can be realised, and its contribution to solving societal and environmental issues can be seen. While these measures identified in this literature review provide an insight into the attitudes, behaviour and knowledge of individuals, they do little to offer understanding of wider, collective social changes encapsulated within the SDGs. To realise the full impact of citizen science on a broader scale, alternative methodologies should be examined and used in conjunction with those identified in this review.

Conclusion

The findings from the literature review link the previously separate fields of environmental attitudes, behaviour and knowledge, with citizen science impact assessment. From the literature review, five approaches which represent the state of the art in their respective fields, were identified as suitable for measuring the impact of participation in citizen science on environmental attitudes, behaviour and knowledge. By bridging the fields of citizen science with environmental psychology and highlighting concrete methods for citizen science projects, this paper provided concrete insights for citizen science practitioners aiming to identify the impact of their project on participants. It also highlights the various efforts, considerations and changes in approach required to embed these impact assessment approaches within citizen science.

This literature review highlights the balance that needs to be struck by citizen science initiatives in impact assessment—namely that generalizable, state-of-the-art assessments should take place, while at the same time providing meaningful, context specific insight into the effects of the initiative [122]. The challenge for citizen science practitioners is to achieve both. Practitioners can accomplish this by using the results from this literature review—as well as the suggested methods of implementation.

The findings from this literature review can also feed in to future frameworks of citizen science, in addition to individual impact assessments. The principles of this review have already been used to inform an updated framework of citizen science impact assessment [122]. Where relevant, impact assessment, and specific tools and scales for measuring impact, should be included in all future citizen science frameworks, as well as an indication of the required resources for their implementation.

While this study outlines a comprehensive view of the current state of the art concerning the measurement of environmental attitudes, behaviour and knowledge, citizen science projects should also be aware of the fast moving nature of the field. These approaches are regularly updated (e.g. [62]), while new and innovative approaches are also often published [125]. It should therefore be ensured that the state of the art is represented when conducting impact assessment.

Availability of data and materials

The data for this article can be found in the Tables section of the document.

Change history

References

  1. Abdelaal F, Guo BH (2021) Knowledge, attitude and practice of green building design and assessment: New Zealand case. Build Environ 201:107960

    Google Scholar 

  2. Abir T, Ekwudu OM, Kalimullah NA, Nur-A Yazdani DM, Al Mamun A, Basak P, Osuagwu UL, Permarupan PY, Milton AH, Talukder SH, Agho KE (2021) Dengue in Dhaka, Bangladesh: Hospital-based cross-sectional KAP assessment at Dhaka North and Dhaka South City Corporation area. PLoS ONE 16(3):e0249135

    CAS  Google Scholar 

  3. Abrahamse W, Matthies E (2012) Informational strategies to promote pro-environmental behaviour: changing knowledge, awareness and attitudes. In: Environmental psychology: An introduction, pp 223–232.

  4. Abrahamse W, Steg L, Vlek C, Rothengatter T (2007) The effect of tailored information, goal setting, and tailored feedback on household energy use, energy-related behaviors, and behavioral antecedents. J Environ Psychol 27(4):265–276

    Google Scholar 

  5. Ajzen I (1991) The theory of planned behavior. Organ Behav Hum Decis Process 50(2):179–211

    Google Scholar 

  6. Alriksson S, Öberg T (2008) Conjoint analysis for environmental evaluation. Environ Sci Pollut Res 15(3):244–257

    Google Scholar 

  7. Al-Shemmeri T, Naylor L (2017) Energy saving in UK FE colleges: The relative importance of the socio-economic groups and environmental attitudes of employees. Renew Sustain Energy Rev 68:1130–1143

    Google Scholar 

  8. Andrade C, Menon V, Ameen S, Kumar Praharaj S (2020) Designing and conducting knowledge, attitude, and practice surveys in psychiatry: Practical guidance. Indian J Psychol Med 42(5):478–481

    Google Scholar 

  9. Ardoin, N. M., Bowers, A. W., & Gaillard, E. (2020). Environmental education outcomes for conservation: A systematic review. Biological Conservation, 241, 108224.

  10. Baird JA, Andrich D, Hopfenbeck TN, Stobart G (2017) Assessment and learning: Fields apart? Assessment in Education: Principles, Policy & Practice 24(3):317–350

    Google Scholar 

  11. Barr S, Gilg AW (2007) A conceptual framework for understanding and analyzing attitudes towards environmental behaviour. Geografiska Annaler: Series B, Human Geography 89(4):361–379

    Google Scholar 

  12. Barrable A, Booth D (2020) Increasing nature connection in children: A mini review of interventions. Front Psychol 11:492

    Google Scholar 

  13. Batel S, Castro P, Devine-Wright P, Howarth C (2016) Developing a critical agenda to understand pro-environmental actions: contributions from Social Representations and Social Practices Theories. Wiley Interdisciplinary Reviews: Climate Change 7(5):727–745

    Google Scholar 

  14. Biel A, Thøgersen J (2007) Activation of social norms in social dilemmas: A review of the evidence and reflections on the implications for environmental behaviour. J Econ Psychol 28(1):93–112

    Google Scholar 

  15. Black JS, Stern PC, Elworth JT (1985) Personal and contextual influences on household energy adaptations. J Appl Psychol 70:3–21

    Google Scholar 

  16. Blythe T, Croft A, Strelec N (1998) Teaching for understanding. Paidós: Teacher’s Guide Buenos Aires

  17. Bonney R, Cooper CB, Dickinson J, Kelling S, Phillips T, Rosenberg KV, Shirk J (2009) Citizen science: a developing tool for expanding science knowledge and scientific literacy. Bioscience 59(11):977–984

    Google Scholar 

  18. Braun T, Dierkes P (2019) Evaluating three dimensions of environmental knowledge and their impact on behaviour. Res Sci Educ 49(5):1347–1365

    Google Scholar 

  19. Brick C, Lewis GJ (2016) Unearthing the “green” personality: core traits predict environmentally friendly behavior. Environ Behav 48(5):635–658

    Google Scholar 

  20. Brossard D, Lewenstein B, Bonney R (2005) Scientific knowledge and attitude change: the impact of a citizen science project. Int J Sci Educ 27(9):1099–1121

    Google Scholar 

  21. Castro P (2006) Applying social psychology to the study of environmental concern and environmental worldviews: Contributions from the social representations approach. Journal of Community & Applied Social Psychology 16(4):247–266

    Google Scholar 

  22. Ceccaroni, L., Bowser, A., & Brenton, P. (2017). Civic education and citizen science: Definitions, categories, knowledge representation. In Analyzing the role of citizen science in modern research (pp. 1–23). IGI Global.

  23. Chang HY, Lin TJ, Lee MH, Lee SWY, Lin TC, Tan AL, Tsai CC (2020) A systematic review of trends and findings in research employing drawing assessment in science education. Stud Sci Educ 56(1):77–110

    Google Scholar 

  24. Chase SK, Levine A (2018) Citizen science: exploring the potential of natural resource monitoring programs to influence environmental attitudes and behaviors. Conserv Lett 11(2):e12382

    Google Scholar 

  25. Cologna, V., & Siegrist, M. (2020). The role of trust for climate change mitigation and adaptation behaviour: A meta-analysis. Journal of Environmental Psychology, 101428.

  26. Corral-Verdugo V (1997) Dual ‘Realities’ of conservation behavior: self-reports vs observations of re-use and recycling behavior. J Environ Psychol 17(2):135–145

    Google Scholar 

  27. Corral-Verdugo V, Figueredo AJ (1999) Convergent and divergent validity of three measures of conservation behavior: The multitrait-multimethod approach. Environ Behav 31(6):805–820

    Google Scholar 

  28. Cruz SM, Manata B (2020) Measurement of environmental concern: A review and analysis. Front Psychol 11:363

    Google Scholar 

  29. De Young R (1996) Some psychological aspects of reduced consumption behavior: The role of intrinsic satisfaction and competence motivation. Environ Behav 28(3):358–409

    Google Scholar 

  30. Dobson A (2003) Citizenship and the environment. OUP Oxford

  31. Duerden MD, Witt PA (2010) The impact of direct and indirect experiences on the development of environmental knowledge, attitudes, and behavior. J Environ Psychol 30(4):379–392

    Google Scholar 

  32. Dunlap RE, Catton WR Jr (1979) Environmental sociology. Ann Rev Sociol 5(1):243–273

    Google Scholar 

  33. Dunlap RE, Van Liere KD, Mertig AG, Jones RE (2000) New trends in measuring environmental attitudes: measuring endorsement of the new ecological paradigm: a revised NEP scale. J Soc Issues 56(3):425–442

    Google Scholar 

  34. Dunlap, R. E., and Jones, R. E. (2002). “Environmental concern: conceptual and measurement issues,” in Handbook of Environmental Sociology, eds R. E. Dunlap, and W. Michelson, (Westport, CT: Greenwood Press), 482–524.

  35. Entwistle N, Smith C (2002) Personal understanding and target understanding: Mapping influences on the outcomes of learning. Br J Educ Psychol 72(3):321–342

    Google Scholar 

  36. Fraisl D, Campbell J, See L, Wehn U, Wardlaw J, Gold M, Moorthy I, Fritz S (2020) Mapping citizen science contributions to the UN sustainable development goals. Sustain Sci 15(6):1735–1751

    Google Scholar 

  37. Franzen A, Vogl D (2013) Two decades of measuring environmental attitudes: A comparative analysis of 33 countries. Glob Environ Chang 23(5):1001–1008

    Google Scholar 

  38. Frick J, Kaiser FG, Wilson M (2004) Environmental knowledge and conservation behavior: Exploring prevalence and structure in a representative sample. Personality Individ Differ 37(8):1597–1613

    Google Scholar 

  39. Gao X, Li P, Shen J, Sun H (2020) Reviewing assessment of student learning in interdisciplinary STEM education. International Journal of STEM Education 7:1–14

    Google Scholar 

  40. Garvill J (1999) Choice of transportation mode: factors influencing drivers’ willingness to reduce personal car use and support car regulations. In: Foddy M, Smithson M, Schneider S, Hogg M (eds) Resolving social dilemmas: dynamic, structural, and intergroup aspects. Psychology Press, Philadelphia, pp 263–279

    Google Scholar 

  41. Gasparatos A (2010) Embedded value systems in sustainability assessment tools and their implications. J Environ Manage 91(8):1613–1622

    Google Scholar 

  42. Gatersleben B, Steg L, Vlek C (2002) Measurement and determinants of environmentally significant consumer behavior. Environ Behav 34(3):335–362

    Google Scholar 

  43. Gatersleben B, Murtagh N, Abrahamse W (2014) Values, identity and pro-environmental behaviour. Contemporary Social Science 9(4):374–392

    Google Scholar 

  44. Grankvist G (2002) Determinants of choice of eco-labeled products. Unpublished Doctoral dissertation, Göteborg University, Göteborg

  45. Grilli, G., & Curtis, J. (2021). Encouraging pro-environmental behaviours: A review of methods and approaches. Renewable and Sustainable Energy Reviews, 135, 110039.

  46. Haklay, M. (2013). Citizen science and volunteered geographic information: Overview and typology of participation. Crowdsourcing geographic knowledge, 105–122.

  47. Ham M, Mrčela D, Horvat M (2016) Insights for measuring environmental awareness. Ekon Vjesnik 29(1):159–176

    CAS  Google Scholar 

  48. Hawcroft LJ, Milfont TL (2010) The use (and abuse) of the new environmental paradigm scale over the last 30 years: a meta-analysis. J Environ Psychol 30(2):143–158

    Google Scholar 

  49. Haywood BK, Parrish JK, Dolliver J (2016) Place‐based and data‐rich citizen science as a precursor for conservation action. Conserv Biol 30(3):476–486

    Google Scholar 

  50. Heberlein TA, Black JS (1976) Attitudinal specificity and the prediction of behavior in a field setting. J Person Soc Psychol 33:474–479

    Google Scholar 

  51. Heimlich JE, Ardoin NM (2008) Understanding behavior to understand behavior change: a literature review. Environ Educ Res 14(3):215–237

    Google Scholar 

  52. Hopper JR, Nielsen JM (1991) Recycling as altruistic behavior: normative and behavioral strategies to expand participation in a community recycling program. Environ Behav 23(2):195–220

    Google Scholar 

  53. Hunecke M, Blöbaum A, Matthies E, Höger R (2001) Responsibility and environment: ecological norm orientation and external factors in the domain of travel mode choice behavior. Environ Behav 33:830–852

    Google Scholar 

  54. Hurst M, Dittmar H, Bond R, Kasser T (2013) The relationship between materialistic values and environmental attitudes and behaviors: A meta-analysis. J Environ Psychol 36:257–269

    Google Scholar 

  55. Irwin A (1995) Citizen science a study of people, expertise and sustainable development. Routledge

  56. Irwin A, Horst M (2015) Engaging in a decentred world: overflows, ambiguities and the governance of climate change. In: Chilvers J, Kearnes M (eds) Remaking participation. Science, Environment and Emergent Publics. https://doi.org/10.4324/9780203797693

  57. Jacquet J, Dietrich M, Jost JT (2014) The ideological divide and climate change opinion: “Top-down” and “bottom-up” approaches. Front Psychol 5:1458

    Google Scholar 

  58. Jagers SC, Martinsson J, Matti S (2016) The environmental psychology of the ecological citizen: comparing competing models of pro-environmental behavior. Soc Sci Q 97(5):1005–1022

    Google Scholar 

  59. Jin H, Mikeska JN, Hokayem H, Mavronikolas E (2019) Toward coherence in curriculum, instruction, and assessment: a review of learning progression literature. Sci Educ 103(5):1206–1234

    Google Scholar 

  60. Jordan RC, Ballard HL, Phillips TB (2012) Key issues and new approaches for evaluating citizen-science learning outcomes. Front Ecol Environ 10(6):307–309

    Google Scholar 

  61. Kaiser FG (1998) A general measure of ecological behavior 1. J Appl Soc Psychol 28(5):395–422

    Google Scholar 

  62. Kaiser FG (2020) GEB-50. General Ecological Behavior Scale

  63. Kaiser FG, Frick J, Stoll-Kleemann S (2001) Zur angemesseheit selbstberichteten verhaltens: Eine validitätsuntersuchung der Skala Allgemeinen Ökologischen Verhaltens. Diagnostica

  64. Kaiser FG, Oerke B, Bogner FX (2007) Behavior-based environmental attitude: development of an instrument for adolescents. J Environ Psychol 27(3):242–251

    Google Scholar 

  65. Kieslinger B, Schäfer T, Heigl F, Dörler D, Richter A, Bonn A (2017) The challenge of evaluation: An open framework for evaluating citizen science activities

  66. Kim JW, Ritter FE (2019) Consideration of a Bayesian hierarchical model for assessment and adaptive instructions. In: International Conference on Human-Computer Interaction. Springer, Cham, pp 521–531.

  67. Knight S, Shum SB (2017) Theory and learning analytics. In: Handbook of learning analytics, pp 17–22.

  68. Koedinger KR, Corbett AT, Perfetti C (2012) The Knowledge-Learning-Instruction framework: bridging the science-practice chasm to enhance robust student learning. Cogn Sci 36(5):757–798

    Google Scholar 

  69. Kormos C, Gifford R (2014) The validity of self-report measures of proenvironmental behavior: a meta-analytic review. J Environ Psychol 40:359–371

    Google Scholar 

  70. Korn R (2014) Summative Evaluation: Citizen Science Program. http://informalscience.org/images/evaluation/2010_CTPR_RKA_Citizen_Science_dist.pdf

  71. Kothe EJ, Ling M, North M, Klas A, Mullan BA, Novoradovskaya L (2019) Protection motivation theory and pro-environmental behaviour: a systematic mapping review. Aust J Psychol 71(4):411–432

    Google Scholar 

  72. Kullenberg C, Kasperowski D (2016) What is citizen science?–A scientometric meta-analysis. PLoS ONE 11(1):e0147152

    Google Scholar 

  73. Lange F, Dewitte S (2019) Cognitive flexibility and pro–environmental behaviour: a multimethod approach. Eur J Pers 33(4):488–505

    Google Scholar 

  74. Larson LR, Usher LE, Chapmon T (2018) Surfers as environmental stewards: understanding place-protecting behavior at Cape Hatteras National Seashore. Leis Sci 40:442–465. https://doi.org/10.1080/01490400.2017.1305306

    Article  Google Scholar 

  75. Launiala A (2009) How much can a KAP survey tell us about people’s knowledge, attitudes and practices? Some observations from medical anthropology research on malaria in pregnancy in Malawi. Anthropol Matters J 11(1):1–13

    Google Scholar 

  76. Lewis CH, Anderson JR (1976) Interference with real world knowledge. Cogn Psychol 8(3):311–335

    Google Scholar 

  77. Liefländer AK, Bogner FX, Kibbe A, Kaiser FG (2015) Evaluating environmental knowledge dimension convergence to assess educational programme effectiveness. Int J Sci Educ 37(4):684–702

    Google Scholar 

  78. Littledyke M (2008) Science education for environmental awareness: approaches to integrating cognitive and affective domains. Environ Educ Res 14(1):1–17

    Google Scholar 

  79. Lownik E, Riley E, Konstenius T, Riley W, McCullough J (2012) Knowledge, attitudes and practices surveys of blood donation in developing countries. Vox Sang 103(1):64–74

    CAS  Google Scholar 

  80. Maloney MP, Ward MP (1973) Ecology: Let’s hear from the people: an objective scale for the measurement of ecological attitudes and knowledge. Am Psychol 28(7):583

    Google Scholar 

  81. Mansilla VB, Duraising ED (2007) Targeted assessment of students’ interdisciplinary work: an empirically grounded framework proposed. J Higher Educ 78(2):215–237

    Google Scholar 

  82. Metcalfe JS, De Liso N, La Mothe D, Paquet J (1996) Innovation, capabilities and knowledge: the epistemic connection

  83. Milfont TL, Duckitt J (2010) The environmental attitudes inventory: a valid and reliable measure to assess the structure of environmental attitudes. J Environ Psychol 30(1):80–94

    Google Scholar 

  84. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Int Med 151(4):264–269

    Google Scholar 

  85. Moore JC (2010) Evaluating the Public Information Campaign for the 1980 Census-Results of the KAP Survey. Surv Methodol 08:9

    Google Scholar 

  86. Nordlund AM, Garvill J (2003) Effects of values, problem awareness, and personal norm on willingness to reduce personal car use. J Environ Psychol 23:339–347

    Google Scholar 

  87. O’Brien EA (2003) Human values and their importance to the development of forestry policy in Britain: a literature review. Forestry 76(1):3–17

    Google Scholar 

  88. Onel N, Mukherjee A (2016) Consumer knowledge in pro-environmental behavior: An exploration of its antecedents and consequences. World J Sci Technol Sustain Develop 8:96

    Google Scholar 

  89. Palmer C, McShane K, Sandler R (2014) Environmental ethics. Annu Rev Environ Resour 39:419–442

    Google Scholar 

  90. Passani A, Janssen AL, Hoelscher K (2021) Impact assessment framework. Zenodo. https://doi.org/10.5281/zenodo.4432132

  91. Phillips T, Bonney R, Shirk J (2012) What is our impact. Citizen science: Public participation in environmental research. pp 82–95

  92. Phillips T, Ferguson M, Minarchek M, Porticella N, Bonney R (2014) Evaluating Learning Outcomes from Citizen Science

  93. Ren J (2018) The influence of cultural values on the environmental attitudes and behaviours of Chinese outbound tourists

  94. Reser JP, Bentrupperbäumer JM (2005) What and where are environmental values? Assessing the impacts of current diversity of use of ‘environmental’ and ‘World Heritage’ values. J Environ Psychol 25(2):125–146

    Google Scholar 

  95. Robina-Ramírez R, Medina-Merodio JA (2019) Transforming students’ environmental attitudes in schools through external communities. J Clean Prod 232:629–638

    Google Scholar 

  96. Roczen N, Kaiser FG, Bogner FX, Wilson M (2014) A competence model for environmental education. Environ Behav 46(8):972–992

    Google Scholar 

  97. Rosa CD, Collado S (2019) Experiences in nature and environmental attitudes and behaviors: Setting the ground for future research. Front Psychol 10:763

    Google Scholar 

  98. Schultz PW (2001) The structure of environmental concern: Concern for self, other people, and the biosphere. J Environ Psychol 21(4):327–339

    Google Scholar 

  99. Scott A, Oates C, Young W (2015) A conceptual framework of the adoption and practice of environmental actions in households. Sustainability 7(5):5793–5818

    Google Scholar 

  100. Shirk JL, Ballard HL, Wilderman CC, Phillips T, Wiggins A, Jordan R, McCallie E, Minarchek M, Lewenstein BV, Krasny ME, Bonney R (2012) Public participation in scientific research: a framework for deliberate design. Ecol Soc 17(2):e20

    Google Scholar 

  101. Sibiya JE, Gumbo JR (2013) Knowledge, attitude and practices (KAP) survey on water, sanitation and hygiene in selected schools in Vhembe District, Limpopo, South Africa. Int J Environ Res Public Health 10(6):2282–2295

    Google Scholar 

  102. Smajgl A, Ward J (2015) Evaluating participatory research: framework, methods and implementation results. J Environ Manag 157:311–319

    Google Scholar 

  103. Staats HJ, Wit AP, Midden CYH (1996) Communicating the greenhouse effect to the public: evaluation of a mass media campaign from a social dilemma perspective. J Environ Manag 46(2):189–203

    Google Scholar 

  104. Steg L, Vlek C (2009) Encouraging pro-environmental behaviour: An integrative review and research agenda. J Environ Psychol 29(3):309–317

    Google Scholar 

  105. Stern PC, Dietz T, Abel T, Guagnano GA, Kalof L (1999) A value-belief-norm theory ;of support for social movements: the case of environmentalism. Hum Ecol Rev 81–97

  106. Thøgersen J (1999) The ethical consumer. Moral norms and packaging choice. J Consumer Policy 22:439–460

    Google Scholar 

  107. Thøgersen J (2003) Monetary incentives and recycling: behavioral and psychological reactions to a performance-dependent garbage fee. J Consumer Policy 26:197–228

    Google Scholar 

  108. Thøgersen J, Ölander F (2002) Human values and the emergence of a sustainable consumption pattern: a panel study. J Econ Psychol 23:605–630

    Google Scholar 

  109. Tianyu J, Meng L (2020) Does education increase pro-environmental willingness to pay? Evidence from Chinese household survey. J Cleaner Prod 275:122713

    Google Scholar 

  110. Toomey AH, Domroese MC (2013) Can citizen science lead to positive conservation attitudes and behaviors? Hum Ecol Rev 50–62

  111. Topal HF, Hunt DV, Rogers CD (2020) Urban Sustainability and Smartness Understanding (USSU)—identifying influencing factors: a systematic review. Sustainability 12(11):4682

    Google Scholar 

  112. Topal HF, Hunt DV, Rogers CD (2021) Exploring urban sustainability understanding and behaviour: a systematic review towards a conceptual framework. Sustainability 13(3):1139

    Google Scholar 

  113. Turaga RMR, Howarth RB, Borsuk ME (2010) Pro-environmental behavior: rational choice meets moral motivation. Ann N Y Acad Sci 1185(1):211–224

    Google Scholar 

  114. Tweddle JC, Robinson LD, Pocock MJO, Roy HE (2012) Guide to citizen science: developing, implementing and evaluating citizen science to study biodiversity and the environment in the UK. NERC/Centre for Ecology & Hydrology

  115. Udall AM, de Groot JI, de Jong SB, Shankar A (2020) How do I see myself? A systematic review of identities in pro-environmental behaviour research. J Consum Behav 19(2):108–141

    Google Scholar 

  116. Vadez V, Reyes-García V, Godoy R, Williams L, Apaza L, Byron E, Wilkie D (2003) Validity of self-reports to measure deforestation: evidence from the Bolivian lowlands. Field Methods 15(3):289–304

    Google Scholar 

  117. Van Brussel S, Huyse H (2019) Citizen science on speed? Realising the triple objective of scientific rigour, policy influence and deep citizen engagement in a large-scale citizen science project on ambient air quality in Antwerp. J Environ Plan Manag 62(3):534–551

    Google Scholar 

  118. Van Liere KD, Dunlap RE (1978) Moral norms and environmental behavior: an application of Schwartz’s norm-activation model to yard burning. J Appl Soc Psychol 8:174–188

    Google Scholar 

  119. Van Liere KD, Noe FP (1981) Outdoor recreation and environmental attitudes: further examination of the Dunlap-Heffernan thesis. Rural Sociol 46:503–513

    Google Scholar 

  120. Vining J, Ebreo A (1992) Predicting recycling behavior from global and speciWc environmental attitudes and changes in recycling opportunities. J Appl Soc Psychol 22:1580–1607

    Google Scholar 

  121. Wang B, Zhou Q (2020) Climate change in the Chinese mind: an overview of public perceptions at macro and micro levels. Wiley Interdiscip Rev Clim Change 11(3):e639

    Google Scholar 

  122. Watling CJ, Ginsburg S (2019) Assessment, feedback and the alchemy of learning. Med Educ 53(1):76–85

    Google Scholar 

  123. Wehn U, Gharesifard M, Ceccaroni L, Joyce H, Ajates R, Woods S, Bilbao A, Parkinson S, Gold M, Wheatland J (2021) Impact assessment of citizen science: state of the art and guiding principles for a consolidated approach. Sustain Sci 2:1–17

    Google Scholar 

  124. Zlatkin-Troitschanskaia O, Schlax J, Jitomirski J, Happ R, Kühling-Thees C, Brückner S, Pant HA (2019) Ethics and fairness in assessing learning outcomes in higher education. High Educ Pol 32(4):537–556

    Google Scholar 

  125. Zsóka Á, Szerényi ZM, Széchy A, Kocsis T (2013) Greening due to environmental education? Environmental knowledge, attitudes, consumer behavior and everyday pro-environmental activities of Hungarian high school and university students. J Clean Prod 48:126–138

    Google Scholar 

  126. Zwickle A, Jones K (2018) Sustainability knowledge and attitudes—Assessing latent constructs. Handbook of sustainability and social science research. Springer, Cham, pp 435–451

    Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This research has received funding from the EU Horizon 2020 research and innovation programme under project MICS Measuring the Impact of Citizen Science (grant agreement No. 824711).

Author information

Authors and Affiliations

Authors

Contributions

UW developed the concept for the paper. LS conducted the literature review for the research and drafted the manuscript. UW supported in the methodology of the literature review, and contributed to the writing of the manuscript. Both authors read and approve the final manuscript.

Corresponding author

Correspondence to Luke Somerwill.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors have no competing interests relevant for this publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: Tables 4, 5, 6 were moved to Supplementary section.

Supplementary Information

Below is the link to the electronic supplementary material.

Additional File 1:

Shortlisted records for literature review.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Somerwill, L., Wehn, U. How to measure the impact of citizen science on environmental attitudes, behaviour and knowledge? A review of state-of-the-art approaches. Environ Sci Eur 34, 18 (2022). https://doi.org/10.1186/s12302-022-00596-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12302-022-00596-1

Keywords

  • Citizen science
  • Impact assessment
  • Environment
  • Attitudes
  • Behaviour
  • Knowledge