How to measure the impact of citizen science on environmental attitudes, behaviour and knowledge? A review of state-of-the-art approaches

Abstract

The effects of citizen science are wide ranging, influencing science, society, the economy, the environment, as well as individual participants. However, in many citizen science projects, impact evaluation is still overly simplistic. This is particularly the case when assessing the impact of participation in citizen science on the environmental attitudes, behaviour and knowledge of citizen scientists. In an attempt to bridge the gap between the state of the art in relevant scientific fields and citizen science, this systematic literature review identified best practices and approaches in the field of environmental psychology for measuring environmental attitudes, behaviour and knowledge. From the literature, five relevant and validated approaches were identified that can be used to measure changes in attitudes, behaviour and knowledge in citizen science projects. This would allow for improved understanding of the impacts of citizen science, as well as for improved project evaluation as a whole.

Introduction

Citizen science is increasingly popular and has been defined and interpreted in various ways. At a basic level, it is a purpose-designed collaboration in which the general public take part in the scientific research process to support knowledge generation [100], or more specifically “the scientific activities in which non-professional scientists volunteer to participate in data collection, analysis and dissemination of a scientific project” ([46], p. 105). Other definitions point to the role of citizen science in the democratisation of science and public engagement [55, 56] and consider it a tool to link science and society, involving citizens more and more in the scientific process [72]. Citizens can play a voluntary, but highly active role in the process, with input ranging from data collection to the co-design of projects [17], and citizen science as a process is increasingly recognised for its multi-stakeholder nature and complexity [122]. Definitions often conflict in various ways, for example regarding the exact extent of public participation, the opt-in or voluntary nature of participation, and the extent to which generated data can be used in science and/or policy [22]. With these subtle nuances in definition, differences in the implementation of citizen science often occur. Nevertheless, with this rise in popularity, the wide ranging impacts and evaluation of citizen science projects are currently of significant interest.

A range of studies have been conducted to assess the impact of citizen science interventions. These studies have covered a variety of impacts, including: feelings of environmental stewardship in participants [92], economic activity and the creation of jobs [60], scientific outcomes and resulting publications [17], and participant learning outcomes [113]. However, the exact impacts of citizen science are still to be fully and comprehensively understood [65], while up to date impact assessment methods and frameworks are not yet fully integrated in practice.

Based on their systematic review of close to 80 impact assessment frameworks and methods for impact assessment of citizen science, Wehn et al. [122] highlighted that the assessment of environmental attitudes, behaviour and knowledge is particularly lacking throughout the citizen science literature. They also noted that when these impacts have been measured, it is often done simplistically, without using approaches and scales considered the state of the art in relevant scientific literature. This can lead to a suboptimal selection of measurement approaches, and an impact assessment procedure lacking in validity. Furthermore, a more soundly grounded impact assessment procedure would allow for enhanced project evaluation, and therefore an improvement in the implementation of citizen science in the long term.

For example, during their assessment of attitude change in citizen science participants, Brossard et al. [20] highlighted the insufficiencies of their measurement methodology, suggesting that more sensitive scales are needed. This deficit in assessment approaches is visible in a range of other impact assessment studies, aiming to identify the influence of participation in citizen science on attitude [24, 49, 110, 126, 102). These studies used a wide variety of approaches (often with little justification for their selection) to measure environmental attitude, displaying the lack of integration of (and consensus on) measurement approaches within citizen science. This is also apparent in the literature focusing on the assessment of knowledge [17] and behaviour [117].

Apart from individual studies and reviews featuring the assessment of environmental attitudes, knowledge and behaviour, various citizen science impact assessment frameworks have also been developed. One such framework, developed by Kieslinger et al. [65] also attempted to capture attitude, behaviour and knowledge in their three-dimensional impact assessment framework, primarily with the ‘citizen scientist’ dimension (impacts on the individual participants). However, no concrete approaches are integrated into the framework to suggest how these concepts should be assessed, highlighting again the lack of integration of state-of-the-art measurement methods in citizen science.

The field of environmental psychology and the literature on the measurement of environmental attitudes, behaviour and knowledge are well developed, with various establish methods and approaches being refined over the past decades. It has long been claimed and documented with anecdotal evidence that experiences with the environment or environmental matters (such as participation in citizen science initiatives) can alter environmental attitudes, behaviour and knowledge over time [7, 37], suggesting that this field of literature should be of keen interest to citizen science. However, despite the availability of such scales, few citizen science projects actually utilised these approaches during impact assessment [122]. Instead, simplistic approaches are often used to assess project impacts on environmental attitudes, behaviour and knowledge. There is therefore a need for an analysis of the state of the art in the assessment of environmental attitudes, knowledge and behaviour with a view to integrating this in the impact assessment of citizen science.

The systematic literature review presented in this paper aimed to assess the state of the art of measuring environmental attitudes, behaviour and knowledge, and analyse to what degree the currently available measurement approaches can be applied to evaluate the impact of citizen science. However, this paper does not intent to be prescriptive in nature, citizen science is broad in nature, as are the needs and requirements of individual projects. This considered, this paper aims to highlight where consistency in the impact of citizen science projects can be developed, and suggest potential ways in which this could be implemented.

Following this introduction, the methodology of this literature review is outlined in the following section, before the results are detailed. In the discussion, the state of the art for measuring environmental attitudes, behaviour and knowledge are presented and the findings applied to the context of citizen science. Finally, conclusions and future avenues for research in the field are highlighted.

Methods

The analysis of the state of the art in the measurement of environmental attitudes, knowledge and behaviour outlined in this paper was developed from a systematic review of relevant academic literature in these fields. As these are three separate, yet closely related concepts, distinct searches were conducted for each. However, due to the degree of overlap and similarity between the concepts, some of the literature identified during this search is relevant for all three.

The selection of relevant literature for this systematic literature review was based on the method outlined by Moher et al. [84]. The purpose of the systematic literature searches was to identify publications that propose or discuss frameworks, approaches and scales for assessing environmental attitudes (also termed environmental concern), behaviour and knowledge.

The literature search was conducted on Web of Science and Wiley’s Online Library (following an initial preliminary search on Google Scholar), from February to April 2021. Keywords were compiled that referred to the concepts of: (1) environmental attitude, (2) environmental behaviour, and (2) (environmental) knowledge (see Tables 1, 2, 3). Similarly, a set of keywords was identified for the second aspect of the search, which related to the measurement or assessment of these concepts. The Boolean operators “AND” and “OR” were used in order to combine the different terms, while the asterisk (*) was used to ensure the inclusion of variations on each of the terms. To specify the exact phrases that should be contained within the search, quotation marks (“”) were used for each of the first column aspect search terms.

Table 1 Parameters used in the Environmental Attitude literature search

Table 2 Parameters used in the Environmental Behaviour literature search

Table 3 Parameters used in the Environmental Knowledge literature search

In order to identify search terms for each of these concepts, an understanding of appropriate definitions, terms and conceptualisations was required. To acquire this understanding, an initial search of the three concepts was done. This was necessary in particular for ‘environmental attitudes’, which is a term that has been defined in a number of conflicting ways, and which has various synonyms (e.g. beliefs, values) partly due to differing or contradictory definitions [93]. This was highlighted by Rosa and Collado [97], who noted that a range of overlapping terms have been used in the literature in place of (or in conjunction with) ‘environmental attitudes’. For example, some researchers [109] conflate ‘environmental willingness’ with ‘environmental attitudes’, using the former as a measure for the latter. Others (e.g. [95]) note the difference of ‘environmental willingness’ and ‘environmental attitudes’, instead investigating the effect of willingness on attitudes. The literature review undertaken for our paper considered these broad definitions, and therefore included terms such as ‘values’ and ‘beliefs’ when searching for literature related to environmental attitudes.

In the literature, the definitions for both ‘environmental behaviour’ and ‘environmental knowledge’ are less varied. The term ‘pro-environmental behaviour’ [114] is a common substitute for ‘environmental behaviour’ in the literature, and is commonly used when discussion determinants of positive behaviour. Additionally, the words ‘actions’ and ‘activities’ are also commonly included in definitions of the concept (e.g. [88, 124]).

The first part of this literature review was conducted by searching the ‘Topic’ section of literature in the core collection of Web of Science. This search includes title, keywords and abstracts of literature. As this literature review was focusing on review articles, the search was further refined to only include reviews.

In the next step, the Wiley Online Library was searched using the same set of keywords. This search was conducted on 3 and 4 March 2021. As it not possible to search the ‘Topic’ of literature on the Wiley Online Library, the keywords within the abstracts of the records were searched. To limit the search to review articles, the term ‘review’ was also included in the search. Several of the same articles seen during Web of Science search were also seen in the Wiley search. These items were ignored and were not double counted.

Firstly, Web of Science was searched. The ‘environmental attitude’ search returned 7558 records (after filtering to only include review articles, the hit number was further narrowed to 330 records). The title, abstract and keywords of these records were then screened, removing articles unrelated to the measurement of environmental attitude. This process resulted in a shortlist of 23 records. The ‘environmental behaviour’ search (see Table 2) returned 2187 records. When filtering for review articles, this was narrowed to 95 records. After screening the title, abstract and keywords of these records based on their relevance to the measurement of environmental behaviour, a shortlist of 14 records remained. The knowledge assessment search was conducted twice, once without the third parameters (“Environment*” and “Sustain*”), in order to capture the wider literature of knowledge assessment, as well as that specifically relating to environmental knowledge. The wider ‘knowledge assessment’ search (see Table 3) returned 1013 records. When filtering for review articles, this was narrowed to 402 records. After screening the title, abstract and keywords of these records based on their relevance to the assessment of knowledge, a shortlist of seven records remained. The more specific ‘environmental knowledge’ search (see Table 3) returned 702 records. When filtering for review articles, this was narrowed to 301 records. After filtering the title, abstract and keywords of these records to ensure relevance to the measurement of environmental knowledge, a shortlist of eight records remained.

Next, Wileys Online Library was searched. The ‘environmental attitude’ search returned 457 records that were similarly screened for relevance and resulted in a shortlist of 13 records. The ‘environmental behaviour’ search returned 360 records that were similarly screened for relevance and resulted in a shortlist of 13 records. The wider ‘knowledge assessment’ search returned 641 records that were similarly screened for relevance and resulted in a shortlist of two relevant records. The ‘environmental knowledge’ search returned 151 records that were similarly screened for relevance and resulted in a shortlist of two relevant records.

In total, the search of both Web of Science and Wileys Online Library resulted in a shortlist of 23 records for the ‘environmental attitudes’ search (see Additional file 1: Table S1), 20 for the ‘environmental behaviour’ search (see Additional file 1: Table S2), and 18 for the ‘knowledge assessment’ search (see Additional file 1: Table S3). Due to the inherent overlap between the environmental attitude and behaviour literatures, these two shortlists shared four of the same records.

Using Google Scholar, a final search was conducted by combining terms across the previous searches. This search aimed to capture scales or approaches that may have been applied to more than one of environmental attitudes, behaviour and knowledge. Additionally, this search attempted to identify tools that have been used to capture attitudes, behaviour and knowledge across other fields.

Results

There is considerable debate in the field of environmental attitude, knowledge and behaviour, particularly concerning the modelling and measurement of these concepts. Naturally, this has significant implications for the evaluation of impact of participation in citizen science.

Environmental attitudes

Defining and conceptualising the terms ‘environment’ and ‘attitude’ (also termed ‘environmental concern’) occupies a large section of the literature in this field. In general, it is agreed that environmental attitudes are comparable to attitudes to other topics. There is now significant consensus that environmental attitudes are multi-dimensional, but reflect a single overall attitude to the environment [34]. Cruz and Mantana ([28], p. 2) therefore term the concept “a hierarchical attitude system that connects and organizes more specific attitudes about a range of environmental topics”. For example, in one of the fields seminal papers, Schultz [98] highlighted three dimensions of environmental attitude: egoistic (concern for self), altruistic (concern for others), and biospheric (concern for the biosphere). Other dimensions have also been postulated by others. Furthermore, despite this general consensus of a hierarchical model of environmental attitude, there are scholars who conceptualise environmental attitudes differently. Dunlap and Jones [34] highlight some papers which suggest that beliefs, intentions, and attitudes are strongly intertwined, and form a key part of (environmental) behaviour.

Difficulty in creating a unified definition and conceptualisation of ‘environmental attitude’ has led to issues with measuring it. These issues are widely reported, with some being directly caused by a poor definition of the term and invalid dimensions [34]. Additionally, the methodology used when measuring environmental attitudes has also been the focus of a large portion of the literature. Self-reporting (in surveys and questionnaires) has been criticised by many, due to the inherent biases caused [69].

Considering this wide ranging conceptual and methodological debate, it is unsurprising that there is a plethora of available approaches, methods and surveys to measure environmental attitude. In a seminal review paper, Cruz and Mantana [28] identified and examined 26 of the most commonly used scales, to identify the most valid. They identified the Ecology Scale from Maloney and Ward [80] and Schultz’s three-dimensional Scale (2001) as the most valid scales for measuring environmental attitudes.

However, arguably the most comprehensive and (currently) widely used scale is based on the New Ecological Paradigm (NEP) model [32, 33]. The NEP scale assesses attitude across a range of environmental topics, in addition to measuring beliefs, intentions and behaviours. This makes scales based on the NEP model highly practical when measuring environmental attitudes, as the model covers a range of topics and concepts relating to attitudes. The NEP model has therefore resulted in a set of scales, which have been widely adapted within the field (e.g. [21, 48, 83]).

Environmental behaviour

Inherently, there is significant overlap between the fields of environmental behaviour and environmental attitude (as seen by the overlap of papers identified during the literature review), and separating the two from each other has proved difficult and contentious. As with environmental attitudes, dimensions of environmental behaviour have often been a source of disagreement in the literature. For example, there is still significant debate as to whether environmental behaviour is uni- or multi-dimensional (Kaiser and Wilson 2004) [74]. For example, from their study of various pro-environmental behaviours, Kaiser and Wilson (2004) developed a six-dimensional model of behaviour (energy conservation, mobility and transportation, waste avoidance, consumerism, recycling, and vicarious conservation behaviours). Furthermore, a range of studies have also attempted to draw parallels between environmental behaviour and various personality traits, such as openness to experience [19], cognitive flexibility [73], and tendency towards abstract thinking []. Regardless, it is generally agreed, however, that while environmental attitudes are linked to behaviour, strong pro-environmental attitudes do not necessarily lead to corresponding behaviours (as other influencing factors are often present). It is therefore necessary to measure the two separately, with separate scales.

One of the most significant recent developments in the field is the increasingly interdisciplinary nature of the research. Previously, psychology and sociology were relatively separated in this area, whereas current literature is now attempting to reconcile these disparate strands of research, and develop coherent frameworks. Batel et al. [13] provide an in-depth review of this literature, highlighting how wider social changes can interact with psychological processes to influence environmental behaviour. In particular, they compare Social Representations Theory with Social Practices Theory and develop a wider theoretical model to understand behaviour change. This literature is also reinforced by research into how environmental behaviours are associated with broader changes in lifestyle and society [99].

The measurement of environmental behaviour (and the scales required to do so) has also generated a significant portion of literature. This can generally be done in three ways: observation in the field; laboratory observation; or self-reporting. Observation in the field has generally taken the shape of retrospectively assessing past behaviours, for example by analysing prior energy usage or transportation choices [4]. Laboratory observations generally refer to situations or choice-making within a controlled environment [26]. However, these two methodologies are rarely used, with self-report questionnaires being the most used approach to measuring environmental behaviour. As with the measurement of environmental attitudes, there is significant debate around the validity of the self-report approach when assessing environmental behaviour. However, some recent studies have suggested that there is a significant correlation between self-reported and directly observed environmental behaviour [69], lending support to the use of such methodology. However, Kormos and Gifford [69] also emphasised that a large portion of variance in the association between self-reported and objective behaviour remains unexplained, meaning that caution is required when utilising this approach.

Nevertheless, self-reporting is necessary for the majority of research in this field. The study from Kormos and Gifford [69] analysed many of the most frequently used methodologies, and highlighted those most highly correlated with directly observed behaviour. Scales utilised by Kaiser et al. [63], Vadez et al. [115] and Corral-Verdugo and Figueredo [27] appeared to provide the most valid results, suggesting that they may form a useful basis for future self-report studies of environmental behaviour. Kormos and Gifford [69] also propose several suggestions for improving self-report scales—these lessons learned could be used to adjust and improve these past scales.

A further issue to consider is that environmental behaviour is not a single, monolithic concept, but is interdimensional (as previously stated). This is revealed when measuring behaviour. Several studies have suggested that individuals can be relatively inconsistent in environmental-related behaviour. For example, an individual may behave in an environment-friendly manner when it comes to dietary choices, but may often select environmentally damaging modes of transport [42]. Therefore, when measuring environmental behaviour (and making claims about the results of studies), one must be wary that a wide range of environmental behavioural dimensions are covered, or the scope of the study should remain context specific. Considering this, most current measures of environmental behaviour take a broad, general approach to measuring environmental behaviour, with a wide range of items.

Knowledge and environmental knowledge

The literature on the assessment of knowledge and learning outcomes is comprehensive, and has often been grounded in broader frameworks of learning. The key goals of learning (and thus assessment) have been captured in Blooms Taxonomy (1956): knowledge, comprehension, application, analysis, synthesis, and evaluation. These goals each represent a different level of learning and understanding; assessments should therefore be designed to identify at which level a person is operating. While there have been new taxonomies developed since Bloom’s Taxonomy (for example, that of Koedinger et al. [68]), these are the key concepts which still ground learning and assessment.

The fundamental goal of the assessment of learning has been conceptualised in a variety of ways, but the definition from Blythe et al. (16, p. 63) is one of the most commonly cited: “Performances of understanding require students to show their understanding in an observable way. They make students’ thinking visible. It is not enough for students to reshape, expand, extrapolate from, and apply their knowledge in the privacy of their own thoughts…Such an understanding would be untried, possibly fragile, and virtually impossible to assess”.

Assessment of knowledge should not simply be seen as a ‘one way street’, however. Although assessment has long been seen as a ‘normative’ process, many argue should be ongoing or formative, providing students with feedback about their work and also allowing both teacher and students to assess progress towards understanding [10]. Watling and Ginsburg [121] highlight how assessment is a learning opportunity for those taking part, as well as for instructors or teachers. It should allow for feedback and an understanding of how to improve learning in the future.

While the broader knowledge assessment literature sheds much needed light on the theoretical background of learning outcome measurement, the literature is largely based on research within academic settings, and is aimed at improving assessment within schools. It is also largely theoretical and does not highlight particular approaches or scales that could be used to measure learning from participation in citizen science. For this reason, a further search was conducted focusing on the measurement of environmental knowledge.

The majority of the literature into environmental knowledge has been written with the aim to assess the influence of knowledge on environmental behaviours. It is generally accepted that environmental knowledge contributes to sustainable or environmentally conscious behaviour [51, 96], but that knowledge alone does not lead to this behaviour [38]. A portion of the literature goes further than this debate, and investigates how environmental knowledge itself can be measured, and how environmental knowledge can change over time.

Traditionally, the educational and psychological fields split knowledge into declarative knowledge (factual knowledge) and procedural knowledge (skills that transform declarative knowledge into action [76]. Frick et al. [38] further developed these dimensions, and specified them for environmental knowledge: system knowledge (e.g. understand the basic structural and functional characteristics of an ecosystem), action-related knowledge (e.g. understand solutions for environmental issues); and effectiveness knowledge (e.g. understand the benefit of sustainable actions). This framework is now commonly used, and lends itself to environmental studies, as it allows the assessment of environmental core knowledge, as well as knowledge relevant for achieving behavioural goals related to sustainability (which is often the desired outcome of a training or intervention).

Braun and Dierkes [18] used these measures to create a framework with which to assess environmental knowledge before and after an intervention. In the study, the framework was used to measure various areas of environmental knowledge (e.g. water, conservation, renewables, etc.) in a group of participants before and after an intervention. This multi-dimensional framework and approach to environmental knowledge measurement has also been successfully implemented in a similar study by Liefländer et al. [77]. Both of these studies have developed similar scales with which to measure environmental knowledge, which can be used and adapted for future studies of environmental knowledge, including in the context of citizen science projects.

A further challenge in (environmental) education is to determine the best way in which knowledge can be self-reported. It is common in environmental research to use confidence or agreement ratings that self-report one’s own knowledge, i.e. “I can explain what the term ecology means,” [31]. It is often suggested that these tests do not measure actual knowledge, and are more just a representation of subjective knowledge [82]. More direct knowledge assessment approaches are now used more frequently.

Knowledge, Attitude and Practice surveys

Knowledge, Attitude and Practice (KAP) surveys measure attitudes, behaviour and knowledge using a single instrument. These surveys have been used across a range of topics and sectors, primarily within health sciences [75], but also in water and sanitation [101], building design [1], and communication sciences [85]. The KAP survey design is a broad conceptual framework, which can be used to develop surveys to assess what is known (knowledge), believed (attitude), and done (practiced) by a target group. As outlined by Andrade et al. [8], the KAP survey framework first requires the implementer to determine what knowledge, attitudes and practices they are expecting from their target group. Based on this, they must then devise several questions (open and/or closed) which they feel examine relevant knowledge, attitudes and practices. For example, one could examine understanding of common misconceptions related to personal hygiene to further understand the knowledge of a participant. Following this, the scoring of the survey needs to be determined according to the type of question, with a separate overall score for each of the three constructs: knowledge, attitude and practice. Implementers are also advised to pilot the survey before widespread use. Using this step-by-step framework, KAP surveys have become widespread to provide a baseline measurement for knowledge, attitudes and practice. They can also be used to measure the impact of an intervention, including of citizen science projects.

Best practice indicators and approaches for assessing attitudes, knowledge and behaviour in citizen science

As highlighted in the previous section, there is a plethora of information and research regarding the measurement of environmental attitudes, behaviour and knowledge. Yet this literature (and the approaches and scales developed within it) has not yet been fully incorporated into the field of citizen science. Data have been collected from individuals that have engaged in citizen science activities in the past [70], and several past studies have even investigated the impact of engagement with citizen science on attitude [17] behaviour and knowledge [113]. However, the scales used within these studies did not represent the current state of the art of the literature (as identified by this literature review).

Future impact evaluation frameworks should endeavour to integrate the findings of this (and similar) literature reviews, to ensure that accessible and high-quality impact assessment approaches are easily available to citizen science practitioners. The analysis within this literature review has already led to the identification of a number of indicators and approaches for measuring environmental attitudes, behaviour and knowledge which have been included in the MICS Conceptual Framework for the evaluation of citizen science impacts on the environment and society [122].

One of the central discussions identified from the literature (and relevant to citizen science) is whether environmental attitudes, behaviour and knowledge should (or even can) be measured with a single, unified scale or approach. Several theories, such as the Theory of Planned Behaviour [5], suggest that attitudes (which are made up of beliefs) are closely related to behaviour, while evidence has also suggested that knowledge and attitudes are linked [125]. However, despite these relatively high-level psychological theories linking attitudes, behaviour and knowledge, the literature focusing on environmental psychology generally separates the concepts. Therefore, the scales and approaches identified in this literature review are generally specific to measuring one of the three concepts: environmental attitudes, environmental behaviour, or environmental knowledge.

In the field of environmental attitude, there is (to a large degree) currently a consensus on the most valid methods to use when measuring attitude. As outlined previously, there are a range of scales that have been used to measure environmental attitudes. Over the history of the literature, the field has been relatively fragmented, with studies often creating new scales with which to measure environmental attitudes. Despite this, the three most commonly used (and adapted) scales are the Ecology Scale from Maloney and Ward [80], Schultz’s three-dimensional Scale (2001) and the New Ecological Paradigm (NEP) scale [32, 33]. These scales (along with other prominent scales in the field) and the attitudinal dimensions that they identified, have been incorporated into the most comprehensive method currently available in the field, the Environmental Attitudes Inventory (EAI) [83]. The EAI offers a 12-dimensional approach to measuring environmental attitudes, has been used across a range of contexts and has been found to be highly consistent and reliable.

Several other scales have been developed recently, and show promise in the measurement of environmental attitudes. One of the most prominent of these is the Sustainability Attitudes Scale (SAS) [125]. This scale used the three-domain definition of sustainability, looking at attitudes to: Ecological Sustainability, Social Sustainability Subscale; and Economic Sustainability. While testing of this scale generally been found it to be valid, it is still relatively new and has not been used as extensively as other scales. While there is a large amount of overlap between the two, this should be considered when selecting a scale. Despite these flaws, this scale could offer a possible method for measuring environmental attitude within citizen science in the future.

Considering the state of the literature, the current use of attitude measurement scales, and the particular needs for citizen science, the Environmental Attitudes Inventory (EAI) [83] was identified as the most applicable for measuring environmental attitudes within citizen science, due to its comprehensive nature and application of the 12 facets of environmental attitudes identified by the authors. However, due to the length of this instrument, citizen science practitioners should consider the added value of such an in-depth scale alongside the additional resources required to implement it. Alternatively, there are several shortened versions of the inventory that can also be utilised. Due to its prevalence across the literature, strong validity, and short, simple nature, Schultz’s Three Dimensional Scale (2001) was identified as an alternative methodology.

A variety of best practices and approaches are also available in the field of measuring environmental behaviour. However, the comprehensive study by Kormos and Gifford [69] listed three approaches that stand out above others. The approaches utilised by Kaiser et al. [63], Vadez et al. [116] and Corral-Verdugo and Figueredo [27] each have benefits for measuring environmental behaviour, and importantly appear to be highly valid when doing so. However, the scales from both Vadez et al. [116] and Corral-Verdugo and Figueredo [27] are highly context specific (measuring behaviour relating to deforestation and recycling, respectively). The scale used by Kaiser et al. [63]—which was adapted from the “General Measure of Ecological Behaviour” [61]—covers a range of different behaviours, and achieved the highest degree of validity in this large study. Variants of the Kaiser et al. [63] scale have been often adopted by following researchers in the field. The most updated of these scales is the ‘General Ecological Behavior Scale—50’ [62]. This scale is generally the most widely used when measuring environmental behaviour, as well as being the most flexible (in terms of the various behaviours and dimensions assessed). For this reason, it provides the most promise to those aiming to measure environmental behaviour within citizen science.

There is also little consensus regarding best practice when measuring environmental knowledge. One of the most commonly used frameworks is the three-dimensional theory of environmental knowledge, separating knowledge into system, action-related and effectiveness dimensions. Assessment should therefore reflect these dimensions. The assessment method used by Braun and Dierkes [18] does this well, and can measure environmental knowledge across a broad range of topics. This scale therefore can be easily adapted to measure environmental knowledge within citizen science. An alternative scale could be the Assessment of Sustainability Knowledge (ASK) [125]. As this scale was developed alongside the Sustainability Attitudes Scale (SAS), use of both scales would allow for the measurement of environmental knowledge and attitude using the same theoretical framework. However, neither have yet been widely tested.

Discussion

This literature review outlined the current state of the art in the measurement of environmental attitudes, behaviour and knowledge. Driven by the need to inform impact evaluation of citizen science, this review built crucial links with the field of environmental psychology, setting up the basis for an improved understanding of the impact of citizen science on environmental attitudes, behaviour and knowledge.

Five scales emerged as relevant for citizen science projects to measure environmental attitudes, behaviour and knowledge, namely: the Environmental Attitudes Inventory (EAI) [83], the Three Dimensional Scale [98], the General Ecological Behavior Scale—50 [62], the three-dimensional theory of environmental knowledge, used by Braun and Dierkes [18], and the Assessment of Sustainability Knowledge (ASK) [125]. These scales were selected based on their positioning as state-of-the-art approaches, as well as their applicability to citizen science projects.

While these selected approaches represent the state of the art in their respective fields, due to the unique context of citizen science projects a variety of factors can influence the requirements of impact assessment tools. For example, with a large number and (often) wide geographical spread of citizens engaging in citizen science projects, face-to-face or workshop style evaluations are often impractical, meaning that questionnaires and surveys can provide an alternative. Additionally, the demographics and expertise of the participants should be considered when selecting an approach. For example, a survey that is appropriate for adult participants in citizen science may not be practical for children. Finally, the (often limited) resources of citizen science projects should also be considered. Assessment approaches should therefore not be resource intensive, usable by non-experts and experts alike, and should be as simple as possible to implement. Considering these various factors, for the purposes of citizen science scales often provide optimum utility when they are flexible—practitioners should be able to adapt these scales, adding or removing questions where required. For example, the environmental knowledge scale provided by Braun and Dierkes [18] covers knowledge of a range of topics, however it does not explicitly measure knowledge of pollution. Naturally for a citizen science initiative with a focus on pollution measurement, this is a key topic that should be measured. Using the theory provided by Braun and Dierkes [18], practitioners can therefore insert questions covering the knowledge of pollution across the three dimensions (system knowledge, action-related knowledge and effectiveness knowledge). In this way, practitioners can keep the generalizable nature of their results, while also generating results meaningful to their specific context.

Being widely used in other fields to assess attitude, knowledge and behaviour, KAP surveys present a further possible option for citizen science practitioners. However, for the purposes of citizen science practitioners, KAP surveys pose several issues. Firstly, various methodological concerns have been raised about KAP surveys since their widespread use, particularly around the relatively simplistic conceptualizations of attitudes, behaviour and knowledge [2], especially when compared with the highly valid models previously highlighted (e.g. [18]). Furthermore, KAP surveys are generally designed to be highly context specific (e.g. [79]). This means that there are very few KAP instruments that practitioners can adopt easily and comparison across survey results is difficult, if not impossible. Additionally, KAP surveys require resources to be designed and implemented [8]. For example, to design a KAP survey a practitioner must have a deep understanding of the field in order to decide what knowledge, attitudes and practices citizen scientists might acquire during the project. However, if an initiative has the resources to develop a KAP survey, this option can provide a generally accepted step-by-step method for developing high quality scales for environmental attitudes, behaviour and knowledge. The exploration of KAP survey approaches by citizen science initiatives, including the construction of generic building blocks for citizen science applications, may therefore offer a future avenue for researchers and practitioners of citizen science.

The findings of this paper provide a clear advancement in the evaluation and assessment of citizen science impact. Due to the cross-disciplinary nature of the topic, citizen science practitioners often need to draw from different fields in order to add structure to outcomes and findings. However, the integration of environmental psychology within citizen science still requires significant work. This review begins this process by highlighting approaches which can significantly improve the impact assessment and understanding of citizen science, and also brings the field of citizen science in line with the state of the art in environmental psychology. Further work is required to fully integrate these approaches into practice in citizen science.

For this integration of new scales and methodologies to properly take place, citizen science practitioners first need to understand and familiarise themselves with these approaches, in order to well implement them. The few prior evaluations that did assess environmental attitude, behaviour or knowledge, did not fully assess the range of assessment approaches available, or simply created a new scale not grounded in the current literature (e.g. [24, 49, 110]). Practitioners should also understand the resources required to implement these assessment approaches, as well as the organizational efforts necessary. Although practitioners benefit from increasingly profound data offered by the highlighted impact assessment methods, it should be noted that the more complex the analysis, the greater the resources and expertise required [65]. The utilization of these methods can be a relatively complex process, some scales may be more advantageous than others in specific contexts (for example, depending on the demographics of the participants), and the analysis of the results can require significant resources.

The findings of this review also emphasise the fact that wider changes to the structure of the citizen science design process (beyond mere resource allocation) are required in order to fully assess the impact of these initiatives. As already implemented by some initiatives (e.g. [90]) impact assessment should first be considered even before the beginning of citizen science projects and from there embedded within the citizen science project management. This is largely because many impact measurement tools (particularly those offering high-quality data) require baseline measurements before participation. This is particularly the case for tools measuring environmental attitudes, behaviour and knowledge, such as those highlighted by this literature review. Therefore, these approaches should not simply be added as an afterthought, but should become a key part of the citizen science process. However, as previously mentioned, the additional resources required by such approaches must also be considered from the beginning of the process.

The integration of these assessment methods within citizen science can be supported by their inclusion within broader impact frameworks, providing a roadmap for the impact assessment of future citizen science initiatives. This is already being done—these tools were highlighted in the impact assessment framework outlined by Wehn et al. [122], allowing for citizen science practitioners to easily select specific approaches for measuring their impact area of interest. Such integration efforts should be continued in the future to further support citizen science practitioners in their evaluation efforts.

By using these approaches to measure citizen science impact, a range of benefits can be expected. With improved approaches to project evaluation, a greater understanding of citizen science could be obtained, supporting relevant policy and research in the future. Furthermore, these approaches can be useful for understanding design principles of citizen science projects, for example the aspects of initiatives that foster environmental attitude, knowledge or behaviour. In future, these aspects could be tailored to ensure that the most benefit is gained.

In the broader picture, citizen science is often highlighted as a route to achieving a range of environmental and social goals, including up to 76 of the 231 SDG indicators [36]. However, to fully assess the influence of citizen science in achieving these goals, the impacts need to be fully understood and accurately measured [90]. By using state-of-the-art evaluation tools of high scientific quality, the real potential of citizen science can be realised, and its contribution to solving societal and environmental issues can be seen. While these measures identified in this literature review provide an insight into the attitudes, behaviour and knowledge of individuals, they do little to offer understanding of wider, collective social changes encapsulated within the SDGs. To realise the full impact of citizen science on a broader scale, alternative methodologies should be examined and used in conjunction with those identified in this review.

Conclusion

The findings from the literature review link the previously separate fields of environmental attitudes, behaviour and knowledge, with citizen science impact assessment. From the literature review, five approaches which represent the state of the art in their respective fields, were identified as suitable for measuring the impact of participation in citizen science on environmental attitudes, behaviour and knowledge. By bridging the fields of citizen science with environmental psychology and highlighting concrete methods for citizen science projects, this paper provided concrete insights for citizen science practitioners aiming to identify the impact of their project on participants. It also highlights the various efforts, considerations and changes in approach required to embed these impact assessment approaches within citizen science.

This literature review highlights the balance that needs to be struck by citizen science initiatives in impact assessment—namely that generalizable, state-of-the-art assessments should take place, while at the same time providing meaningful, context specific insight into the effects of the initiative [122]. The challenge for citizen science practitioners is to achieve both. Practitioners can accomplish this by using the results from this literature review—as well as the suggested methods of implementation.

The findings from this literature review can also feed in to future frameworks of citizen science, in addition to individual impact assessments. The principles of this review have already been used to inform an updated framework of citizen science impact assessment [122]. Where relevant, impact assessment, and specific tools and scales for measuring impact, should be included in all future citizen science frameworks, as well as an indication of the required resources for their implementation.

While this study outlines a comprehensive view of the current state of the art concerning the measurement of environmental attitudes, behaviour and knowledge, citizen science projects should also be aware of the fast moving nature of the field. These approaches are regularly updated (e.g. [62]), while new and innovative approaches are also often published [125]. It should therefore be ensured that the state of the art is represented when conducting impact assessment.