What evidence exists on the performance of nature-based solutions interventions for coastal protection in biogenic, shallow ecosystems? A systematic map protocol

Attached Files:

Systematic Map Protocol

Title
What evidence exists on the performance of nature-based solutions interventions for coastal protection in biogenic, shallow ecosystems? A systematic map protocol

Citation:
Avery B. Paxton, Trevor N. Riley, Camille L. Steenrod, Carter S. Smith, Y. Stacy Zhang, Rachel K. Gittman, Brian R. Silliman, Christine A. Buckel, T. Shay Viehman, Brandon J. Puckett, Jenny Davis. What evidence exists on the performance of nature-based solutions interventions for coastal protection in biogenic, shallow ecosystems? A systematic map protocol: a Systematic Map Protocol. PROCEED-23-00105 Available from:
https://proceedevidence.info/protocol/view-result?id=105
https://doi.org/10.57808/proceed.2023.10

Corresponding author’s email address
avery.paxton@noaa.gov

Keywords
coastal resilience; ecological engineering; green infrastructure; natural infrastructure;

Background
Anthropogenic pressures and climate change threaten the capacity of ecosystems to deliver a variety of services, including protecting coastal communities from hazards like flooding and erosion. Human interventions aim to buffer against or overcome these threats by providing physical protection for existing coastal infrastructure and communities, along with added ecological, social, or economic co-benefits. These interventions are a type of nature-based solution (NBS), broadly defined as actions working with nature to address societal challenges while also providing benefits for human well-being, biodiversity, and resilience. Despite the increasing popularity of NBS for coastal protection, sometimes in lieu of traditional hardened shorelines (e.g., oyster reefs instead of bulkheads), gaps remain in our understanding of whether common NBS interventions for coastal protection perform as intended. To help fill these knowledge gaps, we aim to identify, collate, and map the evidence base surrounding the performance of active NBS interventions related to coastal protection across a suite of ecological, physical, social, and economic outcomes in salt marsh, seagrass, kelp, mangrove, shellfish reef, and coral reef systems. The resulting evidence base will highlight the current knowledge on NBS performance and inform future uses of NBS meant for coastal protection.

Theory of change or causal model
To improve coastal protection, resource managers, governments, local municipalities, tribal nations, non-governmental organizations, and private property owners are increasingly turning to nature-based solutions. Nature-based solutions (NBS) are broadly defined as “actions to protect, conserve, restore, and sustainably use and manage natural or modified terrestrial, freshwater, coastal, and marine ecosystems to address social, economic, and environmental challenges effectively and adaptively, while simultaneously providing human well-being, ecosystem services, resilience, and biodiversity benefits” (UNEP 2022). We focus on the subset of active NBS interventions used to improve coastal resilience to hazards by providing physical protective services, such as wave attenuation and flood reduction.

Stakeholder engagement
This systematic map was initiated by the NOAA National Centers for Coastal Ocean Science to determine the state of knowledge regarding the performance of NBS for coastal resilience. The synthesis was motivated by a federally identified need to understand the evidence base surrounding NBS performance to help inform policy and management decisions about how to monitor NBS and when and where to implement NBS, as well as to identify where additional performance evaluations are warranted. Federal “team leads” for the synthesis effort developed a “core team” of federal researchers and academic scientists who study and implement NBS in estuarine and marine ecosystems. The core team helped refine the protocol scope and will continue to play key roles in compiling the map. We also convened an “advisory team” of additional scientists and managers with expertise in NBS and coastal ecosystems to provide additional direction and feedback.

Objectives and review question
Objective: The objective of this systematic map is to identify, collate, and map the global evidence base on the ecological, physical, social, and economic performance of active NBS interventions related to coastal protection in salt marsh, seagrass, kelp, mangrove, coral reef, and shellfish reef systems. Question: What is the extent and distribution of evidence on the ecological, physical, social, and economic performance of active NBS interventions used in salt marsh, seagrass, kelp, mangrove, coral reef, and shellfish reef systems within the context of coastal protection?

Definitions of the question components
Population: Salt marsh, seagrass, kelp, mangrove, shellfish reef, or coral reef systems where active NBS interventions are used Intervention: Active NBS interventions established within the context of coastal protection Comparator: No comparator required beyond presence of an active NBS intervention Outcome: Ecological, physical, economic, or social performance outcomes evaluated following NBS interventions Study type: Experimental, quasi-experimental, observational, or modeling studies with quantitative or qualitative data on NBS performance outcomes

Search strategy
Searches for primary literature on performance of NBS for coastal protection in shallow, biogenic ecosystems will be conducted using a predefined list of indexing platforms, bibliographic databases, open discovery citation indexes, and organizational databases and websites, as well as an online search engine and novel literature discovery tool. All searches will be conducted in English and will be restricted to literature published from 1980 to present. See full protocol for additional details including search strings.

Bibliographic databases
Scopus: - Subscription: Duke University - Filters: Year 1980 - present Web of Science Core Collection - Indexes: SCI-Expanded, SSCI, CPCI-S, CPCI-SSH, ESCI (all 1980 - present except ESCI 2018 - present) - Subscription: Duke University - Filters: Year 1980 - present; Document type - article, proceeding paper, early access, data paper Ocean Abstracts (1981 - present) - Subscription: NOAA - Filters: Year 1980 - present; Source type - scholarly journals, dissertations and theses, conference papers and proceedings, reports Earth, Atmospheric, & Aquatic Sciences Collection - Databases: Aquatic Sciences and Fisheries Abstracts; Meteorological and Geoastrophysical Abstracts; Earth, Atmospheric, and Aquatic Sciences Database - Subscription: NOAA - Filters: Year 1980 - present; Source type - scholarly journals, dissertations and theses, conference papers and proceedings, reports

Web-based search engines
LENS (lens.org) - Indexes: CORE, Crossref, PubMed, Microsoft Academic - Filters: Year 1980 - present; Document type: journal article, conference proceeding article, conference proceedings, dissertation, report Dimensions - Subscription: NOAA - Filters: Year 1980 - present; Publication type: article, proceeding Google Scholar - Title search - Up to first 1,000 results - Using Publish or Perish (Harzing 2007) Inciteful - Novel literature discovery tool - Up to first 1,000 results - See Weishuhn 2022

Organisational websites
Organization name URL Asian Development Bank https://www.adb.org/ Australian Government Department of Climate Change, Energy, the Environment, and Water https://www.dcceew.gov.au/ Billion Oyster Project https://www.billionoysterproject.org/ Caribbean Natural Resources Institute https://hub.canari.org/ Climate Resilient by Nature https://www.climateresilientbynature.com/ ClimateLinks https://www.climatelinks.org/ Commonwealth Scientific and Industrial Research Organisation https://www.csiro.au/ Conservation International https://www.conservation.org/ UK Government Department for International Development https://www.gov.uk/ USAID Development Experience Clearinghouse https://www.usaid.gov/ Duestsche Gesellschaft fur Internationale Zusammenarbeit https://www.giz.de/ Environmental and Energy Study Institute https://www.eesi.org/ Environmental Defense Fund https://www.edf.org/ European Union / Commission https://op.europa.eu/ Global Facility for Disaster Reduction and Recovery https://www.gfdrr.org/ Global Mangrove Alliance https://www.mangrovealliance.org/ Global Program on Nature-Based Solutions for Climate Resilience https://naturebasedsolutions.org/ iied Publications Library https://www.iied.org/ International Monetary Fund https://www.imf.org/ International Union for Conservation of Nature https://www.iucn.org/ National Fish and Wildlife Foundation https://www.nfwf.org/ National Oceanic and Atmospheric Administration https://www.noaa.gov/ National Science Foundation https://www.nsf.gov/ Oxford Nature Based Solutions Initiative https://www.naturebasedsolutionsinitiative.org/ rare https://rare.org/ Resources for the Future https://www.rff.org/ The Nature Conservancy https://www.nature.org/ United Nations Decade on Restoration https://www.decadeonrestoration.org/ United Nations Development Programme https://www.undp.org/ United Nations Environment Programme https://www.unep.org/ United Nations Environment Programme World Conservation Monitoring Center https://resources.unep-wcmc.org/ United States Army Corps of Engineers https://www.usace.army.mil/ United States Climate Resilience Toolkit https://toolkit.climate.gov/ United States Department of Transportation https://www.transportation.gov/ United States Environmental Protection Agency https://www.epa.gov/ United States Fish and Wildlife Service https://www.fws.gov/ United States Geological Survey https://www.usgs.gov/ University of Georgia Institute for Resilient Infrastructure Systems https://iris.uga.edu/ Wetlands International https://www.wetlands.org/ Wildlife Conservation Society https://library.wcs.org/ World Agroforestry Center https://www.worldagroforestry.org/ World Bank https://www.worldbank.org/ World Resources Institute https://www.wri.org/ World Wildlife Fund https://www.worldwildlife.org/

Comprehensiveness of the search
The stakeholder team identified 55 relevant articles to test our search string against (Additional File 3). These articles, which we refer to as benchmarking articles, were sourced from subject matter experts. Some benchmarking articles were also sourced from Smith et al. (Smith et al. 2020), a recent scoping review of living shorelines. The identified benchmarking articles met the eligibility criteria and would be included at the full text stage. We implemented our search string in the Web of Science Core Collection and tested whether our benchmarking articles were returned by our search strings. Of the 55 benchmarking articles 52 were indexed in Web of Science. Our initial search results failed to identify nine (6 indexed, plus 3 not indexed) of the benchmarking articles. We then adjusted our search string incrementally until it captured all 52 benchmarking articles indexed in Web of Science Core Collection; in total, we conducted five rounds of testing search string variations, improving searches, and refining combinations of substrings during the benchmarking stage. . We verified that the three articles not indexed in Web of Science were returned in searches via open discovery citation indexes like LENS and Dimensions. Following benchmarking, research librarians and subject matter experts peer-reviewed the search strings and strategy to ensure consistent use of syntax like truncations, and the search strings were updated based on reviewer feedback.

Search update
N/A

Screening strategy
Resulting literature will be screened against set inclusion criteria (i.e., population, intervention, outcome, study type) at the level of title and abstract followed by full text. Screening will be facilitated by a web-based active learning tool that incorporates user feedback via machine learning to prioritize articles for review. See full protocol for additional details including on screening strategy.

Eligibility criteria
Population of subjects - Coastal ecosystems with NBS -Salt marsh, seagrass, kelp, mangrove, shellfish reef, or coral reef systems where NBS interventions are used. Intervention - Active NBS intervention related to coastal protection -Interventions must be an active NBS intervention that is used, installed, constructed, or implemented by humans that include: Interventions must be an active NBS intervention that is used, installed, constructed, or implemented by humans. Active interventions include the following (Table 7): --Restore, create, enhance, or rehabilitate natural habitat, ecosystems, or associated services --Create habitat or ecosystem in place of a naturally occurring one -- Add artificial or engineered structure of human origin, natural origin, or hybrid origin to an existing ecosystem -- Retrofit, modify, or remove gray infrastructure -- Stabilize, remove, or place sediment in an ecosystem -- Modify morphology of an ecosystem - Remove or add invasive species to an ecosystem - Interventions must be related to coastal protection. --NBS stated to have goal, aim, or intent of coastal protection --NBS evaluated for coastal protection physical outcomes of any directionality Comparator - NBS performance - No comparator is required for the systematic map because the only requirement is the presence of NBS intervention, which is not a comparator. Outcome - NBS performance outcomes - Ecological, physical, economic, or social performance outcomes of NBS that are measured, observed, or modeled. Study type - Experimental, quasi-experimental, modeling, or observational (e.g., monitoring or assessment) studies with quantitative or qualitative data

Consistency checking
To reduce bias during screening, we will hold two training sessions – one for title and abstract screening and one for full text screening – for all screeners to attend. During the training sessions, we will collaboratively work through screening several articles. We will then assign each screener the same small subset of articles to screen. We will compare screening outcomes, discuss inconsistencies, and may alter eligibility criteria if needed. We will evaluate inter-reviewer consistency for the final training set of articles at the title and abstract stage using the Kappa statistic. Given the high number of expected articles, we will conduct double screening for as many as 5% of articles at the title and abstract or full text screening stages. The exact percentage of articles for which double screening will be conducted will depend on the number of total articles, and we will report this information in the systematic map. We recognize that single screening may introduce bias to the systematic map, but it is necessary because of the high number of expected articles (~30,000) and resource constraints. If a screener is an author of an article, they will not be permitted to screen the article at the title and abstract or full text stage nor permitted to code metadata extraction.

Reporting screening outcomes
The systematic map will conform to the RepOrting standards for Systematic Evidence Synthesis (ROSES) for systematic map protocols

Study validity assessment
Because we are conducting a systematic map to compile a broad evidence base, we do not plan to systematically assess the study validity through conducting critical appraisals as is typical in systematic reviews. We understand that this may have implications for the utility of the systematic map, such as limiting interpretations surrounding gaps and clusters in evidence. We will acknowledge these limitations in the final map. We will, though, code attributes of each study, such as performance assessment frequency and the method used to evaluate NBS performance outcomes. These attributes can assist end users of the systematic map in making preliminary assessments of study validity.

Consistency checking
To reduce bias during screening, we will hold two training sessions – one for title and abstract screening and one for full text screening – for all screeners to attend. During the training sessions, we will collaboratively work through screening several articles. We will then assign each screener the same small subset of articles to screen. We will compare screening outcomes, discuss inconsistencies, and may alter eligibility criteria if needed. We will evaluate inter-reviewer consistency for the final training set of articles at the title and abstract stage using the Kappa statistic. Given the high number of expected articles, we will conduct double screening for as many as 5% of articles at the title and abstract or full text screening stages. The exact percentage of articles for which double screening will be conducted will depend on the number of total articles, and we will report this information in the systematic map. We recognize that single screening may introduce bias to the systematic map, but it is necessary because of the high number of expected articles (~30,000) and resource constraints. If a screener is an author of an article, they will not be permitted to screen the article at the title and abstract or full text stage nor permitted to code metadata extraction.

Data coding strategy
Metadata from studies that meet our inclusion criteria will be entered into a standardized data coding spreadsheet. The extracted metadata will include bibliographic (e.g., publication year, authors, title) attributes, as well as attributes describing the population, intervention, study type and – if applicable – the comparator, and outcome. Population metadata attributes will include the ecosystem type and description. Intervention attributes will include the NBS type and description, as well as whether a coastal protection goal accompanies the NBS intervention and if so a description of the goal. Study type attributes will include the type of study (e.g., observational, experimental, modeling), objective, design, geographic location, and comparator. Outcome attributes will include the category and subcategory of outcome (e.g., social - culture), as well as evaluation method, metrics, duration, and frequency.

Meta-data to be coded
Metadata from studies that meet our inclusion criteria will be entered into a standardized data coding spreadsheet. The extracted metadata will include bibliographic (e.g., publication year, authors, title) attributes, as well as attributes describing the population, intervention, study type and – if applicable – the comparator, and outcome. Population metadata attributes will include the ecosystem type and description. Intervention attributes will include the NBS type and description, as well as whether a coastal protection goal accompanies the NBS intervention and if so a description of the goal. Study type attributes will include the type of study (e.g., observational, experimental, modeling), objective, design, geographic location, and comparator. Outcome attributes will include the category and subcategory of outcome (e.g., social - culture), as well as evaluation method, metrics, duration, and frequency.

Consistency checking
To ensure consistency in data coding, we will hold a training session to train screeners in how to conduct metadata coding; this training session may occur within the full text screening training session (see Screening section above). During the training session, we will collaboratively work through data coding of several articles, including some that are straightforward and others that are more nuanced. We will then assign each screener the same small subset of articles to code. We will compare coding results, discuss inconsistencies, and may alter attributes and instructions if needed. Given the high number of expected articles, we will not conduct double (or side-by-side) data extraction at the full text stage but rather will conduct spot checks on a small percentage of articles. We will compare spot checking results and discuss any inconsistencies with the screening team. The exact percentage of articles for which spot checking will be conducted will depend on the number of total articles, and we will report this information in the systematic map.

Type of mapping
Study mapping and visualization will be conducted to investigate and visualize patterns in the distribution and abundance of evidence surrounding NBS performance. Analyses will be targeted to address our primary and secondary research questions.

Narrative synthesis methods
Following data analyses, we will prepare the final evidence map for peer-reviewed publication in the journal Environmental Evidence. The evidence map will include visual summaries of the evidence base using figures including heat maps, bar plots, and geographic distribution maps, as well as tabular summaries. A core component of the map will be a narrative summary highlighting evidence clusters for which systematic reviews or meta-analyses can be conducted, as well as evidence gaps for which additional research may be warranted. The narrative report will also outline the policy and management implications of the map findings.

Knowledge gap identification strategy
See above

Demonstrating procedural independence
Given the high number of expected articles, we will conduct double screening for as many as 5% of articles at the title and abstract or full text screening stages. The exact percentage of articles for which double screening will be conducted will depend on the number of total articles, and we will report this information in the systematic map. We recognize that single screening may introduce bias to the systematic map, but it is necessary because of the high number of expected articles (~30,000) and resource constraints.

Competing interests
The authors declare that they have no competing interests.

Funding information
This study was supported by the NOAA National Centers for Coastal Ocean Science.

Author’s contributions
AP and TR conceptualized the scope and developed the protocol. TR and AP were the major contributors to the search strategy and string development, bibliographic database selection, source analysis, and reference management. AP, CS, and TR were the main contributors to the organizational literature search strategy and database selection. AP developed the data coding strategy. AP and TR drafted the manuscript with support from CS in the introduction. AP drafted the intervention and outcome typologies with support from BP and JD. All authors helped refine the systematic map scope, protocol methods, and manuscript. All authors reviewed and approved the final manuscript for submission to the journal 'Environmental Evidence.'

Acknowledgements
We thank the NOAA National Centers for Coastal Ocean Science and the NOAA Central Library for supporting the protocol. We thank Duke RESTORE for supporting the protocol, and especially Alyssa Adler, Anjali Boyd, and Josette McLean for their insight during initial protocol scoping. We thank Lisa Clarke, and Hope Shinn from the NOAA Central Library and Brian Voss from the NOAA Seattle Regional Library for reviewing the search string. We thank Leanne Poussard, Alyssa Leclaire, Tomma Barnes, Trevor Meckley, and Rebecca Nicodemus for thoughtful reviews of the manuscript. We thank Samantha Cheng for her feedback on scoping and developing the search and data coding strategies. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the opinions or policies of the US Government, nor does mention of trade names or commercial products constitute endorsement or recommendation for use.

References
Reference for full protocol: Paxton, A.B., Riley, T.N., Steenrod, C.L. et al. What evidence exists on the performance of nature-based solutions interventions for coastal protection in biogenic, shallow ecosystems? A systematic map protocol. Environ Evid 12, 11 (2023). https://doi.org/10.1186/s13750-023-00303-4 References: UNEP. 2022. United Nations Environment Programme. Resolution adopted by the United Nations Enviornment Assembly: Nature-based solutions for supporting sustainable development. United Nations Environment Assembly, Fifth Session. Weishuhn, M. 2022. Inciteful: Citation network exploration. Harzing, A. W. 2007. Publish or perish. https://harzing.com/resources/publish-or-perish. Smith, C. S., M. E. Rudd, R. K. Gittman, E. C. Melvin, V. S. Patterson, J. J. Renzi, E. H. Wellman, and B. R. Silliman. 2020. Coming to terms with living shorelines: A scoping review of novel restoration strategies for shoreline protection. Frontiers in Marine Science 7. Haddaway, N. R., B. Macura, P. Whaley, and A. S. Pullin. 2018. ROSES RepOrting standards for Systematic Evidence Syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. Environmental Evidence 7. Collaboration for Environmental Evidence. 2022. Guidelines and standards for evidence synthesis in environmental management. Version 5.1.in A. S. Pullin, G. K. Frampton, B. Livoreil, and G. Petrokofsky, editors.

Authors and Affiliations

Name	Country	Affiliation
Avery B. Paxton	United States	NOAA National Centers for Coastal Ocean Science
Trevor N. Riley	United States	NOAA Central Library
Camille L. Steenrod	United States	CSS Inc. and NOAA National Centers for Coastal Ocean Science
Carter S. Smith	United States	Duke University
Y. Stacy Zhang	United States	North Carolina State University
Rachel K. Gittman	United States	East Carolina University
Brian R. Silliman	United States	Duke University
Christine A. Buckel	United States	NOAA National Centers for Coastal Ocean Science
T. Shay Viehman	United States	NOAA National Centers for Coastal Ocean Science
Brandon J. Puckett	United States	NOAA National Centers for Coastal Ocean Science
Jenny Davis	United States	NOAA National Centers for Coastal Ocean Science

Submitted: Apr 26, 2023 | Published: Jun 16, 2023

© The Author(s) 2023.
This is an Open Access document distributed under the terms of the Creative Commons Attribution 4.0 International License https://creativecommons.org/licenses/by/4.0/deed.en .