Monitoring Design: Data Analyses and Management

Before-After-Control-Impact Framework

Adaptive Management

Average sediment accumulation (+ standard error) between Aug/Sept 2009 and Feb 2011 (using Surface Elevation Tables) at two restoration sites and one reference site, Nisqually estuary.

The BACI concept is used to examine Before (pre-construction baseline) and After (post- construction) condition of the area, as well as to compare a Control (reference site) with the Impact site (restoration site; Stewart-Oaten et al. 1986, Stewart-Oaten et al. 1992, Underwood 1992, Stewart-Oaten 2003). Before and After sampling can be used to determine how the restoration process changed the site through time from its historical condition. Control and Impact sampling allows effects of restoration actions to be discerned from natural variability, stochastic events, and underlying environmental or species trends in the larger area – for example, sea level rise will increase water levels in the intertidal and adjacent areas to the restoration site. A Control site which has identical environmental conditions to the Impact site is not typically available. Thus, we use the term Reference site (see Smith et al. 1993) to describe areas near the restoration but not part of the area directly affected by the restoration project. The restoration and reference sites are monitored with the same methods, intensity, and frequency to allow for paired comparison of the trajectories of the restored and reference site. However, the lack of replicates for the restoration and reference site can be problematic because differences in trajectories may be due to natural site variation and erroneously attributed as a restoration impact. Conquest (2000) argues that the only way to address this variation is to incorporate replicate control and impact trajectories; however, for most restoration sites this is not possible. In these cases Conquest (2000) recommends to “(1) stare hard at good graphs of the data, (2) use process knowledge in one's argument and (3) avoid having to use a p-value.” Regardless, the BACI approach represents a method to analyze the impact of restoration and final interpretation of results should be interpreted with caution, professional expertise, and a thorough knowledge of the project, site, and related system processes.

Assessing Scaled Objectives

The sampling approach and analysis will be similar for both the restoration and estuary-wide scales in many regards but will differ by spatial and temporal extent, and by the structure of the data. For example, when addressing site-specific rates of sediment accretion at the restoration scale, measure elevation and water levels using methods deployed within the footprint of the restoration site compared to those at the reference site. In contrast, to test a similar hypothesis at the estuary scale, the same methods will be used to measure elevation and water levels, with the comparison occurring across time as well as across the spatial scale of the estuary.

Change Detection

At the most fundamental level, monitoring using consistent methods will yield information that can be summarized by status and trends or changes over time or rates of change. Change detection, although simple, can be rather complicated by the fact that detection may change over time, data may vary by observer bias, and natural variability exists within the ecosystem. The ability to differentiate a change due to the restoration rather than data variability is another complicating factor that can be addressed through a scientific framework of repeated analyses, long term datasets for spatial and temporal trends, and monitoring within a statistical framework.

Sample Sizes

In some cases, it is helpful to assess the sample size needed to detect a certain amount of change using power analysis (Zar 1999). However, most restoration monitoring is determined by the scale of the restoration, the amount of variability across the site, the desired change detection level and practical limits of cost and labor.

The focus of most data analyses for monitoring restoration changes are those that provide trends through time. However, the amount of change that occurs during early restoration is accelerated, so we recommended a 5-10 year period of more intensive monitoring that allows for adaptive management. After the early restoration period, the restoration trajectory is established to a greater extent and a less frequent monitoring schedule of 3-5 years is appropriate.

Data Management

All data should be compiled, checked for quality assurance and control based on monitoring metric, and archived on a data server with mirrored drives, tape backup, and redundant copies at backup location. Field data should be referenced in GIS coverages.

Hard copies of field books and data sheets should be made upon return from the field. Data should be entered immediately and entered data should be checked by a different individual. Raw data from data loggers and field surveys should be backed up onto DVDs monthly. Post-processed data should be stored in readily accessible formats such as excel files, jpegs, and shape files, and should be backed up onto DVDs monthly. In addition, if a system network is available, data should be backed up onto the network to ensure its safety.


Restoring natural estuarine processes are ever-changing, which results in a shifting mosaic of habitat types. Models are useful tools to predict potential end points of restoration actions and incorporate uncertainties. Adaptive modeling approaches can assist landscape scale restorations achieve their goals and understand system-scale processes (Zedler 2005). Given the dynamic processes at play within ecosystems including external regulatory processes, the influence of natural and human disturbances, and random effects (e.g. storm events), conceptual and predictive models are important to understanding the uncertainties and potential trajectories of these open systems (Pickett and Parker 1994; Parker and Pickett 1997; Wissmar and Bisson 2003, Zedler 2005). Models can be used to represent restoration and resilience potential (Bennett et al. 2005), attempt to incorporate uncertainties, and take into account multiple system processes.

Conceptual and predictive models are two of the primary types of models used in restoration science, design and implementation. Conceptual models are a representation of a real world natural system and its interconnected components, often in the form of diagrams. Conceptual models are relatively inexpensive to produce and there are many available for tidal marsh ecosystem processes that can be easily customized for specific projects and study sites. Predictive models incorporate current data to allow you to establish reasonable expectations for physical and biological outcomes from restoration, as well as understand the cumulative effects of multiple projects on long term estuary function and resilience. These models require monitoring data for both calibration and validation and can be very expensive to produce, often requiring specialized training and software.


Conceptual model of the Port Susan Bay ecosystem from Woo et al. 2011. Box shading corresponds to the number of other attributes affected by each component (darker = more attributes affected). Model structure is based on Thom et al. (2004; click to view larger).


Bennett, E.M. 2005. A systems model approach to determining resilience surrogates for case studies. Ecosystems 8:945-957.

Conquest, L. L. 2000. Analysis and Interpretation of Ecological Field Data Using BACI Designs: Discussion. Journal of Agricultural, Biological, and Environmental Statistics 5(3):293-296.

Parker, V.T. and S.T.A Pickett. 1997. Restoration as an ecosystem process: implications of the modern ecological paradigm. Pp17-23 In: Urbanska, K.M., Webb, N.R., Edwards, P.J., editors. Restoration ecology and sustainable development. Cambridge (UK): Cambridge University Press.

Pickett, S.T.A. and Parker, V.T. 1994. Avoiding the old pitfalls: opportunities in a new discipline. Restoration Ecology 2:75-79.

Roman, C.T., Raposa, K.B., Adamowicz, S.C., James-Pirri, M., and J.G. Catena. 2002. Quantifying vegetation and nekton response to tidal restoration of a New England salt marsh. Restoration Ecology 10(3): 450-460.

Thom, R.M., A.B. Borde, N.R. Evans, C.W. May, G.E. Johnson, and J.A. Ward. 2004. A Conceptual Model for the Lower Columbia River Estuary Prepared by Pacific Northwest National Laboratory for the U.S. Army Corps of Engineers, Portland District.

Smith, P.E., Orvos, D.R., and J. Cairns. 1993. Impact assessment using the before-after-control-impact (BACI) model: concerns and comments. Canadian Journal of Fisheries and Aquatic Sciences 50:627-637.

Stewart-Oaten, A., W. W. Murdoch, and K. R. Parker. 1986. Environmental Impact Assessment: “pseudoreplication” in time? Ecology 67:929-940.

Stewart-Oaten, A., J. R. Bence, and C. W. Osenberg. 1992. Assessing effects of unreplicated perturbations: no simple solutions. Ecology 73:1396-1404.

Stewart-Oaten, A. 2003. On rejection rates of paired intervention analysis: comment. Ecology 84:2795-2799.

Underwood, A. J. 1992. Beyond BACI: the detection of environmental impacts on populations in the real, but variable, world. J. Exp. Mar. Biol. Ecol. 161:145-178.

Wissmar, R.C., and P.A. Bisson, editors. 2003. Strategies for restoring river ecosystems: sources of variability and uncertainty in natural and managed systems. Bethesda, MD: American Fisheries Society.

Zar, J.H. 1999. Biostatistical Analysis. Fourth edition. Upper. Saddle River, NJ: Prentice Hall, 663 pp.

Zedler, J.B. 2005. Ecological restoration: guidance from theory. San Francisco Estuary and Watershed Science 3(2):1-31.