Balancing environmental representativeness and biological integrity to establish a

stream reference condition program

Authors: Peter Ode, Andrew Rehn, Raphael Mazor, Kenneth Schiff, Eric Stein, Jason May, Larry
Brown and David Gillett

Peter Ode¹, Andrew Rehn¹, Raphael Mazor 1,2, Kenneth Schiff2, Eric Stein2, Jason May3, Larry Brown3 and
David Gillett2
1
Aquatic Bioassessment Laboratory, Water Pollution Control Laboratory, California Department of Fish
and Game, 2005 Nimbus Road, Rancho Cordova, CA 95670
2
Southern California Coastal Water Research Project, 3535 Harbor Blvd., Suite 110, Costa Mesa, CA
3
United States Geological Survey, 6000 J Street, Sacramento, CA 95819

Abstract: <to be added later>

1
Introduction

The worldwide use of biological indicators in water quality monitoring programs has evolved
rapidly in the last 20 years (Rosenberg and Resh 1993, Gibson et al. 1996, Simpson and Norris
2000, Bonada et al. 2006). Many of the refinements to biological monitoring techniques over
this period have centered on strengthening the theoretical and practical basis for predicting the
biological expectation for sites under low levels of human-derived disturbance, the “reference
state” or “reference condition” (Hughes et al. 1986, Reynoldson et al. 1997, Stoddard et al.
2006 , reviewed by Bonada et al. 2006 and Hawkins et al. 2010a). As a result, the need to
anchor biological expectations in a reference state is now widely regarded as a fundamental
requirement: to the extent possible, the expected biological state of a monitoring site should
be predicted from the biological state observed at sites having similar environmental settings,
but low levels of human disturbance.

Although early efforts to select reference sites often relied on subjective criteria and best
professional judgment (e.g., Wright et al. 1984, Hughes et al. 1986, Barbour et al. 1995, 1996,
Reynoldson et al. 1995, 1999), there is increasing recognition that objective criteria can
enhance the definition of reference condition (Whittier et al. 2007, Yates and Bailey 2010).
Examples of objective site selection are increasingly common (e.g., Ode et al. 2005, Stoddard et
al. 2005, Yates and Bailey 2010). A robust approach to selecting reference sites in
environmentally complex landscapes should account for a variety of potential stressor types.
However, multiple criteria can complicate the achievement of uniform reference definitions in
such complex regions (Statzner et al. 2001, Herlihy et al. 2008, Mykrä et al. 2008, Ode et al.
2008).

Evaluating Performance of Reference Criteria

The central challenge in developing objective criteria for selecting reference sites (i.e.,
reference criteria) for diverse regions is the need to balance two potentially conflicting
demands: 1) reference criteria should select sites that uniformly represent minimally disturbed
conditions throughout the region of interest, 2) reference sites should represent stream types
from the full range of environmental settings in the region with adequate numbers of sites to
support robust scoring tools. Because meeting the second demand usually requires at least
some loosening of reference screening criteria, reference site selection becomes an exercise in
balancing Type I error (the risk of keeping disturbed sites in the reference pool) and Type II
error (unnecessarily rejecting minimally disturbed sites from under-represented stream types).

In a perfect world with a large number of undisturbed streams of all types, we could focus
exclusively on Type I error. However, overly restrictive criteria can result in under-
representation of important natural gradients (Mapstone 2006, Osenberg et al. 2006). Thus,

2
excessive rejection of candidate sites reduces the performance (i.e., accuracy and precision) of
scoring tools. Furthermore, the relationship between reference sites and reference condition is
imperfect. We cannot be omniscient about all sites in our reference pools; errors in reference
assignment can occur in both directions because of inaccurate or incomplete information about
natural and anthropogenic stressors. An effective screening process must therefore balance
Type I and Type II error, with a goal of defining the most restrictive set of reference criteria that
allow us to represent all major natural gradients.

Evaluating the performance of reference criteria allows scientists and resource managers to
make informed decisions about this balance. This paper outlines the use of one approach to
evaluate performance in California, an environmentally complex region of the USA. This work
built on previous attempts to identify reference in similarly complex regions (e.g., Chaves et al.
2006, Coller et al. 2007, Sánchez-Montoya et al. 2009, Falcone et al. 2010, Yates and Bailey
2010) We drew on these efforts to identify an initial suite of screens and thresholds, expanded
them to accommodate a broad array of stressors known to be important in California (Ode et al
in review), then evaluated the degree to which we balanced our objectives.

The objectives of this paper are: 1) to describe the development process we used for
establishing reference criteria and 2) to describe the approach we used to evaluate the degree
to which the needs of representativeness and biological integrity are balanced in the final
reference pool.

Methods

A set of 1637 candidate sites representing a wide range of stream types was assembled to
support the development of screening criteria. Each site was characterized with a suite of
landuse and landcover metrics for both its natural characteristics and for potential
anthropogenic stressors. Sites were then screened with a subset of metrics using thresholds
that represented low levels of anthropogenic stress. Finally, the pool of passing reference sites
was evaluated to assess whether the objectives of balancing Type I and Type II error were
achieved to a degree sufficient to support defensible biocriteria.

Setting – California’s stream network is approximately 280,000 km long (approximately 30% of

which consists of perennial streams) and drains a large (424,000 km²) and remarkably diverse
landscape. Spanning latitudes between 33° and 42° (N), California’s geography is characterized
by its extremes. California boasts both the highest and lowest elevations in the continental US
and its ecoregions range from temperate rainforests in the Northwest to deserts in the
Northeast and Southeast, with the majority of the state having a Mediterranean climate
(Omernik 1987). California’s geology is also complex, ranging from southern coast ranges
comprised of recently uplifted and poorly consolidated marine sediments to granitic batholiths

3
along the eastern border to recent volcanism in the northern mountains. This geographical
diversity is associated with a large degree of biological diversity and endemism in the stream
fauna (Erman 1996, Moyle et al. 1996, Moyle and Randall 1996). California’s natural diversity is
further complicated by an equally complex pattern of landuse. The native landscapes of some
regions of the state have been nearly completely converted to agricultural or urban land uses
(e.g., the Central Valley, the San Francisco Bay area and the South Coast) (Sleeter et al. 2011).
Other regions are still largely natural but contain pockets of agricultural and urban landuse and
also support timber harvest, livestock grazing, mining and recreational uses.

To facilitate analysis, the state was divided into six regions based on ecoregional, hydrological,
and political boundaries (Figure 1): The North Coast, Chaparral, South Coast, Central Valley,
Sierra Nevada, and combined Deserts + Modoc Plateau (see Ode and Schiff 2009) and [will cite
state PSA summary report when it is released]. Three of these regions were subdivided further
to reflect known or suspected differences in biological communities: the Sierra Nevada was
subdivided into the Western Sierra and Central Lahontan; the Chaparral was subdivided into
Interior and Coastal regions, and the South Coast was divided into the Mountains and low
elevation Xeric region. The first two regional subdivisions are described in Ode and Schiff
(2009), and the subdivisions in the South Coast follow Omernik (1987).

Assemblage of site data

More than 20 federal, state, and regional monitoring programs were inventoried to assemble
data sets used for screening reference sites (Table 1). From the population of several
thousands of CA sites with bioassessment data, sites were prioritized for inclusion if they met at
least one of two criteria: 1) they were at least reasonably likely to pass screening thresholds
(e.g., ones identified as reference in previous IBI or O/E development), 2) they were sampled
under probabilistic survey designs. When multiple programs sampled identical candidate sites
or sites in close proximity (within 300 m), data were aggregated under a single site
identification code to minimize redundancy. All unique sites sampled between 2000 and 2009
were aggregated into a single database (Figure 1).

Assembled data included benthic macroinvertebrate (BMI) taxa lists, water chemistry and
physical habitat characteristics. Field protocols often varied among programs and not all
programs collected all data types, but most analytes were available for most sites (Table 3).
The majority of BMI data were collected using the reachwide protocol of the US EPA’s
Environmental Monitoring and Assessment Program (EMAP, Peck et al. 2006), but some were
collected with targeted riffle protocols (Herbst and Silldorff 2006, Rehn et al. 2007). Previous
studies have documented the comparability of these protocols (Ode et al. 2005, Gerth and
Herlihy 2006, Herbst and Silldorff 2006, Rehn et al. 2007 ). BMI taxa lists were standardized for
calculation of two different types of scoring tools widely used by California monitoring

4
programs: multi-metric indices (IBI) and multivariate predictive models (O/E). Full description
of these indices and their development is beyond the scope of this paper, but for IBIs can be
found in Ode et al. (2005), Rehn et al. (2005) and Rehn (2009). O/E scores were calculated via
an unpublished predictive model developed by C. Hawkins at Utah State University, but see
Ode et al. 2008 and for partial documentation. Calculation of IBIs required 500-count BMI
samples identified to standard taxonomic effort levels defined for bioassessment in California
(Richards and Rogers 2006). Calculation of O/E required 300-count BMI samples with taxa lists
converted to operational taxonomic units (OTUs) used by the predictive model (Hawkins
unpublished).

For calculation of physical habitat metrics, preference was given to programs that used
quantitative field protocols (e.g., Peck et al. 2006, Ode 2007) and allowed calculation of
quantitative variables defined by Kaufmann et al. (1999). One of Kaufmann’s metrics,
“W1_HALL”, a composite index of human disturbance activity integrated over each sampling
reach, was emphasized here. The measure includes evidence of many types of near stream
human-related activity (channel alteration, trash, grazing, roads, etc.) and each variable is
weighted by proximity to the wetted channel. Water chemistry protocols varied widely, with
some programs collecting only in situ probe data (pH, dissolved oxygen, conductivity and
temperature) and other programs collecting water samples for lab analysis of nutrients,
chloride, metals, total suspended solids, etc. This restricted the sample sizes available for some
analyses (Table 3).

Integration of probability data sets

A subset of the data set consisting of sites collected under probabilistic survey designs was used
to evaluate whether our final pool of reference sites adequately represented the full range of
natural stream types occurring in California. Data from 10 probabilistic surveys were combined
for this effort. Although most surveys had similar design characteristics, they were different
enough to require synchronization before they could be integrated. First a common sample
frame was created so that weights (which define the relative contribution of each site to the
overall distribution) could be calculated for each site in the combined data set. All probabilistic
sites were registered to a common stream network (NHD 1:100,000), which was attributed with
strata defined by the design parameters of all integrated programs (e.g., land use, stream
order, survey boundaries, etc.). Since not all design variables were present in each survey,
many strata (e.g., 1st order, forested sites in the Central Valley) were not represented by sites,
or had very few sites. Strata with fewer than 5 sites were aggregated with similar sites until all
strata contained at least 5 sites. Weights were calculated for each site by dividing total stream
length in each stratum by the number of site evaluations in that stratum. All weight

5
calculations were conducted using the spsurvey package (Kincaid and Olsen 2009) in R v 2.11.1
(The R Foundation for Statistical Computing 2010).

GIS data and metric calculation

A large number of spatial data sources were aggregated to characterize natural and
anthropogenic gradients that may affect biological condition at each site, such as land cover
and land use, population density, road density, hydrologic alteration, mining, geology, elevation
and climate (Table 3). Data sets were evaluated for statewide consistency and layers with poor
or variable reliability were avoided. All spatial data sources were publicly available except for
the roads layer, which was customized for this project by appending unimproved and logging
roads obtained from the United States Forest Service and California Department of Forestry
and Fire Protection to a base roads layer (TeleAtlas 2009).

Land cover, land use and other measures of human activity were quantified into metrics (Table
3) that were calculated at three spatial scales: within the entire upstream drainage area
(watershed), within 5 km upstream and within 1 km upstream. Polygons defining these spatial
analysis units were created using ArcGIS tools (ESRI 2009). Upstream watershed polygons were
aligned to NHD polygons and the downstream portion of each watershed was adjusted with
standard flow direction and flow accumulation techniques using 30 m digital elevation models
(NED). The local (5k and 1k) scales were created by intersecting a 5km or 1km radius circle with
the primary watershed polygon. “Point” metrics associated with each sampling location also
were calculated based on each sites latitude and longitude (e.g., mean annual temperature,
elevation, NHD+ attributes, etc.).

Selection of screening metrics and thresholds

In defining criteria for acceptable amounts of human disturbance, our goal was to balance
two needs: 1) the need for reference condition to be adequately protective of stream biota,
and 2) the need for sufficient numbers of sites to support adequate characterization of
reference condition across the full range of natural stream settings occurring in California.

A primary set of screening metrics was selected based on land use frequently associated with
impairment to the biological condition in streams and rivers. The specific metrics and
thresholds were initially identified from a combination of prior reference development (Ode et
al. 2005; Stoddard et al. 2005) or values obtained from literature (e.g., Collier et al. 2007,
Angradi et al. 2009, Falcone et al. 2010). This initial list was augmented after examining the
distribution of stressors in watersheds in California (Ode et al. In Prep). Best professional
judgment was used to set thresholds for metrics or particular spatial scales (e.g., 1k or 5k) that
lacked published values.

6
A set of secondary thresholds, or “kill switches”, was established to further refine reference site
selection (i.e., to exclude sites that passed primary screens yet still showed some indication of
stress). In contrast to our primary screens, secondary thresholds were not chosen to minimize
biological response to stressor gradients but to eliminate highly stressed sites that were not
eliminated by primary metrics. This approach was used instead of strict reliance on primary
thresholds for all metrics to help achieve a balance of Type I and Type II errors. Secondary
thresholds were applied in the same manner as primary screens but were intentionally set at
higher values: 1) for nitrogen and phosphorous concentrations which can be naturally high in
certain geological settings and do not necessarily indicate human disturbance, 2) for land use at
the watershed scale because distant disturbance generally has less impact on biological
condition than near-site disturbance (Munn et al. 2009), and 3 ) for number of upstream road
crossings because inaccuracies in GIS layers (specifically, the line work that forms stream
networks and road layers) make this metric difficult to quantify accurately.

Performance Measures

Exploration of metric thresholds – Two performance characteristics of the screening thresholds

were evaluated directly: 1) the relative influence of different metrics in the screening process,
2) the amount of biological impact observed below reference thresholds. Both factors were
evaluated in each of six major ecological regions of the state (Figure 1).

Regions of the state differ in the relative dominance of different stressors, thus the relative
contribution of these stressors to overall disturbance at candidate sites varies regionally. To
explore these patterns, thresholds for each primary metric (and several secondary metrics)
were adjusted individually while all others were held constant and the number of passing sites
(i.e., sensitivity) was plotted per region. Examination of these partial-dependence curves was
used to evaluate the number of sites that could be added by relaxing thresholds for each
screening metric in each region.

Responsiveness of biological indices (IBI and O/E) and representative metrics to disturbance
levels allowed by our screens were evaluated because all thresholds allowed at least some
degree of upstream disturbance (i.e., none were set at zero disturbance). If Pearson’s R 2 was <
0.1 for correlations between individual stressors and BMI metrics, IBI and O/E, the biological
response to disturbance levels below reference thresholds was considered to be negligible and
thresholds were considered to be adequately protective of biological integrity. The variance in
BMI metrics and indices explained by the minimal disturbance gradient that remained in
reference sites was compared to the variance explained within the overall data set to examine
the extent to which reference thresholds minimized the impact of major stressors.

7
Evaluation of reference pool Reference sites were evaluated to ensure that the resulting data
set would be adequate for use as the foundation of statewide biocriteria. Evaluations focused
on two properties: 1) the number of reference sites identified, both statewide and within major
regions of California (i.e., adequacy), and 2) the effectiveness of those reference sites at
capturing important environmental gradients (i.e., representativeness).

The adequacy of the reference site density for developing biological integrity models was
assessed by counting the number of reference sites statewide and within major sub-regions. A
target minimum number of sites was not set, but 30-50 reference sites are likely to be
adequate to capture natural biological variability at the scale of our sub-regions. If fewer than
30 reference sites were available in a given region, these sites would need to be aggregated
with another region or the region would need to be excluded from subsequent reference-based
analyses.

The representativeness of reference sites was evaluated by comparing the distribution of

reference sites along important natural gradients against the distribution of all sites, as inferred
from our probabilistic data. Natural gradients were chosen for analysis based on their expected
relationship with biological community structure. These variables included: watershed area,
elevation, predicted conductivity (Olsen and Hawkins, in review), slope, % CaO geology, %
sedimentary geology, and precipitation.

Note: We might add a section here comparing O/E scores in different regions to the final paper, but we’ll
use our new O/E model for this (currently under construction). This might also go in the O/E
development and application paper.

Results

Reference status by region

Of the 1637 sites included in analysis, 615 (i.e., 38%) were identified as potential reference sites
after applying screening thresholds (Table 4). The density of reference sites varied widely by
region, with highest concentrations in mountainous regions (e.g., the Sierra Nevada, the North
Coast and South Coast Mountains), which also contain the majority of the state’s stream length
(Ode et al. 2010). Lower elevation, drier sub-regions generally had few reference sites (South
Coast Xeric = 22, Interior Chaparral = 30), and only a single reference site was identified in the
Central Valley.

Based on the probability data, 36% (± 3% standard error) of California’s stream-length was
estimated to meet our reference criteria (Table 6). Reference quality streams were
predominant in mountainous regions of California, comprising approximately 70% of the
stream length in the Central Lahontan and South Coast Mountain regions. Only 2% of streams

8
in the Central Valley and the South Coast Xeric regions were estimated to be in reference,
whereas 50% of the Sierra Nevada and Deserts / Modoc streams met our reference criteria.
Perhaps surprisingly, only 28% of North Coast streams were estimated to meet reference
criteria (similar to levels seen in Chaparral regions), suggesting that the abundance of reference
sites in the North Coast is due more to the overall large extent of streams than the lack of
anthropogenic stressors in the region.

Threshold sensitivity

There were strong regional differences in the number and types of stressor metrics that
contributed to the removal of individual candidate sites from the reference pool (Table 5). For
example, whereas most non-reference sites in the Sierra Nevada and the South Coast
Mountains failed only one or two metrics, a large majority (i.e., 85%) of non-reference sites in
the Central Valley and the South Coast Xeric regions failed five or more metrics. The other
regions had intermediate values. 28% of Chaparral sites were rejected on the basis of only one
or two stressors, whereas 47% of Chaparral sites failed 5 or more criteria. North Coast sites and
Desert – Modoc sites were generally less stressed than Chaparral sites, with 35% and 47% of
sites failing one or two criteria, respectively, and only 17% and 19% failing more than 5 criteria,
respectively.

Related patterns were reflected in threshold sensitivity plots (Figure 2), which contrast the
relationships of the number of passing sites as a function of changing stressor thresholds using
four example metrics. Thresholds for the two landuse metrics (% agricultural land and % urban
landuse) were relatively insensitive to changes in the screening threshold values, indicating that
other metrics were limiting or co-limiting in all regions. This pattern was common for most
metrics. In contrast, road density and Code21 (an NLCD landcover class closely associated with
roadside and urban vegetation) were more sensitive to changing thresholds with strong
regional differences in this pattern. Even modest relaxation of these metrics resulted in
increased numbers of sites passing our reference screens. For road density, this was true for all
regions, but especially the North Coast and Chaparral. For Code 21, this was true for the North
Coast, Chaparral and South Coastal Mountains. We used this sensitivity to increase the
screening thresholds for road density and Code21 and thereby increased the number of sites in
several regions, improving a critical shortage in the Interior Chaparral. Thus, relaxation of these
screening thresholds improved the representation of sites in several regions.

Validation: Reference site representativeness

The large number of sites in our integrated probability data set (~750 sites) allowed us to
produce well-resolved distribution curves for a suite of natural gradients in each region (Figure
3 illustrates a handful of biologically-important gradients). For nearly all of the natural

9
gradients and regions we examined, the distribution of reference sites was very similar to the
overall distribution of sites. Large watersheds were consistently under-represented, but these
tend to be associated with non-wadeable rivers, which were not part of the scope of this effort.
Most of the other minor gaps were associated with a class of streams that represented the tails
of distributions for several related environmental variables (low elevation, low-gradient, low
precipitation, large watersheds). These gaps were largest in the South Coast region, followed by
the Chaparral, and (to a lesser extent) the North Coast. Gaps were most conspicuous for nearly
all gradients in regions with few reference sites (i.e., the Central Valley and Deserts / Modoc),
but these examples represented minor exceptions to the overall high degree of concordance
between the reference and overall distributions.

Validation: Biological response to stressors

Nearly all stressors investigated had strong, negative relationships with selected bioassessment
metrics when evaluated against the full screening data set (Table 7, Figure 3). However, these
relationships were always weaker (and frequently absent) when only reference sites were
examined. Examination of scatterplots (Figure 3) of biological scores against primary stressor
gradients indicated that biological scores did not decrease significantly below the level of our
thresholds.

The amount of biological variance in our reference sites explained by various stressors (as
contrasted to the variance in the whole dataset) is a demonstration of the amount of residual
anthropogenic impairment in our reference pool (Table 7, Figure 5). Although reference
thresholds did not completely eliminate the influence of disturbance on biological metrics in
our reference pools, this influence was greatly reduced across all metrics. Furthermore,
thresholds successfully reduced the influence of stressors that were not specifically included in
reference screens, such as population density or grazing, presumably because these stressors
are associated with other stressors included in screens (Figure 5). The low amounts of biological
variability at our reference sites that was associated with anthropogenic sources indicates that
we did not have to sacrifice significant amount of biological integrity (i.e., reduction in Type I
error) in order to achieve adequate gradient representation.

Discussion

To ensure optimal use of reference-based tools, programs need to evaluate whether selection
criteria produce a set of reference sites that are suitable for the intended uses of these data
(Bailey et al. 2004). As the focus of water quality monitoring programs shifts toward greater
emphasis on ecological condition (Rosenberg and Resh 1993, Davies and Jackson 2006),

10
reference concepts can enhance multiple components of watershed management programs,
including non-biological endpoints. Thus, thorough evaluation of performance characteristics
can pay dividends that extend beyond immediate bioassessment applications. Although
reference programs traditionally tend to focus on minimizing degradation of reference site
quality, representativeness also affects several aspects of model performance. In particular, we
argue that explicit attention to environmental representativeness could help improve overall
accuracy and reduce prediction bias (see Hawkins 2010a) in all reference applications.

Performance summary

Our reference thresholds yielded an unexpectedly large data set, with over 600 unique
reference sites distributed throughout the state. With the exception of certain regions and sub-
regions of the state (e.g., the Central Valley and the South Coastal Xeric regions), sites in the
reference pool represent nearly the full range of all the natural gradients we evaluated. Thus,
we have confidence that the vast majority of stream types present in California are well
represented in the reference pool. Although our thresholds did not eliminate all anthropogenic
disturbances from the pool of reference sites, we demonstrated that the influence of these
disturbances on the reference pool fauna has been greatly reduced, suggesting that impacts on
ecological integrity are likely to be negligible, and that Type I error will not be inflated.
Furthermore, although we anticipated that we might need to make regional adjustments in
either the choice of stressors or specific thresholds, we were able to achieve adequate
representation for most regions of the state with a common set of stressors and thresholds.
Our performance evaluations give us confidence that the balance of environmental
representativeness and biological integrity is sufficient to support robust regulatory
applications in California, and that our approach can be applied to other regions or projects
where reference sites are required.

Using the terminology of Stoddard et al. (2005), our reference approach represents a hybrid of
the “minimally disturbed” and “least disturbed” concepts. That is, our approach was anchored
in the minimally disturbed objective, but we relaxed our criteria to allow low levels of
disturbance in order to achieve our representativeness goals. We then evaluated the degree to
which our relaxed criteria introduced anthropogenic variation into our reference pool. Thus,
although we have not followed a strict interpretation of the minimally disturbed model, this
modification allowed us to achieve the balance we sought.

Managing inter-regional complexity

Most programs attempting to apply a consistent set of criteria for ecological benchmarks across
a diverse geographical and anthropogenic landscape are faced with a common problem (Herlihy
11
et al. 2008, Mykrä et al. 2008). Because regions vary in extent of disturbance (Ode et al. In
Prep), a uniform approach is likely to provide unsatisfactory results. In these cases,
classification of regions can improve the process of achieving benchmark comparability. Our
data suggest three categories that are likely to be widely applicable outside California:

Class I (Minimally-altered Regions) – Regions where minimally disturbed streams occur in

high enough frequency that it is relatively easy to find low stress sites that represent the
full range of environmental settings. In California, much of the Sierra Nevada and South
Coast Mountains meet these criteria, primarily because they include an extensive network
of state and federal parks and wilderness areas. In these regions, applying stringent
reference criteria is unlikely to reduce representativeness or increase Type II error.

Class II (Moderately-altered Regions) – Regions where minimally disturbed sites are rare
(either because streams are sparse or one or two stressors are widespread), but
additional sites can be added with additional sampling effort, or only moderate relaxation
of selection criteria. The Chaparral regions and the Desert / Modoc regions have low
densities of perennial streams, as is typical of arid regions, but even slight relaxation of
road density criteria greatly increased the number of passing sites in these regions
(especially in the Chaparral). In contrast, the North Coast region has low development in
much of the region and might be expected to meet the Class I definition, but pervasive
timber harvest in the late 19th and early 20th centuries have left a legacy of impacts,
including extensive road networks that were detected by our screens. In these regions, it
may be impossible to use stringent reference criteria without unacceptably increasing
Type II error, so impacts on Type I error should be evaluated if criteria are relaxed
throughout much of the region.

Class III (Pervasively-altered Regions) – Regions where anthropogenic alteration is so

pervasive that even moderate relaxation of reference criteria is not sufficient. In these
cases, the choice is to radically relax the definition of reference (and weaken the link to
natural condition) or to develop a separate process for setting condition benchmarks. In
California, the highly developed Central Valley and South Coast Xeric regions both require
alternate approaches to setting benchmarks. In the South Coast Xeric, the link to natural
condition may be weakened by the paucity of reference sites, but in the Central Valley,
the link may be abandoned altogether. That is, the traditional approach to defining
reference is not likely to achieve a desirable balance between Type I and Type II error in
these regions.

This categorization strategy allows programs to use a common framework for reference
streams in Class I and Class II regions while minimizing the need to compromise biological
integrity. Because relaxing thresholds potentially degrades biological integrity, it is critical that

12
impacts to biological integrity be quantified in Class I and Class II regions (as we did in this
study). Class III regions must be treated differently. Several authors describe “non-standard”
approaches, such as synthetic approaches to model reference sites based on empirical or
theoretical relationships with stress (e.g., Carter and Fend 2005, Chessman 1999, Chessman
and Royal 2004). Regardless of which alternate approach is used, benchmarks in Class III
regions will need to be related to those used for Class I/II regions in order to make sensible
state-wide assessments and management decisions (see Herlihy et al. 2008, Bennett et al.
2011).

Applications of the reference condition approach

A well-established reference pool has several applications for stream and watershed
management. Reference concepts provide defensible regulatory frameworks for protecting
and managing aquatic resources, and providing a “common currency” for the integration of
multiple biological indicators (e.g., algal and fish assemblages). Beyond perennial streams, the
approach outlined in this paper can be used to define reference sites for a wide range of habitat
types, including non-perennial streams, lakes, depressional wetlands, and estuaries (e.g., Solek
et al. 2010). Further, the process of defining reference criteria can be used to identify streams
and watersheds deserving of special protections and application of anti-degradation policies,
which are often under-applied in the United States and globally (Linke et al. 2011, Collier 2011).
Two general applications extend these uses to management of non-biological parameters:

Objective Regulatory Thresholds for Non-Biological Indicators – The process of establishing

regulatory standards for management water quality parameters with non-zero expected
values (e.g., nutrients, chloride, conductivity, fine sediment) is more subjective than for
novel pollutants like pesticides. The range of parameter values found at reference sites can
help standardize the way regulatory benchmarks are set for these pollutants. Examples of
this concept have appeared in peer-reviewed literature (Yates and Bailey 2010, see Hawkins
et al. 2010 for a review variety of physical and chemical endpoints), but management
applications are rare.

Context for Interpreting Targeted and Probabilistic Monitoring Data – Comparisons of

reference to the full range of stressor values affecting a site (i.e., as obtained from
probability surveys, see Ode et al. In Prep) can establish a framework for evaluating the
success of site-specific restoration projects. This framework gives management programs
the ability to distinguish between relatively small differences in pollutant concentration and
environmentally meaningful differences.

13
Limits of this analysis

Two major types of data limitations have potentially large impacts on any approach to identify
reference sites:1) Inadequate or inaccurate GIS layers; and 2) lack of information about reach
scale stressors. Although improvements in availability and accuracy of spatial data over the last
two decades have greatly enhanced our ability to apply consistent screening criteria across
large areas, reliance on these screens can underestimate impairment (Yates and Bailey 2010).
The most accurate and uniform spatial data tend to be associated with urban and agricultural
stressors (e.g., landcover, roads, hydrologic alteration), so impacts in non-agricultural rural
areas (like timber harvest, recreation, grazing, invasive species) are typically underestimated.
Reach scale stressors (proximate stressors) have a large influence on aquatic assemblages (e.g.,
Waite et al. 2000, Munn et al. 2009, ), but are challenging to assess unless adequate
quantitative data were collected along with biological samples, as this context is often essential
for interpreting proximate sources of stress (e.g., Poff et al. 2009). We were fortunate to have
access to good reach scale chemical and physical habitat data at many sites, but we
undoubtedly missed locally important variables in some cases.

Managing the Reference Pool

Once a reference pool is established, it needs to be monitored over time to track inter-and
intra-annual variability due to natural or anthropogenic changes (Mazor et al. 2009). Programs
need long-term, routine monitoring of reference sites to incorporate the impacts of rare (e.g.,
debris flows, Cover et al. 2010, Herbst and Cooper 2010) and common (e.g., wildfires, Minshall
2003) natural disturbances, and anthropogenic changes that have very widespread impacts (like
climate change, invasive species, aerial deposition, ozone depletion) (Jackson and Füreder
2006, Lawrence et al. 2010, Resh et al. in review). Large scale monitoring of the reference pool
over time (i.e., 50 sites/ year) may have sufficient resolution to document issues of specific
management interest (e.g., climate change), but it would be more cost effective to target
routine reference site sampling to key areas of interest (Mazor et al. 2009, Herbst and Cooper
2010, Hamilton et al. 2010).

Likewise, a reference management plan should develop guidance for waiting periods post-
natural disturbance events (e.g., wildfires, debris flows), during which reach scale data (habitat,
chemistry, biology) are not included in the data summaries. Very large reference sets may be
robust enough to establish post-disturbance waiting periods, or to allow explicit incorporation
of these impacts in model development.

14
Acknowledgements <to be added later>

15
Literature Cited

Abramson, M., J. Topel, and H. Burdick. 2009. New Zealand Mudsnail Surveys: July 2006, July
2007, October 2008, and April 2009: Santa Monica Mountains. Santa Monica Bay
Restoration Commission. Santa Monica, CA.

Angradi, T.R., M.S. Pearson, T.M. Jicha, D.L. Taylor, D.W. Bolgrien, M.F. Moffett, K.A. Blocksom,
and B.H. Hill. 2009. Using stressor gradients to determine reference expectations for great
river fish assemblages. Ecological Indicators. 9: 748-764.

Bailey, R.C., R.H. Norris, and T.B. Reynoldson. 2004. Bioassessment of freshwater ecosystems:
Using the reference condition approach. Kluwer Academic Publishers. Boston, MA.

Baker, M.E., M.J. Wiley, M.L. Carlson, and P.W. Seelbach. 2003. A GIS model of subsurface
water potential for aquatic resource inventory, assessment, and environmental
management. Environmental Management. 32: 706-719.

Barbour, M.T., J.B. Stribling, and J.R. Karr. 1995. Multimetric approach for establishing
biocriteria and measuring biological condition. Pages 63-77 in W.S. Davis and T.P. Simon
(editors). Biolgoical Assessment and Criteria: Tools for water resource planning and decision
making. Lewis Pubishers. Boca Raton, FL.

Barbour, M.T., J. Gerritsen, G.E. Griffith, R. Frydenborg, E. McCarron, J.S. White, and M.L.
Bastian. 1996. A framework for biological criteria for Florida streams using benthic
macroinvertebrates. JNABS. 15: 185-211.

Bennett, C., R. Owen, S. Birk, A. Buffagni, S. Erba, N. Mengin, J Murray-Bligh, G. Ofenböck, I.

Pardo, W. van de Bund, F. Wagner, and J.-G. Wasson. 2011. Bringing European river quality
into line: an exercise to intercalibrate macro-invertebrate classification methods.
Hydrobiologia. 667: 31-48.

Bonada, N., N. Prat, V.H. Resh, and B. Statzner. 2006. Developments in Aquatic Insect
Biomonitoring: A comparative analysis of recent approaches. Annual Review of Entomology.
51: 495 – 523.

Carter, J.L and S.V. Fend. 2005. Setting limits: the development and use of factor-ceiling
distributions for an urban assessment using benthic macroinvertebrates. Pages 179 – 191 in
L.R. Brown, R.H. Gray, R.M. Hughes, and M.R. Meador, editors. Effects of urbanization on
stream ecosystems. American Fisheries Society Symposium 47. Bethesda, MD.X: (in press).

Chessman, B.C. 1999. Predicting the macroinvertebrate faunas of rivers by multiple regression
of biological and environmental differences. Freshwater Biology 41: 747-757.

16
Chessman, B.C and M.J. Royal. 2004. Bioassessment without reference sites: use of
environmental filters to predict natural assemblages of river macroinvertebrates. Journal of
the North American Benthological Society 23: 599-615.

Collier, K. 2011. The rapid rise of streams and rivers in conservation assessment. Aquatic
Conservation: Marine and Freshwater Ecosystems. 21: 397 – 400.

Collier, K.J., A. Haigh, and J. Kelly. 2007. Coupling GIS and multivariate approaches to reference
site selection for wadeable stream monitoring. Environmental Monitoring and Assessment.
127: 29-45.

Cover, M.R., J.A. de la Fuente, and V.H. Resh. 2010. Catastrophic disturbances in headwater
streams: The long-term ecological effects of debris flows and debris floods in the Klamath
Mountains, northern California. Canadian Journal of Fisheries and Aquatic Science. 67:
1596-1610.

Davies, S.P. and S.K. Jackson. 2006. The biological condition gradient: A descriptive model for
interpreting change in aquatic ecosystems. Ecological Applications. 16: 1251-1266.

Falcone, J.A., D.M. Carlisle, and L.C., Weber. 2010. Quantifying human disturbance in
watersheds: Variable selection and performance of a GIS- based disturbance index for
predicting the biological condition of perennial streams. Ecological Indicators. 10: 264-273.

Fetscher, E.A., L. Busse, and P.R. Ode. 2009. Standard operating procedures for collecting
stream algae samples and associated physical habitat and chemical data for ambient
bioassessment in California. Surface Water Ambient Monitoring Program. Sacramento, CA.

Gerth, W.J. and A.T. Herlihy. 2006. The effect of sampling different habitat types in regional
macroinvertebrate bioassessment surveys. The Journal of the North American Benthological
Society. 25: 501-512.

Gibson, J.R., M.T. Barbour, J.B. Stribling, J. Gerritsen, and J.R. Karr. 1996. Biological criteria:
Technical guidance for streams and rivers (revised edition). EPA 822-B-96-001. Office of
Water. US Environmental Protection Agency. Washington, DC.

Hamilton, A.T., J.D. Stampl, and B.G. Bierwagen. 2010. Vulnerability of biological metrics and
multimetric indices to the effects of climate change. Journal of the North American
Benthological Society. 29: 1379-1396.

Hawkins, C.P., R.H. Norris, J. Gerritsen, R.M. Hughes, S.K. Jackson, R.K. Johnson, and R.J.
Stevenson. 2000a. Evaluation of the use of landscape classifications for the prediction of

17
 freshwater biota: Synthesis and recommendations. Journal of the North American
Benthological Soceity. 19: 541-556.

Hawkins, C.P., R.H. Norris, J.N. Hogue, and J.W. Feminella. 2000b. Development and evaluation
of predictive models for measuring the biological integrity of streams. Ecological
Applications 10: 1456-1477.

Hawkins, C.P. J.R. Olson, and R.A. Hill. 2010a. The Reference Condition: Predicting benchmarks
for ecological water-quality assessments. Journal of the North American Benthological
Society. 29: 312-343.

Hawkins, C.P., Y. Cao, and B. Roper. 2010b. Method of predicting reference condition biota
affects the performance and interpretation of ecological indices. Freshwater Biology. 55:
1066-1085.

Herbst, D.B. and E.L. Silldorff. 2006. Comparison of the performance of different bioassessment
methods: similar evaluations of biotic integrity from separate programs and procedures.
Journal of the North American Benthological Society. 25: 513-530.

Herbst, D.B. and S.D. Cooper. 2010. Before and after the deluge: Rain-on-snow flooding effects
on aquatic invertebrate communities of small streams in the Sierra Nevada, California.
Journal of the North American Benthological Society. 29: 1354-1366.

Herlihy, A.T., S.G. Paulsen, J. Van Sickle, J.L. Stoddard, C.P. Hawkins, and L. Yuan. 2008. Striving
for consistency in a national assessment: The challenges of applying a reference condition
approach on a continental scale. Journal of the North American Benthological Society. 27:
860-877.

Hughes, R.M., D.P. Larsen, and J.M. Omernik. 1986. Regional reference sites: A method for
assessing stream potentials. Environmental Management. 10: 629-635.

Kaufmann, P.R., P. Levine, E.G. Robinson, C. Seeliger, and D.V. Peck. 1999. Quantifying physical
habitat in wadeable streams. EPA/620/R-99/003. US Environmental Protection Agency.
Research Ecology Branch. Corvallis, OR.

Kincaid, T. and T. Olsen. 2009. SPSurvey package for R. Available from

http://www.epa.gov/nheerl/arm

Lawrence, J.E., K.B. Lunde, R.D. Mazor, L.A. Bêche, E.P. McElravy, and V.H. Resh. Long-term
macroinvertebrate responses to climate change: Implications for biological assessment in
Mediterranean-climate streams. Journal of the North American Benthological Society. 49:
1424-1440.

18
Linke, S. , E. Turak, and J. Neale. 2011. Freshwater conservation planning: The case for
systematic approaches. Freshwater Biology. 56: 6 – 20.

Mapstone, B.D. 2006. Scalable decision criteria for environmental impact assessment: Effect
size, Type I, and Type II errors. Pages 67-82 in R.J. Schmitt and C.W. Osenberg, editors.
Detecting Ecological Impacts. Academic Press. New York, NY.

Mazor, R.D., T.B. Reynoldson, D.M. Rosenberg, and V.H. Resh. 2006. Effects of biotic
assemblage, classification, and assessment method on bioassessment performance.
Canadian Journal of Fisheries and Aquatic Sciences. 63: 394-411.

Mazor R.D., A.H. Purcell, and V.H. Resh. 2009. Long-term variability in bioassessments: A
twenty-year study from two Northern California streams. Environmental Management. 43:
1269-1286.

Meinshausen, N. 2006. Quantile regression forests. Journal of Machine Learning Research. 7:

983-999.

Minshall, G.W. 2003. Responses of stream benthic macroinvertebrates to fire. Forest Ecology
and Management. 178: 155-161.

Munn, M.D., I.R. Waite, D.P. Larsen, and A.T. Herlihy. 2009. The relative influence of geographic
location and reach-scale habitat on benthic invertebrate assemblage in six ecoregions.
Environmental Monitoring and Assessment. 154: 1-14.

Mykrä H., J. Aroviita, J. Kotanen, H. Hämäläinen, and T. Muotka. 2008. Predicting the stream
macroinvertebrate fauna across regional scales: influence of geographical extent on model
performance. Journal of the North American Benthological Society. 27: 705-716.

Ode, P.R. 2007. Standard operating procedures for collecting benthic macroinvertebrate
samples and associated physical and chemical data for ambient bioassessment in California.
Surface Water Ambient Monitoring Program. Sacramento, CA.

Ode, P.R., and K. Schiff. 2009. Recommendations for the development and maintenance of a
reference condition management program (RCMP) to support biological assessment of
California’s wadeable streams. Report to the State Water Resources Control Board’s Surface
Water Ambient Monitoring Program (SWAMP). Technical Report 581. Southern California
Coastal Water Research Project. Costa Mesa, CA.

Ode, P.R., A.C. Rehn, and J.T. May. 2005. A quantitative tool for assessing the integrity of
Southern California coastal streams. Environmental Management. 35: 493-504.

19
Ode, P.R., C.P. Hawkins, and R.D. Mazor. 2008. Comparability of biological assessments derived
from predictive models and multimetric indices of increasing geographic scope. Journal of
the North American Benthological Society. 27: 967-985.

Ode, P.R., K. Schiff, R.D. Mazor, D.J. Gillett, K. Ritter, A. Rehn, E. Stein, J. May, and L. Brown. In
Prep. Distribution of stressors in California’s perennial wadeable streams.

Olson, JR. and C.P. Hawkins. In review. Predicting natural base-flow stream water chemistry in
the western United States.

Osenberg, C.W. , R.J. Schmitt, S.J. Holbrook, K.E. Abu-Saba, and A.R. Flegal. 2006. Detection of
environmental impacts: Natural variability, effect size, and power analysis. Pages 83 – 108 in
R.J. Schmitt and C.W. Osenberg, editors. Detecting Ecological Impacts. Academic Press. New
York, NY.

Peck, D.V., A.T. Herlihy, B.H. Hill, R.M. Hughes, P.R. Kaufmann, D.J. Klemm, J.M. Lazorchak, F.H.
McCormick, S.A. Peterson, S.A. Ringold, T. Magee, and M. Cappaert. 2006. Environmental
Monitoring and Assessment Program—Surface Waters Western Pilot study: Field operations
manual for wadeable streams. EPA/620/R-06/003. US Environmental Protection Agency.
Office of research and Development. Corvallis, OR.

Poff, N.L, B.D. Richter, A.H. Arthington, S.E. Bunn, R.J. Naiman, E. Kendy, M. Acreman, C. Apse,
B.P. Bledsoe, M.C. Freeman, J. Henriksen, R.B. Jacobson, J.G. Kennen, D.M. Merritt, J.H.
O’Keefe, J.D. Olden, K. Rogers, R.E. Tharme, and A. Warner. 20009. The ecological limits of
hydrological alteration (ELOHA): A new framework for developing regional environmental
flow standards. Freshwater Biology, 55: 147-170.

The R Foundation for Statistical Computing. 2010. R. Version 2.11.1. Available from
http://www.r-project.org/

Rehn, A.C. 2008. Benthic macroinvertebrates as indicators of biological condition below

hydropower dams on west slope Sierra Nevada streams, California, USA. River Research and
Applications.

Rehn, A.C., P.R. Ode, and J.T. May. 2005. Development of a benthic index of biotic integrity (B-
IBI)for wadeable streams in northern coastal California and its application to regional 305(b)
assessment. Report to the State Water Resources Control Board. California Department of
Fish and Game. Aquatic Bioassessment Laboratory. Rancho Cordova, CA.

20
Rehn, A.C., P.R.Ode, and C.P. Hawkins. 2007. Comparison of targeted-riffle and reach-wide
benthic macroinvertebrate samples: implications for data sharing in stream-condition
assessments. Journal of the North American Benthological Society. 26: 332-348.

Resh, V.H. [Review of long term variability in Mediterranean climate streams]

Reynoldson, T.B. 1995. Biological guidelines for freshwater sediment based on BEnthic
Assessement of SedimenT (BEAST) using a multibariate approach for predicting biological
state. Australian Journal of Ecology. 20: 198-219.

Reynoldson, T.B., R.H. Norris, V.H. Resh, K.E. Day, and D.M. Rosenberg. 1997. The reference
condition: A comparison of multimetric and multivariate approaches to assess water quality
impairment using benthic macroinvertebrates. Journal of the North American Benthological
Society. 16: 833-852.

Richards, A.B. and D.C. Rogers. 2006. List of freshwater macroinvertebrate taxa from California
and adjacent states including Standard taxonomic effort levels. Southwest Association of
Freshwater Invertebrate Taxonomists. Chico, CA. Available from www.safit.org.

Rosenberg, D.M., T.B. Reynoldson, and V.H. Resh (1999). Establishing reference conditions for
benthic invertebrate monitoring in the Fraser River Catchment. British Columbia, Canada.
Fraser River Action Plan. Environment Canada. Vancouver, B.C. FRAP Rep. No. DOE-FRAP
1998-32.

Rosenberg, D.M., and V. Resh. 1993. Freshwater biomonitoring and benthic macro-
invertebrates. Chapman and Hall, New York.

Solek, C.W., E. Stein, J.N. Collins, L. Grenier, J.R. Clark, K. O’Connor, C.Clark, and C Roberts.
2010. Developing a statewide network of reference wetlands for California: Conceptual
approach and process for prioritizing and selecting reference sites. Southern California
Coastal Water Research Project. Costa Mesa, CA.

Stevens, D.L. and A.R. Olsen. 2004. Spatially balanced sampling of natural resources. Journal of
the American Statistical Association 99 (465): 262-278.

Waite, I.R., A.T. Herlihy, D.P. Larsen, and D.L. Klemm. 2000. Comparing strengths of geographic
and nongeographic classifications of stream benthic macroinvertebrates in the Mid-Atlantic
Highlands, USA. Journal of the North American Benthological Society. 19: 429-441.

Whittier, T.R., J.L. Stoddard, D.P. Larsen, and A.T. Herlihy. 2007. Selecting reference sites for
stream biological assessments: Best professional judgment or objective criteria. Journal of
the North American Benthological Society. 26: 349-360.

21
Wright, J.F., D. Moss, P.D. Armitage, and M.T. Furse. 1984. A preliminary classification of
running water sites in Great Britain based on macroinvertebrate species and the prediction
of community types using environmental data. Freshwater Biology. 14:221-256.

Wright, J.F. 1995. Development and use of a system for predicting macroinvertebrate fauna in
flowing waters. Australian Journal of Ecology. 20: 181-197.

Wright, J.F., D.W. Sutcliffe, and M.T. Furse, 2000. Assessing the biological quality of fresh
waters: RIVPACS and other techniques. The Freshwater Biological Association. Ambleside,
UK.

Yates, A.G. and R.C. Bailey. 2010. Selecting objectively defined reference streams for
bioassessment programs. Environmental Monitoring and Assessment. 170: 129-140.

22
Table 1. Biomonitoring programs that provided site locations and assessment data for reference site
screening. EMAP: Environmental Monitoring and Assessment Program. CMAP: California Monitoring and
Assessment Program. RCMP: Reference Condition Monitoring Program. TMDL: Total Maximum Daily
Load. IBI: Index of Biotic Integrity. EPA: Environmental Protection Agency. SWAMP: Surface Water
Ambient Monitoring Program. SMC: Stormwater Monitoring Coalition. LASGRWC: Los Angeles and San
Gabriel Rivers and Watersheds Council. SCCWRP: Southern California Coastal Water Research Project.
TRPA: Tahoe Regional Planning Agency. DFG: Department of Fish and Game.

Program Agency Geographic scope Design Sites Years

EMAP EPA Statewide Probabilistic 169 4
Targeted 40 2
EMAP Central Coast
Supplement EPA Coastal Chaparral Probabilistic 23 1
State Board (Non-
CMAP Point Source) Statewide Probabilistic 179 4
PSA SWAMP Statewide Probabilistic 135 3
SMC SMC South Coast Probabilistic 121 1
Statewide (forested
Forest Service Forest Service regions) Targeted 155 2
RCMP SWAMP Statewide Targeted 125 3
San Gabriel River Regional
Monitoring Program LASGRWC South Coast Probabilistic 23 5
Targeted 6

Los Angeles River Watershed

Monitoring Program LASGRWC South Coast Probabilistic 10 1
Santa Ana River Regional
Monitoring Program Region 8 South Coast Probabilistic 51 5
Santa Clara Watershed
Monitoring Program Region 4 South Coast Probabilistic 6 1
West Slope Sierra Hydropower CA Energy
IBI Commission Sierra Foothills Targeted 5 1
Coastal Chaparral and
Sediment TMDL Development R3 and R6 Sierras Targeted 88 3

Lahontan Regional Monitoring Region 6 Sierras Targeted 200 10

Coastal Chaparral and
Bay Area Regional Monitoring Region 2 North Coast Targeted 33 2
Central Coast Regional
Monitoring Region 3 Coastal Chaparral Targeted 41 9
Coastal Chaparral and
Algae reference development SCCWRP and others South Coast Targeted 89 2

23
Table 1 (continued).

Program Agency Geographic scope Design Sites Years

Central Coast Algae CSU Monterey Bay Coastal Chaparral Targeted 88 2

Tahoe Basin Regional
Monitoring Program TRPA Central Lahontan Probabilistic 37 1
Sacramento River Watershed
Program DFG Sierra Foothills Targeted 30 1
South Coast and North
IBI Development DFG Coast Targeted 20 4
Other programs Various Statewide Targeted 33 5

24
Table 2. Sources of spatial data used in this analysis.

Type of spatial data Source or Model Reference Code

Climate PRISM http://www.prism.oregonstate.edu a
Population and housing density US Census 2000 http://www.census.gov/ b
Geology and mineral content Generalized geology and Olson and Hawkins (in Review) c
mineralogy data
Atmospheric deposition National Atmospheric http://nadp.sws.uiuc.edu/ntn/ d
Deposition Program National
Trends Network
Predicted surface water conductivity Quantile regression forest Olson and Hawkins (in Review) e
model (Meinshausen 2006)
Soil Natural Resources http://soils.usda.gov/survey/geography/statsgo f
Conservation Service State
Soil Geographic Database
(STATSGO)
Vegetation MODIS Satellite Imagery from http://lpdaac.usgs.gov g
Land Processes Distributed
Active Archive Center
Groundwater MRI-Darcy Model (Baker et al. Olson and Hawkins (in Review) h
2003)
Waterbody location and attribute data NHD Plus http://www.horizon-systems.com/nhdplus/ i

Dam location, storage National Inventory of Dams http://geo.usace.army.mil/ j

Land cover, imperviousness National Land Cover Dataset http://www.epa.gov/mrlc/nlcd-2006.html k
(2001)
Watershed boundaries Major watershed boundaries http://www.ca.nrcs.usda.gov/features/calwater/ l

Elevation National Elevation Dataset http://ned.usgs.gov/ m

Mine location and attribute data Mineral Resource Data http://tin.er.usgs.gov/mrds/ n
System

25
Table 2 (continued)
Type of spatial data Source or Model Reference Code

Discharge location and attribute data California Integrated Water http://www.swrcb.ca.gov/ciwqs/ o

Quality System
TMDL location and type 303d list of impaired [PETE] p
waterbodies
Road location and attribute data [PETE] [PETE] q
[PETE] [PETE]
Railroad location and attribute data [PETE] [PETE] r
Ecoregion EPA Level III and IV http://www.epa.gov/wed/pages/ecoregions/level_iii_iv.htm s
Ecoregions of the United
States
Federal Grazing Allotments US Forest Service Grazing http://www.fs.fed.us/r5/rsl/clearinghouse/gis- t
Allotments download.shtml
BLM Grazing Allotments http://www.blm.gov/ca/st/en/res/index/data_page.html

Invasive invertebrate records CA Aquatic Bioasssessment http://www.dfg.ca.gov/abl/ u

Lab
University of Montana http://www.esg.montana.edu/aim/mollusca/nzms/index.html

Santa Monica Baykeeper Abramson et al. (2009)

USGS Non-indigenous Aquatic http://nas.er.usgs.gov
Species Database

26
27
Table 3. Metrics and stressors. Codes refer to Table Sources. Unless otherwise noted under n, sample size for all metrics and stressors was 1637.

Scales
Metric Description n Source(s) Unit Point WS 5k 1k
Natural gradient
Location
AREA Area of the unit of analysis l, m m2 X
ELEV Elevation m m X
Climate
PPT 30-y (1961-1990) average annual precipitation a
mm X
TEMP 30-y (1961-1990) average monthly temperature a
°C X
AtmCa Catchment mean of mean 1994-2006 annual d
ppt-weighted mean Ca concentration % X
AtmMg Catchment mean of mean 1994-2006 annual d
ppt-weighted mean Mg concentration % X
AtmSO4 Catchment mean of mean 1994-2006 annual d
ppt-weighted mean SO4 concentration
% X
MINP_WS Catchment mean of mean 1971-2000 min d
monthly ppt % X
MEANP_WS Catchment mean of mean 1971-2000 annual ppt d
% X
SumAve_P Catchment mean of mean June-Sep 1971-2000 d
monthly ppt % X
TMAX_WS Catchment mean of mean 1971-2000 max d
temperature % X
XWD_WS Catchment mean of mean 1961-1990 annual d
number of wet days % X

28
 MAXWD_WS Catchment mean of 1961-1990 annual max d
number of wet days % X
Geology
CaO_Avg Calcite mineral content c % X
MgO_Avg Magnesium oxide mineral content c % X
N_Avg Nitrogenous mineral content c % X
P_Avg Phosphorus mineral content c % X
S_Avg Sulphur mineral content c % X
UCS_Mean Catchment mean unconfined Compressive f
Strength % X
LPREM_mean Catchmeant mean log geo mean hydraulic h
conductivity % X
BDH_AVE Catchment mean bulk density f % X
KFCT_AVE Catchment mean soil erodability (K) factor f % X
PRMH_AVE Catchment mean soil permeability f % X
PCT_CENOZ Percent cenozoic sediments c % X
PCT_NOSED Percent non-sedimentary or volcanic geology c
(gneiss, granitic, mafic, ultramafic). = "Other"
geology
% X
PCT_QUART Percent quarternary geology c % X
PCT_SEDIM Percent sedimentary geology c % X
PCT_VOLCNC Percent volcanic geology c % X
Hydrology
Length Total stream length, excluding pipes i km X
Other
EVI_MaxAve Catchment max of the long term average g
Enhanced Vegetation Index None X
CondQRm Median predicted conductivity 1155 e µS/cm X
Stressor

29
Biological
CorbiculaDist Inverse distance from the nearest Corbicula u
record km X
DreissenaDist Inverse distance from the nearest Dreissena u
record km X
InvasiveInvertDist Inverse distance from the nearest Corbicula, u
Dreissena, or Potamopyrgus record km X
PotamopyrgusDist Inverse distance from the nearest Potamopyrgus u
antipodarum record km X
Census
HousingDens2000 Housing units density in 2000 b Units/km2 X X X
PopDens2000 Population density in 2000 b People/km2 X X X
Hydrology
CANALS Percent canals or pipes at the 100k scale i % X
CanalDist Distance to nearest canal or pipe (100k) in i
watershed km X
DamCount Number of dams j Number X X X
DamDensArea Density of dams, by area j Dams/km 2
X X X
DamDist to nearest upstream dam in catchment j km X
DamStorage Total normal dam storage j Acre-feet X
Land use
Ag % Agricultural (row crop and pasture, NLCD k
codes 81 and 82) % X X X
CODE_21 % Urban/Recreational Grass (NLCD code 21) k
% X X X
FOREST % Forest (NLCD codes 41, 42, 43) k % X X X
IMPERVMEAN Imperviousness k % X X X
NAT_BARREN % Natural barren (NLCD code 31) k % X X X
NGRASSLAND % Natural grassland (NLCD code 71) k % X X X

30
 PASTURE % Natural pasture (NLCD code 81) k % X X X
ROW_CROPS % Row crops (NLCD code 82) k % X X X
SHRUB % shrublands (NLCD code 52) k % X X X
URBAN % urban (NLCD codes 22, 23, 24) k % X X X
WETLANDS % wetlands (NLCD codes 90, 95) k % X X X
GRAZING % allotted to grazing on USFS and BLM lands in t
CA % X X X
Mining
GravelMinesDensL Linear density of gravel mines n mines/km X X X
MinesDens Density of mines (producers only) n mines/km2 X X X
Regulatory
CIWQS Count of CIWQS discharges o Count X X X
303dSeg Site is on a 303d listed segment p None X
303dShed Site is downstream of a 303d listed segment or p
waterbody None X X X
Transportation
PavedRoadCross Number of paved road crossings q, r Count X X X
RDDENSC123R Road density (includes rail) q, r km/km2 X X X
PHAB
P_SAFN Percent sands and fines 1191 Field measurements % X
W1_HALL Weighted human influence 964 Field measurements None X
XEMBED Average percent embeddedness 1059 Field measurements % X
Water chemistry
Cl Chloride microequivalents 809 Field measurements uEq X
COND Specific conductivity 1155 Field measurements uS/cm X
NTL Total Nitrogen 1147 Field measurements ug/L X
PTL Total Phosphorous 1080 Field measurements ug/L X

31
Table 4. Thresholds used to identify reference sites

Variable Scale Threshold Unit

% Agricultural 1k and 5k 3 %
WS 10 %
% Urban 1k and 5k 3 %
WS 10 %
% Ag + Urban 1k and 5k 5 %
% Code 21 1k and 5k 7 %
WS 10 %
Road density 1k, 5k, and WS 2 km/km2
Road crossings 1k 5 Crossings/km2
5k 10 Crossings/km2
WS 50 Crossings/km2
Dam distance 1 km
% Canals, pipes 10 %
Instream gravel mines 5k 0.1 Mines/km
Producer mines 5k 1 Mines
Total N 3000 µg/L
Total P 500 µg/L
Conductivity 99/1* Prediction interval
W1_Hall 1.5 None
* The 99th and 1st percentile of predictions was used to generate site-specific thresholds for conductivity.
Because the model was observed to under-predict at higher levels of specific conductivity (data not
shown), a threshold of 2000 µS/cm was used as an upper bound if the prediction interval included 1000
µS/cm.

32
Table 5. Number (n) and percent (%) of reference, and non-reference sites, by region and sub-region.

Non-
reference Reference % of non-reference sites failing
Total
length 1 to 2 5 or
Region (km) n % n % thresholds 3 to 5 more
North Coast 9,278 128 62 79 38 35 48 17
Chaparral 8,126 270 70 117 30 28 24 47
--Coastal Chaparral 5,495 226 72 87 28 27 25 48
--Interior Chaparral 2,631 44 59 30 41 34 23 43
South Coast 2,945 351 75 118 25 24 10 66
--South Coast 1,123
Mountains 89 48 96 52 66 21 12
--South Coast Xeric 1,821 262 92 22 8 9 6 85
Central Valley 2,407 55 98 1 2 0 15 85
Sierra Nevada 11,313 186 41 273 59 54 27 19
--Western Sierra Nevada 8577 88 40 131 60 60 26 14
--Central Lahontan 2,736 98 41 142 59 49 28 23
Deserts / Modoc 2,531 32 54 27 46 47 34 19
Total 36,599 1022 62 615 38 20 14 29

33
Table 6. Extent of streams estimated to be reference by region.

n n probabilistic % reference Standard

Region probabilistic and reference (length) error
North Coast 139 33 28 4
Chaparral 144 29 23 4
--Coastal Chaparral 106 16 18 5
--Interior Chaparral 38 13 33 9
South Coast 334 74 27 4
--South Coast Mountains 138 68 68 4
--South Coast Xeric 196 6 2 1
Central Valley 52 1 2 2
Sierra Nevada 80 34 56 6
--Western Sierra Nevada 53 18 50 8
--Central Lahontan 27 16 74 6
Deserts / Modoc 40 14 56 10
Total

34
Table 7. Percent of variance explained (Pearson’s R2) in selected bioassessment metrics by stressors within reference sites (R) and within the total data set (T).
For all GIS metrics, correlations were based on 489 reference sites, and 1306 total sites. For conductivity, the estimates were based on 317 and 971 sites, and
for W1_Hall, on 241 and 836 sites, respectively. Metrics are abbreviated as follows: O/E: Observed to expected number of taxa. O/E50: Observed to expected
number of taxa, 0.5 capture probability. IBI: Regionally specific index of biotic integrity. EPT Tx: Number of Ephemeroptera, Plecoptera, and Trichoptera taxa.
Dipt Tx: Number of Diptera taxa. Pred Tx: Number of predator taxa. % EPT Tx. Percent of Ephemeroptera, Plecoptera, and Trichoptera taxa. % Intol: Percent
intolerant individuals. % Coll: % collector-filterers and collector-gatherers. % Tol Tx. Percent tolerant taxa. % NI Tx: Percent non-insect taxa.

O/E O/E50 IBI EPT Tx Coleo Tx Dipt Tx Pred Tx % EPT Tx % Intol % Coll % Tol Tx
Stressor Scale R T R T R T R T R T R T R T R T R T R T R T
% Agriculture 1k 0.00 0.06 0.00 0.03 0.01 0.07 0.01 0.07 0.00 0.02 0.01 0.04 0.00 0.03 0.01 0.08 0.00 0.06 0.00 0.02 0.01 0.08
5k 0.01 0.06 0.02 0.03 0.01 0.07 0.01 0.08 0.00 0.03 0.01 0.05 0.01 0.04 0.00 0.09 0.01 0.06 0.00 0.02 0.01 0.10
WS 0.00 0.04 0.01 0.03 0.01 0.04 0.01 0.06 0.00 0.02 0.00 0.04 0.00 0.03 0.02 0.07 0.02 0.04 0.00 0.01 0.02 0.09
% Urban 1k 0.00 0.22 0.00 0.27 0.00 0.20 0.00 0.24 0.01 0.14 0.00 0.12 0.00 0.18 0.00 0.28 0.00 0.14 0.01 0.11 0.00 0.24
5k 0.00 0.19 0.01 0.24 0.03 0.15 0.00 0.22 0.01 0.13 0.00 0.12 0.00 0.18 0.00 0.25 0.00 0.12 0.01 0.09 0.01 0.23
WS 0.01 0.16 0.01 0.23 0.03 0.11 0.02 0.19 0.00 0.12 0.00 0.11 0.01 0.18 0.02 0.21 0.02 0.10 0.00 0.09 0.04 0.18
% Ag + Urban 1k 0.00 0.27 0.00 0.28 0.01 0.27 0.01 0.32 0.00 0.16 0.00 0.16 0.01 0.20 0.00 0.36 0.01 0.20 0.01 0.12 0.01 0.32
5k 0.01 0.24 0.02 0.25 0.04 0.22 0.01 0.29 0.01 0.15 0.01 0.17 0.00 0.21 0.00 0.33 0.01 0.17 0.00 0.11 0.02 0.33
WS 0.02 0.20 0.02 0.21 0.04 0.17 0.03 0.25 0.00 0.13 0.01 0.15 0.02 0.19 0.02 0.28 0.02 0.15 0.00 0.10 0.05 0.26
% Code 21 1k 0.00 0.14 0.03 0.16 0.00 0.13 0.01 0.19 0.01 0.06 0.00 0.05 0.01 0.11 0.02 0.19 0.02 0.16 0.01 0.09 0.03 0.18
5k 0.00 0.18 0.01 0.20 0.00 0.16 0.03 0.26 0.03 0.09 0.01 0.07 0.01 0.14 0.07 0.27 0.02 0.18 0.00 0.10 0.04 0.26
WS 0.01 0.17 0.01 0.20 0.00 0.13 0.06 0.26 0.04 0.07 0.01 0.05 0.02 0.15 0.11 0.28 0.06 0.16 0.00 0.09 0.08 0.23
% Impervious 1k 0.00 0.20 0.00 0.25 0.03 0.19 0.00 0.23 0.02 0.13 0.00 0.12 0.00 0.17 0.00 0.26 0.00 0.13 0.00 0.11 0.00 0.23
5k 0.00 0.18 0.01 0.22 0.04 0.14 0.00 0.21 0.02 0.12 0.01 0.11 0.00 0.17 0.00 0.23 0.01 0.11 0.01 0.09 0.01 0.22
WS 0.00 0.16 0.01 0.22 0.03 0.11 0.02 0.18 0.00 0.11 0.00 0.10 0.00 0.17 0.04 0.20 0.03 0.09 0.00 0.08 0.06 0.17
Road density 1k 0.00 0.17 0.00 0.21 0.01 0.15 0.01 0.18 0.00 0.08 0.00 0.08 0.00 0.12 0.00 0.24 0.00 0.13 0.00 0.07 0.00 0.19
5k 0.01 0.16 0.00 0.20 0.00 0.13 0.01 0.19 0.01 0.08 0.00 0.09 0.01 0.13 0.00 0.25 0.00 0.13 0.00 0.06 0.00 0.22
WS 0.00 0.12 0.00 0.16 0.00 0.09 0.00 0.15 0.05 0.04 0.00 0.06 0.00 0.11 0.03 0.21 0.02 0.10 0.01 0.04 0.01 0.16
Road crossings 1k 0.02 0.10 0.01 0.11 0.03 0.08 0.02 0.11 0.01 0.04 0.00 0.03 0.01 0.08 0.01 0.12 0.01 0.08 0.01 0.06 0.03 0.11
5k 0.01 0.12 0.02 0.20 0.06 0.09 0.02 0.19 0.00 0.08 0.01 0.06 0.01 0.13 0.02 0.20 0.02 0.14 0.02 0.07 0.04 0.22
WS 0.02 0.04 0.04 0.04 0.02 0.06 0.04 0.06 0.00 0.02 0.01 0.02 0.01 0.03 0.05 0.07 0.04 0.06 0.00 0.02 0.08 0.05

35
 O/E O/E50 IBI EPT Tx Coleo Tx Dipt Tx Pred Tx % EPT Tx % Intol % Coll % Tol Tx
Stressor Scale R T R T R T R T R T R T R T R T R T R T R T

% Grazing 1k 0.00 0.04 0.00 0.03 0.01 0.03 0.00 0.08 0.00 0.01 0.01 0.01 0.00 0.03 0.00 0.08 0.00 0.04 0.00 0.02 0.00 0.06
5k 0.00 0.05 0.00 0.04 0.00 0.04 0.00 0.09 0.00 0.01 0.01 0.01 0.00 0.03 0.00 0.09 0.00 0.05 0.00 0.02 0.00 0.07
WS 0.00 0.04 0.00 0.04 0.00 0.03 0.00 0.08 0.00 0.01 0.00 0.01 0.00 0.04 0.00 0.07 0.00 0.04 0.00 0.02 0.00 0.06
Population density 1k 0.00 0.14 0.00 0.17 0.00 0.11 0.00 0.15 0.00 0.08 0.00 0.07 0.00 0.12 0.00 0.18 0.00 0.08 0.00 0.06 0.00 0.13
5k 0.00 0.16 0.00 0.21 0.00 0.12 0.01 0.19 0.00 0.11 0.01 0.10 0.00 0.15 0.01 0.22 0.00 0.10 0.00 0.06 0.00 0.19
WS 0.00 0.14 0.00 0.20 0.00 0.09 0.01 0.17 0.00 0.10 0.01 0.09 0.00 0.16 0.01 0.19 0.00 0.09 0.00 0.06 0.00 0.16
Dam distance 0.01 0.04 0.00 0.08 0.01 0.03 0.00 0.07 0.01 0.04 0.01 0.02 0.00 0.05 0.00 0.05 0.00 0.04 0.01 0.04 0.01 0.06
Dam count 0.01 0.04 0.00 0.03 0.01 0.06 0.01 0.06 0.01 0.02 0.01 0.04 0.00 0.03 0.00 0.06 0.00 0.04 0.01 0.02 0.01 0.04
Canal distance 0.00 0.07 0.00 0.07 0.01 0.07 0.00 0.07 0.00 0.05 0.01 0.03 0.00 0.05 0.00 0.06 0.00 0.05 0.00 0.04 0.01 0.08
% Canal 0.00 0.06 0.01 0.05 0.01 0.06 0.00 0.05 0.00 0.03 0.00 0.05 0.00 0.04 0.00 0.05 0.01 0.04 0.00 0.03 0.02 0.06
Gravel mine density 1k 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.01
5k 0.00 0.01 0.00 0.01 0.00 0.01 0.00 0.01 0.00 0.01 0.00 0.01 0.02 0.00 0.00 0.02 0.00 0.01 0.00 0.00 0.00 0.02
WS 0.00 0.03 0.00 0.03 0.00 0.03 0.00 0.04 0.00 0.01 0.00 0.01 0.01 0.02 0.00 0.05 0.00 0.02 0.00 0.01 0.00 0.04
Discharge density 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.01 0.00 0.01 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00
Conductivity-O/E 0.01 0.04 0.02 0.04 0.01 0.02 0.01 0.04 0.00 0.01 0.01 0.00 0.00 0.03 0.02 0.06 0.00 0.02 0.02 0.01 0.03 0.05
W1_Hall 0.02 0.21 0.05 0.25 0.06 0.18 0.04 0.22 0.02 0.17 0.01 0.15 0.05 0.21 0.01 0.22 0.03 0.14 0.01 0.09 0.05 0.20

36
Figure 1. Distribution of candidate sites screened for inclusion in California’s reference pool. White
triangles represent passing sites and black dots represent sites that failed one or more screening criteria.
Thick solid lines indicate boundaries of major ecological regions referred to in the text. Lighter dotted
lines indicate sub-regional boundaries referred to in the text (not labeled).

37
Figure 2. Partial dependence curves showing the relationship between numbers of reference sites and
thresholds for selected stressors (% Urban, Road Density, % Agricultural, and % Code 21). All other
stressors were held constant using the thresholds listed in Table 4.

38
Figure 3. Scatterplots of the relationships between two stressors and two common biological scoring
indices. A-B) O/E vs. % Urban at the 5k scale; panel A shows the full range of response, and panel B
shows the range below the threshold in Table Threshold. C-D) % EPT Taxa vs. % Code 21 at the
watershed scale. In both plots, reference sites are dark circles, and non-reference sites are light circles;
panel C shows the full range of response, and panel D shows the range below the threshold in Table
Threshold. In all panels, dashed lines show thresholds used to identify reference sites.

39
Figure 4. Distributions of natural gradients within reference sites (top, open curves) and within
probabilistic sites (bottom, shaded curves) within major regions of California. Curves show kernel
density estimates. Values of individual reference sites are shown as small semi-transparent vertical lines.
Regions are abbreviated as follows: SN: Sierra Nevada. SC: South Coast. NC: North Coast. DM: Deserts /
Modoc. CV: Central Valley. CH: Chaparral, Cal = statewide.

40
Figure 5. Heatmap of correlations between several bioassessment indicators and stressors. Lighter
colors correspond to weaker correlations, ranging in values of R2 from 0.0 to 0.4 and above, as indicated
by the color key. The left half of the plot shows correlations with reference sites, and the right half of
the plot shows correlations with all sites.

41