Chapter 2

Construct Maps

2.0 Chapter Overview

This chapter expands on the concept of the construct map introduced in the previous
chapter. The aim is to enrich the reader’s understanding of this particular approach to
conceptualizing a construct, an approach that has been found to be very useful as a basis for
measuring (for example, see the many publications listed in Appendix 1B). There is no claim
being made here that this approach will satisfy every possible measurement need (this point is
expanded on at the end of the chapter). However, both for didactic purposes, and because it will
prove a useful tool in many applications, the chapter will concentrate on just this one type of
construct, as will the rest of the book. The chapter mainly consists of a series of examples of
construct maps, illustrating the main different types: two partial types, respondent maps, item
maps, and the full construct map, consisting of both respondent and item locations. All of the
examples are derived from published applications1. These examples show construct maps that
have been through several iterations, as well as some that might be best described as proto-
construct maps.

Key concepts: construct and construct maps.

2.1 The Construct Map

What are the essential aspects of construct maps that I need to know about?

The construct map is the first building block in the Bear Assessment System (BAS). It
has already been introduced, lightly, in Chapter 1, and its relationship to the other building
blocks was illustrated there too—see Figure 2.1. In this chapter, it is the main focus.

The idea of a construct that will be described in this chapter is one that is particularly
suitable for a visual representation and is called a construct map. Its most important features are
that:
(a) there is a coherent and substantive definition for the content of the construct;
(b) there is an idea that the construct is composed of an underlying continuum—this can be
manifested in two ways, in terms of the respondents, and/or in terms of item responses.

The two different aspects of the construct, the respondents and their responses, lead to two
different sorts of construct maps:
(a) a respondent construct map where the respondents are ordered from more to less (on the
construct)—and qualitatively may be grouped into an ordered succession of waypoints;
and
(b) an item response construct map where the item responses are ordered from more to less (on
the construct)—and qualitatively may also be grouped into an ordered succession of
waypoints.
1
The interested reader can also find examples of construct maps within each of the Worked Cases in the Examples
Archive (Appendix 1) on the website associated with this book.

1
And also:
(c) a full construct map which consists of both respondent and item locations (which will most
often be shortened to just “construct map”).
Of course, words like “construct” and “map” have many other usages in other contexts, but they
will be reserved in this book for just the purposes described above.

Figure 2.1 The Construct Map, the first building block in the BEAR Assessment System
(BAS)

Construct Items
Map Design

Calibration Outcome
Model Space

A generic construct map is shown in Figure 2.2—the generic variable being measured is
labelled “X” for this Figure. The depiction shown here will be used throughout this book, so a
few lines will be used to describe its parts before moving on to examine some concrete
examples. The arrow running up and down the middle of the map indicates the continuum of the
construct, running from “low” to “high.” The left-hand side will indicate qualitatively distinct
groups of respondents, each occupying a waypoint, and ranging from those with high “X” to
those with low “X.” A respondent construct map would include only the left side. The right-
hand side will indicate qualitative differences in item responses, each occupying a waypoint, and
ranging from responses that indicate high “X” to those that indicate low “X.” An item response
construct map would include only the right side. A full construct map will have both sides
represented.

2
Figure 2.2 A generic construct map in construct “X”.

3
 Note that this Figure depicts an idea rather than being a technical representation. Indeed,
later this idea will be related to a specific technical representation, but for now, just concentrate
on the idea. Certain features of the construct map concept are worth pointing out. As before,
there are waypoints marked off by the circles on the line running up and down the middle. But
now we will consider the line itself—this is crucially important, as this is where the individual
respondents and individual items are located. In theory, respondents and items can be located
anywhere on the line—some will be on top of waypoints, others will scatter around the
waypoints.

4
1. In general, there is no a priori limit on the density of the potential locations on the construct
continuum2 that could be occupied by a particular respondent. This corresponds to the
idea that no matter where a respondent is on the continuum, there could be another
respondent arbitrarily close just above and/or just below that respondent. An example
could be where a reader was responding to a reading comprehension test on a Tuesday,
and then responding to a second (different but equivalent3) one on a Wednesday but was
distracted on the Wednesday by a loud leaf-blower outside the classroom, leading to a
small diminution of the underlying reading comprehension ability. Of course, one might
expect that there will be limitations of accuracy in identifying that location, caused by
limitations of data, but that is another matter (see “estimation” and “error” in Chapters 5
and 6).

2. Similarly, there is no a priori limit on the density of the potential locations on the construct
continuum that could be occupied by a particular item. For example, this corresponds to
the idea that no matter where an item is on the continuum, there could be another
(probably similar) item arbitrarily close just above and/or just below that item. An
example could be where a single word was replaced within an item with a synonym that
had approximately the same reading difficulty. And, of course, issues of estimation and
error will be involved here too.

3. The labels of individual items on the construct map are actually labels of item responses. It is
important to keep in mind that the locations of the labels are not the locations of items
per se but are really the locations of certain types of responses to the items. Thus,
effectively the items’ locations will be estimated via the respondents’ reactions to them.

4. In any specific context, just a few waypoints will be identified along the construct map, and
these can be expressed in terms of typicalrespondents or typical item responses for a
certain waypoint, or both. Thus, the waypoints can be seen as relating to groups of
respondents and classes of item responses.

Examples of constructs that can be mapped in this way abound: In attitude surveys, for
example, there is always something that the respondent is agreeing to or liking or some other
action denoting an ordering; in educational achievement testing, there is most often an
underlying idea of increasing correctness, of sophistication or excellence; in marketing, there are
always some products that are more attractive or satisfying than others; in political science, there
are some candidates who are more likely to be voted for than others; in health outcomes
research, there are better health outcomes, and worse health outcomes. In almost any domain
there are important contexts where the type of construct that can be mapped (as defined above)

2
Note that the word “continuum” is not being used here in the mathematical sense (i.e., as the nondenumerable set
of real numbers). So far there are no numbers attached to the construct or its construct map, so that would be
premature. Instead, the term continuum is being used here to signify the idea that the construct is ordered and dense
in the way defined in this paragraph. Later, in Chapter 5, this will be made more concrete in that we will find that
observations on the continuum can, at least in theory, be thought of as being as dense as the rational numbers, in the
sense that these observations will be indicated by ratios of numbers of observations.
3
Although the general topic of establishing that two instruments are equivalent is beyond the scope of this book,
further comments are made about it in Chapter X, and it is in the topics list for the follow-up volume.

5
needs to be measured. The next section of this chapter contains numerous examples of construct
maps to help convey the range of possibilities.

As should be clear from the description above, a construct can be most readily expressed
as a construct map where the intended construct has a single underlying continuum—and this
implies that, for the intended use of the instrument, the measurer wants to spread the respondents
from high to low, or left to right, etc., in some context where that distribution is substantively
interpretable4. This makes good sense as a basis for instrument construction (See Section 6.1).

There are several ways in which the idea of a construct map can exist in the more
complex reality of usage—a construct is always an ideal, we use it because it suits our theoretical
approach and/or our practical aims. If the theoretical approach is inconsistent with the idea of
mapping a construct onto an ordered continuum, then it is hardly sensible to want to use a
construct map as the fundamental structural approach. There can be several situations that are
not expressly suited for construct map interpretation, though some can be adapted to do so.
Examples are given of such situations in each of the next four paragraphs.

Latent Classes. An example of this could a case where a student was engaging in
problem-solving regarding 3-dimensional shapes, and it had been established that there were two
typical approaches, say one based on a geometrical (Euclidean) interpretation, and one based on
a topological interpretation (i.e., by using qualitative features of the shapes). Then, the aim of a
measurement might be to classify the students into the two different classes who used those
problem-solving strategies. Here the concepts of “more” and “less” do not apply.

Ordered Partition. However, this situation might be subsumed into a context that is more
like a construct map. For example, suppose that the problem-solving described in the previous
paragraph was set in a more complex assessment situation where there was a waypoint at a less
sophisticated level (e.g., the student did not even understand the problem), and also a waypoint
that was more sophisticated—e.g., where the student solved the problem, but also adapted it to a
new situation. Here the construct can be seen as an ordered partition of the set of categories as is
illustrated in Figure 2.3. In this situation, the partial order itself can be used to simplify the
problem so that it is collapsed into a construct map. In this case there will be a loss of
information, but this simplified construct may prove useful, and the extra complications can be
added back in later (Wilson & Adams, 1993). Consider the MoV example introduced in the
previous chapter. Here the construct described in Figure 1.12 can be re-expressed as a construct
map as in Figure 2.4. This Figure makes clear that the waypoints given in Figure 1.12 were
essentially different levels of student thinking, i.e., respondent locations, and so consequently
they are given on the left-hand side of the Figure. Note also that this shows an example of an
ordered partition, specifically at MoV2/MoV3.

Multiple Strands. But there are also constructs that are more complex than what is
involved in a single construct map, yet contain construct maps as a component. Probably the
most common would be a construct with multiple “strands”—for example the six Data Modeling

4
Some might express this as the construct being “unidimensional,” but that form of expression will be reserved in
this book for the situation where the construct has been associated with a quantitative representation, which is not
yet the case here (but see Chapter 5).

6
strands mentioned in Example 1 in Chapter 1. In this sort of situation where a simple construct
map would be inadequate to fully match the underlying constructs, a reasonable approach would
be to use the construct map approach consecutively for each of the strands5.

Multiple Strands but also a Composite. An interesting added complexity in this multiple
strand case is where the measurements are to be made at two grain sizes—within each of the
multiple strands, and also across all the strands—this multilevel measurement context demands a
different type of calibration model than what will be used in this book (although it is a topic in
the follow-on volume—see Wilson & Gochyyev (2020)).

For other examples of more complex structures, see the “Resources” section at the end
of this chapter.

Figure 2.3 Illustration of a simple ordered partition

5
This may well involve a multidimensional calibration model—see Section 9.1.

7
Figure 2.4 A sketch of the construct map for the MoV construct of the Data Modeling
instrument.

8
2.2 Examples of Construct Maps
What are some interesting and instructive examples of construct maps?

The idea of a construct map is very natural in the context of educational testing, such as
Example 1 in the previous chapter. It is also just as amenable to use in other domains. For
example, in attitude measurement one often finds that the underlying idea is one of increasing or
decreasing amounts of a human attitudinal attribute, and that attribute (i.e., the construct) might
be satisfaction, liking, agreement, etc. In the sections below, multiple examples of construct
maps are described, across a wide range of topics in the social sciences and professional practice,
as well as beyond. Just as Example 1 will be used in several places in the following chapters,
several of these Examples will also be returned to help illustrate applications of the BAS.

Figure 2.5 Three constructs represented as three strands spanning a theory of learning (from
Wilson, 2009)

2.2.1 Example 1: The Six Constructs in the Data Modeling Curriculum

What is an example of a construct map in the area of educational achievement?

In the example used in Chapter 1, the construct map produced by the Data Modeling
project for Models of Variability (MoV) construct was used to illustrate the central ideas and
some of the ways that a construct map can be used in developing an instrument. In this section,
the account of the work of the Data Modeling project is broadened to include the rest of the six
constructs that it deployed and deepened by describing a hypothesized learning progression (see
Chapter 10) based on the six Data Modeling constructs (plus the seventh, which is based on the
substantive topic in which the data modelling takes place). An illustration of multiple construct
maps is given in Figure 2.5. In this representation, only 3 strands are illustrated, but the idea is
readily generalized. The three construct maps are shown as mapping out (in analogy to latitude

9
and longitude), the somewhat less specific ideas about student thinking that might be envisaged
by the researcher, shown in the lower left-hand corner.

The Data Modeling learning progression was developed in a series of classroom design
studies, first conducted by the designers of the progression (e.g., Lehrer, 2017; Lehrer, Kim &
Schauble, 2007; Lehrer & Kim, 2009; Lehrer, Kim & Jones, 2011) and subsequently elaborated
by teachers who had not participated in the initial iterations of the design (e.g., Tapee, Cartmell,
Guthrie, & Kent, 2019). Six constructs were generated that delineate the desired conceptual
changes in data modeling thinking and practices, and which could be supported by student
participation in instructional activities (Lehrer, Kim, Ayers & Wilson, 2014). These constructs
were developed as part of the design studies just mentioned—these studies together gave
evidence for common patterns of conceptual growth as students learned about data modeling in
particular substantive contexts ranging from repeated measurements in classroom contexts, to
manufacturing production (e.g., different methods for making packages of toothpicks) to
organismic growth (e.g., measures of plant growth). Conceptual pivots to promote change were
structured and instantiated into a curriculum, most especially inducting students into statistical
practices of visualizing. The curriculum includes rationales for particular tasks, tools, and
activity structures, guides for conducting mathematically productive classroom conversations,
and a series of formative assessments that teachers could deploy to support learning. The six
constructs are described in the next few paragraphs. First, recall that Models of Variability (MoV)
was already described in Chapter 1, so that will not be repeated here.

Visualizing data. Two of the six constructs have waypoints in student progress towards
ways of thinking that typically are seen as students begin to learn the practices of visualizing
data. The first is Data Display (DaD) which describes conceptions of data that inform how
students construct and interpret representations of data. These conceptions range along a
dimension that starts with students interpreting data through the lens of individual cases to
students viewing data as statistical distributions. DaD is closely associated with a second
construct, Meta-Representational Competence (MRC), which incorporates the waypoints of
understanding as students learn to design and adjust data representations to illustrate how the
data support the claims that they want to make. A crucial step here is where students begin to
consider trade-offs among potential representations with respect to particular claims.

Conceptions of statistics. A third construct, Conceptions of Statistics (CoS), describes

how students’ understandings of statistics change when they have multiple opportunities to
invent and critique indicators of characteristics of a distribution, such as its center and spread.
Initially, students begin by thinking of statistics as the result of computations, not as indicators of
features of a distribution. The invention and critique of measures of distribution is viewed as
another conceptual pivot. The upper anchor of this construct entails recognition of statistics as
subject to sample-to-sample variation.

Conceptions of chance. Chance (Cha) describes the progress in student understanding

about how probability operates to produce distributions of outcomes. Students’ beginning forms
of understandings are intuitive and rely on conceptions of agency (e.g., “lucky numbers”). Later
they begin to understand the idea of a trial, which means that students must abandon the idea of
personal influence on selected outcomes, and which leads the way to a profound perspectival

10
sea-change that now frames chance as being associated with a long-term process, a necessity for
a frequentist view of probability (Thompson, Liu & Saldanha, 2007). Further experiences
eventually lead to a new kind of distribution, that of a sampling distribution of sample statistics,
reflecting the upper anchor of the CoS construct. The CoS and Cha constructs are related in that
the upper anchor of CoS relies on conceptions of sample-to-sample variation attributed to
chance.

Informal inference. The sixth and final construct, Informal Inference (InI), describes
important changes in students’ reasoning about inference. The term “informal” is intended to
convey that the students are not expected to reach conceptions of probability density and related
statistical ideas that would be typical of professionals. Rather, students are engaged in
generalizing or predictions beyond the specific data at hand. The beginner waypoints describe
types of inferences that are based on personal beliefs and experiences (in many cases here, data
do not play a role, other than perhaps as confirmation of what the student believes). At the upper
anchor, students can conceptualize a “sample” as an empirical sample that is just one instance of
a possibly infinite collection of samples generated by a long-term, repeated process (Saldanha &
Thompson, 2014). Informal inference is then based upon this understanding of sample, which is
a keystone of professional practice of inference (Garfield, Le, Zieffler, & Ben-Zvi, 2015).

More information about measurement using the Data Modeling constructs is given in
Chapters 9 and 10.

Figure 2.6 The construct map for the RIS (from Koo et al, 2021)

11
2.2.2 Example 2: A Social and Emotional Learning Example (RIS-Researcher Identity
Scale).
What is an example of a construct map in the area of socio-emotional learning?

The San Francisco Health Investigators (SFHI) Project (Koo et al, 2021) developed the
Researcher Identity Scale (RIS), with application focused on students at the high school level.
The developers considered researcher identity as “one unified idea made up of four strands:
Agency, Community, Fit and Aspiration, and Self” (Koo et al. p. 5). For more information on
the latent variable, its components, and its construct map, see Bathia et al. (2020). The construct
map for this construct is shown in Figure 2.6. The hypotheses represented by the construct map
can be thought of as having a lowest extreme Waypoint (0) which is at the bottom, below the
other waypoints—here the student is not aware of what research entails and has no consideration
for their possible role(s) in such research. Going up to the next waypoint (1), the student is
considered a novice to the idea of research. At Waypoint 2, the student is in the process of
exploring different aspects of research. Beyond that, at Waypoint 3, the student starts to be
comfortable with their identity as a researcher. At the highest extreme (4), the student self-
identifies as a researcher and integrates this into their larger identity.

Note that Figure 2.6 is formatted in a somewhat different way than the earlier figures
showing construct maps. In particular, it is shown in a “table” format that does not so clearly
make reference to the idea of an underlying continuum of possible locations. This format is
convenient on the printed page, but can lead to confusion, where the reader thinks that the
construct map is no more than a sequence of categories6. A second variant is indicated on the
right-hand column, where items rather than item responses are shown. Again, this can lead to
confusion, where the waypoints are seen as being associated with items rather than item
responses. Nevertheless, the Figure does indicate that, at least in broad conception, this was
conceptualized as a full construct map by the SFHI project.

More information about measurement using the RIS construct is given in Chapter 8.

2.2.3 Example 3: An Attitude Example (GEB- General Ecological Behavior).

What is an example of a construct map in the area of attitudes?

The General Ecological Behavior (GEB) scale (Kaiser, 1998; Kaiser & Wilson, 2004) is
an instrument meant to measure environmental attitude (see, e.g., Kaiser et al., 2010). It is based
on self-reports of past environmentally protective behavior. The full set of items in the GEB is
shown in Appendix 2A. The GEB is based on the Campbell paradigm for developing and
interpreting attitudinal measurements (Kaiser & Wilson, 2019), whereby

... people disclose their attitude levels—their valuations of the attitude object (e.g.,
environmental protection) or their commitment to the attitude-implied behavioral goal
(e.g., protecting the environment)—in the extent to which they engage in verbal and

6
Note that this type of formatting is also associated with the use of the term “level” which has also been found to be
associated with a similar confusion.

12
 nonverbal behaviors that involve increasing levels of behavioral costs (Kaiser & Lange,
2021) [emphases added]

All behaviors involve costs, and so that is true for behaviors aimed at environmental protection.
Thus, the extent of a person’s environmental attitude can be inferred from the environmentally
protective activities they engage in (e.g., when people publicly boycott companies with a poor
ecological record, buy products in refillable packages, wash dirty clothes without prewashing
and/or ride a bicycle, walk or take public transportation to work or school) or engaged in in the
past. These activities can be regarded as the behavioral means necessary to pursue a specific goal
(e.g., protecting the environment). Thus, the more committed people are to protecting the
environment, the more they engage in protecting the environment (Kaiser, 2021).

These behavioral costs can be minor and/or common in the community in which the
person lives, or they can be large and/or much less common in their community:

... people generally favor more convenient and socially accepted over more demanding,
socially prohibited, or otherwise costly environmentally protective behaviors ...
Engagement in a specific behavior involves costs in terms of time, money, effort,
courage, inconvenience, etcetera. Such costs have a chance of being endured by people
only when these people’s levels of environmental attitude at least match the costs ...
Consequently, people who engage in a comparatively demanding environmentally
protective behavior (e.g., became members in environmental organizations) reveal higher
levels of environmental attitude than people who fail to engage in that behavior. (Kaiser
& Lange, 2021)

The GEB is an instrument that has been developed without the a priori creation of a
construct map. However, consistent with the Campbell paradigm, one can seek to generate an
item ordering in terms of waypoints that is quite consistent with the concept of a construct map.
In fact, a construct map can be hypothesized in an a posteriori way in this context. Following a
review of the item set, the ordering of the GEB items shown in Figure 2.7 was postulated7, where
only the positively oriented items have been used to make the interpretation easier. These
subsets can then be shown as in the construct map in Figure 2.88. According to Florian Kaiser:
we can recognize behaviors that are "distinct (and explicit) expressions of commitment to
environmental protection" (A), behaviors that represent "active (but only indirectly
connected with) commitment to environmental protection" (B), behaviors that stand for
"weak commitment to environmental protection" (D)--what everybody does in Germany
to actively protect the environment, and behaviors that we regard as the absolute
essentials "(if even these two would be avoided) there would be an absolute lack of
commitment to actively protect the environment" (E). (C) is the messy middle ground of
behaviors. (F. G. Kaiser, personal communication)
Note that, in this construct map, no waypoint is included for Waypoint C (the “messy middle”),
as, in this formulation, it has not been identified as a specific point, but rather as a location
somewhere between Waypoints B and D.

7
For a German context.
8
Note that the relative distances between the waypoints in Figure 2.5 has not been set by a technical method, though
they are roughly equivalent to the differences that are found using the methods described in Chapter 5.

13
Figure 2.7 The GEB items arrayed into four consecutive sets

14
Figure 2.8 Sketch of a construct map in general ecological behavior

15
 The development steps for the GEB were similar to the design-process used in
developing the Program for International Student Assessment (PISA) tests; there it is referred to
there as developing “described scales” (OECD, 2015, p. 265 and ff). This multi-step process may
be briefly summarized as follows:
(a) a committee of curriculum experts in the relevant subject matter develop a set of items that
they judge to be relevant to the curriculum area to be tested,
(b) a survey is conducted to try-out the items, so that they can be arrayed from empirically
easiest to most difficult, and
(c) another committee of experts examines the ordered set of items, splitting them into
interpretable classes of the items that order the content, and then labels/describes those
classes.
As noted above, the interesting difference here is in the order of development, where in the
described scale, the items are developed and calibrated first, and the construct is “described”
later, whereas in the construct mapping procedure, the construct (map) is established first, and
the items and the outcome space are developed based on that. Of course, in either case, there are
overlaps: for example, (a) the items developed in the described variable approach could be seen
as being generated in view of an implicit construct definition (latent in the educational curricula
in the PISA case); and (b) the items developed in the construct map case may, via BAS
iterations, have an influence on the final version of the construct map.

2.2.4 Example 4: A 21st Century Skills Example (LPS-Argumentation)

What is an example of a construct map in the area of 21st Century Skills?

Traditional achievement testing in education has been augmented in the last 20 years with
a range of new types of achievement constructs called “habits of mind” and “21st Century
Skills” (e.g., Griffin, McGaw, & Care, 2012).). These have proven amenable to description using
construct maps, especially following the “learning progression” concept. The Learning
Progressions in Science (LPS) project built on a view of scientific argumentation as a complex
competency of reasoning utilized in situations that require scientific content knowledge to
construct and/or critique proposed links between claims and evidence. (Osborne et al., 2016).
The learning progression (hypothesized as a complex construct map progression) draws on
Toulmin’s (1958) model for the structure of practical or informal arguments. This model starts
with a claim, which is a “conclusion whose merits we are seeking to establish” (p. 90) and which
must be then supported by relevant data or evidence. The relation between the evidence and the
claim is provided by a warrant that forms the substance of the justification for the claim. These
warrants can in turn be dependent on (implicit or explicit) assumptions that are referred to as
backing. In addition, claims may also be circumscribed by the use of qualifiers that define the
limits of validity for the claim.

Figure 2.9 shows the hypothesized learning progression for argumentation. In it there are
three broad levels of argumentation differentiated by intrinsic cognitive load, where each level is
conceived as being distinguished by having more connections between the claims and the
various pieces of evidence. The initial waypoints (shown at the top of the Figure) are seen as
being in the zeroth level (indicated by “0”) to signify that assessment items relating to these
levels do not require explicit connections between claim and evidence. At these levels, the
connections (i.e., the warrants according to Toulmin) are not specifically required, and thus

16
success is possible by demonstrating identification/critique of an isolated claim, warrant, or
evidence without making a logical connection between them. Put differently, these levels require
zero degrees of coordination.

Going higher on the construct map (i.e., lower in Figure 2.9) to the next set of waypoints
(indicated by “1”), one will find items that require the construction of relationships between
claims and evidence (i.e., warrants). These require only one degree of coordination – i.e., in
responding, a student would need to make one explicit logical connection between claim and
evidence by way of a warrant. Thus, here a student must not only be able to identify a claim or a
piece of evidence, but also must know how to construct or critique a relationship between claims
and evidence. The highest levels of the construct map require two or more degrees of
coordination. These items at the highest waypoint (“2”) involve students in explicating or
comparing two or more warrants. Figure 2.6 provides much more information than in the above
summary, including the distinction between the strands of constructing and critiquing (which
effectively provides a second level of distinction for this construct), as well as brief descriptions
for the sublevels (which are not explicitly mentioned in the summary above), and a graphical
representation of the structure of each of these sublevels of argument. More detail is available in
Osborne et al. (2012) and concrete examples of how various progress levels are operationalized
with assessment items is included in the Supplementary Materials for that paper.

As previously, the presentation format of the construct map has been adapted for this
project: (a) the usual orientation (from bottom to top) has been reversed, (b) the waypoints are
formatted as a table, and (c) a handy addition has been the use of a graphic representation for the
structures of the different types of arguments. The question of whether this is a respondent
construct map or an item construct map is left somewhat ambiguous by the project—the labels of
the sub-levels are explicitly expressed in terms of student cognitions, but the textual descriptions
included (which are summarized above) are given in terms of item responses.

More information about measurement using the LPS-Argumentation construct is given in

Chapter 3.

17
 18
Figure 2.9 The argumentation construct map (from Osborne et al., 2016)
2.2.5 Example 5: A Process Measurement Example: Collaborative Problem Solving (CPS)
What is an example of a construct map in the area of process measurement?

A new collaborative problem solving (CPS) process framework (Zhang et al., submitted)
has been developed as part of the ATC21S project (Griffin, McGaw & Care, 2012; Griffin &
Care, 2015; Care, Griffin & Wilson, 2017). This framework proposed a new view of CPS as a
unified construct composed of an interaction of levels of sophistication between collaboration
and problem solving. The waypoints in the first strand of a multi-strand framework are shown in
Figure 2.10. In this construct, the collaborative problem solvers will begin their problem-solving
efforts by Exploring both the social space and the problem space inherent in the materials that
they are given. The lowest Waypoint is Focus, where they independently engage with and assess
their own resources. At Waypoint 2 they share with one another what they have found and thus
Contribute to the group understanding of the available resources. With this accomplished, at the
third Waypoint, they can each take advantage of the other’s resources, and thus Benefit from
them. This is then complemented by questioning about one another’s resources, so that they
come to Depend on one another. Finally, at the highest Waypoint, they jointly examine and
discuss the available resources.

Figure 2.10 Exploring: the first strand in the CPS Process Framework

1. Exploring
D. Depend E. Metacognitive

Examines shared resources

Asks others questions; Asks others

about their resources
B. Contribute C. Benefit

Take and uses others resources;

Responds to others questions

Give own resources / information to

others; Describes own resources to
others
(Independent)
A. Focus

Engage with own resources

19
 The full CPS process is seen as being composed of five strands (i.e., the columns in
Figure 2.8). Thus, when a group is collaborating proficiently, and according to the framework,
the collaborative problem solvers will begin by Exploring (first column, as described above) both
the social space and the problem space. They will then define (second column) the problem with
respect to their joint resources, thus constructing a shared understanding of the problem.
Together, they will then come up with a plan (third column) and implement the plan (fourth
column) followed by evaluating their progress and reflecting and monitoring their outcomes,
considering alternative hypotheses (fifth column). Within that, the collaborators will carry out
these five processes with differing levels of sophistication and success (i.e., the rows of the five
columns). In practice, these processes would be iterative—with collaborators returning to
previous process steps and correcting errors and omissions.

Figure 2.11 The Complete CPS Process Framework

Measurement of a process, such as is described in the CPS Process Framework demands

a form of observation that is focused on the processes of the collaboration. In order to facilitate
this, the ATC21S project team developed a set of interactive computerized game-like tasks that

20
afforded group members a range of virtual activities designed to help them reach a successful
conclusion. Logfiles of each collaborator’s moves in the games were recorded and used to
develop a process-based coding and scoring of their moves (Awwal et al., 2021).

More information about the CPS example, including examples of items,the outcome
space and a Wright map, is given in Section 9.3.

2.2.6 Example 6: A Health Assessment Example (PF-10—Physical Functioning 10)

What is an example of a construct map in the topic area of health sciences?

A health sciences example of a self-report behavioral construct that can be mapped in this
way is the Physical Functioning subscale (PF-10; Raczek, et al, 1998)) of the SF-36 health
survey (Ware & Gandek, 1998). The SF-36 instrument is used to assess generic health status,
and the PF-10 subscale assesses the physical functioning aspect of that. The items of the PF-10
consist of descriptions of various types of physical activities to which the respondent may
respond that they are “limited a lot”, “a little”, or “not at all.” The 10 items in this instrument are
given in Table 2.1. An initial construct map for the PF-10, developed using an informal version
of the “described scale” procedure discussed above (and based on empirical item difficulties
from earlier studies—Raczek et al., 1998), is shown in Figure 2.12. In this case, the succession
of increasing ease of physical functioning was indicated by the order of the item responses. This
sequence ranges from very strenuous activities, such as those represented by the label “Vigorous
Activities”, down to activities that take little physical effort for most people. Note that the order
shown here is based on the relative difficulty of self-reporting that the respondents’ activities are
“not limited at all.”

Note that, in this case, the top waypoint (“Vigorous activities”) is associated with just one
summarizing item, whereas the other two are associated with several each—this make sense, as
the focus of the survey is on people with some health concerns, who would be expected to not be
very involved in those vigorous activities.

More information about measurement using the PF-10 construct is given in Chapters 5-8.

21
Table 2.1 Items in the PF-10 (in difficulty order)

Item Item Label Item

Number
1 VigAct Vigorous activities, such as running, lifting heavy objects,
participating in strenuous sports
4 SevStair Climbing several flights of stairs
7 WalkMile Walking more than a mile
6 Bend Bending kneeling, or stooping
2 ModAct Moderate activities, such as moving a table, pushing a
vacuum cleaner, bowling, or playing golf
3 Lift Lifting or carrying groceries
8 WalkBlks Walking several blocks
5 OneStair Climbing one flight of stairs
9 WalkOne Walking one block
10 Bath Bathing or dressing yourself

22
Figure 2.12. A sketch of the construct map for the Physical Functioning subscale (PF-10) of the
SF-36 Health Survey.

Direction of increasing ease of

physical functioning.
Responses to Items
Respondents

“Not limited at all” to

Vigorous Activities

“Not limited at all” to

Moderate Activities

“Not limited at all” to

Easy Activities

Direction of decreasing ease of

physical functioning.

23
2.2.7 Example 7: An Interview Example (CUE- Conceptual Underpinnings of Evolution).
What is an example of a construct map in an area where interviews are the main source
of information?

Using a cognitive interview technique, the Conceptual Underpinnings of Evolution

(CUE) project investigated elementary students’ abilities to reason in biology, specifically
focusing on microevolution for second and third grade students (i.e., 7-9 years old) (Metz, et al.,
2019). The curriculum incorporated modules about animals and plants and their natural ecology
and diversity. Students were involved in activities that focused on the organisms’ habitats, traits
and needs, and their structures and functions, including strategically designed thought
experiments about environmental stresses. The project developed a learning progression focused
on this grade-span, but also included deeper understandings that come into play at subsequent
grade levels. The progression aimed to build student understandings of the following important
concepts: variation between-species and within-species, structure/function, limiting factors,
survival advantage, and change over time.

The project assessed student placement along the learning progression using a structured
interview protocol. This was developed from the literature using prior classroom-based research
in these domains, which was also the source of the instructional modules that were developed for
this project. Each question in the interview was related to certain waypoints within the learning
progression, and quotations from student responses were systematically documented that
exemplified the learning progression levels.

The learning progression went through several iterations, and based on both qualitative
and quantitative data, a final version was developed (shown in Figure 2.13; Cardace, Wilson &
Metz, in press). This learning progression consists of two construct maps, one for “State” and
one for “Process.” The State strand focusses on students’ explanation about how organisms fit
into their environment (labeled as “S” on the left-hand side in the Figure). Here, students’
explanations advance from exclusively considering organisms’ needs (Waypoint S2), to
observing the affordances of their physical characteristics, to initial explanations about structure-
function-environment relationships (S3 and S4), and, ultimately, they incorporate natural
selection as an explanation for the organism’s good fit in their environment (S4+P7). The
Process strand (labeled as “P” on the right-hand side in the Figure) focusses on the mechanism
by which organisms become well-fitted to their environments. For this, it is crucial to identify
variability in traits (P3 and P4), and also the survival value that may be attached to those traits
(P5A and P5B). As they proceed along the progression, student explanations incorporate the
shifting of distributions between generations (P6) and ultimately include natural selection as the
specific mechanism for the attainment of good fit (S4+P7). Both strands are described in more
detail in publications about this project (Cardace, Wilson & Metz, in press; Metz et al., 2019).

24
Figure 2.13 The final CUE learning progression

25
2.2.8 Example 8: An Observational Instrument: Early Childhood (DRDP)
What is an example of a construct map In a topic area where structured observations are
the main source of information?

The California Department of Education (CDE) has developed the Desired Results
Developmental Profile9 (DRDP), an observation-based formative child assessment system used
in early care and education programs throughout California (Information about DRDP can be
found at https://www.cde.ca.gov/sp/Cd/ci/desiredresults.asp; and a recent technical report is
available at https://www.desiredresults.us/research-summaries-drdp-2015-domain). The DRDP is
designed to be used across the full developmental continuum for children from birth through
kindergarten, and the observations are organized as “measures” across several important domains
of child development. For example, at the kindergarten level, there are 55 measures across 11
domains or sub-domains. The representation of each measure is designed as a combination of
the construct map and the guidelines for the item itself: for example, the kindergarten measure
“Identity of Self in relation to Others” (SED1) is illustrated in Figure 2.14. This is a view of the
scoring guide that early childhood teachers see when they log in to rate their students in this
measure. The lowest waypoint is at the left-hand side at the top: Child ... “Responds in basic
ways to others.” The highest in this view is on the right at the top: Child ... “Compares own
preferences or feelings to those of others.” Each also includes several examples of what a
teacher might see from children in their early childhood classroom at that waypoint. These brief
examples are supplemented by a video library structured by waypoints, as well as training
materials and online and face-to-face teacher professional development workshops.

Some commentators have criticized the use of observations as the basis for consequential
measurement in educational settings. The project has published reports supporting its usage—
and the case has been much debated. One evaluator has noted that educators should prefer
“to invest in training teachers to be better observers and more reliable assessors than to spend
those resources training and paying for outside assessors to administer on-demand tasks to young
children in unfamiliar contexts that will provide data with the added measurement error inherent
in assessing young children from diverse backgrounds” (Atkins-Burnett, 2007).

In comparison to other construct maps, this one clearly is represented in a different way
—this orientation is what the teachers found most useful. Give that this instrument is used for
assessment of every student in California in publicly funded pre-schools, this represents that
largest use of construct maps so far.

More information about measurement using the DRDP constructs are given in Chapters 3
and 8. Research reports are available at https://www.desiredresults.us/research-summaries-drdp-
2015-domain.

9
Developed in concert with early childhood assessment experts from WestEd and the Berkeley Evaluation and
Assessment Research (BEAR) Center at the University of California, Berkeley.

26
Figure 2.14 A view of the DRDP measure SED1

27
2.2.9 The Issues Evidence and You Science Assessment (IEY)
What is an example of a construct map for a science assessment in the topic area
of “Evidence and Tradeoffs”?

This example is an assessment system built for a middle school science

curriculum, Issues, Evidence and You (IEY; SEPUP, 1995). The Science Education for
Public Understanding Project (SEPUP) at the Lawrence Hall of Science was awarded a
grant from the National Science Foundation in 1993 to create year-long issues-oriented
science courses for the middle school and junior high grades (Thier, 1992). In issues-
oriented science, students learn science content and procedures, but they are also required
to recognize scientific evidence and weigh it against other community concerns with the
goal of making informed choices about relevant contemporary issues or problems. The
goal of this approach is the development of an understanding of the science and problem-
solving approaches related to social issues without promoting an advocacy position. The
course developers were interested in trying new approaches to assessment in the IEY
course materials for at least two reasons. First, they wanted to reinforce the problem
solving and decision-making aspects of the course--to teachers and to students.
Traditional "fact-based" chapter tests would not reinforce these aspects and, if included as
the only form of assessment, could direct the primary focus of instruction away from the
course objectives the developers thought were most important. Second, the developers
knew that in order to market their end product, they would need to address questions
about student achievement in this new course, and traditional assessment techniques were
not likely to demonstrate student performance in the key objectives (problem solving and
decision making).

Both the IEY curriculum, and its assessment system is built (which, like the Data
Modeling example, uses the BEAR Assessment System as its foundation--Wilson &
Sloane, 2000) on four constructs. The Understanding Concepts construct is the IEY
version of the traditional “science content.” The Designing and Conducting
Investigations construct is the IEY version of the traditional “science process.” The
Evidence and Tradeoffs construct was a relatively new one in science education at the
time the curriculum was developed and is composed of the skills and knowledge that
would allow one to evaluate, debate and discuss a scientific report such as an
environmental impact statement and make real-world decisions using that information.
The Communicating Scientific Information construct is composed of the communication
skills that would be necessary as part of that discussion and debate process. The four
constructs are seen as four dimensions on which students will make progress during the
curriculum and are the target of every instructional activity and assessment in the
curriculum. The dimensions are positively related, because they all relate to “science”,
but are educationally distinct.

The Evidence and Tradeoffs (ET) construct was split into two parts (called
“elements”) to help relate it to the curriculum. An initial idea of the Using Evidence
element of the ET construct was built up by considering how a student might increase in
sophistication as they progressed through the curriculum. A sketch of the construct map

28
for this case is shown in Figure 2.15: On the right side of the continuum is a description
of how the students are responding to the ET items.

Figure 2.11 A sketch of the construct map for the Using Evidence construct of the IEY
Evidence and Tradeoffs constructs.

29
2.3 Using Construct Mapping to Help Develop an Instrument.
How can the concept of construct mapping help in the development of an
instrument?

The central idea in using the construct mapping concept at the initial stage of
instrument development is for the measurer to focus on the essential features of what it is
that is to be measured—in what way does an individual show more of it, and less of it? It
may be expressed as from “higher to lower”, “agree to disagree”, “weaker to stronger” or
“more often to less often”—the particular wording will depend on the context. But the
important idea is that there is an order for the waypoints inherent in the construct—and
underlying that there is a continuum running from more to less—that is what allows it be
thought of as a construct map. A tactic that can help is the following:
(a) think first of the extremes of that continuum (say “novice” and “expert”, or in
the context of an attitude toward something, “loathes” to “loves”),
(b) make the extremes concrete through descriptions, and then
(c) develop some intermediate waypoints between the two extremes.
It will be helpful also to start thinking of typical responses that respondents at each level
would give to first drafts of items (more of this in the next chapter).

Before this can be done successfully however, the measurer will often have to
engage in a process of “variables clarification” where the construct to be measured is
distinguished from other, closely related, constructs. Reasonably often, the measurer will
find that there were several constructs lurking under the original idea—the 4 building
blocks method can still be applied by attempting to measure them one at a time. One
informal tactic for disentangling possibly different constructs is to consider respondents
who are “high” on one construct and “low” on the other. If such respondents are
common and are interestingly different in a theoretical sense, then it is likely that the
constructs are interestingly different. However, similar to the adage “correlation is not
causation,” the opposite does not hold generally, that is respondents may tend to be
similar on two different constructs, but that does not necessarily imply that the constructs
are indeed the same.

The nine Examples discussed above, which are based on published cases, tend to
discuss only the final resolution of this process. However, the CUE example explicitly
addresses one successful effort at variable clarification, and this especially so in the
publication Cardace et al. (2019). Unfortunately, editors in the social science literature
tends to seek to eliminate such discussions as being superfluous to “scientific
advancement” in the discipline—this is, of course, nonsense—in fact it is essential that
such discussions and investigations be made public, so that social science disciplines can
advance in their measurement practices. There are more examples of this in the Examples
Archive (Appendix A).

In creating a construct map, the measurer must be clear about whether the
construct is defined in terms of who is to be measured, the respondents, or what
responses they might give, the item responses. Eventually both will be needed, but often
it makes sense in a specific context to start with one rather than the other. For instance,

30
on the one hand, when there is a developmental theory of how individuals increase on the
construct, or a theory of how people array themselves between the extremes of an
attitude, then probably the respondent side will be first developed. On the other hand, if
the construct is mainly defined by a set of items and the responses to those items, then it
will probably be easier to start by ordering the item responses.

2.4 Resources

Many examples of construct maps are given in the series of references cited in
Appendix 1. However, relatively few of them incorporate both the respondent and item
response sides of the continuum.

One important issue is that one needs to distinguish constructs that are amenable
to the use of construct mapping and constructs that aren’t. Clearly any construct that is
measured using a single score or code for each person will be a candidate for construct
mapping. If a more complex construct is composed of a series of strands (such as the
learning progression in the CUE example), then each in turn can be seen as a construct
map. Also exemplified above were constructs that are partially ordered—these too can
be simplified so that they can be treated as construct maps.

The major type of construct that is not straightforwardly seen as a candidate for
mapping is one where there is no underlying continuum, where, for example, there is
assumed to be just a set of discrete unordered categories. This is seen in areas such as
cognitive psychology where one might assume that there are only a few strategies
available for solving a particular problem. Latent class analysis (e.g., Vermunt, 2010) is
an approach that posits just such a construct and should be used when the measurer is
seriously wanting to use that as the basis for reporting.

When there is an ordered partition among the latent classes, such as, say, an
increasing complexity in the nature of the strategies, then other possibilities arise, such as
is illustrated in Figures 2.3 and 2.4. For example, one could have the strategies treated as
observed categories mapped to an underlying latent continuum of increasing
sophistication (e.g., Wilson, 1992).

Or one could try and combine the two types of constructs, adding a construct map
within classes (e.g., Wilson, 1989, or Mislevy & Wilson, 1996) or adding a dimension as
a special class (e.g., Yamamoto & Gitomer, 1993). Increasingly complex combinations
of all of these are also possible, leading to some very complicated possibilities (see, for
example, Jeon et al. 2018; Junker, 2001).

2.5 Exercises and Activities

(following on from the exercises and activities in Chapter 1)

31
1. Lay out the different constructs involved in the area you have chosen to work in.
Clarify the relationships amongst them and choose one to start concentrating on.

2. For your chosen construct, write down a brief (1-2 sentences) definition of the
construct. If necessary, write similar definitions of related constructs to help distinguish
amongst them.

3. Describe the different waypoints of the construct—as noted above, start with the
extremes and then develop qualitatively distinguishable waypoints in between.
Distinguish between waypoints for the respondents, and in potential item responses.
Write down the successive waypoints in terms of both aspects, if possible at this point in
the development of the construct map.

4. Open up the BASS application and name your construct map, enter your definition,
and add useful background information and links (be sure to save your work).

5. Take your description of the construct (and any other clarifying material) to a selected
subset of your informants and ask them to critique it.

6. Try to think through the steps outlined above in the context of developing your
instrument and write down notes about your plans for how the responses may be mapped
to the waypoints.

7. Share your plans and progress with others—discuss what you and they are succeeding
on, and what problems have arisen.

32
 Appendix 2A
The Constructing Measures Examples

1 The Data Modelling (MoV) Example

1A The Data Modelling (6 Constructs) Example
2 A Social and Emotional Learning Example (RIS-Researcher Identity Scale).
3 An Attitude Example (GEB- General Ecological Behavior)
4 A 21st Century Skills Example (LPS-Argumentation)
5 A Process Measurement Example: Collaborative Problem Solving (CPS)
6 A Health Assessment Example (PF-10—Physical Functioning 10)
7 An Interview Example (CUE- Conceptual Underpinnings of Evolution).
8 An Observational Instrument: Early Childhood (DRDP)
9 The Issues Evidence and You Science Assessment (IEY)

# Name Chapters Sections Website, etc.

1 Data 1, 2 1.3.1, 1.4.1, 1.5.1, 1.6.1, 2.1
Modeling
(MoV)
1 Data 2, 9 2.2.1, 9.1, 9.2
A Modeling
(6
Constructs)
2 RIS-G 2, 4, 8 2.2.2, 3.2.4, 8.2.1, 8.4.2, 8.4.4, 8.5.1,
8.7.2
3 GEB 2 2.2.3
4 LPS-Arg 2, 3 2.2.4, 3.2.1, 3.4
5 CPS 2, 9 2.2.5, 9.3
6 PF-10 2, 5, 6, 7, 8 2.2.6, 5.3, 6.2.1, 6.3.1, 6.3.2, 6.3.3, 7.2,
8.4.6, 8.5.1
7 CUE 2 2.2.7, 3.2.3
8 DRDP 2, 8 2.2.8, 3.2.1, 8.3.1
9 IEY 2, 4 2.2.9, 4.3.2

33
 Appendix 2B
The Complete Set of GEB Items
3910 I am a member of an environmental organization.
15 I contribute financially to environmental organizations.
24 I read about environmental issues.
47 I am a member of a carpool.
50 I am a vegetarian or vegan
43 I own solar panels.
9 I drive on freeways at speeds under 100kph (= 62.5 mph).
14 I have pointed out unecological behavior to someone.
46 I refrain from owning a car.
6 (inverse) I drive my car in or into the city.
25 I talk with friends about environmental pollution, climate change, and/or energy consumption.
1 I ride a bicycle or take public transportation to work or school.
26 (inverse) For longer journeys (more than 6 hours of travel time by car), I take an airplane.
20 I buy domestically grown wooden furniture.
28 (inverse) At red traffic lights, I keep the engine running.
21 I boycott companies with an unecological background.
2 I buy meat and produce with eco-labels.
11 In nearby areas (around 30 kilometers; around 20 miles), I use public transportation or ride a bike.
49 I own a fuel-efficient automobile (less than 6 liters per 100 kilometer).
44 I have looked into the pros and cons having a private source of solar power.
35 (inverse) I use fabric softener with my laundry.
19 I buy products in refillable packages.
18 (inverse) I buy convenience foods.
45 I have a contract for renewable energy with my energy provider.
4 (inverse) I use an oven cleaning spray to clean my oven.
10 (inverse) If I am offered a plastic bag in a store, I take it.
30 In winter, I turn down the heat when I leave my apartment for more than 4 hours.
38 (inverse) I use a chemical air freshener in my bathroom.
23 (inverse) I use a clothes dryer.
31 (inverse) I drive to where I want to start my hikes.
17 (inverse) I buy bleached or colored toilet paper.
34 (inverse) In the winter, I keep the heat on so that I do not have to wear a sweater.
3 (inverse) I buy beverages in cans.
7 (inverse) In the winter, I air rooms while keeping on the heat and leaving the windows open, simultane
29 (inverse) I kill insects with a chemical insecticide.
27 (inverse) I keep the engine running while waiting in front of a railroad crossing or in a traffic jam.
22 I buy seasonal produce.
8 I wash dirty clothes without prewashing.
16 I buy beverages and other liquids in returnable bottles.
37 (inverse) After meals, I dispose of leftovers in the toilet.
12 I collect and recycle used paper.
41 I own an energy efficient dishwasher (efficiency class A+ or better).
48 I drive in such a way as to keep my fuel consumption as low as possible.
13 I bring empty bottles to a recycling bin.
32 I shower (rather than to take a bath).
40 (inverse) In hotels, I have the towels changed daily.
36 (inverse) I put dead batteries in the garbage.
5 I wait until I have a full load before doing my laundry.
42 After a picnic, I leave the place as clean as it was originally.
33 I reuse my shopping bags.

10
In difficulty order.

34