This article was downloaded by: [McGill University Library]

On: 20 December 2012, At: 01:14

Publisher: Psychology Press
Informa Ltd Registered in England and Wales Registered Number:
1072954 Registered office: Mortimer House, 37-41 Mortimer Street,
London W1T 3JH, UK

Multivariate Behavioral
Research
Publication details, including instructions for
authors and subscription information:
http://www.tandfonline.com/loi/hmbr20

Changing a Causal
Hypothesis without
Changing the Fit: some
Rules for Generating
Equivalent Path Models
Ingeborg Stelzl
Version of record first published: 10 Jun
2010.

To cite this article: Ingeborg Stelzl (1986): Changing a Causal Hypothesis

without Changing the Fit: some Rules for Generating Equivalent Path Models,
Multivariate Behavioral Research, 21:3, 309-331

To link to this article: http://dx.doi.org/10.1207/s15327906mbr2103_3

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/

terms-and-conditions

This article may be used for research, teaching, and private study
purposes. Any substantial or systematic reproduction, redistribution,
reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make
any representation that the contents will be complete or accurate
or up to date. The accuracy of any instructions, formulae, and drug
doses should be independently verified with primary sources. The
publisher shall not be liable for any loss, actions, claims, proceedings,
demand, or costs or damages whatsoever or howsoever caused arising
 directly or indirectly in connection with or arising out of the use of this
material.
Downloaded by [McGill University Library] at 01:14 20 December 2012
 Multivariate Behavioral Research, 1986,21,309331

Changing a Causal Hypothesis without Changing the

Fit: Some Rules for Generating Equivalent Path Models
Ingeborg Stelzl
University of Marburg, West Germany

Since computer programs have been available for estimating and testing linear caiusal
models (e.g. LISREL by J6reskog & Stirborn) these models have been used increasi~igly
in the behavioral sciences. This paper discusses the problem that very different caiusal
structures may fit the same set of data equally well. Equivalence of models is defined as
Downloaded by [McGill University Library] at 01:14 20 December 2012

undecidability in principle and four rules are presented showing how equivalent models
may be generated. Rules I and If refer to inversions in the causal order of variables and
Rules III and IV to replacements of paths by the assumption of correlated residuals.
Three examples are given to illustrate how the N ~ are S applied.

Within the last decade considerable progress has been made in

developing covariance structure analysis as a multivariate method,
combining the ideas of classical path analysis and confirmatory factor
analysis. Programs such as LISREL VI by Joreskog and Sbrbom (1984)
or COSAN by McDonald (1978,1980) can be used to estimate complex
models for linear causal relationships and to test their fit. Since these
programs are available at most universities, covariance struct~ure
analysis has found increasing attention among psychologists (for a
review of psychological applications see Bentler, 1980).
The data analysed by covariance structure analysis usually are
correlation matrices. The model to be estimated can be a model with
assumed causal relationships either between observables or between
latent variables, with the observables taken as indicators. The re-
searcher usually starts from a model in form of a path diagram, which
is then translated in the program's language of equation systems. The
output of the analysis consists of the estimates for the parameters ~tnd
several indices for the fit of the model.
In order to obtain consistent and unbiased estimates for ithe
parameters the niodel must be
1. specified correctly; i.e., the assumed causal relationship be-
tween the variables must be correct and no relevant variable
must be omitted.
2. identified; i.e., given the covariance matrix of the variables rthe
solution for the parameters of the model must be unique.

The author thanks the editor and two anonymous reviewers for their helpful
comments on an earlier version of this paper.
All oomspondence to the author should be addressed to: Ingeborg Stelzl, Fachber-
eich Psychologie der Universiat Marburg, Gutenbergstr. 18, D 355 Marburg, F.R.G.

JULY 1986 :309

 lngeborg Stelzl

While the second question, whether the parameters are identified,

has been discussed in some detail by statisticians (Hanushek &
Jackson, 1977; Malinvaud, 1980; Joreskog & Sarbom, 1982), the first
problem has been left mainly to the researcher to solve. It is his task to
develop the correct model. That the same set of data may be fitted
equally well by quite different models has never been denied but has
not been subjected to systematic investigation either.
This paper deals with one aspect of the problem. In the follrswing
section equivalence of two path models is defined as undecidability in
Downloaded by [McGill University Library] at 01:14 20 December 2012

principle (that is the impossibility to decide between the models

whatever the data look like). Then we turn to the question of when the
reversal of the direction of a path leads to an equivalent model (Rule I
and Rule II). Inverting the direction of a path means a change in the
causal hypothesis, and this usually will have remarkable conse-
quences on interpreting the results and drawing psychological conclu
sions. Another modification of a structural model might be to replace a
directed path by the assumption of correlated residuals. That would
mean that no direct influence is assumed between the two variables,
but an unknown external variable produces (part of) the covariance
between these variables. Two further rules (Rule III and IV) sta* the
conditions under which replacing a directed path by correlated residu-
als leads to an equivalent model.
The discus~ionwill be restricted to recursive models, but it applies
to models both for latent variables and for observables.
Before starting with the definition and the rules for equivalence it
will be useful to summarize some basic ideas of path analysis and to
introduce a simple notation for path diagrams.

Path Models: Basic Assumptions and Notation

In a complete recursive path diagram a variable, say A, deter-

mines a second variable B; A and B determine a third variable C; A
and B and C determine D and so on (see Van der Geer, 1971). If we
denote the variable in the first place of the causal chain by X1, the
second by X2 and so on, we have the equation system:
[I] A = X1
B = X 2 = P21X1+ E2
c = x3 = P31x1 + $3&2 + E3
310 MULTIVARIATE BEHAVIORAL RESEARCH
 where the coefficients p are path coefficients (= partial
Downloaded by [McGill University Library] at 01:14 20 December 2012

regression weights) and E2, E3....En are residuals from

the regression.
The solution for the path coefficients P can be obtained from the
covariance matrix I: of all variables by successive multiple regression.
The path coefficientsfor the paths arriving at the i-th variable are the
partial regsession coefficients of the i-th variable on the i-1 variables
preceding it in causal order. Arranging the rows and columns of' Z
according to the causal order of the variables, we get the coefficients P
for the i-th variable by multiple regression using the submatris formled
from the first i-1 rows and columns of X as covariance matrix of the
predictors and the first i-1 elements af column i as vector of the
covariances of the predictors with the criterion.

where
Pi = vector of path coefficients for variable i, containing the
partial regression weights of the i-th variable on the
variables XI, X2...Xi-1
Z(i-l, = covariance matrix of the variables XI, X2..Xid1
Ziill = inverse of $i-ll
= vector of covariances of Xi with XI, X2..Xi-I
ofi)
A recursive path diagram as given in Equation 1 is called
complete if all paths are free parameters. A complete model, however,
is not testable, because the number of parameters is equal to tlhe
number of elements in the covariance matrix 2. To get a testablle
model some restrictions must be introduced to reduce the number of
parameters. For instance, some parameters might be replaced by kcd
constants, if their numerical values are known a priori from theory.
" The number of parameters might also be reduced by constraining some
parameters to be equal. In practice, however, the most common
JULY 1986 311
 lngeborg Stelzl

restriction is to assume that some path coefficients are zero and to omit
these paths in the path diagram. Since the structural equations for
these models are special cases of the equational system given as
Equation 1, Equation 2 applies for these models too.
If, for instance, a psychologist assumes that Xg (= level of
education) has no direct influence on Xh (= neuroticism) and therefore
omits the path from Xgto Xhthat means that in Equation 1
Phg =0
Downloaded by [McGill University Library] at 01:14 20 December 2012

and, referring to Equation 2, that the g-th element of the solution-

vector &) is zero:
phg = P(xhXglx~..xg-l~g+l..xh-l)
=0
which also means that
Phg =0
= ~(x~&/x1~.x~-1~~+l..xh-l)
If S, the covariance matrix for the population is known, and the
path model is correct, solving for the path coefficients P according to
Equation 2 is straight forward.
With a sample covariance matrix S only, there are of course
problems of estimation. But we will not discuss this topic here. A
general introduction to estimation procedures for structural equation
models is given for example by Malinvaud (1980) or Kmenta (1971).
The special case of recursive path diagrams with observable variables
only is discussed by Land (1969,1973) and Wermuth (1980). A broader
review of more recent literature concerning estimation procedures in
covariance structure analysis is given by Bentler & Bonnet (1980).
Our further discussion will be restricted to the type of path models
described above, that is, to recursive path models with zero-con-
straints. We will not consider non-recursive models because theoreti-
cal problems seem to be much more involved for this class of models.
We will concentrate upon zero constraints because this type of con-
straint is the one which is by far the most widely used and the most
important for empirical research.

Notation
The general structure of a complete recursive path diagram is
determined when the causal order of the variables is known. For the
general structure of a model with zero constraints to be known, the
causal order and the zero paths must be given, as shown in Figure 1.
We will use this way of notation for the equation system of a path
diagram with zero constraints.
312 MULTIVARIATE BEHAVIORAL RESEARCH '
 lngeborg Stelzl

Variable:
I
I
A, B, C , D - - . . - - - Q
Position: 1, 2 , 3 , U . . .n ....
Figure 1. Path diagram with three zero-paths: AC, AD, BC.

There may he models in which the order of the variables is not

Downloaded by [McGill University Library] at 01:14 20 December 2012

specified completely: LISREL for exmple allows the definition of same

variables as "independent". That means that their correlations are not
analysed and that they are set all together in the first place in the
causal chain. Another reason for ambiguity may stem from zero paths:
Zero paths between variables on adjacent positions leave the ordering
of these variables undetermined. For the rules given below, however,
no problems arise: If the order is undetermined, one may choose any
order one likes. Equivalence of the other way of ordering imrnedia1;ely
follows from the rules.

Compatibility of Path Models with Covariance Matrices

If Z is a nonsingular covariance matrix and P is a complete path

diagram, solving Equation 2 always yields a unique solution for the
path coefficients p. If in this solution some path coefficients fi turn out
to be zero, evidently S has a solution also for a path model Po that
requires just these coefficients to be zero. We may say that X is
compatible with Po, or that I:may be generated from Poby substiitut-
ing special numerical values for its parameters. More generally
speaking, we call a nonsingular covariance matrix X compatible 4 t h a
path diagram Po, if the solution of Equation 2 meets all restrictions ofPo.
Nonsingularity of Z is assumed to make the solution unique. No
variable must be a linear combination of others. This condition is nnet,
for instance, if all residuals are independent and all variances of the
residuals are greater than zero.

Equivalence of Path Models

Two path models Poand PI are called equivalent if they generate

(are compatible with) the same set of covariance matrices (Jiireskog &
Siirbom, 1984, p. 1.21). That means, that for every nonsingular covari-
JULY 1986 313
 lngeborg Stelzl

ance matrix which may be generated by substituting numerical values

for Po, there is a solution for the parameters under the assumptions of
P1 and vice versa,
When two models are equivalent we cannot decide between them,
even if we know the population covariance matrix for our variables.
Whatever the population matrix looks like, it will be compatible with
both models or with neither of them.
Prom an empirical investigation, however, only a sample covari-
ance matrix S is known. Therefore one cannot give a definite answer to
Downloaded by [McGill University Library] at 01:14 20 December 2012

the question of whether the population matrix E, is compatible with

some path diagram Po, but instead one must ~udgewhether S is likely
to come from a population matrix that is compatible with Po.Several
indices for goodness of fit have been suggested for this purpose
(Jiireskog & Sorbom, 1984, chap. I J 2 ; Bentler & Bonnett, 1980).It will
be shown below that two equivalent models must both produce the
same values for all goodness of fit indices.
Let us denote the set of matrices X that are compatible with Poas
noand an element from this set as BooTo estimate the parameters for
Po a loss function has to be chosen, which is defined as a measure of
closeness between a hypothetical population matrix X and the sample
matrix S. Well known types of loss functions, which are used for
example, by the programs LISREL and COSAN, correspond to the
principles of Maximum Likelihood, Ordinary Least Squares, and
Generalized Least Squares. After a special loss function has been
selected, those values for the parameters that minimize the loss
fuadion are takRn as estimates. We will denote these estimates with $
and the covariance matrix that is generated from Poby the values $ as
*4.
Two equivalent models Po and PI generate the same set of
covariance matrices, that is no= n1.Therefore, no matter whether Po
or PI is fitted to the data the same matrix So= 81 will be selected as
the mati+ that minimizes the loss function(Theparameters Oo and O1
of course will be different and will depend on the dBerent equation
systeims that correspond to Poand PI.)
Since fitting equivalent models Po and P1 results in the sqme
estimated covariance matrix Zo = el, all indices of goodness of fit that
are defined to compare the fitted covariance matrix to the sample
matrix S must be the same for Po and PI.
Thus, as has been stated before, there is no way to decide between
equivalent madeis by exqmiaing S or even the papulation matrix X.
Unfortunately, that does not mean that it is always possible to decide
between nion-equivalent models. If one takes two non-equivalent
314 MULTIVARIATE BEHAVIORAL RESEARCH
 lngeborg St13lzl

models and looks at the set of covariance matrices which can be

generated from them, one will usually find that these sets are clot
mutually exclusive. Thus, considering any particular covariance ma-
trix from the set of all nonsingular covariance matrices, this matrix
may be compatible with both models, with only one of them, or with
neither. If S is compatible with both models it must of course sati~ify
both sets of restrictions and in thie case the researcher will usually try
to find a new model that combines the constraints of the two models
and reduces the number of parameters as far as possible.
Downloaded by [McGill University Library] at 01:14 20 December 2012

Thus with twu,non-equivalent models, the simplest outcome fra~m

an empirical investigation would be that one of the two models is
supported by the data while the other one is rejected. But tlhe
researcher may also end up rejecting both models or neither of them.

Rules for Generating Equivaknt Models

A complete recursive path diagram is compatible with all nonsin-

gular covariance matrices (Wermuth, 1980). So all complete path
diagrams are trivially equivalent. A path diagram Po, in which some
paths are omitted, however, will generate only those covariance
matrices for which the partial regression coefficients (and partial
correlations) corresponding to the omittad paths are zero. Thus, iDo
divides the set of all nonsingular covariance matrices into two distinct
subsets: those that are generable from the model, as all restrictio~ns
concerning the partial correlations are met, and those which are na~t.
Evidently, two path models Po and PI which constrain the same
partial correlations to zero generate the same set of covariancce
matrices. Hence they are equivalent. On the other hand, if one of the
two path models, for example Po, constrains a partial correlation to
zero that is not constrained in PI, then PI will generate some
covariance matrices which are not compatible with Po.Hence, in this
case the two models cannot be equivalent. Thus we can state that for
two recursive path models with zero constraints for some paths to be
equivalent, it is necessary and sufficient that they constrain the same
partial correlations to zero.
Note, that the same partial correlation means the partial correlia-
tion between the same two variables (for example, level of education
and neuroticism) with the same set of other variables (for exampbe,
parental dominance, number of siblings, and social competence) par-
tialled out. It does not mean that these variables must hold the same
positions in the causal order of the two models. Indeed, the question of
JULY 1986 315
 lngeborg Stelzl

when different path models lead to the same restrictions in the partial
correlation will be our main concern in the following.

Changing Causal Order

The first two rules answer the question: which inversions in the
causal order of the variables will result in equivalent models.

Rule I
Downloaded by [McGill University Library] at 01:14 20 December 2012

As has already been stated above, two path diagrams Po and PI

are equivalent if they constrain the same partial correlations to zero.
Since in a partial correlation the causal order of the variables par-
tialled out is irrelevant, all changes in causal order will lead to
equivalent models as long as for every constrained partial correlation
the set of variables partialled out is the same in Po and P I .
As a first example let us look at a path diagram with only one
constraint, From this example, we will specify three special cases of
Rule I which will be denoted as Rule Ia, Ib, and Ic later in the text.
Let Po be a recursive path diagram of n variables, as in Equation
1, and assume that we know that there is no direct effect of variable E
on variable M. Let E be at position g and M be at position h (with g <
h), as shown in Figure 2.
We now obtain a large number of path diagrams equivalent to Po
if we keep the restriction of a zero-path from E to M but interchange
the order of the variables in any one of the follawing ways:
(Ia) Interchapge the positions of the variables to the left of M
(places 1 up to h-1) in any sequence.
(Ib) Interchange the pasitions of the variables to the right of M
(places h+l up to n) in any sequence.
(Ic) Interchange the positions of E and M.
Model Po requires, that, if M is regressed on all causally prior
variables (to the left), $he partial regression coefficient for E should
turn out to be zero; that is, p(MEIA.J)P.L) shiould be zero. In this
condition the order of the variables from A to L to the left of M and

Variable: A , B.. D, E. F.. ..

L . M * N. Q
n
.. .. *..
Position: 1.2. .$-1, g, g+1.. h-1*h, h+1. .n

Figure 2. Path diagram with one zero-path: EM. All other path coefficients are free parameters.

316 MULTIVARIATE BEHAVIORAL RESEARCH

 lngeborg Stelzl

from N to Q to the right of M is irrelevant. Furthermore whether 1%is

at position g and M is a t position h or the other way around1 is
irrelevant. Both constraints p(ME1A.Df.L) = 0 or p(EMIA..D,F.J;f)=
0 mean the same partial correlation p(EM1A.DJ.L) to be zero.
Now let us have a look on the solution for the resulting path
diagrams P I .
Using Rule Ia: By changing the positions of the variables from 1to
h-1, the set of predictors for these variables is changed. Therefore
their path coefficientsmust be re-computed by regressing each vari-
Downloaded by [McGill University Library] at 01:14 20 December 2012

able on all variables preceding it in causal order according to the

assumptions of PI.For variable M and all variables to the right of M
the set of predictors is the same in Po and PI and therefore their path
coefficients remain the same in Po and P I . In particular the path
coefficient from E to M will be zero also in P1, as required.
Using Rule Ib: Here the multiple regression equations for the
variables in position 1 up to h remain the same in Po and P1, So their
path coefficients (including the constrained path from E to M) will not
change. Only in the regression equations fbr the variables to the right
of M are predictors addedldeleted. Hence only for these variables do
path coefficients have to be recomputed.
Using Rule Ic: The coefficientsfor the variables to the left ofE (1to
g- 1) and to the right of M (h+1 to n) are unchanged. For the varialbles
E and M and for the variables between them (g+l to h-1) predictors
are addedldeleted and the path coefficients have to be re-computed.
Because of p(ME1A.DP.L) = 0 the path from M to E, that is
p(EMIA.Df.L), will turn out to be zero.
Combining the possibilities from (Ia) to (Ic) yields (h-l)!(h-g)!2
equivalent models. But usually a path diagram has more than one
zero-path and so the number of equivalent models is considerably
reduced.
Let us consider a path diagram with two zero-paths, as shown in
Figure 3.

Variable: A, B.. . . .E.. . I.. .K.. . . . P.. ...V

Position: 1.2 .....k...l...s .....t .....n
Border-Positions:.......... t.....?
Figure 3. Path diagram with two zero-paths: EK, IP. All other path coefficients are free parameters.

JULY 1986 317

 lngeborg Stelzl

If we look for changes in the causal order that can be made without
changing the set of variables partialled out in the critical partial
regression equations for K and P, we will find that we may:
Interchange the variables on position 1 to s-1;
Interchange the variables on positions s+ 1 to t-1;
Interchange the variables on positions t+ 1 to n.
These changes correspond to Rules Ia and Ib given above. Further-
more, one may interchange E and K according to Rule Ic. Interchang-
ing the positions of I and P, however, would change the set of variables
Downloaded by [McGill University Library] at 01:14 20 December 2012

in the regression equation for K, which is a critical one, and therefore

would not lead to an equivalent model.
In general, if there is more than one constraint, take each
constraint alone and work out a11 permutations that. are allowed with
regard to this constraint. Having done so, pick out those permuta@ions
which are allowed with regard to all constraints*As a result it will be
found that the variables for which zero-paths are prescribed play a
special role as border positions: Rule I permits change in causal order
to the left or to the right of border positions, or between them.
Moreover, it permits interchanges in "startpoints" and "endpoints9' of
zero-paths, provided that in doing so border positions are not crossed.
The conditions of Rule I are sufficient, but not necessary for the
resulting model PI to be equivalent. Further possibilities, where
interchanges of variables are also possible acrass border positions are
given in Rule 11.

Rule 11:
In Rule I only those changes in causal orderings were considered
which leave the set of predictors in the critical regression equation
(that is the equation with the zero restriction) unchanged. We now ask
for cases in which a partial correlation remains zero, in spite of
deleting or adding a variable to the set of variables, which are
partialled out. The answer is given by Rule 11.
Let Po be a recursive path diagram in which the variable E has no
direct influence on the variables M and N. Let E be at position g and M
and N on the adjacent positions h and h+l (with g < h), as shown in
Figure 4.
When we invert the positions of M and N and keep the restriction
of zero.paths from E to M and N, the new model PI is equivalent to 9 0 .
This rule can be derived fmm the well known formula for the
change in partial correlations that occurs, when predictors are added.
The restriction of Po are:
318 MULTIVARIATE BEHAVIORAL RESEARCH
 lngeborg Stelzl

The restrictions of Plare:

p(MEIA.. D,F..L,N) -
p(NEIA..D,F..L) = 0 or p(NE/A..D,F..L) = 0.
0 or p(ME1A.. D,F..L,N) = 0.

From the general formula for partial correlations (Lord & Novick,
1968, p.280) we have
Downloaded by [McGill University Library] at 01:14 20 December 2012

Starting from the restrictions of Po we get

Hence

as required in the first restriction of P1.Deleting M from the

predictors of N does not affect the zero-path EN.

Furthermore ,

p(ME/A.D,F. L,N) =
.
[p(MEIA.D,F..L) - p(MN1A.DP.L) p(ENIA.DF.L)I/Denorn. =
EO - p(IJ1A.DF.L) .O]/Denom. = 0,
as required in the second restriction of P I . Adding N to the
predictors of M does not effect the zero-path EM.

So the restrictions of Plcan be derive;d from the restrictions of Po.

By the same argument, the restrictions of Po follow from the restric-
tions of P1,so that equivalence is shown.
The rule given so far only refers to the case where M and N lhave
only one zero-restriction relating to the same predictor (E).If M and N
have more than one zero-path, with these zero-paths coming from the
same predictors, the same argument can be used to show that the
positions of M and N are interchangeable and the resulting model P1 is
still equivalent to model Po. It is important however, that M and N
occupy adjacent pcrsitions. Otherwise interchanging the positions of M
and N would mean that the variables between M and N would be
addedldeleted from the regression equations and the resulting model
P1would in general not be equivalent to Po.
JULY 1986 319 .
 lngeborg Stelzl

Replacing Paths by Correlating Residuals

While in classical path analysis residuals are assumed to be
independent, models of the LISREL-type provide for correlations in the
residuals ~r errors. Thus the same portion of covariance can either be
explained by a directed path or by correlating residuals, and in each
case one parameter is fitted. We now discuss the conditions under
which replacing path-coefficients by correlated residuals (or vice versa)
will lead to equivalent models.
Downloaded by [McGill University Library] at 01:14 20 December 2012

Rule 111:
Again let Po be a recursive path diagram from n variables. Let
variable E be at position g and variable M at position h, with g < h:
Variable A,B ....B....M .....Q
Position 1,2 ....g ...h .....a
All%path coefficients for paths leading to M are free parameters. All
other path coefficients may be free or constrained.
The substitution of the path from E to M by assuming correlated
residuals results in an equivalent model if all path coefficients for M
are free parameters.
If you omit the path from E to M and replace it with correlated
residuals, the path coefficients for all other variables (besides M)
remain unchanged, because they are regressed on the same predictors
in Po and P I . Only the regression equation for M is changed, since E is
no longer taken as a determiner of M and is deleted from the set of
variables on which M is regressed. Thereby the regression coeacients
of the remaining predictors of M may be changed and thus must be re-
computed. The remaining portion of covariance between E and M,
which cannot be attributed to common predictors, is now "explained"
as covariance of residuals. If there are no constraints concerning the
path coefficients of M this way of substituting always produces a
solution for PI.Conversely, if there are no constraints in the path
coefficients for M (the variable for which a predictor is added), one can
always substitute correlated residuals by a directed path.

Rule IV:
Let Po again be a recursive path diagram in which variable E has
no direct effect on variable M and N.Let E be at position g and M and
N at the adjacent positions h and h + 1,with g < h, as has been shown
in Figure 4.
320 MULTIVARIATE BEHAVIORAL RESEARCH
 lngeborg Stelzl

Variable: A , B.. D. E. P.. . ..

L . M . N.. . . Q . . ..
Position: 1.2.. g-1.8, g+l.. h-1. h, h+1. n . .
Figure 4. Path diagram with two zero-paths, EM and EN, leading to paths on adjacent positions. All olher
path coefficients are free parameters.

Concerning the paths other than EM and EN, we now make the
Downloaded by [McGill University Library] at 01:14 20 December 2012

following assumptions: The path coefficients for all other paths leading
to N are free. The path coefficientsfor all other paths leading to M may
be free or constrained. The path coefficients for all other variables
(besides M and N)may be free or constrained.
The path from M to N can be replaced by the assumption of
correlated residuals, if all variables (in this example variable E) ibr
which fixed zero-paths are assumed for variable N have fixed zero-
paths also for variable M. (Since the reverse need not be true, this
condition is somewhat weaker than the condition given in Rule 111 .1
When the path from M to N is replaced by correlated residuals
only the regression equation for N is changed. Therefore the path
coefficients (including the fixed zeros) for all other variables, besid,es
N, will remain unchanged.
In the equation for N the predictor M is deleted. But as has alrea~dy
been shown in deriving Rule 11, this will not affect the zero-path EIV:
The constraint of Pl that

can be derived from the constraint of Po, that

p(ME1A.DP.L) = 0 and p(NEIA.DP.Lil4) = 0
and the constraint of Po, that

can be derived from the constraint of P1, that

p(ME1A.DP.L) = 0 and p(NEIA.D,F.L) = 0.
So the models Poand PI are equivalent.
The reverse form of the rule may be stated as follows: Correlated
residuals between M and N can be replaced by a directed path from M
to N if M and N are on adjacent positions, M before N, and N has zero-
paths originating only from variables from which M also has zero-
paths.
JULY 1986 321
 lngeborg Stelzl

Examples

The following section gives some examples of how Rules I through

may be applied. The first example is a fictitious one: a miniature
structure with only four variables is analysed. The second example
uses real data taken from the literature. In the last example Patent
variables are involved.
We always start &om a situation where the researcher already has a model. Our
rules for generating models which are equivalent to the given one are meant to
Downloaded by [McGill University Library] at 01:14 20 December 2012

help one avoid overlooking a rival hypothesis that is equally well supported by the
data and to prevent unwarranted conclusions.
It rnay happen, however, that a researcher has a covariance matrix but haq no
model. In this case our rules will not help. Dempster (1972) and Wermuth (1976,
1980)present a technique, which they call covariance selection analysis, and give
backward and forward strategies for finding partial correlations that are zero in a
covariance matrix. This technique might be used in such situations. A broader
discussion of methods for developing and modifying a model with the data already
at hand is given by Learner (1978).

Example 1
In Figure 5 a path-diagram involving four variables is shown. The
order of the variables is A, B, C, D with zero-constraints for the paths
from A to C and D.
Rule Ia allows the inversion of A and B
Rule I1 allows the inversion of C and D
Rule Ic allows the inversion of A and C
(or A and D, if CD have been inverted first).
The combination of these possibilities results in eight different
permutations, as shown in Figure 6.
These eight permutations correspond to only six different path
diagrams, which are given in Figure 7. Two diagrams (7 and 8) are
identical to diagrams already given (3 and 4) because they differ only
in the positions of variables, which are on adjacent positions and have
zero-paths between them.
The 6 path diagrams generated by the rules given above are
equivalent. That means that there is no way of deciding between them,
whatever the covariance matrix of the variables may be.
Besides inverting positions of variables, there are further possibil-
ities for generating equivalent models by assuming covariances in the
residuals instead of directed paths, according to Rules III and IV.In
diagrams 1 to 4, you may replace the paths between A and B and
between C and D, and in diagrams 5 and 6 any of the paths between C,
B, and D by the assumption of correlated errors.
322 MULTIVARIATE BEHAVIORAL RESEARCH
 A
- B
lngeborg Ste~lzl

C D
I

Figure 5. Path diagram fmm four variables.

Downloaded by [McGill University Library] at 01:14 20 December 2012

Exchange AB and C D Exchange A C or AD

II B A
LD Cl
Figure 6. Models resulting from the path diagram given in Figure 5 by permutationaccording to Rule I and
II.

Example I1
The data are taken from Sewell, Haller and Ohlendorf (1970) and
have already been reanalysed by other authors (Wiley, 1973). In the
LISREL-manual (Joreskog & Sijrbom, 1984, p. 111.40)a simple model is
suggested from which the reader may start with his modificatio~is.
This model and the LISREL-estimates for its parameters are given in
Figure 8. The model has three zero-paths: AE, BE and BC. We will now
apply our rules to see which models are equivalent to the model given
in Figure 8.
Rule I: The two zero-paths BE and AE do not impose any
restrictions on the order of variables A to D. Were these zero-paths the
only ones, the sequence of variables A, B, C, D would be completely
free. The zero-path BC, however, sets considerable limits to the
admissable permutations: D and E must stay to the right of B or C, so
that the path from C to D for example cannot be inverted. Indeed, these
JULY 1986 323
 lngeborg Stelzl
Downloaded by [McGill University Library] at 01:14 20 December 2012

7 same as 3

8 same as 4

Figure 7. Equivalent path diagrams corresponding to the permutations given in Figure 6.

are only two inversions left: the inversion of A and B, which is trivial,
and the inversion of B and C, which does not make sense considering
the contents of the variables.
Rule I I The conditions for applying this rule are not met.
Rule 111: Any of the paths leading to D might be replaced by
correlating residuals. The path coefficients for the other paths leading
to D would thereby be changed.
Rule IV: The conditions for applying this rule are not met.
Yet, the model given in Figure 8 does not fit the data and each of
the equivalent models would of course give the same bad fit. The
modification indices suggest adding a path from B to E. If one does so,
324 MULTIVARIATE BEHAVIORAL RESEARCH
 lngeborg Stelzl
Mental
A=~bility - .589 Academic
C = Performance

Educational

-
=Aspiration

tatus us -
Socioeconomic .245 Signif i c a n t
D=~ther's
Downloaded by [McGill University Library] at 01:14 20 December 2012

A B C D E
Figure 8. Path diagramand estimated parametersfor me model given in the LISREL-Manualas Computer
Exercise 5 "Educational Attainment".

the chi-square value is reduced by a considerable amount. Giving up

the zero-constraint for path BE does not lead to any new possibilities of
inversions and the directions of the paths are still well defined.
But the model still does not fit the data. The next modification,
suggested by the modification-indices,is to introduce a path from A to
E. But when this is done, the zero-path BC is the only constraint left.
Now Rule Ic can be used to change the order DE into ED. The resulting
two equivalent path models are shown in Figures 9 and 10.
Now the model fits the data. Giving up the last constraint would
have made the complete order unidentified.
Using Rule 111, further models, which are equivalent to the models
in Figures 9 and 10, can be generated by substituting correlating
residuals for paths. With respect to the content of the variables, tlhis

A c-B D E df = 1 X2= 3.47

Figure 9. Path diagram and estimated parameters for the model given in Figure 8 with two paths adtled.

JULY 1986 :325

 lngeborg Stelzl

Mental
,,*=Ability - 589 Academic
t =P e r f o r m a n c ~
I-

B=~tatus -
Socioeconomic .127 Sianif i c a n t
U = ~ 6 e r ' s~ n f l u e i c e .-?73/
Downloaded by [McGill University Library] at 01:14 20 December 2012

Figure 10. Path diagram and estimatedparametersfor the model given in Figure 9 with the direction of the
path between D and E inverted.

may be plausible for the path from C to E, because both variables are
influenced by scholastic motivation. The results are given in Figure
11.Similarly, any of the paths leading to D or E might be replaced by
correlated residuals. This would lead to many equivalent models with
varying coefficients for the paths left, in the model.

Example ZZZ
In our last example latent variables are involved. The model given
in Figure 12 is part of a more complex model which Lohmoller (1984)
fitted to the data of Marjoribanks (1972). The numerical values

Performance

Figure 11. Path diagram and estimated parametersfor the model given in Figure 9 with the path from C to
E replaced by mrrelated residuals.

326 MULTIVARIATE BEHAVIORAL RESEARCH

 lngeborg Stelzl

assumed for the paths in Figure 12 correspond to his results (Loh-

moller, 1984,p. 3-09 and p. 4-05).
The model contains three latent variables, each of which is
measured by several observables. Joining Joreskog & Sorbom (1984,
p.I.3), we call that part of the model which is made up by the relations
among the latent variables the structural equation model, or the ininer
model, and that part of the model which combines the latent variables
to the observables the measurement model. In our model we have 15
parameters (12from the measurement model and 3 path coefficientrr in
Downloaded by [McGill University Library] at 01:14 20 December 2012

the inner model), which account for the correlations between the
observables. Since there are 66 correlations but only 15 paramett?rs,
there are 41 degrees of freedom and the model seems to he highly
restrictive.

Father Mother

I
I
Dominance ..J,
-\

Test 4
Activities

Figure 12. Pam diagram with three latent variable*. The numerical values for the path wefflcients i r ~the
inner model and for the factor loadings are taken from LohrnoIIer (1984, p.3-09 and p.4-05).

JULY 1986 :327

 lngeborg Stelzl

But with further examination of the structure of the model, one

may easily see that it is only the measurement model that is restric-
tive; the inner part of the model is saturated with parameters. Indeed,
the structural equation model is a complete recursive path diagram
and each of the 3! = 6 permutations which can be made up from the
letters A, B, C yields an equivalent model. (This rather trivial result
also follows from Rule I: Since there are no border positions, free
permutation is allowed.) Furthermore (according to Rule 111) each of
the paths may be replaced by correlating residuals. As is easily
Downloaded by [McGill University Library] at 01:14 20 December 2012

verified from the tracing rules of path aqalysis (Van de Geer, 1971, p.
124) or the equations given by Jiireskog and Siirbom (1984, p.I,8), two
equivalent structural equation models combined with the same mea-
surement model produce the same covariance matrices and therefore
the two combined models are equivalent too. The numerical values for
the path coefficients which result when the ordering of the latent
variables is changed to AA,C, B are given in Figure 13.
Looking at Lohmoller's solution, an interesting empirical result
might be that direct dominance pressure does harm to the intellectual
abilities of the child, while indirect pressure, which raises the child's
intellectual activities, has positive effects. But in the model shown in
Figure 13, which as an equivalent model must give exactly the same
fit, the path from parental dominance to mental ability is almost zero.
Therefore, without further information, one cannot draw the psgcho-
logical conclwion that parental dominance has a direct harmful effect
on IQ, because the mqdel of Figure 13, which is a plausible rival
hypothesis and is in conflict with this result, cannot be excluded.
As has been stated above, Figure 12 presents only part of Loh-
miillex's model. His complete model contains 6 latent variables and 18
observables. In this model there are many zero-constraintsand invert-
ing the path fkom "Parental Dominance" to "intellectual Activity"
would not result in an equivalent model, But also in his model this
path may be replaced by carrelating residuals, and this modifloation
again leads to a solytion with a path coefficientof about zero for the
path from "'Parental Dominanioe" to "Mental Ability."
So, which kind of information woqld help US to decide about the
direction of the path between B and C? We neeld a more complex model
with more prior knowledge. In Figure 14 two madels (A and B ) are
presented to &ow how a further variable X qight be added, so thpt in
the resulting model changiqg; t h position
~ of $ and It:would no longer
lead to eqvivalence. Mowever, according to Rule 111replacing the path
from B ta C would still had, m equivalent moddLTherefwe in model C
a further variable Y is intmduc~dand t h , ~path f?am Y to C is fixed to
328 MULTIVARIATE BEHAVIORAL RESEARCH
 lngeborg Stelzl

Father Mother

Parental
A =
Dominance
Test 1
Downloaded by [McGill University Library] at 01:14 20 December 2012

B=
Activities

C I ( m
~ a

Figure 13. Path diagram, equivalent b the diagram given in Figure 12.

Model A Model B Model C

n
X A B C
n
A B X C
m
Y X A B C

Figure 14. Three mcdels, where inverting BC doss not lead to equivalence

zero. Since there is now a constraint in the equation for C Rule 111does
not apply any longer. Since now the two models generated by inverting
the path BC or by replacing it by correlated errors are no longer
equivalent, one may at least hope that the data will aIIow acceptrmce
of one model and rejection of the others.
JULY 1986 329
 lngeborg Stelzl

So far we have been concerned with only the inner model and have
not questioned the measurement model. Further equivalent models
might be found when the latent variables are redefined, for example by
rotating factors, by defining second order factors and the like. Yet,
with a highly restrictive factor pattern which contains many zero-
loadings, rotation will not lead to more simplicity but only destroy a lot
of zero-loadings. So we will not go into further detail.

Discussion
Downloaded by [McGill University Library] at 01:14 20 December 2012

The examples given above were chosen to show how the rules for
generating equivalent models are applied. As already stated, equiva-
lence means that there is no way to decide between different models,
whatever the covariance matrix looks like. But psychologists who are
engaged in applied research are not interested in all theoretically
possible covariance matrices, but in the ones they actually have. Hence
they are not interested only in equivalent models but in all models that
might fit their data-irrespective of the question of whether these
models might be discernible from other covariance matrices and
therefore not be equivalent. Their problems go beyond what has been
discussed here. Furthermore, researchers do not know the population
covariance matrix exactly but only have estimates for it. Therefore
those models which could be exduded if the population matrix were
known. but cannot be excluded with the data at hand remain in the pool
of possible alternatives.
Thus we have three kinds of undecidability:
1. Undecidability in principle by equivalence of the models,
2. Undecidability given a special covariance matrix for the popu-
lation, and
3. Undecidability with the data at hand.
This paper only dealt with the first of these issues, indecidability
in principle. Yet, our discussion appears to be relevant for every
researcher using covariance structure analysis: Before gathering data
for a hypothetical model, it will be usefhl to generate equivalent
models according to the rules given above. By doing so, one will see
beforehand what will definitely not be decided by the data. Perhaps
one will find that there is undecidedness only in some minor details.
Perhaps one will be able to modify one's model to get a more stable
structure. In some cases, one may even conclude that one should not
gather the data a t all. In any case, it will not be a waste of time to
consider these problems before carrying out an empirical investiga-
tion.
330 MULTIVARIATE BEHAVIORAL RESEARCH
 lngeborg Stelzl

References

Bentler, P. M. (1980). Multivariate analysis with latent variables: Causal model~ng.

Annual Review of Psychology, 31,419--456.
Bentler, P.M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the
analysis of covariance structures. Psychological Bulletin, 88, 588-606.
Dempater, A. P. (1972). Covariance Selection. Riometrics, 28, 157-175.
Hanusek, E. A., & Jackson, J. E. (1977). Statistical methods for social scientists. New
York: Academic Press.
J6reskog, K. G., & Sarbom, D. (1984). LISREL 171. Analysis of linear structirral
relationships by the method of maximum likelihood. Usefs guide. Mooresville:
Scientific S o h a r e , Inc.
Kmenta, J. (1971). Elements of econometrics. New York: Macrnillan.
Downloaded by [McGill University Library] at 01:14 20 December 2012

Lord, F.M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading,
MA: Addison-Wealey.
Land, K. C. (1969). Principles of path analysis. In E. F. Borgatta (Ed.), Sociologrical
methodology. 4 . 3 - 3 7 . San Francisco: Jossey-Bass.
Land, K. C. (1973). Identification, parameter estimation, and hypothesis testing in
recursive sociological models. In A. S. Goldberger & 0.D. Duncan (Eds.), Structural
equntion models in the social sciences. 4.1 9 4 9 . New York: Seminar Press.
Learner, E. E. (1978). SpeciF:atwn searches--Ad hot inference with nowperimental
dnta, New York: Wiley.
LohmGller, J. B. (198.4). LVPLS, Progmm Manual. Latent variables path analysis with
partial least-squares estimation. Koeln: Zentralarchiv fiir empirische Sozialifors-
chung.
Malinvaud, E. (19801. Statistical methods of econometrics (3rd ed.). Amsterdam: Ncrrth-
Holland Publishing Company.
Majoribanks, K. (1972). Environment, social class, and mental abilities. Jourmzl of
Educational Psychology, 63,103-109.
McDonald, R. P. (19'78).A simple comprehensive model for the analysis of covariimce
structures. British Journal of Mathematical and Statistical Psychology, 31, 59--72.
McDonald, R. P. (1980). A simple comprehensive model for the analysis of covari,ance
structures: Some remarks on applications. British Jourml of Mathematical and
StcrtiStical Psychology, 33,161-188.
Sewell, W. H., Haller, A. O., & Ohlendorf, G. W. (1970). The educational and eaarly
occupational attainment process: Revisions and Replications. American Sociolo~~ical
Review, 35,1014-1027.
Van de Geer, J. P. (1976). Introduction to multwariate analysis for the social sciences.
San Francisco: Freeman.
Wermuth, N. (1976). Model search among multiplicative models. Biometries, 32, 253-
263.
~ e G u t hN.
, (1980). Linear recursive equations, covariance selection, and path analy-
sis. Journal of the American Statistical Association, 75, 963-972.
Wiley, D. E. (1973). The identification problem for structural equation models with
unmeasured variables. h A. S. Goldberger & 0.D. Duncan (Eds.), Structural
equation h l s in the social sciences (pp. 69-89). New York: Seminar Press.

JULY 1986 331