METHODS ARTICLE

A MULTICOLLINEARITY AND MEASUREMENT ERROR

STATISTICAL BLIND SPOT: CORRECTING FOR
EXCESSIVE FALSE POSITIVES IN
REGRESSION AND PLS
Dale L. Goodhue
Terry College of Business, MIS Department, University of Georgia,
Athens, GA 30606 U.S.A. {dgoodhue@terry.uga.edu}

William Lewis
{william.w.lewis@gmail.com}

Ron Thompson
School of Business, Wake Forest University,
Winston-Salem, NC 27109 U.S.A. {thompsrl@wfu.edu}

Multiple regression has a previously unrecognized “statistical blind spot” because when multicollnearity and
measurement error are present, both path estimates and variance inflation factors are biased. This can result
in overestimated t-statistics, and excessive false positives. PLS has the same weakness, but CB-SEM’s estima-
tion process accounts for measurement error, avoiding the problem. Bringing together partial insights from
a range of disciplines to provide a more comprehensive treatment of the problem, we derive equations showing
false positives will increase with greater multicollinearity, lower reliability, greater effect size in the dominant
correlated construct, and, surprisingly, with higher sample size. Using Monte Carlo simulations, we show that
false positives increase as predicted. We also provide a correction for the problem. A literature search found
that of IS research papers using regression or PLS for path analysis, 33% were operating in this danger zone.
Our findings are important not only for IS, but for all fields using regression or PLS in path analysis.

Keywords: Multicollinearity, measurement error, M+ME, multiple regression, partial least squares, PLS, CB-
SEM, false positives, Type I error, statistical power, variance inflation factor, VIF, path estimate bias

To purchase this paper, go to

https://misq.org/a-multicollinearity-and-measurement-error-statistical-blind-spot-correcting-for-
excessive-false-positives-in-regression-and-pls.html

MIS Quarterly Vol. 41 No. 3—Appendices/September 2017 A1

Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Appendix A
Deriving Equations for M+ME Biases and
for t-statistic Overestimations

Determining the Impact of VIF Bias on the t-statistic

Consider first the situation where we have no measurement error. Using standard equations for the estimated standard error of β2 from any
regression textbook, an unbiased and consistent estimate of the standard error is

s2 = Σε2i / (N – K) (A1-1)

and the estimate of the variance of β2 is

Var (β^2) = [s2 / Σx2i2] ∗ [1 / (1-ρ12Underlying2)] (A1-2)

When the error terms are normally distributed, the following has a t distribution:

t-stat = (β^2 - β2) / {Var(β^2)}.5 ~ t N-K (A1-3)

Substituting in the equations for Var ( β^2) and s2 from above (A1-1 and A1-2), we can see the impact of correlated predictor variables on the
t-stat.

t-stat = (β^2 - β2) / { [s2 / Σx2i2 ] ∗ [ 1 /(1- ρ12 Underlying2)] }.5

t-stat = (β^2 - β2) / { [{Σεi / (N – K)}] / Σx2i ]) ∗ [ 1 /(1- ρ12 Underlying2)] }.5
2 2

Rearranging terms:

t-stat = (β^2 - β2) / { [Σεi2 /{(N – K) ∗ Σx2i2 } ] ∗ [ 1 /(1- ρ12 Underlying2)] }.5

Squaring both sides:

(t-stat)2 = (β^2 - β2) 2 / {[Σεi2 /{ (N – K) ∗ Σx2i2 } ] ∗ [ 1 /(1- ρ12 Underlying2)] }

(t-stat)2 = (β^2 - β2) 2 ∗ { [ (N – K) *Σx2i2 ] / Σεi2 ] * [ (1- ρ12 Underlying2) / 1] }
2
(t-stat) 2 = (β^2 - β2) 2 ∗ { [ (N – K) ∗ Σx2i ∗ (1 – ρ12 Underlying2 )] } / Σεi2

Rearranging terms:

(t-stat) 2 = {[(β^2 − β2) 2 ∗ (N – K) ∗ Σx2i2 ∗ (1) / Σεi2}

– {[(β^2 – β2) 2 ∗ (N – K) ∗ Σx2i2 ∗ (ρ12 Underlying2)] /Σεi2} (A1-4)

Note that the first half of equation (A1-4) is the squared t-statistic if there were no correlation between X1 and X2. The second half is the
correction for when there is a correlation (at this point all assuming no measurement error).

As suggested by Goodhue et al. (2011), the problem with the above is that the estimate for the correlation (ρ12Underlying) assumes perfect
measurement (i.e., that the Xi values have no random measurement error). In regression, estimates of the two constructs are based on averaged
or summed indicator scores, and any estimate of the ρ12Underlying correlation will be attenuated (reduced) by the random measurement error, as
shown below:

ρ12Underlying = ρ12Overt / (α1 ∗ α2)1/2 (Equation 2 from the paper proper) (A1-5)

A2 MIS Quarterly Vol. 41 No. 3–Appendices/September 2017

 Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

When we do have measurement error, ρ12-Overt is not equal to ρ12Underlying. When the t-stat calculated by regression has not been corrected for
attenuation, A1-4 becomes:

(t-statOvert)2(incorrect) = {[(β^2 – β2)2 ∗ (N – K) ∗ Σx2i2 ]/Σεi2}

– {[(β^2 – β2)2 ∗ (N – K) ∗ Σx2i2 ∗ (ρ12-Overt2)]/Σεi2} (A1-6)

Below (in equation A1-7) we show how the t-statistic value is biased by the incorrect VIF, assuming we know the reliabilities for X1 and X2.
The first two lines are equation A1-6, assuming there is no correction to the VIF for random measurement error. The third line adds back the
amount that was incorrectly subtracted out to correct for the X1 and X2 correlation (incorrect because not taking into account random
measurement error attenuation), and then the fourth line subtracts the proper amount to correct for X1 and X2 correlation (taking into account
random measurement error attenuation).

(t-statCorrected)2 = {[(β^2 – β2)2 ∗ (N – K) ∗ Σx2i2]/Σεi2}

− {[(β^2 – β2)2 ∗ (N – K) ∗Σx2i2 ∗ (ρ12-Overt2)]/Σεi2}
+ {[(β^2 – β2)2 ∗ (N – K) ∗Σx2i2 ∗ (ρ12-Overt2)]/Σεi2}
− {[(β^2 – β2)2 ∗(N – K) ∗ Σx2i2 ∗ (ρ12Underlying2)]/Σεi2} (A1-7)

The amount by which regression is overestimating the “t-statistic squared” due to the incorrect VIF is the following. This is the last two terms
of equation A1-7 with the signs (and order) reversed:

(t-stat)2 overestimation = (t-statOvert)2 – (t-statCorrected)2 =

+ {[(β^2 – β2)2 ∗(N – K) ∗ Σx2i2 ∗ (ρ12Underlying2)]/Σεi2}
− {[(β^2 – β2)2 ∗ (N – K) ∗Σx2i2 ∗ (ρ12-Overt2)]/Σεi2} (A1-8)

By inserting the square of equation A1-5 for the ρ12Underlying2 term in A1-8, we get the following:

(t-stat)2 overestimation = [(β^2 – β2) 2 (N – K) Σx2i2(ρ12Overt2) / (α1∗ α2)] / Σεi2

- [(β^2 – β2) 2 (N – K) Σx2i2 (ρ12Overt2)] / Σεi2 (A1-9)

Gathering common terms, the amount the t-statistic squared in regression is overestimated due to the VIF bias when there are both correlated
predictors and random measurement error is1

(t-stat)2 overestimation = [ (1/(α1∗ α2)) – 1] ∗ [(β^2 – β2) 2 (N – K) Σx2i2 (ρ12-0vert2) ] / Σε2i (A1-10)

Equation A1-10 is helpful in seeing the impact of various factors on the t-statistic overestimation due to bias in the VIF. It was one of the
insights that led to our hypothesis generation. Of course it does not tell the full story, which would also require taking into account the path
bias embodied in equations 6 and 7 from the paper, and determining the corrected standard deviation for the corrected estimated underlying
path. We explain the logic of that below.

Correcting the Path Estimates for the M+ME Bias

Johnston’s (1972) equations. Consider the following regression equation:

Yi = β0 + β1 X1i + β2 X2i + . . . . + βk Xki + εi (A1-11)

Though the essential results extend to the full equation above, for simplicity we will ignore all the βk Xki terms above when k is greater than
2, giving

1
We note that it would be incorrect to suggest that we can take the square root of both sides of equation (A1-10) and therefore determine that

t-stat overestimation = {[(1/α1∗ α2) – 1] ∗ [(β2^ – β2)2 (N – K) Σx2i (ρ12-Overt2) ] / Σε2i }.5 (not correct)

The (t-stat)2 overestimation term is not the same as [t-stat overestimation]2.

MIS Quarterly Vol. 41 No. 3—Appendices/September 2017 A3

Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Yi = β0 + β1 X1i + β2 X2i + εi (A1-12)

Following Johnston (1972, pp. 160-162), if we first look at the impact of multicollinearity on bias in the regression path estimates of β1 and
β2 (that is, β^1 and β^2), we see the following:

β^1 – β1 = Σux1 – (ρ12 ∗ Σuv)/(1 – ρ122) (A1-13)

and

β^2 – β2 = (Σuv)/(1 – ρ122) (A1-14)

where u is the vector of error terms in Y; v is the vector of error terms in the equation for X2 as a function of X1; and ρ12 is the underlying
correlation between X1 and X2.

It is not necessary for the reader to understand the two equations in depth, but only to understand that mathematically, when X1 and X2 are
correlated and the error terms u and v are non-zero,2 regression estimates of β^1 and β^2 will be biased away from the underlying value (the true
value absent any systematic error). We note that as ρ12 increases, the size of the negative bias for β^1 increases. Likewise, as ρ12 increases, the
size of the positive bias for β^2 increases. Johnston goes on to say that a large correlation between independent variables “is thus likely to
produce large and opposite errors in β^1 and β^2; if β^1 underestimates β1, then β^2 is likely to overestimate β2 and vice versa. It is thus very
important that the standard errors should alert one to the presence of multicollinearity” (p. 162).

Green and Kiernans’s (1989, pp. 359-363) equations. Green and Keirnan carried the analysis of the impact of multicollinearity and random
measurement error on regression path biases further than Johnston. In particular, Green and Kiernan make a clear distinction between what
we are calling the “overt” (ρ12overt) and the “underlying” (ρ12underlying) correlation—the overt correlation is the correlation based on the “signal-
plus-noise” or the correlation between the “error containing” measures of the two constructs, while the “underlying” correlation is the
correlation based only on the “signal,” without the noise (Green and Kiernan 1989, p. 360.)

Under fairly general conditions (equal reliability {α1 = α2} for β1 and β2; β1 > 0), Green and Keirnan gave equations for what they call
“proportional inconsistencies” or (PIi) defined as (βi – plim β^i ) / βi. If PIi is positive, then plim β^i is less than $i (i.e., an underestimation);
If PIi is negative, then plim β^i is greater than βi; (i.e., an overestimation). We will reproduce two of their equations, folding in several other
additional reasonable assumption that are appropriate to our analysis here.3 With this assumption Green and Kiernan’s equations for the
proportional inconsistency show us more about the impact of M+ME on regression estimates than Johnston’s treatment.

PIβ1 = (β1 – plim β^1) / β1 = [(1 – α1)/(1 – ρ122)] ∗ [1 – (ρ12∗(β2/β1))] (Equation 6 repeated) (A1-15)

PIβ2 = (β2 – plim β^2) / β2 = [(1 – α1)/(1 – ρ122)] ∗ [1 – (ρ12 ∗ (β1/β2))] (Equation 7 repeated) (A1-16)

Here ρ12 is the “overt” correlation between X1 and X2, α1 is the reliability of X1 and of X2 (α1 is assumed be equal to α2).4 PIi indicates the
proportional amount the plim β î estimate has been biased away from the true (or underlying) value of βi. The “plim” indicates that plim β î
would be the estimate if there were an infinite number of data points. (As stated earlier in the paper, we found that with sample sizes of 100
to 200, the equations predicted the values of regression path estimates reasonably accurately, and for collections of 500 datasets at those sample
sizes, quite accurately in the aggregate).

These equations can give us a feel for how M+ME will affect regression path estimates. Assume for the moment that β1, β2, and ρ12 are all
positive and β1 is greater than β2 (a not uncommon situation). Under these conditions, given that α1, ρ12, and β2/β1 are all less than one, it can
be seen that PIβ1 will always be greater than zero. (Recall that: PIi > 0  β^i is underestimated.) Therefore given our assumptions, when there
is multicollinearity and random measurement error, the dominant β1 will always be underestimated.

2
The u and v error components of Johnston’s equations might contain more than only random measurement error, but both would certainly increase as random
measurement error in the X and Y values increased. Green and Kiernan’s equations focus more precisely on measurement error and its implications.

3
See footnote 15 in the paper proper.

4
Note that when α1 is not equal to α2, we use the approximation of αcombination = (α1 ∗ α2)1/2 so that we can continue to use Green and Kiernan’s equations.

A4 MIS Quarterly Vol. 41 No. 3–Appendices/September 2017

 Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Depending on the values of ρ12 and β1/β2, the non-dominant β2 could be under- or over-estimated. In Appendix C we present the logic that
shows that except when β1 and β2 have close to the same value, M+ME will tend to push the β^2 estimate higher than β2 if β2 is positive.

Equations for Correcting the Path Estimate Bias. Although the algebra is tedious, equations A1-15 and A1-16 can be turned around to allow
us to calculate estimates of the underlying β values, given the β^1 and β^2 estimates, as follows:

Let C = (1-α1) / (1-ρ122) (Recall that when α1 is not equal to α2 we use

α1 = αcombination = (α1 ∗ α2)1/2. See footnote #16.)

(β1 – β^1)/β1= C ∗ [1 – ρ12 ∗ β2/ β1] (Green and Kiernan’s Equation for PIβ1)
(β1 – β^1) = β1{C ∗ [1 – ρ12 ∗ β2/ β1] }
β1 = β^1 + β1C ∗ [1 – ρ12∗ β2/ β1]
β1 = β^1 + β1C – C ∗ ρ12 ∗ β2]
β1 – β1C = β^1 – C ∗ ρ12 ∗ β2]
β1 = β^1/(1-C) – [C ∗ ρ12 ∗ β2] /(1-C)

Similarly

β2 = β^2/(1-C) – [C ∗ ρ12 ∗ {β1}] /(1-C)

Substituting in (β1) and collecting terms,

β2 = β^2/(1-C) – [C ∗ ρ12 ∗ {β^1/(1-C) – [Χ ∗ ρ12 ∗ β2] /(1-C)}] /(1-C)

β2 = β^2/(1-C) – C ∗ ρ12 ∗ β^1/(1-C)2 + [(C ∗ ρ12) 2 ∗ β2] /(1-C) 2
β2 – (C ∗ ρ12) 2 ∗ β2 /(1-C) 2 = β^2/(1-C) – C ∗ ρ12 ∗ β^1/(1-C)2
β2 [ 1– (C ∗ ρ12) 2 /(1-C) 2 ] = β^2/(1-C) – C ∗ ρ12 ∗ β^1/(1-C)2
β2 = [β^2/(1-C) – C ∗ ρ12 ∗ β^`/(1-C)2 ] / [ 1 – (C ∗ ρ12) 2 /(1-C) 2 ]
β2 = β^2/(1-C) – C ∗ ρ12 ∗ β^1/(1-C)2 / [(1-C) 2 – (C ∗ ρ12) 2] / (1-C) 2

β 2^ ∗ (1 − C) − C∗ ρ12 ∗ β1^
β2 = (A1-17)
(1 − C)2  − (C∗ ρ )2
  12

Similarly

β1^ ∗ (1 − C) − C∗ ρ12 ∗ β ^2
β1 = (A1-18)
(1 − C)2  − (C∗ ρ )2
  12

Correcting the Standard Deviation of the Corrected Path Estimate. One final insight is needed in correcting for the M+ME path bias.
To determine the proper estimate of the path standard deviation, we have to recognize that the path bias created by the M+ME comes with a
change in the standard deviation. To calculate the corrected standard deviation for the true path we need to, in a sense, undo that change. Once
the variances of the β^1 and β^2 paths have been corrected for the VIF bias (Equation A1-2), the variance of the estimates for the above underlying
β1 and β2 paths can be calculated using a standard result from statistics:

if β2 = a ∗ β^2 + b ∗ β^1
then Var(β2) = a2 ∗ Var(β^2) + b2 ∗ Var(^1) (A1-19)

where in our case “a” is (1-C) /{ (1-C)2 – (C ∗ ρ12)2}, and “b” is C ∗ ρ12 / { (1-C)2 – (C ∗ ρ12)2} from equation A1-17.

MIS Quarterly Vol. 41 No. 3—Appendices/September 2017 A5

Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Appendix B
Will PLS with Bootstrapping Correct for M+ME?

For PLS with bootstrapping to correct for the M+ME blindspot seen in regression, it would need to overcome the deficiency noted in our
equation 3 versus our equation 4. That is, it would need to somehow incorporate random measurement error into its estimate for the standard
deviations of the X1 and X2 paths leading to Y1. We see two possible ways that PLS with bootstrapping could do this. First, bootstrapping
could determine the reliability of the two constructs (in our case the X1 and X2 constructs). It could then adjust its (bootstrapping determined)
standard deviation of the path using those reliabilities, similarly to our equation 4. However, we see no point in the PLS bootstrapping process
where the reliability of the two construct measures is taken into account. For each bootstrapping resample, after the indicator weights are
determined and the proxy construct scores calculated, all information about the indicator values is discarded. What occurs is that OLS
regression is used with the proxy construct scores to determine another set of path values. Nowhere in the process is the information (for
explicitly determining the measurement reliability of the constructs) used by PLS or its bootstrapping process.

A second possibility could be that bootstrapping automatically takes M+ME into account. The central assumption of bootstrapping in general
(Mooney and Duval 1993) is that the variation contained in a given sample is representative of the variation existing in the larger population.
If this were true for the M+ME blind spot, then bootstrapping could correctly incorporate the extra variation due to M+ME and suggest
appropriately larger standard deviations.

If in some bootstrapping resamples M+ME led to additional overestimations of the path values, and in others M+ME led to additional
underestimations, then the total distribution of the bootstrapping resample path values would be appropriately wider, and estimations for the
path standard deviations would have increased accordingly. However, recall that each PLS bootstrapping resample is drawn from the original
sample and therefore contains roughly the same underlying β1, β2, ρ12, and other characteristics as the original sample. We showed in Appendix
A that Green and Kiernan’s equations (A1-15 and A1-16) clearly indicate that when there is M+ME and both path estimates are positive and
not too close together, regression will tend to systematically underestimate β^1 and systematically overestimate β^2 based on the reliability and
the values of β1, β2, and ρ12.5 The bias seen in Green and Kiernan’s equations (6 and 7) for regression will then be apparent in each of the
bootstrapping regression results.

Therefore, it is reasonable to assume that when PLS uses OLS to estimate paths for each of its bootstrapping samples, it will tend to
systematically underestimate each of those β^1 path estimates by roughly the same amount, and to systematically overestimate each of those
β^2 estimates by roughly the same amount. If that is true, instead of having the collection of bootstrapping β^2 path estimates more widely
dispersed, they will be biased but about as closely spaced as if there were no M+ME bias.

PLS’s bootstrapping distributions should therefore incorporate the same variance inflation factor as seen in equation 3, mirroring the results
seen in regression. We see no argument for a way in which the PLS bootstrapping resamples will incorporate the correction shown in our
equation 4.

Appendix C
Further Exploration of Green and Kiernan’s Equations:
Impact of Negative Paths or Negative Correlations

Mela and Kopalle (2002) have argued that positive versus negative correlations would have very different (asymmetric) impacts on path biases
and path estimate variances. For us (with multicollinearity and random measurement error) that question is answered by looking again at Green
and Kiernan’s (1989, p. 360) equations for the bias (or proportional inconsistency) in the estimates for β^1 and β^2:

PIβ1 = (β1 – plim β^1) / β1 = [(1 – α1)/(1-ρ122)] ∗ [1-(ρ12 ∗ (β2/β1))] (Eq. 6 repeated)
PIβ2 = (β2 – plim β^2) / β2 = [(1 – α1)/(1-ρ122)] ∗ [1-(ρ12 ∗ (β1/β2))] (Eq. 7 repeated)

5
See Appendix C for the impact of relaxing these assumptions about the signs of β1^, β2^, and ρ12.

A6 MIS Quarterly Vol. 41 No. 3–Appendices/September 2017

 Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Equations 6 and 7, repeated above, can tell us quite a bit about the behavior of β^1 and β^2. First, remember that if PIβi is > 0, then |β^1| is an
underestimation of |βi|. If PIβi is < 0, then |β^i| is an overestimation of| βi|. If |β^i| is an underestimation of |βi|, this means that |β^i| is closer to
zero than |βi|. Whether β1 or β2 are over- or under-estimated depends upon the sign of PIβ1 or PIβ2. Note that in both equations, the [(1 -
α1)/(1-ρ122) ] term before the asterisk is always positive. Therefore whether β1 or β2 are over or under-estimated depends upon the sign of [1-(ρ12
∗ (β2/β1))] or [1-(ρ12 ∗ (β1/β2))]. Table C1 shows the impact of different combinations of negative and positive signs on [1-(ρ12 ∗ (β2/β1))] or
[1-(ρ12 ∗ (β1/β2))], and therefore on the value and sign of the proportional inconsistencies.

One interesting outcome is apparent in the column labeled “Decision Condition” of Table C1. For β1 there is no decision: if | β^1 | > | β^2 |, we
would say that β^1 is dominant. In this case | β^1 | will always be an underestimation of | β1 |. More interesting is the case for | β^2 |. If | β^2| is
relatively large (i.e. , greater than | ρ12 ∗ $^1 |), then | β^2| will also be an underestimate of | β2 |. Otherwise, | β^2| will be an overestimate of | β2|.
This last can lead to false positives.

Relative to Mela and Kopalle’s arguments, in fact it can be seen that having zero or two negative signs for ρ12, β1,and β2 creates a quite different
configuration of the results than having one or three negatives. Having exactly two of the signs negative keeps the same configuration of
results, though it does cause one or more path estimate biases to switch (symmetrically) from positive to negative. This all suggests a slightly
different reading of the Mela and Kopalle paper. Although they focused on the impact of omitted but correlated variables, behavior similar
to their findings can be created without omitted variables, by adding measurement error and collinearity. The impacts Mela and Kopalle seek
to show have the same relationship as those pointed out above from Green and Kiernan’s equations.

Table C1. Behavior of Proportional Inconsistencies (Green and Kiernans’s PI) (Assuming |β1| > |β2|)
# Minus What What
Focus on Decision Relationship of Signs among Happens to 1 Happens to
PIβ1 or PIβ2 Condition |ρ12 ∗ β2/β1| to “1” ρ12, β2, β1 − (ρ12 ∗ β2/β1) PIβ1 or PIβ2
PIβ1= none |ρ12 ∗ β2/β1| always 0 or 2 Minus (ρ12∗β2/β1) PIβ1 is always |β^1| is an under-
(1−ρ1)/(1−ρ122) ∗ <1 signs subtracts from positive estimate of |β1|
[1−(ρ12 ∗ β2/β1)] one
1 or 3 Minus (ρ12∗β2/β1) PIβ1 is more |β^1| is a bigger
signs adds to one positive underestimate of
|β1|
PIβ2 =
(1−ρ1)/(1−ρ122) ∗
[1−(ρ12∗β1/β2)]
|ρ12∗β1| < |ρ12 ∗ β1/β2| always
|β2| <1

(|β2| is
relatively
large)
0 or 2 Minus (ρ12 ∗ β1/β2) PIβ2 is always |β^2| is an
signs subtracts from positive underestimate of
one |β2|
1 or 3 Minus (ρ12 ∗ β1/β2) PIβ2 is more |β^2| is a bigger
signs adds to one positive underestimate of
|β2|
|ρ12 ∗ β1| > |ρ12∗β1/β2| always
|β2| >1

(β2 is
relatively
small)
0 or 2 minus (ρ12 ∗ β1/β2) PIβ2 is always |β^2| is an
signs subtracts from negative overestimate of
one |β2|
1 or 3 Minus (ρ12 ∗ β1/β2) PIβ2 is |β^2| is an under-
signs adds to one positive estimate of |β2|

MIS Quarterly Vol. 41 No. 3—Appendices/September 2017 A7

Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Appendix D
Correcting for M+ME Biases — Step by Step

When might an analysis be in the M+ME danger zone? For the sake of illustration, consider the model and correlations shown in Figure
D1. Here we have nine constructs connected by hypothesized paths, showing selected (overt) path estimates and selected (overt) inter-construct
correlations. How would a researcher recognize that any particular pair of these constructs is near enough to the M+ME danger zone to possibly
require the correction? Note that Figures 3 through 7 and Figure 9 in the paper show underlying correlations. If a researcher compares their
own results to Figures 3 through 7, they need to use equation 2 to convert overt correlations to underlying correlations. Note also that the
M+ME Correction Application requires “overt” correlations, not underlying correlations.

A-to-G Path = .257, t = 3.50

B-to-G Path = .170, t = 2.05

A
.40 G
small B G-to-I Path = .462, t = 2.89

small
C
.40 I

D
.380 H G-to-I Path = .240, t = 2.9
.304 E
.320
F D-to-H Path = .232, t = 3.25
E-to-H Path = .138, t = 2.13
F-to-H Path = .381, t = 6.41

Figure D1. Overt Correlations in Hypothetical Model with N = 200, Reliability = .80

Step One: Identify Correlations of Interest. First, recognize that in Figure D1, only the seven overt correlations or paths with a value (or
the word “small”) added to the link are of concern to us. Note that the path between G and H is of concern even though it is modeled as a path
rather than a correlation, because G and H are correlated, and both participate in the regression equation predicting construct I. Note also that
even if the overt correlation between construct C and construct D were very high, say.72, that would not be of concern, because constructs C
and D do not participate in the same regression equation.

Categorize all high correlation situations into two groups. Notice that the upper part of the figure shows two of what we will call an “isolated”
high correlation—only two highly correlated constructs participating in the same regression. The bottom part of Figure D1 shows a different
situation—three highly correlated constructs all participating in the same regression, specifically D, E, and F are correlated and all predict H.
This latter situation we will call a “combination” of high correlations. These “combination” situations present more of a challenge than the
isolated high correlation and will be dealt with in Appendix E.

Step Two: Assess and perhaps correct the “isolated” correlations of interest. There are two isolated correlations in Figure D1 that should
be examined: A and B predicting G, and G and H predicting I (both having an overt correlation of .40).

To rule out obviously non-problematic correlated pairs, one can do a quick (very approximate) back of the envelope calculation to determine
if there is even any cause for concern, as follows. For example, if the reliability of constructs A and B in Figure D1 is .80, using equation 2,
we conclude that the underlying correlations are about .50 for A with B (and for G with H). To get a general feeling for whether A and B

A8 MIS Quarterly Vol. 41 No. 3–Appendices/September 2017

 Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

predicting G might involve a substantive M+ME bias, we can refer to Figure 7 (rather than Figures 4A or 4B, because the N is 200). If the
sample size were closer to n = 100, then Figures 4A or 4B might be appropriate.

In Figure 7 we see that an analysis with a β1 about .292, sample size of N = 200, and an underlying correlation of .50 gives us about a 7%
likelihood of excessive false positives, near the top of the 95% confidence interval around 5% false positives. This suggests that we should
use the M+ME Correction Application to check the possibility of an M+ME bias.

The M+ME correction application will correct for the bias to the VIF, correct the path biases, and adjust the standard deviations for the
corrected path values. This downloadable application is available on the MISQ web site (Online Supplements: M+ME Correction Application).
Figure 8 of the article proper displays the front end of the application, showing the input required (on the left side) and the results of the
correction calculations on the right side. First, be sure that you are using the standardized regression (or PLS) results. Though here we will
assume that β1 is the dominant path (the construct whose path has the highest absolute value), that is not necessary. With this understanding,
enter the regression results for the two (overt) path estimates from regression or PLS, the two overt t-statistics, the two reliabilities (for X1 and
X2) and the overt (or apparent) correlation between X1 and X2. The corrected path values, t-statistics and p values will be displayed by the
M+ME correction application. Because it is relatively easy to use, the M+ME correction application can be used without referring to the figures
in the paper as a “back-of-the-envelope” approximation.

Table D1 shows the results of applying the corrections. The first set of rows in Table D1 shows the A and B predicting G situation when it
is entered into the M+ME correction application. The input values are to the left, and the corrected path values and t-statistics are shown to
the right. In this case the path correction has increased the B  G path slightly6 (to .184) and the recalculated standard deviation has decreased
the t-statistic a good bit (to 1.561). The result is that the corrected B to G path is no longer statistically significant.

The second “isolated correlation” from Figure D1 is G and H predicting I, with G and H correlated at .40. This situation is depicted in the
second set of rows of Table D1. There the H to I path is shown to be statistically significant even with the M+ME correction, though note that
the t-statistic has dropped from 2.90 to 2.02. Finally, the third example in Table D1 has the same input as the second, but a correlation of .60
instead of .40. This increase in multicollinearity takes its toll, and under these circumstances the H to I path is no longer statistically significant
after we apply the M+ME correction.

Note that in all three examples, the uncorrected results show that the questionable path is statistically significant (i.e., different from zero with
95% confidence). In two of those, the correction shows that the questionable path is not actually significant, and should not be considered
different from zero.

We suggest that there is no place for optimists in questions of statistical significance. When M+ME for a particular path is not clearly ruled
out by displays such as those in Figures 4A, 4B, or 7, the correction should be applied by inputting the relevant data into the M+ME correction
application. Applying this correction to regression or PLS results when M+ME is not a problem will not create new problems. Instead, it will
give the researcher a more accurate value for the t-statistic.

6
As described in Appendix C and Table C1, this is an example where the true β2 path is close enough to the true β1 path that the M+ME bias will decrease both
values, rather than decreasing the β1 path and increasing the β2 path. In this case the correction will result in an increase from the regression value to the true value
for both paths. When the β2 path is much smaller than the β1 path, the correction will increase the β1 path and decrease the β2 path from what is seen in the
regression results.

MIS Quarterly Vol. 41 No. 3—Appendices/September 2017 A9

Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Table D1. Input and Output (Corrected Path Values and t-statistics) for the M+ME Correction
Application
Needed Input Output
Significance Significance
Relation- Without Corrected Corrected with path and
ship N β^1 β^2 β^1t β^2t α1 α2 ρ12 Path Corrections Path Est t-stat VIF correction
A, B ö G; 200 .257 .170 3.50 2.05 .80 .80 .40 BG yes 0.184 1.56 Not Sig!
ρa,b = .40
AG yes 0.314 3.00 yes

G,H ö I; 200 .462 .240 2.89 2.90 .80 .80 .40 HI yes 0.243 2.02 yes
ρg,h = .40 GI yes 0.576 2.55 yes
G,H ö I; 200 .462 .240 2.89 2.90 .80 .80 .60 HI yes 0.179 1.01 Not Sig!
ρg,h = .60
GI yes 0.623 2.03 yes

Appendix E
The Challenge of Combinations of High Correlations

In the lower part of Figure D1 we see a “combination” of three constructs (D, E, and F) connected with one high and one only moderately high
overt correlation, both of which affect the E to H path (that is .380 and .320 overt correlations, suggesting underlying correlations of .475 and
.400). The third overt correlation (between D and F) is .304, suggesting an underlying value of .380. We acknowledge at the outset that we
do not fully understand all the issues relating to multiple high correlations situations. The problem is that Green and Kiernan’s (1989) equations
do not extend to three intercorrelated constructs. In fact both pairs of correlated constructs have an impact on the estimated E to H path estimate
and their impact could be additive or in some cases more than additive. This means that except for turning to CB-SEM, we do not have an
effective way to correct for the path biases in regression or PLS, when faced with such a situation. This is an area where additional research
would be quite valuable.

A key insight in understanding combinations of high correlations is that there are three general archetypes of the underlying “causes” of these
three way configurations of correlations, as shown in Figure E1. Of course the cause may contain a mixture of several of these types, plus the
possibility of direct causal links between the focal constructs (in this case Constructs D, E or F.

On the top left of Figure E1 (Panel A, single underlying cause) we see that the correlations between all three constructs are due to relationships
with a single underlying construct. As it turns out, there is a reasonable correction approach when the combination of high correlations is
produced by a single underlying construct. In those situations the M+ME biases can be considered additive, and it is possible to correct for
the largest correlation as we have shown in the previous section for isolated high correlations. Once this is done, the researcher can determine
(very roughly) from the size of the remaining correlation (for example using Figures 3, 4, or 7), whether that remaining correlation by itself
would put the analysis into the M+ME danger zone. If not, then the single correlation correction method is sufficient. One definitely should
not apply the M+ME correction application a second time.

Unfortunately we know of no way to determine, from the data a researcher would have, whether a combination of correlations was caused by
two, or even three, background constructs. Thus we have no way to safely assume a single underlying cause. This poses a difficult problem.

In Figure E2 we show the uncorrected results from data generated by each of the archetypal possible causes, along with the results after the
highest correlation has been corrected. The figure does show that correcting for the dominant correlation always improves the situation. But
unfortunately, even with the correction, some of the lines are well above the 95% confidence interval around .05.

Since the researcher cannot know which situation they are working with, and since it is not appropriate to optimistically “assume the best
possible situation” in hypothesis testing, we cannot recommend using the M+ME correction with combinations of high correlations.

A10 MIS Quarterly Vol. 41 No. 3–Appendices/September 2017

 Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Panel A. Single Underlying Cause Panel B. Two Distinct Underlying Causes

Panel C. Three Distinct Underlying Causes

Figure E1. Three Different Archetypal Causes of Combinations of High Correlations

No Correction Versus Dominant Correlation Corrected, α = .80, N = 200, X1  Y1 = .420

Figure E2. Regression False Positives Results from Combinations of High Correlations

MIS Quarterly Vol. 41 No. 3—Appendices/September 2017 A11

Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

α = .80, N = 200, X1  Y1 = .420

Figure E3. CB-SEM False Positives Results From Combinations of High Correlations

One safe method to use in combinations of high correlation situations would be to convert the analysis over to CB-SEM. Figure E3 shows the
results from that method. We see that with CB-SEM, the large numbers of false positives never appear in the first place.7 Though it may
require extra work for those not well-versed in the use of CB-SEM, this clearly provides a solution to the combination of high correlation
M+ME situation. Alternatively (or in conjunction), it may be appropriate to change the underlying model (e.g., using higher-order constructs).

As stated earlier, we do not fully understand the combinations of high correlation situation. Because combinations of high correlations do
appear in IS research, additional research in this area could be helpful.

Appendix F
Testing Whether M+ME False Positives Could Be Due
to Discriminant Validity Problems

One alternative explanation for the results we obtained might be framed within the larger context of measurement model misspecification, and
specifically the presence of a lack of discriminant validity among constructs. To address this possibility, we conducted some ancillary analyses.
We found the following. First, using chi square difference tests of discriminant validity and one of our Monte Carlo simulation datasets (500
samples each of N = 100, reliability = .80, ρ12 = .80) we did find evidence of discriminant validity problems. Specifically, 56 (or about 10%)
of the samples had discriminant validity problems, as compared to 59 (also about 10%) samples with M+ME false positives. Although these
numbers are quite close, only 4 samples had both discriminant validity problems and excessive false positives.

We note further that although both false positives and discriminant validity problems are exacerbated by increasing correlations and decreasing
reliabilities, those two phenomena react quite differently to increasing sample size. As shown in Figure F1, larger sample sizes decrease
discriminant validity problems to virtually zero, while they increase M+ME false positives substantially. Thus it is clear that the two
phenomena share some causal factors but are actually quite distinct. Knowing that a dataset suffers from one of these phenomena does not
provide much knowledge about the likelihood that it also suffers from the other.

7
The reader may be concerned that in the .50/.50 combination correlation situation in Figure E3, the percent of false positives for CB-SEM (7.2%) is above the
6.9% limit for the confidence interval. We were concerned as well, but recognized that since we have used so many tests against that limit, it is highly likely that
some values will be a little higher than the stated limit for an individual test. To give us more confidence in that explanation, we generated an additional 100,000
data points (500 new datasets of 200 cases each), and reran the analysis. In that run we found that CB-SEM resulted in 5.4% false positives. With 1,000 datasets
(the combined total) we now have an average of 6.3% false positives, within the confidence interval of 3.65% to 6.35% for a sample size of 1,000.

A12 MIS Quarterly Vol. 41 No. 3–Appendices/September 2017

 Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

(β1 path = .292, N = 100, α = .80, ρ12 = .80)

Figure F1. Sample Size and Discriminant Validity Problems Versus M+ME False Positives

Appendix G
Pictorial Depiction: M+ME Impact on Regression and CB-SEM

Some readers may find it helpful to see a graphical representation of the impact of M+ME on regression and CB-SEM results for 500 samples
as in our Monte Carlo simulations. Throughout we will be looking at an underlying X1 to Y1 path of .600, an X2 to Y1 path of zero, a sample
size of N = 200, and reliability of .80 for both X1 and X2. For both regression and CB-SEM, the average path estimates across the 500 samples
is shown below in Figures G1, G2, and G3, for three different scenarios. Along with the average path estimates, also shown is a representation
of the distribution of those 500 estimates around that average estimate. The distribution curves shown comes from calculating the standard
deviation of the 500 path estimates, and displaying them as a curve anchored at the average path estimate plus 2 times the standard deviation,
and the average path estimate minus 2 times the standard deviation.

We first look at the results when the underlying reality is a zero correlation between X1 and X2. That is, when there is no M+ME. We will
then shift our focus to the situation where M+ME is extreme, with a correlation between X1 and X2 of .90.

Figure G1 shows the results for β2 (a zero path) when there is no correlation between X1 and X2. The average β2 path estimates for regression
and CB-SEM are both about zero, and the distribution curves for both span from about -1.3 to + 1.5. These results are what we might have
expected.

In Figure G2 we show the results when the correlation between X1 and X2 is .90, a very high correlation. Though M+ME will rarely or never
be this extreme in practice, this scenario allows us to see more clearly what the specific effects of M+ME are, and why this is a problem. Under
these conditions, CB-SEM returns a path estimate for β2 of near zero, and a standard deviation that is about 7 times as large as it was when ρ12
was .00 (.497 versus .068). The very large spread of the β2 path estimations suggests that the high correlation between X1 and X2 makes the
resulting path estimates very dependent upon random variance in the data. CB-SEM recognizes this and increases the standard deviation it
uses appropriately.

For regression under these conditions, the average path estimate is quite skewed (from near zero at ρ12 = 0 to a value of .175 at ρ12 = .90. This
is as predicted by the Green and Kiernan equations. The distribution around the .175 based on the standard deviation of the regression estimates
is about as wide as in Figure G1, but now shifted up, so that most of the estimates are now significantly different from zero. This is of course
misleading, since the true path is zero. This makes the M+ME bias very apparent.

In Figure G3, we have used the Green and Kiernan equations to correct for the M+ME path bias in regression, and at the same time corrected
the standard deviation values in regression, also based on the VIF and Green and Kiernan equations. These corrections have removed the path
bias, and now the bias and the dispersion of the path values are roughly equal for regression and for CB-SEM, and both reflect the uncertainly
created by the large correlation between X1 and X2.

MIS Quarterly Vol. 41 No. 3—Appendices/September 2017 A13

Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

Figure G1. Regression and CB-SEM When Correlation Between X1 and X2 is 0.00

Figure G2. Regression and CB-SEM When Correlation Between X1 and X2 is 0.90

Figure G3. Regression and CB-SEM When Correlation Between X1 and X2 is 0.90, After VIF and Green
and Kiernan Path Bias Corrections

A14 MIS Quarterly Vol. 41 No. 3–Appendices/September 2017

 Goodhue et al./Correcting for Excessive False Positives in Regression & PLS

References

Goodhue, D., Lewis, W., and Thompson, R. 2011. “A Dangerous Blind Spot in IS Research: False Positives Due to Multicollinearity
Combined with Measurement Error,” in Proceedings of the 17th Americas Conference on Information Systems, Detroit, MI, August 4-7.
Green, C. J., and Kiernan, E. 1989. “Multicollinearity and Measurement Error in Econometric Financial Modelling,” The Manchester School
(57:4), pp. 357-369.
Johnston, J. 1972. Econometric Methods (2nd ed.), New York: McGraw-Hill.
Mela, C., and Kopalle, P. 2002. “The Impact of Collinearity on Regression Analysis: the Asymmetric Effect of Negative and Positive
Correlations,” Applied Economics (34:6), pp. 667-677.
Mooney, C. Z., and Duval, R. D. 1993. Bootstrapping: A Nonparametric Approach to Statistical Inference, Beverly Hills, CA: Sage
Publications.

MIS Quarterly Vol. 41 No. 3—Appendices/September 2017 A15