Principle of marginality
The article's lead section may need to be rewritten. (December 2021) |
In statistics, the principle of marginality, sometimes called hierarchical principle, is the fact that the average (or main) effects of variables in an analysis are marginal to their interaction effect—that is, the main effect of one explanatory variable captures the effect of that variable averaged over all values of a second explanatory variable whose value influences the first variable's effect.[1] The principle of marginality implies that, in general, it is wrong to test, estimate, or interpret main effects of explanatory variables where the variables interact or, similarly, to model interaction effects but delete main effects that are marginal to them.[2] While such models are interpretable, they lack applicability, as they ignore the dependence of a variable's effect upon another variable's value.
Nelder[3] and Venables[4] have argued strongly for the importance of this principle in regression analysis.
Regression form
[edit]If two independent continuous variables, say x and z, both influence a dependent variable y, and if the extent of the effect of each independent variable depends on the level of the other independent variable then the regression equation can be written as:
where i indexes observations, a is the intercept term, b, c, and d are effect size parameters to be estimated, and e is the error term.
If this is the correct model, then the omission of any of the right-side terms would be incorrect, resulting in misleading interpretation of the regression results.
With this model, the effect of x upon y is given by the partial derivative of y with respect to x; this is , which depends on the specific value at which the partial derivative is being evaluated. Hence, the main effect of x – the effect averaged over all values of z – is meaningless as it depends on the design of the experiment (specifically on the relative frequencies of the various values of z) and not just on the underlying relationships. Hence:
- In the case of interaction, it is wrong to try to test, estimate, or interpret a "main effect" coefficient b or c, omitting the interaction term.[5]
In addition:
- In the case of interaction, it is wrong to not include b or c, because this will give incorrect estimates of the interaction.[6][7]
See also
[edit]References
[edit]- ^ James, Gareth; Witten, Daniela; Hastie, Trevor; Tibshirani, Robert (2021). An introduction to statistical learning: with applications in R (Second ed.). New York, NY: Springer. p. 103. ISBN 978-1-0716-1418-1. Retrieved 29 October 2024.
- ^ Fox, J. Regression Notes.
- ^ Nelder, J. A. (1977). "A Reformulation of Linear Models". Journal of the Royal Statistical Society. 140 (1): 48–77. doi:10.2307/2344517. JSTOR 2344517. (Section 2.1: The Neglect of Marginality)
- ^ Venables, W.N. (1998). "Exegeses on Linear Models". Paper presented to the S-PLUS User's Conference Washington, DC, 8–9 October 1998.
- ^ See Venables, p.13: "... testing main effects in the presence of an interaction is a violation of the marginality principle".
- ^ See Venables, p.14/15, about the S-Plus command
drop1, which does not drop the main effect terms from a model with interaction: "To my delight I see that marginality constraints between factor terms are by default honoured". In R, the marginality requirement of thedroptermfunction (in package MASS) is stated in the Reference Manual. - ^ The above regression model, with two independent continuous variables, is presented with a numerical example, in Stata, as Case 3 in What happens if you omit the main effect in a regression model with an interaction?.