View source for Analysis of variance

{{short description|Collection of statistical models}}
{{redirect|ANOVA|other uses|Anova (disambiguation)}}
{{Use dmy dates|date=March 2020}}

'''Analysis of variance''' ('''ANOVA''') is a family of [[statistical methods]] used to compare the [[Mean|means]] of two or more groups by analyzing variance. Specifically, ANOVA compares the amount of variation ''between'' the group means to the amount of variation ''within'' each group. If the between-group variation is substantially larger than the within-group variation, it suggests that the group means are likely different. This comparison is done using an [[F-test]]. The underlying principle of ANOVA is based on the [[law of total variance]], which states that the total variance in a dataset can be broken down into components attributable to different sources. In the case of ANOVA, these sources are the variation between groups and the variation within groups. 

ANOVA was developed by the [[statistician]] [[Ronald Fisher]]. In its simplest form, it provides a [[statistical test]] of whether two or more population [[mean]]s are equal, and therefore generalizes the [[Student's t-test#Independent two-sample t-test|''t''-test]] beyond two means.
{{TOC limit}}

==History==
While the analysis of variance reached fruition in the 20th century, antecedents extend centuries into the past according to [[Stephen Stigler|Stigler]].<ref>Stigler (1986)</ref> These include hypothesis testing, the partitioning of sums of squares, experimental techniques and the additive model. [[Pierre-Simon Laplace|Laplace]] was performing hypothesis testing in the 1770s.<ref>Stigler (1986, p 134)</ref> Around 1800, Laplace and [[Carl Friedrich Gauss|Gauss]] developed the least-squares method for combining observations, which improved upon methods then used in astronomy and [[geodesy]]. It also initiated much study of the contributions to sums of squares. Laplace knew how to estimate a variance from a residual (rather than a total) sum of squares.<ref>Stigler (1986, p 153)</ref> By 1827, Laplace was using [[least squares]] methods to address ANOVA problems regarding measurements of atmospheric tides.<ref>Stigler (1986, pp&nbsp;154–155)</ref> Before 1800, astronomers had isolated observational errors resulting 
from reaction times (the "[[personal equation]]") and had developed methods of reducing the errors.<ref>Stigler (1986, pp&nbsp;240–242)</ref> The experimental methods used in the study of the personal equation were later accepted by the emerging field of psychology <ref>Stigler (1986, Chapter 7 – Psychophysics as a Counterpoint)</ref> which developed strong (full factorial) experimental methods to which randomization and blinding were soon added.<ref>Stigler (1986, p 253)</ref> An eloquent non-mathematical explanation of the additive effects model was available in 1885.<ref>Stigler (1986, pp&nbsp;314–315)</ref>

[[Ronald Fisher]] introduced the term [[variance]] and proposed its formal analysis in a 1918 article on theoretical population genetics, ''[[The Correlation between Relatives on the Supposition of Mendelian Inheritance|The Correlation Between Relatives on the Supposition of Mendelian Inheritance]]''.<ref>''The Correlation Between Relatives on the Supposition of Mendelian Inheritance''. Ronald A. Fisher. ''Philosophical Transactions of the Royal Society of Edinburgh''. 1918. (volume 52, pages 399–433)</ref> His first application of the analysis of variance to data analysis was published in 1921, ''Studies in Crop Variation I''.<ref>{{cite journal | title=) Studies in Crop Variation. I. An Examination of the Yield of Dressed Grain from Broadbalk | first1=Ronald A. | last1=Fisher | journal=Journal of Agricultural Science | volume=11  | pages=107–135| year=1921 | issue=2 | doi=10.1017/S0021859600003750 | hdl=2440/15170 | s2cid=86029217 | hdl-access=free }}</ref> This divided the variation of a time series into components representing annual causes and slow deterioration. Fisher's next piece, ''Studies in Crop Variation II'', written with [[Winifred Mackenzie]] and published in 1923, studied the variation in yield across plots sown with different varieties and subjected to different fertiliser treatments.<ref>{{cite journal | title=) Studies in Crop Variation. II. The Manurial Response of Different Potato Varieties | first1=Ronald A. | last1=Fisher | journal=Journal of Agricultural Science | volume=13  | pages=311–320| year=1923 | issue=3 | doi=10.1017/S0021859600003592 | hdl=2440/15179 | s2cid=85985907 | hdl-access=free }}</ref>  Analysis of variance became widely known after being included in Fisher's 1925 book ''[[Statistical Methods for Research Workers]]''.

Randomization models were developed by several researchers. The first was published in Polish by [[Jerzy Neyman]] in 1923.<ref>Scheffé (1959, p 291, "Randomization models were first formulated by Neyman (1923) for the completely randomized design, by Neyman (1935) for randomized blocks, by Welch (1937) and Pitman (1937) for the Latin square under a certain null hypothesis, and by Kempthorne (1952, 1955) and Wilk (1955) for many other designs.")</ref>

==Example==
[[File:Anova, no fit..png|thumb|No fit: Young vs old, and short-haired vs long-haired]][[File:ANOVA fair fit.svg|thumb|Fair fit: Pet vs Working breed and less athletic vs more athletic]][[File:ANOVA very good fit.jpg|thumb|Very good fit: Weight by breed]]The analysis of variance can be used to describe otherwise complex relations among variables. A dog show provides an example. A dog show is not a random sampling of the breed: it is typically limited to dogs that are adult, pure-bred, and exemplary. A histogram of dog weights from a show is likely to be rather complicated, like the yellow-orange distribution shown in the illustrations. Suppose we wanted to predict the weight of a dog based on a certain set of characteristics of each dog. One way to do that is to ''explain'' the distribution of weights by dividing the dog population into groups based on those characteristics. A successful grouping will split dogs such that (a) each group has a low variance of dog weights (meaning the group is relatively homogeneous) and (b) the mean of each group is distinct (if two groups have the same mean, then it isn't reasonable to conclude that the groups are, in fact, separate in any meaningful way).
In the illustrations to the right, groups are identified as ''X''<sub>1</sub>, ''X''<sub>2</sub>, etc. In the first illustration, the dogs are divided according to the product (interaction) of two binary groupings: young vs old, and short-haired vs long-haired (e.g., group 1 is young, short-haired dogs, group 2 is young, long-haired dogs, etc.). Since the distributions of dog weight within each of the groups (shown in blue) has a relatively large variance, and since the means are very similar across groups, grouping dogs by these characteristics does not produce an effective way to explain the variation in dog weights: knowing which group a dog is in doesn't allow us to predict its weight much better than simply knowing the dog is in a dog show. Thus, this grouping fails to explain the variation in the overall distribution (yellow-orange).

An attempt to explain the weight distribution by grouping dogs as ''pet vs working breed'' and ''less athletic vs more athletic'' would probably be somewhat more successful (fair fit). The heaviest show dogs are likely to be big, strong, working breeds, while breeds kept as pets tend to be smaller and thus lighter. As shown by the second illustration, the distributions have variances that are considerably smaller than in the first case, and the means are more distinguishable. However, the significant overlap of distributions, for example, means that we cannot distinguish ''X''<sub>1</sub> and ''X''<sub>2</sub> reliably. Grouping dogs according to a coin flip might produce distributions that look similar.

An attempt to explain weight by breed is likely to produce a very good fit. All Chihuahuas are light and all St Bernards are heavy. The difference in weights between Setters and Pointers does not justify separate breeds. The analysis of variance provides the formal tools to justify these intuitive judgments. A common use of the method is the analysis of experimental data or the development of models. The method has some advantages over correlation: not all of the data must be numeric and one result of the method is a judgment in the confidence in an explanatory relationship.

== Classes of models ==
There are three classes of models used in the analysis of variance, and these are outlined here.

===Fixed-effects models===
{{Main|Fixed effects model}}
The fixed-effects model (class I) of analysis of variance applies to situations in which the experimenter applies one or more treatments to the subjects of the experiment to see whether the [[response variable]] values change. This allows the experimenter to estimate the ranges of response variable values that the treatment would generate in the population as a whole.
[[File:Fixed_effects_vs_Random_effects.pdf|thumb|291x291px|Fixed effects vs Random effects]]

===Random-effects models===
{{Main|Random effects model}}
Random-effects model (class II) is used when the treatments are not fixed. This occurs when the various factor levels are sampled from a larger population. Because the levels themselves are [[random variable]]s, some assumptions and the method of contrasting the treatments (a multi-variable generalization of simple differences) differ from the fixed-effects model.<ref>Montgomery (2001, Chapter 12: Experiments with random factors)</ref>

===Mixed-effects models===
{{Main|Mixed model}}
A mixed-effects model (class III) contains experimental factors of both fixed and random-effects types, with appropriately different interpretations and analysis for the two types.

===Example===
Teaching experiments could be performed by a college or university department to find a good introductory textbook, with each text considered a treatment. The fixed-effects model would compare a list of candidate texts. The random-effects model would determine whether important differences exist among a list of randomly selected texts. The mixed-effects model would compare the (fixed) incumbent texts to randomly selected alternatives.

Defining fixed and random effects has proven elusive, with multiple competing definitions.<ref>Gelman (2005, pp. 20–21)</ref>

==Assumptions==
The analysis of variance has been studied from several approaches, the most common of which uses a [[linear model]] that relates the response to the treatments and blocks. Note that the model is linear in parameters but may be nonlinear across factor levels. Interpretation is easy when data is balanced across factors but much deeper understanding is needed for unbalanced data.

===Textbook analysis using a normal distribution===
The analysis of variance can be presented in terms of a [[linear model]], which makes the following assumptions about the [[probability distribution]] of the responses:<ref>{{cite book |title = Statistical Methods
| last1 = Snedecor | first1 = George W. 
| last2 = Cochran | first2 = William G.
| year = 1967 | edition = 6th | page = 321
}}</ref><ref>Cochran & Cox (1992, p 48)</ref><ref>Howell (2002, p 323)</ref><ref>
{{cite book | last1 = Anderson | first1 = David R.
| last2 = Sweeney | first2 = Dennis J.
| last3 = Williams | first3 = Thomas A.
| title = Statistics for business and economics 
| publisher = West Pub. Co | location = Minneapolis/St. Paul 
| year = 1996 | edition = 6th| isbn = 978-0-314-06378-6 | pages = 452–453}}
</ref>
* [[Statistical independence|Independence]] of observations – this is an assumption of the model that simplifies the statistical analysis.
* [[normal distribution|Normality]] – the distributions of the [[Residual (statistics)|residuals]] are [[Normal distribution|normal]].
* Equality (or "homogeneity") of variances, called [[homoscedasticity]]—the variance of data in groups should be the same.

The separate assumptions of the textbook model imply that the [[errors and residuals in statistics|errors]] are independently, identically, and normally distributed for fixed effects models, that is, that the errors (<math>\varepsilon</math>) are independent and
<math display="block">\varepsilon \thicksim N(0, \sigma^2).</math>

===Randomization-based analysis===
{{See also|Random assignment|Randomization test}}
In a [[Randomized controlled trial|randomized controlled experiment]], the treatments are randomly assigned to experimental units, following the experimental protocol. This randomization is objective and declared before the experiment is carried out. The objective random-assignment is used to test the significance of the [[null hypothesis]], following the ideas of [[Charles Sanders Peirce|C. S. Peirce]] and [[Ronald Fisher]]. This design-based analysis was discussed and developed by [[Francis J. Anscombe]] at [[Rothamsted Experimental Station]] and by [[Oscar Kempthorne]] at [[Iowa State University]].<ref>Anscombe (1948)</ref> Kempthorne and his students make an assumption of ''unit treatment additivity'', which is discussed in the books of Kempthorne and [[David R. Cox]].<ref>{{cite book |last1=Hinkelmann |first1=Klaus |last2=Kempthorne |first2=Oscar |title=Design and Analysis of Experiments, Volume 2: Advanced Experimental Design |date=2005 |publisher=John Wiley |page=213 |isbn=978-0-471-70993-0 |url=https://books.google.com/books?id=GiYc5nRVKf8C&pg=PA213 |language=en}}</ref><ref>{{cite book |last1=Cox |first1=D. R. |title=Planning of Experiments |date=1992 |publisher=Wiley |isbn=978-0-471-57429-3 |language=en}}</ref>

====Unit-treatment additivity====
In its simplest form, the assumption of unit-treatment additivity<ref group="nb">Unit-treatment additivity is simply termed additivity in most texts. Hinkelmann and Kempthorne add adjectives and distinguish between additivity in the strict and broad senses. This allows a detailed consideration of multiple error sources (treatment, state, selection, measurement and sampling) on page 161.</ref> states that the observed response <math>y_{i,j}</math> from experimental unit <math>i</math> when receiving treatment <math>j</math> can be written as the sum of the unit's response <math>y_i</math> and the treatment-effect <math> t_j</math>, that is <ref>Kempthorne (1979, p 30)</ref><ref name="Cox">Cox (1958, Chapter 2: Some Key Assumptions)</ref><ref>Hinkelmann and Kempthorne (2008, Volume 1, Throughout. Introduced in Section 2.3.3: Principles of experimental design; The linear model; Outline of a model)</ref>
<math display="block">y_{i,j}=y_i+t_j.</math>
The assumption of unit-treatment additivity implies that, for every treatment <math>j</math>, the <math>j</math>th treatment has exactly the same effect <math>t_j</math> on every experiment unit.

The assumption of unit treatment additivity usually cannot be directly [[Falsifiability|falsified]], according to Cox and Kempthorne. However, many ''consequences'' of treatment-unit additivity can be falsified. For a randomized experiment, the assumption of unit-treatment additivity ''implies'' that the variance is constant for all treatments. Therefore, by [[contraposition]], a necessary condition for unit-treatment additivity is that the variance is constant.

The use of unit treatment additivity and randomization is similar to the design-based inference that is standard in finite-population [[survey sampling]].

====Derived linear model====
Kempthorne uses the randomization-distribution and the assumption of ''unit treatment additivity'' to produce a ''derived linear model'', very similar to the textbook model discussed previously.<ref>Hinkelmann and Kempthorne (2008, Volume 1, Section 6.3: 
Completely Randomized Design; Derived Linear Model)</ref> The test statistics of this derived linear model are closely approximated by the test statistics of an appropriate normal linear model, according to approximation theorems and simulation studies.<ref name="HinkelmannKempthorne">Hinkelmann and Kempthorne (2008, Volume 1, Section 6.6: Completely randomized design; Approximating the randomization test)</ref> However, there are differences. For example, the randomization-based analysis results in a small but (strictly) negative correlation between the observations.<ref>Bailey (2008, Chapter 2.14 "A More General Model" in Bailey, pp.&nbsp;38–40)</ref><ref>Hinkelmann and Kempthorne (2008, Volume 1, Chapter 7: Comparison of Treatments)</ref> In the randomization-based analysis, there is ''no assumption'' of a ''normal'' distribution and certainly ''no assumption'' of ''independence''. On the contrary, ''the observations are dependent''!

The randomization-based analysis has the disadvantage that its exposition involves tedious algebra and extensive time. Since the randomization-based analysis is complicated and is closely approximated by the approach using a normal linear model, most teachers emphasize the normal linear model approach. Few statisticians object to model-based analysis of balanced randomized experiments.

====Statistical models for observational data====
However, when applied to data from non-randomized experiments or [[observational study|observational studies]], model-based analysis lacks the warrant of randomization.<ref>
Kempthorne (1979, pp 125–126, 
"The experimenter must decide which of the various causes that he feels will produce variations in his results must be controlled 
experimentally. Those causes that he does not control experimentally, because he is not cognizant of them, he must control by the device of randomization." "[O]nly when the treatments in the experiment are applied by the experimenter using the full randomization procedure is the chain of inductive inference sound. It is ''only'' under these circumstances that the experimenter can attribute whatever effects he observes to the treatment and the treatment only. Under these circumstances his conclusions are reliable in the statistical sense.") 
</ref> For observational data, the derivation of confidence intervals must use ''subjective'' models, as emphasized by [[Ronald Fisher]] and his followers. In practice, the estimates of treatment-effects from observational studies generally are often inconsistent. In practice, "statistical models" and observational data are useful for suggesting hypotheses that should be treated very cautiously by the public.<ref>Freedman {{full citation needed|date=November 2012}}</ref>

===Summary of assumptions===
{{See also|Shapiro–Wilk test|Bartlett's test|Levene's test}}
The normal-model based ANOVA analysis assumes the independence, normality, and homogeneity of variances of the residuals. The randomization-based analysis assumes only the homogeneity of the variances of the residuals (as a consequence of unit-treatment additivity) and uses the randomization procedure of the experiment. Both these analyses require [[homoscedasticity]], as an assumption for the normal-model analysis and as a consequence of randomization and additivity for the randomization-based analysis.

However, studies of processes that change variances rather than means (called dispersion effects) have been successfully conducted using ANOVA.<ref>Montgomery (2001, Section 3.8: Discovering dispersion effects)</ref> There are ''no'' necessary assumptions for ANOVA in its full generality, but the ''F''-test used for ANOVA hypothesis testing has assumptions and practical 
limitations which are of continuing interest.

Problems which do not satisfy the assumptions of ANOVA can often be transformed to satisfy the assumptions. 
The property of unit-treatment additivity is not invariant under a "change of scale", so statisticians often use transformations to achieve unit-treatment additivity. If the response variable is expected to follow a parametric family of probability distributions, then the statistician may specify (in the protocol for the experiment or observational study) that the responses be transformed to stabilize the variance.<ref>Hinkelmann and Kempthorne (2008, Volume 1, Section 6.10: Completely randomized design; Transformations)</ref> Also, a statistician may specify that logarithmic transforms be applied to the responses which are believed to follow a multiplicative model.<ref name="Cox" /><ref>Bailey (2008)</ref>
According to [[Cauchy's functional equation]] theorem, the [[logarithm]] is the only continuous transformation that relates multiplication operations to addition operations over the real numbers.<ref>{{cite web | title=Cauchy Functional Equation | url=https://mathworld.wolfram.com/CauchyFunctionalEquation.html }}</ref><ref>{{cite book |last1=Cauchy |first1=Augustin Louis |title=Cours d'analyse de l'École royale polytechnique |trans-title=Analysis course at the Royal Polytechnic School |language=fr |date=1821 |publisher=Imprimerie royale |ol=6477125M }}{{pn|date=August 2025}}</ref><ref>{{cite book |title=Lectures on Functional Equations their and Applications |chapter=Solution of Equations by Determining the Values of the Unknown Function on a Dense Set |series=Mathematics in Science and Engineering |date=1966 |volume=19 |pages=31–139 |doi=10.1016/S0076-5392(09)60208-3 |isbn=978-0-12-043750-4 }}</ref>

==Characteristics==
ANOVA is used in the analysis of comparative experiments, those in which only the difference in outcomes is of interest. The statistical significance of the experiment is determined by a ratio of two variances. This ratio is independent of several possible alterations to the experimental observations: Adding a constant to all observations does not alter significance. Multiplying all observations by a constant does not alter significance. So ANOVA statistical significance result is independent of constant bias and scaling errors as well as the units used in expressing observations. In the era of mechanical calculation it was common to subtract a constant from all observations (when equivalent to dropping leading digits) to simplify data entry.<ref>Montgomery (2001, Section 3-3: Experiments with a single factor: The analysis of variance; Analysis of the fixed effects model)</ref><ref>Cochran & Cox (1992, p 2 example)</ref> This is an example of data [[Coding (social sciences)|coding]].

==Algorithm==
The calculations of ANOVA can be characterized as computing a number of means and variances, dividing two variances and comparing the ratio to a handbook value to determine statistical significance. Calculating a treatment effect is then trivial: "the effect of any treatment is estimated by taking the difference between the mean of the observations which receive the treatment and the general mean".<ref>Cochran & Cox (1992, p 49)</ref> 

===Partitioning of the sum of squares===
{{main|Partition of sums of squares}}
[[File:Example ANOVA Table.png|thumb|324x324px|One-factor ANOVA table showing example output data]]
{{see also|Lack-of-fit sum of squares}}
ANOVA uses traditional standardized terminology. The definitional equation of sample variance is <math display="inline">s^2 = \frac{1}{n-1} \sum_i (y_i-\bar{y})^2</math>, where the divisor is called the degrees of freedom (DF), the summation is called 
the sum of squares (SS), the result is called the mean square (MS) and the squared terms are deviations from the sample mean. ANOVA estimates 3 sample variances: a total variance based on all the observation deviations from the grand mean, an error variance based on all the observation deviations from their appropriate treatment means, and a treatment variance. The treatment variance is based on the deviations of treatment means from the grand mean, the result being multiplied by the number of observations in each treatment to account for the difference between the variance of observations and the variance of means.

The fundamental technique is a partitioning of the total [[sum of squares (statistics)|sum of squares]] ''SS'' into components related to the effects used in the model. For example, the model for a simplified ANOVA with one type of treatment at different levels.

<math display="block">SS_\text{Total} = SS_\text{Error} + SS_\text{Treatments}</math>

The number of [[Degrees of freedom (statistics)|degrees of freedom]] ''DF'' can be partitioned in a similar way: one of these components (that for error) specifies a [[chi-squared distribution]] which describes the associated sum of squares, while the same is true for "treatments" if there is no treatment effect.

<math display="block">DF_\text{Total} = DF_\text{Error} + DF_\text{Treatments}</math>

===The ''F''-test===
{{Main|F-test}}
[[File:F-Distribution Table.png|thumb|338x338px|To check for statistical significance of a one-way ANOVA, we consult the F-probability table using degrees of freedom at the {{Math|0.05}} alpha level. After computing the F-statistic, we compare the value at the intersection of each degrees of freedom, also known as the critical value. If one's F-statistic is greater in magnitude than their critical value, we can say there is statistical significance at the {{Math|0.05}} alpha level.]]
The [[F-test|''F''-test]] is used for comparing the factors of the total deviation. For example, in one-way, or single-factor ANOVA, statistical significance is tested for by comparing the F test statistic

<math display="block">F = \frac{\text{variance between treatments}}{\text{variance within treatments}}</math>
<math display="block">F = \frac{MS_\text{Treatments}}{MS_\text{Error}} = {{SS_\text{Treatments} / (I-1)} \over {SS_\text{Error} / (n_T-I)}}</math>

where ''MS'' is mean square, <math>I</math> is the number of treatments and <math>n_T</math> is the total number of cases to the [[F-distribution|''F''-distribution]] with <math>I - 1</math> being the numerator degrees of freedom and <math>n_T - I</math> the denominator degrees of freedom. Using the ''F''-distribution is a natural candidate because the test statistic is the ratio of two scaled sums of squares each of which follows a scaled [[chi-squared distribution]].

The expected value of F is <math>1 + {n \sigma^2_\text{Treatment}} / {\sigma^2_\text{Error}}</math> (where <math>n</math> is the treatment sample size) which is 1 for no treatment effect. As values of F increase above 1, the evidence is increasingly inconsistent with the null hypothesis. Two apparent experimental methods of increasing F are increasing the sample size and reducing the error variance by tight experimental controls.

There are two methods of concluding the ANOVA hypothesis test, both of which produce the same result:
* The textbook method is to compare the observed value of F with the critical value of F determined from tables. The critical value of F is a function of the degrees of freedom of the numerator and the denominator and the significance level (''α''). If F ≥ F<sub>Critical</sub>, the null hypothesis is rejected.
* The computer method calculates the probability (p-value) of a value of F greater than or equal to the observed value. The null hypothesis is rejected if this probability is less than or equal to the significance level (''α'').
The ANOVA ''F''-test is known to be nearly optimal in the sense of minimizing false negative errors for a fixed rate of false positive errors (i.e. maximizing power for a fixed significance level). For example, to test the hypothesis that various medical treatments have exactly the same effect, the [[F-test|''F''-test]]'s ''p''-values closely approximate the [[permutation test]]'s [[p-value]]s: The approximation is particularly close when the design is balanced.<ref name="HinkelmannKempthorne" /><ref>Hinkelmann and Kempthorne (2008, Volume 1, Section 6.7: Completely randomized design; CRD with unequal numbers of replications)</ref> Such [[permutation test]]s characterize [[uniformly most powerful test|tests with maximum power]] against all [[alternative hypothesis|alternative hypotheses]], as observed by [[Paul R. Rosenbaum|Rosenbaum]].<ref group="nb">Rosenbaum (2002, page 40) cites Section 5.7 (Permutation Tests), Theorem 2.3 (actually Theorem 3, page 184) of [[Erich Leo Lehmann|Lehmann]]'s ''Testing Statistical Hypotheses'' (1959).</ref> The ANOVA ''F''-test (of the null-hypothesis that all treatments have exactly the same effect) is recommended as a practical test, because of its robustness against many alternative distributions.<ref>Moore and McCabe (2003, page 763)</ref><ref group="nb">The ''F''-test for the comparison of variances has a mixed reputation. It 
is not recommended as a hypothesis test to determine whether two ''different'' samples have the same variance. It is recommended for ANOVA where two estimates of the variance of the ''same'' sample are compared. While the ''F''-test is not generally robust against departures from normality, it has been found to be robust in the special case of ANOVA. Citations from Moore & McCabe (2003): "Analysis of variance uses F statistics, but these are not the same as the F statistic for comparing two population standard deviations." (page 554) "The F test and other procedures for inference about variances are so lacking in robustness as to be of little use in practice." (page 556) "[The ANOVA ''F''-test] is relatively insensitive to moderate nonnormality and unequal variances, especially when the sample sizes are similar." (page 763) ANOVA assumes homoscedasticity, but it is robust. The statistical test for homoscedasticity (the ''F''-test) is not robust. Moore & McCabe recommend a rule of thumb.</ref>

===Extended algorithm===
ANOVA consists of separable parts; partitioning sources of variance and hypothesis testing can be used individually. ANOVA is used to support other statistical tools. Regression is first used to fit more complex models to data, then ANOVA is used to compare models with the objective of selecting simple(r) models that adequately describe the data. "Such models could be fit without any reference to ANOVA, but ANOVA tools could then be used to make some sense of the fitted models, and to test hypotheses about batches of coefficients."<ref name="Gelman">Gelman (2008)</ref> "[W]e think of the analysis of variance as a way of understanding and structuring multilevel models—not as an alternative to regression but as a tool for summarizing complex high-dimensional inferences&nbsp;..."<ref name="Gelman" />

==For a single factor==
{{Main|One-way analysis of variance}}
The simplest experiment suitable for ANOVA analysis is the completely randomized experiment with a single factor. More complex experiments with a single factor involve constraints on randomization and include completely randomized blocks and [[Latin square|Latin squares]] (and variants: [[Mutually orthogonal Latin squares|Graeco-Latin squares]], etc.). The more complex experiments share many of the complexities of multiple factors.

There are some alternatives to conventional one-way analysis of variance, e.g.: Welch's heteroscedastic F test, Welch's heteroscedastic F test with trimmed means and Winsorized variances, Brown-Forsythe test, Alexander-Govern test, James second order test and Kruskal-Wallis test, available in [https://cran.r-project.org/web/packages/onewaytests/index.html onewaytests] [[R package|R]]

It is useful to represent each data point in the following form, called a statistical model:
<math display="block">Y_{ij} = \mu + \tau_j + \varepsilon_{ij}</math>
where
* ''i'' = 1, 2, 3, ..., ''R''
* ''j'' = 1, 2, 3, ..., ''C''
* ''μ'' = overall average (mean)
* ''τ''<sub>''j''</sub> = differential effect (response) associated with the ''j'' level of X; {{pb}} this assumes that overall the values of ''τ''<sub>''j''</sub> add to zero (that is, <math display="inline">\sum_{j = 1}^C \tau_j = 0</math>)
* ''ε''<sub>''ij''</sub> = noise or error associated with the particular ''ij'' data value

That is, we envision an additive model that says every data point can be represented by summing three quantities: the true mean, averaged over all factor levels being investigated, plus an incremental component associated with the particular column (factor level), plus a final component associated with everything else affecting that specific data value.

==For multiple factors==
{{Main|Two-way analysis of variance}}
ANOVA generalizes to the study of the effects of multiple factors. When the experiment includes observations at all combinations of levels of each factor, it is termed [[Factorial experiment|factorial]]. Factorial experiments are more efficient than a series of single factor experiments and the efficiency grows as the number of factors increases.<ref name="Montgomery">Montgomery 
(2001, Section 5-2: Introduction to factorial designs; The advantages of factorials)</ref> Consequently, factorial designs are heavily used.

The use of ANOVA to study the effects of multiple factors has a complication. In a 3-way ANOVA with factors x, y and z, the ANOVA model includes terms for the main effects (x, y, z) and terms for [[Interaction (statistics)|interactions]] (xy, xz, yz, xyz). 
All terms require hypothesis tests. The proliferation of interaction terms increases the risk that some hypothesis test will produce a false positive by chance. Fortunately, experience says that high order interactions are rare.<ref>Belle (2008, Section 8.4: High-order interactions occur rarely)</ref> {{verify source|date=December 2014}}
The ability to detect interactions is a major advantage of multiple factor ANOVA. Testing one factor at a time hides interactions, but produces apparently inconsistent experimental results.<ref name="Montgomery" />

Caution is advised when encountering interactions; Test interaction terms first and expand the analysis beyond ANOVA if interactions are found. Texts vary in their recommendations regarding the continuation of the ANOVA procedure after encountering an interaction. Interactions complicate the interpretation of experimental data. Neither the calculations of significance nor the estimated treatment effects can be taken at face value. "A significant interaction will often mask the significance of main effects."<ref>Montgomery (2001, Section 5-1: Introduction to factorial designs; Basic definitions and principles)</ref> Graphical methods are recommended to enhance understanding. Regression is often useful. A lengthy discussion of interactions is available in Cox (1958).<ref>Cox (1958, Chapter 6: Basic ideas about factorial experiments)</ref> Some interactions can be removed (by transformations) while others cannot.

A variety of techniques are used with multiple factor ANOVA to reduce expense. One technique used in factorial designs is to minimize replication (possibly no replication with support of [[Tukey's test of additivity|analytical trickery]]) and to combine groups when effects are found to be statistically (or practically) insignificant. An experiment with many insignificant factors may collapse into one with a few factors supported by many replications.<ref>Montgomery (2001, Section 5-3.7: Introduction to factorial designs; The two-factor factorial design; One observation per cell)</ref>

==Associated analysis==
Some analysis is required in support of the ''design'' of the experiment while other analysis is performed after changes in the factors are formally found to produce statistically significant changes in the responses. Because experimentation is iterative, the results of one experiment alter plans for following experiments.

===Preparatory analysis===
====The number of experimental units====
In the design of an experiment, the number of experimental units is planned to satisfy the goals of the experiment. Experimentation is often sequential.

Early experiments are often designed to provide mean-unbiased estimates of treatment effects and of experimental error. Later experiments are often designed to test a hypothesis that a treatment effect has an important magnitude; in this case, the number of experimental units is chosen so that the experiment is within budget and has adequate power, among other goals.

Reporting sample size analysis is generally required in psychology. "Provide information on sample size and the process that led to sample size decisions."<ref>Wilkinson (1999, p 596)</ref> The analysis, which is written in the experimental protocol before the experiment is conducted, is examined in grant applications and administrative review boards.

Besides the power analysis, there are less formal methods for selecting the number of experimental units. These include graphical methods based on limiting the probability of false negative errors, graphical methods based on an expected variation increase (above the residuals) and methods based on achieving a desired confidence interval.<ref>Montgomery (2001, Section 3-7: Determining sample size)</ref>

====Power analysis====
[[Statistical power|Power analysis]] is often applied in the context of ANOVA in order to assess the probability of successfully rejecting the null hypothesis if we assume a certain ANOVA design, effect size in the population, sample size and significance level. Power analysis can assist in study design by determining what sample size would be required in order to have a reasonable chance of rejecting the null hypothesis when the alternative hypothesis is true.<ref>Howell (2002, Chapter 8: Power)</ref><ref>Howell (2002, Section 11.12: Power (in ANOVA))</ref><ref>Howell (2002, Section 13.7: Power analysis for factorial experiments)</ref><ref>Moore and McCabe (2003, pp 778–780)</ref>
[[File:Effect_size.png|thumb|Effect size]]

====Effect size====
{{Main|Effect size}}
Several standardized measures of effect have been proposed for ANOVA to summarize the strength of the association between a predictor(s) and the dependent variable or the overall standardized difference of the complete model. Standardized effect-size estimates facilitate comparison of findings across studies and disciplines. However, while standardized effect sizes are commonly used in much of the professional literature, a non-standardized measure of effect size that has immediately "meaningful" units may be preferable for reporting purposes.<ref name="Wilkinson">Wilkinson (1999, p 599)</ref>

====Model confirmation====
Sometimes tests are conducted to determine whether the assumptions of ANOVA appear to be violated. Residuals are examined or analyzed to confirm [[homoscedasticity]] and gross normality.<ref>Montgomery (2001, Section 3-4: Model adequacy checking)</ref> Residuals should have the appearance of (zero mean normal distribution) noise when plotted as a function of anything including time and 
modeled data values. Trends hint at interactions among factors or among observations.

====Follow-up tests====
A statistically significant effect in ANOVA is often followed by additional tests. This can be done in order to assess which groups are different from which other groups or to test various other focused hypotheses. Follow-up tests are often distinguished in terms of whether they are "planned" ([[A priori and a posteriori|a priori]]) or [[Post-hoc analysis|"post hoc]]." Planned tests are determined before looking at the data, and post hoc tests are conceived only after looking at the data (though the term "post hoc" is inconsistently used).

The follow-up tests may be "simple" pairwise comparisons of individual group means or may be "compound" comparisons (e.g., comparing the mean pooling across groups A, B and C to the mean of group D). Comparisons can also look at tests of trend, such as linear and quadratic relationships, when the independent variable involves ordered levels. Often the follow-up tests incorporate a method of adjusting for the [[multiple comparisons problem]].

Follow-up tests to identify which specific groups, variables, or factors have statistically different means include the [[Tukey's range test]], and [[Duncan's new multiple range test]].  In turn, these tests are often followed with a [[Compact Letter Display (CLD)]] methodology in order to render the output of the mentioned tests more transparent to a non-statistician audience.

==Study designs==
There are several types of ANOVA. Many statisticians base ANOVA on the [[experimental design|design of the experiment]],<ref>Cochran & Cox (1957, p 9, "The general rule [is] that the way in which the experiment is conducted determines not only whether inferences can be made, but also the calculations required to make them.")</ref> especially on the protocol that specifies the [[random assignment]] of treatments to subjects; the protocol's description of the assignment mechanism should include a specification of the structure of the treatments and of any [[blocking (statistics)|blocking]]. It is also common to apply ANOVA to observational data using an appropriate statistical model.<ref>{{Cite web |title=ANOVA Design |url=https://bluebox.creighton.edu/demo/modules/en-boundless-old/www.boundless.com/statistics/textbooks/boundless-statistics-textbook/estimation-and-hypothesis-testing-12/one-way-anova-57/anova-design-283-2741/ |access-date=2023-01-23 |website=bluebox.creighton.edu}}</ref>

Some popular designs use the following types of ANOVA:
*[[One-way ANOVA]] is used to test for differences among two or more [[statistical independence|independent]] groups (means), e.g. different levels of urea application in a crop, or different levels of antibiotic action on several different bacterial species,<ref>{{Cite web|url=http://www.biomedicalstatistics.info/en/multiplegroups/one-way-anova.html|archive-url=https://web.archive.org/web/20141107211953/http://www.biomedicalstatistics.info/en/multiplegroups/one-way-anova.html|url-status=dead|title=One-way/single factor ANOVA|archive-date=7 November 2014}}</ref> or different levels of effect of some medicine on groups of patients. However, should these groups not be independent, and there is an order in the groups (such as mild, moderate and severe disease), or in the dose of a drug (such as 5&nbsp;mg/mL, 10&nbsp;mg/mL, 20&nbsp;mg/mL) given to the same group of patients, then a [[linear trend estimation]] should be used. Typically, however, the one-way ANOVA is used to test for differences among at least three groups, since the two-group case can be covered by a [[t-test]].<ref>{{Cite journal | doi = 10.1093/biomet/6.1.1 | title = The Probable Error of a Mean | journal = Biometrika | volume = 6 | pages = 1–25 | year = 1908 | hdl = 10338.dmlcz/143545 }}</ref> When there are only two means to compare, the [[t-test]] and the ANOVA [[F-test|''F''-test]] are equivalent; the relation between ANOVA and ''t'' is given by {{math|1=''F'' = ''t''<sup>2</sup>}}.
*[[Factorial experiment|Factorial]] ANOVA is used when there is more than one factor.
*[[Repeated measures]] ANOVA is used when the same subjects are used for each factor (e.g., in a [[longitudinal study]]).
*[[Multivariate analysis of variance]] (MANOVA) is used when there is more than one [[dependent variable|response variable]].

==Cautions==
Balanced experiments (those with an equal sample size for each treatment) are relatively easy to interpret; unbalanced experiments offer more complexity. For single-factor (one-way) ANOVA, the adjustment for unbalanced data is easy, but the unbalanced analysis lacks both robustness and power.<ref>Montgomery (2001, Section 3-3.4: Unbalanced data)</ref> For more complex designs the lack of balance leads to further complications. "The orthogonality property of main effects and interactions present in balanced data does not carry over to the unbalanced case. This means that the usual analysis of variance techniques do not apply. Consequently, the analysis of unbalanced factorials is much more difficult than that for balanced designs."<ref>Montgomery (2001, Section 14-2: Unbalanced data in factorial design)</ref> In the general case, "The analysis of variance can also be applied to unbalanced data, but then the sums of squares, mean squares, and ''F''-ratios will depend on the order in which the sources of variation are considered."<ref name="Gelman" />

ANOVA is (in part) a test of statistical significance. The American Psychological Association (and many other organisations) holds the view that simply reporting statistical significance is insufficient and that reporting confidence bounds is preferred.<ref name="Wilkinson" />

==Generalizations==
ANOVA is considered to be a special case of [[linear regression]]<ref>Gelman (2005, p.1) (with qualification in the later text)</ref><ref>Montgomery (2001, Section 3.9: The Regression Approach to the Analysis of Variance)</ref> which in turn is a special case of the [[general linear model]].<ref>Howell (2002, p 604)</ref> All consider the observations to be the sum of a model (fit) and a residual (error) to be minimized.

The [[Kruskal-Wallis test]] and the [[Friedman test]] are [[nonparametric]] tests which do not rely on an assumption of normality.<ref>Howell (2002, Chapter 18: Resampling and nonparametric approaches to data)</ref><ref>Montgomery (2001, Section 3-10: Nonparametric methods in the analysis of variance)</ref>

===Connection to linear regression===
Below we make clear the connection between multi-way ANOVA and linear regression.

Linearly re-order the data so that <math>k</math>-th observation is associated with a response <math>y_k</math> and factors <math>Z_{k,b}</math> where <math>b \in \{1,2,\ldots,B\}</math> denotes the different factors and <math>B</math> is the total number of factors. In one-way ANOVA <math>B=1</math> and in two-way ANOVA <math>B = 2</math>. Furthermore, we assume the <math>b</math>-th factor has <math>I_b</math> levels, namely <math>\{1,2,\ldots,I_b\}</math>. Now, we can [[one-hot]] encode the factors into the <math display="inline"> \sum_{b=1}^B I_b</math> dimensional vector <math>v_k</math>.

The one-hot encoding function <math>g_b : \{1,2,\ldots,I_b\} \mapsto \{0,1\}^{I_b}</math> is defined such that the <math>i</math>-th entry of <math>g_b(Z_{k,b})</math> is
<math display="block">g_b(Z_{k,b})_i = \begin{cases}
1 & \text{if } i=Z_{k,b} \\
0 & \text{otherwise}
\end{cases}</math>
The vector <math>v_k</math> is the concatenation of all of the above vectors for all <math>b</math>. Thus, <math>v_k = [g_1(Z_{k,1}), g_2(Z_{k,2}), \ldots, g_B(Z_{k,B})]</math>. In order to obtain a fully general <math>B</math>-way interaction ANOVA we must also concatenate every additional interaction term in the vector <math>v_k</math> and then add an intercept term. Let that vector be <math>X_k</math>.

With this notation in place, we now have the exact connection with linear regression. We simply regress response <math>y_k</math> against the vector <math>X_k</math>. However, there is a concern about [[identifiability]]. In order to overcome such issues we assume that the sum of the parameters within each set of interactions is equal to zero. From here, one can use ''F''-statistics or other methods to determine the relevance of the individual factors.

====Example====
We can consider the 2-way interaction example where we assume that the first factor has 2 levels and the second factor has 3 levels.

Define <math>a_i = 1</math> if <math>Z_{k,1}=i</math> and <math>b_i = 1</math> if <math>Z_{k,2} = i</math>, i.e. <math>a</math> is the one-hot encoding of the first factor and <math>b</math> is the one-hot encoding of the second factor.

With that,
<math display="block">
X_k = [a_1, a_2, b_1, b_2, b_3 ,a_1 \times b_1, a_1 \times b_2, a_1 \times b_3, a_2 \times b_1, a_2 \times b_2, a_2 \times b_3, 1]
</math>
where the last term is an intercept term. For a more concrete example suppose that
<math display="block">\begin{align}
Z_{k,1} & = 2 \\
Z_{k,2} & = 1
\end{align}</math>
Then, <math display="block">X_k = [0,1,1,0,0,0,0,0,1,0,0,1]</math>

==See also==
{{Portal|Mathematics}}
<!-- Please keep entries in alphabetical order & add a short description [[WP:SEEALSO]] -->
{{div col |colwidth=25em}}
*[[ANOVA on ranks]]
*[[ANOVA-simultaneous component analysis]]
*[[Analysis of covariance]] ('''ANCOVA''')
*[[Analysis of molecular variance]] (AMOVA)
*[[Analysis of rhythmic variance]] (ANORVA)
*[[Expected mean squares]]
*[[Explained variation]]
*[[Linear trend estimation]]
*[[Mixed-design analysis of variance]]
*[[Multivariate analysis of covariance]] ('''MANCOVA''')
*[[Permutational analysis of variance]]
*[[Variance decomposition]]
{{div col end}}
<!-- please keep entries in alphabetical order -->

==Footnotes==
{{reflist|group="nb"}}

==Notes==
{{reflist|30em}}

==References==
* {{cite journal|doi=10.2307/2984159|title=The Validity of Comparative Experiments|author-link=Francis J. Anscombe|first=F. J.| last=Anscombe|journal=Journal of the Royal Statistical Society. Series A (General)|volume=111|issue=3|year=1948|pages=181–211| jstor=2984159|mr=30181}}
* {{cite book |last=Bailey|first=R. A.|author-link=Rosemary A. Bailey|title=Design of Comparative Experiments|publisher=Cambridge University Press|year=2008 |isbn=978-0-521-68357-9|url=http://www.maths.qmul.ac.uk/~rab/DOEbook}} Pre-publication chapters are available on-line.
* {{cite book | last = Belle | first = Gerald van
| title = Statistical rules of thumb | publisher = Wiley 
| location = Hoboken, N.J | year = 2008 | edition = 2nd 
| isbn = 978-0-470-14448-0 }}
* {{cite book | last1 = Cochran | first1 = William G.
| author-link1= William Gemmell Cochran
| last2 = Cox | first2 = Gertrude M. 
| author-link2=Gertrude Mary Cox
| title = Experimental designs | publisher = Wiley | location = New York 
| year = 1992 | isbn = 978-0-471-54567-5 | edition = 2nd }}
* Cohen, Jacob (1988). ''Statistical power analysis for the behavior sciences'' (2nd ed.). Routledge {{ISBN|978-0-8058-0283-2}}
* {{Cite journal | doi = 10.1037/0033-2909.112.1.155 | author = Cohen, Jacob 
| year = 1992 | title = Statistics a power primer | journal = Psychological Bulletin | volume = 112 | issue = 1| pages = 155–159 | pmid=19565683| s2cid = 14411587 }}
*[[David R. Cox|Cox, David R.]] (1958). ''Planning of experiments''. Reprinted as {{ISBN|978-0-471-57429-3}}
*{{cite book | last = Cox | first = David R.
| title = Principles of statistical inference 
| publisher = Cambridge University Press 
| location = Cambridge New York | year = 2006 
| isbn = 978-0-521-68567-2 }}
* [[David A. Freedman (statistician)|Freedman, David A.]](2005). ''Statistical Models: Theory and Practice'', Cambridge University Press. {{ISBN|978-0-521-67105-7}}
* {{Cite journal| last1 = Gelman | first1 = Andrew | doi = 10.1214/009053604000001048 | title = Analysis of variance? Why it is more important than ever | journal = The Annals of Statistics | volume = 33 | pages = 1–53 | year = 2005 | arxiv = math/0504499 | s2cid = 13529149 }}
*{{cite book | last = Gelman | first = Andrew| title = The new Palgrave dictionary of economics | publisher = Palgrave Macmillan | location = Basingstoke, Hampshire New York| chapter=Variance, analysis of |edition=2nd | year = 2008 | isbn = 978-0-333-78676-5}}
*{{cite book |author=Hinkelmann, Klaus |author2=Kempthorne, Oscar 
|name-list-style=amp|year=2008|title=Design and Analysis of Experiments |volume=I and II|edition=Second|publisher=Wiley|isbn=978-0-470-38551-7|author2-link=Oscar Kempthorne }}
* {{cite book | last = Howell | first = David C. 
| title = Statistical methods for psychology | publisher = Duxbury/Thomson Learning | location = Pacific Grove, CA | year = 2002 | edition = 5th | isbn = 978-0-534-37770-0 | url-access = registration | url = https://archive.org/details/statisticalmetho0000howe }}
*{{cite book
|author=Kempthorne, Oscar
|author-link= Oscar Kempthorne
|year=1979
|title=The Design and Analysis of Experiments
|edition=Corrected reprint of (1952) Wiley
|publisher=Robert E. Krieger
|isbn=978-0-88275-105-4
}}
* [[Erich Leo Lehmann|Lehmann, E.L.]] (1959) Testing Statistical Hypotheses. John Wiley & Sons.
* {{cite book | last = Montgomery | first = Douglas C.
| title = Design and Analysis of Experiments 
| publisher = Wiley | location = New York 
| year = 2001 | edition = 5th | isbn = 978-0-471-31649-7}}
* [[David S. Moore|Moore, David S.]] & McCabe, George P. (2003). Introduction to the Practice of Statistics (4e). W H Freeman & Co. {{ISBN|0-7167-9657-0}}
* [[Paul R. Rosenbaum|Rosenbaum, Paul R.]] (2002). ''Observational Studies'' (2nd ed.). New York: Springer-Verlag. {{ISBN|978-0-387-98967-9}}
* {{cite book |title=The Analysis of Variance
|last=Scheffé |first=Henry 
|author-link= Henry Scheffé
|location=New York
|publisher=Wiley |year=1959}}
*{{cite book | last = Stigler | first = Stephen M. |author-link=Stephen Stigler| title = The history of statistics : the measurement of uncertainty before 1900 | publisher = Belknap Press of Harvard University Press | location = Cambridge, Mass | year = 1986 | isbn = 978-0-674-40340-6 | url-access = registration | url = https://archive.org/details/historyofstatist00stig |oclc=1422544327 }}
* {{cite journal |last1=Wilkinson |first1=Leland |author-link=Leland Wilkinson |title=Statistical methods in psychology journals: Guidelines and explanations |journal=American Psychologist |date=August 1999 |volume=54 |issue=8 |pages=594–604 |doi=10.1037/0003-066X.54.8.594 }}

==Further reading==

* {{cite book
|last1=Freedman |first1=David A. |author-link1=David A. Freedman (statistician)
|last2=Pisani |first2=Robert
|last3=Purves |first3=Roger
|year=2007
|title=Statistics
|edition=4th
|publisher=W.W. Norton & Company
|isbn=978-0-393-92972-0
}}
* {{cite book
|last1=Tabachnick |first1=Barbara G.
|last2=Fidell |first2=Linda S.
|year=2006
|title=Using Multivariate Statistics
|edition=5th
|publisher=Allyn & Bacon, Inc.
|publication-place=Needham, MA
|series=Pearson International Edition
|isbn=978-0-205-45938-4
}}

* {{cite book

|last=Wichura|first=Michael J.
|title=The coordinate-free approach to linear models
|series=Cambridge Series in Statistical and Probabilistic Mathematics
|publication-place=Cambridge University Press
|year=2006
|pages=xiv+199
|isbn=978-0-521-86842-6
|mr=2283455
}}
* {{cite book
|last=Christensen |first=Ronald
|title=Plane Answers to Complex Questions: The Theory of Linear Models
|publication-place=New York
|publisher=Springer
|year=2002
|edition=Third
|isbn=978-0-387-95361-8
}}
* {{cite book |title=Block Designs: A Randomization Approach |series=Lecture Notes in Statistics |date=2000 |volume=150 |doi=10.1007/978-1-4612-1192-1 |isbn=978-0-387-98578-7 |first1=Tadeusz |last1=Caliński |first2=Sanpei |last2=Kageyama }}
*{{cite book |last1=Cox |first1=D.R. |last2=Reid |first2=Nancy |title=The Theory of the Design of Experiments |date=2000 |doi=10.1201/9781420035834 |isbn=978-0-429-12628-4 }}
* {{cite book
|last1=Hettmansperger |first1=T. P.
|last2=McKean |first2=J. W.
|title=Robust nonparametric statistical methods
|edition=1st
|series=Kendall's Library of Statistics
|volume=5
|publication-place=New York
|publisher=A Hodder Arnold Publication
|year=1998
|pages=xiv+467
|isbn=978-0-340-54937-7
|mr=1604954
}}
* {{cite book
|first1=Marvin |last1=Lentner
|last2=Bishop |first2=Thomas
|title=Experimental design and analysis
|edition=2nd
|publisher=Valley Book Company
|publication-place=Blacksburg, VA
|year=1993
|isbn=978-0-9616255-2-8
}}
* {{cite book
|last=Phadke |first=Madhav S.
|title=Quality Engineering using Robust Design
|publisher=Prentice Hall PTR
|publication-place=New Jersey
|year=1989
|isbn=978-0-13-745167-8
}}
* {{cite journal
|last=Box |first=G. E. P.
|title=Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, II. Effects of Inequality of Variance and of Correlation Between Errors in the Two-Way Classification
|doi=10.1214/aoms/1177728717
|journal=The Annals of Mathematical Statistics
|volume=25
|issue=3
|page=484
|year=1954
|doi-access=free
}}
* {{cite journal
|last=Box |first=G. E. P.
|title=Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification
|doi=10.1214/aoms/1177728786
|journal=The Annals of Mathematical Statistics
|volume=25
|issue=2
|page=290
|year=1954
|doi-access=free
}}

* {{cite journal
|last=Box |first=G. E. P. |author-link=George E. P. Box
|title=Non-Normality and Tests on Variances
|journal=Biometrika
|volume=40
|issue=3/4
|pages=318–335
|year=1953
|jstor=2333350
|doi=10.1093/biomet/40.3-4.318
}}
* {{cite journal |last1=Fisher |first1=R. A. |title=Studies in crop variation. I. An examination of the yield of dressed grain from Broadbalk |journal=The Journal of Agricultural Science |date=April 1921 |volume=11 |issue=2 |pages=107–135 |id={{CORE output|162101431}} |doi=10.1017/S0021859600003750 |hdl=2440/15170 |hdl-access=free }}

==External links==
{{Commons category|Analysis of variance}}
{{wikiversity}}
* [[SOCR]]: [http://wiki.stat.ucla.edu/socr/index.php/AP_Statistics_Curriculum_2007_ANOVA_1Way ANOVA Activity]
* [https://www.southampton.ac.uk/~cpd/anovas/datasets/index.htm Examples of all ANOVA and ANCOVA models with up to three treatment factors, including randomized block, split plot, repeated measures, and Latin squares, and their analysis in R] (University of Southampton)
* NIST/SEMATECH e-Handbook of Statistical Methods, [http://www.itl.nist.gov/div898/handbook/prc/section4/prc43.htm section 7.4.3: "Are the means equal?"]
*[https://web.archive.org/web/20150405053021/http://biostat.katerynakon.in.ua/en/multiplegroups/anova.html Analysis of variance: Introduction]

{{Statistics|correlation|state=collapsed}}
{{Experimental design|state=collapsed}}
{{Least squares and regression analysis}}
{{Public health}}
{{Authority control}}

{{DEFAULTSORT:Analysis Of Variance}}
[[Category:Analysis of variance| ]]
[[Category:Design of experiments]]
[[Category:Statistical tests]]
[[Category:Parametric statistics]]