How we change what others think, feel, believe and do

ANOVA

Explanations > Social Research > Analysis > ANOVA

t-test problems

A significant problem with the t-test is that we typically accept significance with each t-test of 95% (alpha=0.05). For multiple tests these accumulate and hence reduce the validity of the results.

ANalysis Of VAriance (ANOVA) overcomes these problems by using a single test to detect significant differences between the treatments as a whole.

ANOVA assumes parametric data.

F-ratio

Like the t-test, ANOVA produced a test statistic that compares the means of variables, testing them for equality (or, hopefully, not). This is the F-ratio, which compares the amount of unsystematic variance in the data (SS_M) to the amount of systematic variance (SS_R).

This is a problem in that the F-ratio only says that there is a difference in means, but does not say which ones differ or which are the same. This may be addressed with additional post-hoc tests.

Bonferroni condition

In multiple tests, you could go back to the t-test problem of deteriorating alpha (the probability of type 1 error). This is addressed with the Bonferroni correction, where alpha is divided by the number of tests.

Thus if you have set alpha=0.05, then with five ad-hoc tests, you revise it to 0.01 and require the test statistic to be less than this.

Test types

Types of ANOVA have 'X-way' (or 'X-factor') in the title. This indicates the number of independent variables that were manipulated in the study. Thus:

'One way' means one independent variable.
'Two way' means two independent variables.
etc.

The second part of the title tell how the independent variables are measured:

'Independent' means different subjects take part in different conditions.
'Repeated measures' means the same people take part in all treatments.
'Mixed' means at least one independent variable will be measured using different subjects, and at least one independent variable will be measured using the same subjects.

Reporting

The ANOVA statistic is reported like this:

The results shows that sucking lollipops significantly increases IQ of college men, F(3,17) = 2.87, p = .007.

Where:
F - the test statistic
3 - the model degrees of freedom (numerator)
17 - residual degrees of freedom (denominator)
2.87 - value of F, the test statistic
.007 - the probability of H₀ being true

Post-tests

After an ANOVA test, a number of additional tests may be used to further understand the data.

Bonferroni, as above, gives strict control but can give 'false positive' results.
REGWQ and Tukey HSD have good power to handle Type 1 errors.
If the sample size varies a little, use Gabriel's procedure. If it varies significantly, use Hochberg's GT2.
If the population may not have homogeneous variance, then Games-Howell may be used.
Newman-Keuls and Scheffe tests are the most conservative.

These are all available in SPSS.

The effect and power may also be measured, to determine the value

Discussion

ANOVA is also known as Fisher's ANOVA or Fisher's analysis of variance after its originator, R. A. Fisher in the 1920s.

When comparing two groups to see if they are similar, ANOVA compares only the means, in the same way as the t-test. Despite its name, it does not assess the whole distribution (in fact it requires a similar variance across the groups being assessed!).