Comparing all pairwise comparisons in an anova assuning equality of group variances

Ramsey PH and Ramsey PP (2008) recommend using the Tukey-Kramer procedure to compare all possible group means assuming homogeneity of group variances and allowing different group sizes. It is found to give the best any-pair power if the overall F test is not significant. This procedure is computed for upto 10 means using this [attachment:mcneq+hf.xls spreadsheet.] It may also be computed using the mc or mcneq procedures found by typing mc and mcneq at a UNIX prompt. Ramsey and Ramsey (2008) further recommend using the more conservative Hayter-Fisher modification of the Tukey-Kramer procedure for maximizing any-pair power in the presence of a significant overall F value. This is also computed in the spreadsheet.

The Games-Howell approach is recommended by Field, 2005 and Howell, 2002 for all pairwise comparisons when there are heterogeneous group variances. The Games-Howell approach may be computed in SPSS using the GLM:univariate procedure or by using this [attachment:gamesh.xls spreadsheet.]

A flow chart detailing issues involved in the choice of post-hoc tests for between subject designs is [attachment:flow.pdf here.] The chart suggests two alternative tests to the Tukey-Kramer mentioned above for groups with homogeneous variances but of different size.

References

Field A. (2005) Discovering statistics using SPSS. Second edition. Sage:London.

Howell DC (2002) Statistical methods for psychologists. Fifth edition. Wadsworth:Pacific Grove, CA.

Ramsey PH and Ramsey PP (2008) Power of pairwise comparisons in the equal variance and unequal sample size case. British Journal of Mathematical and Statistical Psychology 61(1) 115-131.