Comparing all pairwise comparisons in an anova assuning equality of group variances
Ramsey PH and Ramsey PP (2008) recommend using the Tukey-Kramer procedure to compare all possible group means assuming homogeneity of group variances and allowing different group sizes. It is found to give the best any-pair power if the overall F test is not significant. This procedure is computed for upto 10 means using this [attachment:mcneq+hf.xls spreadsheet.] It may also be computed using the mc or mcneq procedures found by typing mc and mcneq at a UNIX prompt. Ramsey and Ramsey (2008) further recommend using the more conservative Hayter-Fisher modification of the Tukey-Kramer procedure for maximizing any-pair power in the presence of a significant overall F value. This is also computed in the spreadsheet.
The Games-Howell approach is recommended by Field, 2005 and Howell, 2002 for all pairwise comparisons when there are heterogeneous group variances. The Games-Howell approach may be computed in SPSS using the GLM:univariate procedure or by using this [attachment:gamesh.xls spreadsheet.]
A flow chart detailing issues involved in the choice of post-hoc tests for between subject designs is [attachment:flow.pdf here.] The chart suggests two alternative tests to the Tukey-Kramer mentioned above for groups with homogeneous variances but of different size. In particular an adjustment suggested by Gabriel to the Tukey test is suggested when the sample sizes are close. The REGWQ test incorporating Gabriel's adjustment is computed by this [attachment:tuk+regwq.xls spreadsheet.] Tukey's (and the REGWQ) test (see pages 399-400 of Howell (2002)) may also be used for comparing all pairwise group difference based on a single repeated measures factor. For a worked example of using Tukey's HSD with groups from repeated measures data see [http://www.uvm.edu/~dhowell/StatPages/More_Stuff/RepMeasMultComp/RepMeasMultComp.html here.]
Note (just in case you were wondering!): The above spreadsheets quote the, perhaps, more familiar t-statistic rather than the studentised range statistic, q, (which is quoted in the output of e.g. the mc and mcneq UNIX programs at CBSU) although we, equivalently, still use the studentised range statistic for testing. In fact, as Howell(2002) points out, q and t can be used interchangeably since q = $$sqrt(2)$$t.
Hochberg's GT2 should be appearing in due course.
References
Field A. (2005) Discovering statistics using SPSS. Second edition. Sage:London.
Howell DC (2002) Statistical methods for psychologists. Fifth edition. Wadsworth:Pacific Grove, CA.
Ramsey PH and Ramsey PP (2008) Power of pairwise comparisons in the equal variance and unequal sample size case. British Journal of Mathematical and Statistical Psychology 61(1) 115-131.