A note on confidence intervals and statistical significance

If two 95% confidence intervals overlap this does not imply that the two statistics on which they are based (e.g. means, odds ratios) differ at the 5 percent level. In other words it is possible for the difference between two statistics to be statistically non-zero and for their respective confidence intervals still to overlap. This is usually the case when the difference between the means has moderate significance.

It is true, however, that if a pair of confidence intervals do not overlap the difference between the two statistics is statistically non-zero.

The rationale behind the above discrepancy is explained in this [attachment:cis.pdf article] taken from the Cornell University website. See also [attachment:cis2.pdf here.]

Reference

Wolfe R and Hanley J (2002) If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2. CMAJ 166 65-66