|
Size: 744
Comment:
|
Size: 1706
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| Line 4: | Line 3: |
| If two 95% confidence intervals overlap this does ''not'' imply that the two statistics (e.g. means, odds ratios) in question differ at the 5 percent level. In other words it is possible for the difference between two statistics to be statistically non-zero and for their respective confidence intervals to overlap. This is usually the case when the difference between the means has moderate significance. | If two 95% confidence intervals overlap this does ''not'' imply that the two statistics on which they are based (e.g. means, odds ratios) differ at the 5 percent level. In other words it is possible for the difference between two statistics to be statistically non-zero and for their respective confidence intervals still to overlap. This is usually the case when the difference between the means has moderate significance. |
| Line 6: | Line 5: |
| It is true, however, that if a pair of confidence intervals do not overlap the difference between the two statistics is statistically non-zero. | It ''is'' true, however, that if a pair of confidence intervals do not overlap the difference between the two statistics is statistically non-zero. |
| Line 8: | Line 7: |
| For the rationale behind the above see this [attachment:cis.pdf article] taken from the Cornell University website. | The rationale behind the above discrepancy is explained in this [[attachment:cis.pdf|article]] taken from the Cornell University website. See also [[attachment:cis2.pdf|here.]] As a special case if two independent group means have the same standard error (se) then the standard error of the difference in the two means equals sqrt(se^2 ^+ se^2 ^) = sqrt(2 se^2 ^) = sqrt(2)se. Now if both groups have large sample sizes then the group difference is approximately statistically significant if the abs[difference in group means] / (sqrt(2) se) is greater than 2 ie if the abs(difference in group means) > 2 x sqrt(2) se = approximately 2.8 se. Now, 2.8 se is greater than 2 se which suggests that in this special case of equal mean standard errors two intervals about two statistically non-significant means could not overlap if the interval widths about the mean are equal to one standard error of each mean. __Reference__ Wolfe R and Hanley J (2002) If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2. CMAJ 166 65-66 |
A note on confidence intervals and statistical significance
If two 95% confidence intervals overlap this does not imply that the two statistics on which they are based (e.g. means, odds ratios) differ at the 5 percent level. In other words it is possible for the difference between two statistics to be statistically non-zero and for their respective confidence intervals still to overlap. This is usually the case when the difference between the means has moderate significance.
It is true, however, that if a pair of confidence intervals do not overlap the difference between the two statistics is statistically non-zero.
The rationale behind the above discrepancy is explained in this article taken from the Cornell University website. See also here.
As a special case if two independent group means have the same standard error (se) then the standard error of the difference in the two means equals sqrt(se2 + se2 ) = sqrt(2 se2 ) = sqrt(2)se.
Now if both groups have large sample sizes then the group difference is approximately statistically significant if the abs[difference in group means] / (sqrt(2) se) is greater than 2 ie if the abs(difference in group means) > 2 x sqrt(2) se = approximately 2.8 se. Now, 2.8 se is greater than 2 se which suggests that in this special case of equal mean standard errors two intervals about two statistically non-significant means could not overlap if the interval widths about the mean are equal to one standard error of each mean.
Reference
Wolfe R and Hanley J (2002) If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2. CMAJ 166 65-66