Overlapping Confidence Intervals and Statistical Significance

459 Words2 Pages
StatNews # 73: Overlapping Confidence Intervals and Statistical Significance October 2008 In this issue of StatNews, we address the question: can we judge whether two statistics are significantly different depending on whether or not their confidence intervals overlap? The answer is: not always. If two statistics have non-overlapping confidence intervals, they are necessarily significantly different but if they have overlapping confidence intervals, it is not necessarily true that they are not significantly different. We can illustrate this with a simple example. Suppose we are interested in comparing means from two independent samples. The mean of the first sample is 9 and the mean of the second sample is 17. Let’s assume that the two group means have the same standard errors, equal to 2.5. The 95 percent confidence interval for the first group mean can be calculated as: 9 ± 1.96 × 2.5 where 1.96 is the critical t-value. The confidence interval for the first group mean is thus (4.1, 13.9). Similarly for the second group, the confidence interval for the mean is (12.1, 21.9). Notice that the two intervals overlap. However, the t-statistic for comparing two means is: t= 17 − 9 2.5 2 + 2.5 2 = 2.26 which reflects that the null hypothesis, that the means of the two groups are the same, should be rejected at the α = 0.05 level. To verify the above conclusion, consider the 95 percent confidence interval for the difference between the two group means: (17 − 9 ) ± 1.96 × 2.5 2 + 2.5 2 which yields (1.09, 14.91). The interval does not contain zero, hence we reject the null hypothesis that the group means are the same. Generally, when comparing two parameter estimates, it is always true that if the confidence intervals do not overlap, then the statistics will be statistically significantly different. However, the converse is not true. That is, it is erroneous

More about Overlapping Confidence Intervals and Statistical Significance

Open Document