Why Perform Multiple Comparison Tests After ANOVA?

Why Perform Multiple Comparison Tests After ANOVA?

 ## Why Perform Multiple Comparison Tests After ANOVA?

After conducting a one-way **Analysis of Variance (ANOVA)** and finding a significant overall difference among the group means, it is necessary to perform multiple comparison tests for the following reasons:

1. **ANOVA only tells you that at least one pair of means is significantly different**, but it does not specify which pairs differ. Multiple comparison tests help identify which specific pairs of means are significantly different from each other.

2. **Without multiple comparisons, it is not possible to control the family-wise error rate (FWER)**, which is the probability of making one or more Type I errors (false positives) when performing several statistical tests simultaneously. Multiple comparison tests adjust the significance level to maintain the desired FWER.

3. **Multiple comparison tests provide more detailed information about the patterns of differences among the groups**, allowing for a better understanding of the relationships between the groups.

## Tukey's Honestly Significant Difference (HSD) Test

**Tukey's HSD** is a commonly used multiple comparison test that controls the FWER. It is particularly useful when all pairwise comparisons are of interest and the sample sizes are equal across groups.

The steps involved in Tukey's HSD test are as follows:

1. **Calculate the test statistic q for each pair of means**:

   $$q = \frac{|\bar{X}_i - \bar{X}_j|}{\sqrt{\frac{MSE}{n}}}$$

   where $\bar{X}_i$ and $\bar{X}_j$ are the means of the $i$th and $j$th groups, $MSE$ is the mean square error from the ANOVA table, and $n$ is the sample size per group.

2. **Compare the calculated q values to the critical value** from the Studentized Range distribution table, which depends on the desired significance level ($\alpha$) and the number of groups ($k$).

3. **If the calculated q value for a pair of means exceeds the critical value**, the difference between those means is considered statistically significant at the specified $\alpha$ level.

4. **Tukey's HSD test maintains the FWER at $\alpha$ level** by using a more conservative critical value compared to conducting multiple individual t-tests.

## Interpreting Tukey's HSD Results

After performing Tukey's HSD test, the results can be interpreted as follows:

1. **If the difference between two means is significant**, it indicates that those two groups are significantly different from each other at the specified $\alpha$ level.

2. **If the difference between two means is not significant**, it suggests that those two groups are not detectably different from each other at the specified $\alpha$ level.

3. **The results can be presented using a compact letter display (CLD)**, where groups that are not significantly different from each other are assigned the same letter.

In summary, Tukey's HSD is a powerful multiple comparison test that helps identify which specific pairs of means are significantly different after a significant ANOVA result. It controls the FWER and provides a clear interpretation of the relationships among the groups.

Citations:

[1] https://www.youtube.com/watch?v=NeNWMIU9gWw

[2] https://stats.libretexts.org/Bookshelves/Introductory_Statistics/Inferential_Statistics_and_Probability_-_A_Holistic_Approach_%28Geraghty%29/13:_One_Factor_Analysis_of_Variance_%28ANOVA%29/13.06:_Posthoc_Analysis__Tukeys_Honestly_Significant_Difference_%28HSD%29_Test85

[3] https://en.wikipedia.org/wiki/Tukey%27s_range_test

[4] https://arc.lib.montana.edu/book/statistics-with-r-textbook/item/59

[5] https://real-statistics.com/one-way-analysis-of-variance-anova/unplanned-comparisons/tukey-hsd/

[6] https://www.isixsigma.com/dictionary/tukeys-1-way-anova/

[7] https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient

[8] https://www.raybiotech.com/learning-center/t-test-anova/