Lecture Notes: Chapter 16 – Analysis of Variance (ANOVA)

Introduction to ANOVA

Analysis of Variance (ANOVA) is a statistical method used to compare the means of three or more groups simultaneously. While a t-test can compare two means, performing multiple t-tests to compare many groups increases the risk of making a Type I error (incorrectly rejecting the null hypothesis). ANOVA uses an F-test to determine if there is a significant difference between the means of the groups being compared by examining the ratio of variability between groups to variability within groups.

Section 1: The F-Distribution

The F-distribution is a probability distribution used in ANOVA and other statistical tests. Use the sliders below to explore how different degrees of freedom affect the shape of the distribution:

Section 2: One-Way ANOVA Framework

One-Way ANOVA is used when you have one categorical independent variable (factor) with three or more levels (groups) and one quantitative dependent variable. The test determines if there is a statistically significant difference between the means of these groups.

Assumptions for One-Way ANOVA:

For the results of a one-way ANOVA to be valid, the following assumptions should ideally be met:

• Independence: The samples are simple random samples, and observations within and between groups are independent. This is crucial and often depends on the study design.
• Normality: The population from which each group sample is drawn is approximately normally distributed. This assumption is less critical with larger sample sizes due to the Central Limit Theorem.
• Homogeneity of Variances: The populations from which the samples are drawn have equal variances (or equal standard deviations). This is also known as homoscedasticity. This can be formally tested using tests like Levene's test. If this assumption is violated, alternative procedures like Welch's ANOVA or nonparametric tests may be more appropriate.

ANOVA Identity:

The total variation in the data (SST) can be partitioned into the variation between groups (SSTR) and the variation within groups (SSE). The fundamental equation of ANOVA is:

• \(SST\): Total Sum of Squares - measures the total variability in the data.
• \(SSTR\): Sum of Squares due to Treatment (or Between-Groups) - measures the variability between the means of the different groups.
• \(SSE\): Sum of Squares due to Error (or Within-Groups) - measures the variability within each group. This represents the random error.

Section 3: ANOVA Procedure Step-by-Step

The procedure involves calculating the sum of squares, degrees of freedom, mean squares, and finally the F-statistic, which are typically summarized in an ANOVA table.

ANOVA Table Format:

Where:

• k = number of groups
• N = total sample size
• SSTR = between-groups sum of squares
• SSE = within-groups sum of squares
• SST = total sum of squares
• MSTR = between-groups mean square
• MSE = within-groups mean square (error)

Section 4: Multiple Comparisons

If the One-Way ANOVA test results in rejecting the null hypothesis \(H_0\) (i.e., you conclude that at least one group mean is different), you typically want to know *which* specific pairs of group means are different. Performing simple t-tests between all pairs of groups increases the family-wise error rate (the probability of making at least one Type I error among all comparisons). Multiple comparison procedures are designed to control this error rate.

Tukey's Honestly Significant Difference (HSD) test is a common method for pairwise comparisons when you have equal (or nearly equal) sample sizes per group and the equal variance assumption holds. It is based on the studentized range distribution.

Tukey’s Procedure Steps:

1. Choose a family confidence level \(1 - \alpha\). This is the probability that all confidence intervals constructed for all pairwise comparisons contain the true difference. The typical \(\alpha\) from the ANOVA is often used here.
2. Find the studentized range critical value \(q_{\alpha,\,k,\,n-k}\) from the studentized range table (or using software) with \(\alpha\), number of groups \(k\), and error degrees of freedom \(n-k\).
3. For each pair of groups \(i\) and \(j\) (where \(i \ne j\)), compute the confidence interval for the difference between their population means (\(\mu_i - \mu_j\)). The confidence interval is given by:
   \[ (\bar{x}_i - \bar{x}_j) \pm \dfrac{q_{\alpha,\,k,\,n-k}}{\sqrt{2}} \sqrt{MSE \left(\dfrac{1}{n_i} + \dfrac{1}{n_j}\right)} \] (Note: An equivalent and common form when sample sizes are equal, \(n_i = n_j = n'\), is \((\bar{x}_i - \bar{x}_j) \pm q_{\alpha,\,k,\,n-k} \sqrt{\dfrac{MSE}{n'}}\). The first formula is more general for unequal \(n_i\).)
4. If the confidence interval for a pair of means does not contain 0, you declare that the population means \(\mu_i\) and \(\mu_j\) are significantly different at the chosen family confidence level. If the interval contains 0, you do not have sufficient evidence to conclude they are different.

Section 5: The Kruskal–Wallis Test

The Kruskal–Wallis H test is a nonparametric alternative to the One-Way ANOVA. It is used when the assumptions for ANOVA, particularly normality or homogeneity of variances, are not met. It tests whether samples originate from the same distribution, which is often interpreted as testing for differences in medians or locations when the shape of the distributions is similar across groups.

Assumptions for the Kruskal–Wallis Test:

• The samples are independent random samples.
• The distributions of the variable in the populations have the same shape (though they may differ in location/median).
• Sample sizes for each group are generally recommended to be 5 or more for the chi-square approximation to be reliable.

Test statistic:

The test involves ranking all observations from all groups together. The Kruskal-Wallis test statistic, \(H\) (often denoted \(K\)), is calculated based on the sum of the ranks for each group:

Where \(k\) is the number of groups, \(n_i\) is the sample size of the \(i\)-th group, \(R_i\) is the sum of the ranks for the \(i\)-th group, and \(n = \sum n_i\) is the total sample size.

For reasonably large sample sizes (\(n_i \ge 5\)), the distribution of the \(H\) statistic can be approximated by a chi-squared distribution (\(\chi^2\)) with \(k-1\) degrees of freedom.

Procedure using the Kruskal–Wallis Test:

1. State hypotheses:
   \(H_0:\) The distributions of the variable are the same for all groups.
   \(H_a:\) At least one group's distribution is different from the others (often interpreted as different medians/locations).
2. Select significance level \(\alpha\).
3. Combine all data and rank them from smallest (1) to largest (n). If there are ties, assign the average rank to the tied observations.
4. Calculate the sum of ranks (\(R_i\)) for each group.
5. Compute the \(H\) test statistic using the formula above.
6. Find the critical value \(\chi^2_{\alpha,\,k-1}\) from the chi-squared distribution table (or using software) with \(k-1\) degrees of freedom. Or, compute the p-value associated with the calculated \(H\) statistic.
7. Compare \(H\) to the critical value (or compare p-value to \(\alpha\)) and conclude. If \(H > \chi^2_{\alpha,\,k-1}\) (or p-value \(< \alpha\)), reject \(H_0\).

Section 6: Real-World Examples with R

Effect Size Calculations

Start → Are there outliers? → Yes → Consider robust methods or nonparametric tests
                           → No  → Check normality
                                     → Normal → Check variance equality
                                                 → Equal → Use One-Way ANOVA
                                                 → Unequal → Use Welch's ANOVA
                                     → Not Normal → Sample size > 30?
                                                    → Yes → Use One-Way ANOVA
                                                    → No → Use Kruskal-Wallis
    

Worked Example: One-Way ANOVA

Practice Problems with R

Conceptual Understanding Check