Statistics › 🔬 Confidence & Inference

t-Tests and Comparing Means

When you don't know the true spread, the t-distribution keeps your conclusions honest.

Intro StatisticsAP Statistics level

💡

The big idea: In real data you rarely know the population's true standard deviation σ, so you estimate it with the sample's s. That single substitution adds uncertainty — and to stay honest about it, the standardized statistic follows the t-distribution instead of the normal: the same bell shape, but with heavier tails that shrink back toward the normal as your sample grows. t-tests use this idea to compare a sample mean to a fixed value, or to compare the means of two groups.

🎯 By the end, you'll be able to

Explain why an unknown population σ forces us to use the t-distribution instead of the normal
Describe how degrees of freedom control the heaviness of the t-distribution's tails
Compute and interpret a one-sample t statistic, t = (x̄ − μ₀)/(s/√n)
Set up a two-sample t-test to compare the means of two groups
State what a p-value does and does not tell you

📎 You should already know

Sampling distributions and standard error
The normal distribution and z-scores
The basic logic of a hypothesis test

You almost never know σ

When we first standardized a value, we divided by the population standard deviation \(\sigma\). But here is the awkward truth about real data: you almost never know \(\sigma\). To know it you would already have had to measure the whole population — and if you had the whole population, you would not be taking a sample in the first place.

So we do the natural thing: we estimate \(\sigma\) from the sample itself, using the sample standard deviation \(s\). That small substitution has a surprisingly large consequence, and it is the whole reason the t-distribution exists.

🔑 Estimating σ costs you something

When you replace the true \(\sigma\) with the sample estimate \(s\), you add a second source of uncertainty: \(s\) itself wobbles from one sample to the next. To stay honest, the standardized statistic no longer follows the normal curve — it follows the slightly wider t-distribution, which has heavier tails that make room for that extra uncertainty.

Degrees of freedom control the shape

The t-distribution is not a single curve but a family, indexed by a number called the degrees of freedom (df). For a one-sample test, \(\text{df} = n - 1\).

With few degrees of freedom (small samples), \(s\) is a shaky estimate of \(\sigma\), so the tails are fat — extreme standardized values are more likely than the normal curve would suggest. As df grows, \(s\) settles down toward the true \(\sigma\), the tails thin out, and the t-curve becomes almost indistinguishable from the standard normal \(z\)-curve. By around df = 30 the two are already very close.

\[ t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}} \]

The one-sample t statistic: how many estimated standard errors the sample mean x̄ sits from the hypothesized mean μ₀. Compare it against a t-distribution with n − 1 degrees of freedom.

Reading the null distribution

A hypothesis test starts by assuming the null hypothesis is true — for a one-sample test, that the population mean really equals \(\mu_0\). Under that assumption the t statistic has a known distribution: the t-curve for its degrees of freedom. That curve is the null distribution.

The simulation below draws that null distribution. Move the degrees of freedom to watch the tails fatten and thin. Move the observed t to a value your data might produce, and the area in the tail beyond it gets shaded — that shaded area is the p-value. Switch between a one-sided and a two-sided test to see how the shading, and therefore the p-value, changes.

🎮 Interactive: the null distribution and its tails LIVE

The bell is the t-distribution the standardized statistic follows when the null hypothesis is true. The vertical line is your observed t; the shaded tail area is the p-value. Lower the degrees of freedom to see the tails grow heavier than the normal; raise it and the curve tightens toward the standard normal.

✨ What the p-value does and does not say

The shaded tail is the p-value: if the null hypothesis were true, it is the chance of getting a t statistic at least as extreme as the one you observed. A small p-value means your data would be surprising under the null. It is not the probability that the null hypothesis is true, and not the probability that you made a mistake — those are different things the test simply cannot hand you. A large t reaches into the thin tail (small p); a t near zero sits under the fat middle (large p).

Comparing two groups

The most common question in practice is not 'does this mean equal a fixed number?' but 'do these two groups differ?' — treatment versus control, method A versus method B. The two-sample t-test answers it by standardizing the difference of the two sample means by its own standard error:

\[ t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}}} \]

The two-sample (Welch) t statistic. The denominator combines the uncertainty of both group means; the degrees of freedom are worked out from the two sample sizes and spreads.

⚠️ Big samples: t and z nearly agree

Because the t-curve slides toward the standard normal as df grows, for large samples a t-test and a z-test give almost the same answer. The t-distribution matters most for small samples, where estimating \(\sigma\) with \(s\) is genuinely shaky. When \(\sigma\) is unknown, reaching for t is the safe habit — it self-corrects toward z automatically as your sample grows.

📝 Worked example: A sample of n = 16 measurements has mean x̄ = 53 and sample standard deviation s = 8. Test the null hypothesis that the population mean is μ₀ = 50, against the two-sided alternative that it is not.

Set up the hypotheses: null \(\mu = 50\) versus the two-sided alternative \(\mu \ne 50\).
Estimated standard error of the mean: \(s/\sqrt{n} = 8/\sqrt{16} = 8/4 = 2\).
Standardize: \(t = (\bar{x} - \mu_0)/(s/\sqrt{n}) = (53 - 50)/2 = 1.5\), with \(\text{df} = n - 1 = 15\).
Compare against the null t-curve: for a two-sided test at the 0.05 level with df = 15, the cutoff is about 2.13. Our t = 1.5 does not reach it — the tail area (the p-value) is roughly 0.15.

✓ t = 1.5 on 15 degrees of freedom. The sample mean sits only about 1.5 estimated standard errors above 50 — inside the fat middle of the null curve, not out in a thin tail — so this is not strong evidence against μ = 50, and we do not reject the null at the 0.05 level.

Check your understanding

1. Why do we use the t-distribution instead of the normal when testing a mean?

Replacing the unknown σ with the sample estimate s adds a second source of uncertainty, so the standardized statistic follows the heavier-tailed t-distribution rather than the normal.

2. As the degrees of freedom increase, the t-distribution…

More degrees of freedom mean s estimates σ more reliably, so the tails thin out and the t-curve converges to the standard normal.

3. A sample of n = 25 has mean x̄ = 20 and sample standard deviation s = 5. Testing the null hypothesis μ₀ = 18, the one-sample t statistic is…

The standard error is s/√n = 5/√25 = 5/5 = 1, so t = (20 − 18)/1 = 2.0.

4. A two-sided t-test returns p = 0.03. Which statement is correct?

A p-value is computed assuming the null is true: it is the chance of a result at least this extreme under the null — not the probability the null is true or false, and not a replication rate.

✅ Key takeaways

You rarely know the population σ, so you estimate it with the sample standard deviation s.
Substituting s for σ adds uncertainty, so the standardized statistic follows the t-distribution — a bell with heavier tails than the normal.
The t-distribution is a family indexed by degrees of freedom (n − 1 for one sample); more df means thinner tails, approaching the standard normal z-curve.
A one-sample t statistic is t = (x̄ − μ₀)/(s/√n); a two-sample t-test standardizes the difference of two group means by its own standard error.
The p-value is the tail area under the null distribution — the chance of a result at least this extreme if the null were true, not the probability the null is true.