Statistics › 🧪 Collecting Data

Observational Studies vs Experiments

Before you trust a claim about cause and effect, ask how the data was produced.

Intro StatisticsCollege intro statistics level
Observational Studies vs Experiments — illustration
Illustrative image (AI-generated).
💡
The big idea: Before a single number is computed, the way data was produced already decides what conclusions are allowed. An observational study watches subjects as they naturally are; an experiment deliberately imposes a treatment, usually through random assignment. That one design choice — was there random assignment or not — is what separates 'these are associated' from 'this causes that'. Skipping this question is the single most common way real-world data claims go wrong.
🎯 By the end, you'll be able to
  • Classify a study as observational or experimental from a description of its design
  • Identify the explanatory variable, response variable, and treatment in a study
  • Distinguish a lurking variable from a confounding variable
  • Explain why an observational study, no matter how large, cannot establish causation
  • Recognize that 'association is not causation' is a statement about study design, not about the size of a correlation
📎 You should already know
  • Population, sample, and variable (basic vocabulary)
  • Categorical vs quantitative variables

Two ways data get produced

Before you compute a single number, ask how the data was produced. There are exactly two designs to choose between.

In an observational study, researchers watch subjects and record what they see. Nobody is told to exercise, take a pill, or change anything about their life — the researcher simply measures people (or animals, or plots of land) as they already are.

In an experiment, researchers deliberately impose a treatment on subjects, usually by randomly assigning who gets which treatment, and then measure the result. The deliberate imposition — not just the measuring — is what makes it an experiment.

Example: comparing the heart health of people who already exercise regularly to people who do not is observational. Randomly assigning volunteers to a 12-week exercise program or to no program, then comparing heart health, is an experiment.

🔑 Explanatory variable, response variable, treatment

Every study has a variable thought to explain differences in another:

  • The explanatory variable is the one believed to influence or explain the outcome (exercise habit).
  • The response variable is the outcome being measured (a heart-health score).
  • In an experiment, the specific condition imposed on a subject is called the treatment — the 12-week exercise program is one treatment, and 'no program' is also a treatment: the control condition.
⚠️ Wrong: 'This observational study showed X causes Y.'

Wrong: 'This observational study showed that regular exercise causes better heart health.'

Right: The study shows exercise and heart health are associated. It cannot establish causation, because the groups were not randomly assigned — people who already exercise may differ in dozens of other ways (diet, income, age, access to healthcare) from people who don't. Any of those could be the real reason for the difference. A study without random assignment can show association; it cannot show cause.

⚠️ Wrong: 'A large enough sample proves causation.'

Wrong: 'With a large enough sample, an observational study can prove causation.'

Right: No sample size rescues an observational study from confounding. A bigger n makes the estimate of the association more precise — it does nothing to fix the underlying design problem. Only random assignment licenses a causal claim; n does not. A confounded study with a million subjects is still confounded.

Lurking variables and confounding variables

Both terms describe an outside variable the researcher did not build into the comparison — but they are not interchangeable.

A lurking variable is any variable, outside the ones being studied, that could influence the result. It is a broad, cautionary label: something you have not measured or accounted for that might matter.

A lurking variable becomes a confounding variable specifically when it is associated with both the explanatory variable and the response variable — so its effect on the response cannot be separated from the treatment's effect. Confounding is the specific, diagnosable failure mode; 'lurking' is the general warning sign that should make you go looking for one.

⚠️ Wrong: 'A confounding variable and a lurking variable are the same thing.'

Wrong: 'A confounding variable and a lurking variable are the same thing.'

Right: A lurking variable is any outside variable not accounted for. It becomes a confounder specifically when it is associated with both the explanatory variable and the response — at that point its effect and the treatment's effect are tangled together and cannot be pulled apart statistically.

📝 Worked example: People who take a daily vitamin supplement are observed to be healthier, on average, than people who do not. A headline claims: 'Vitamin supplements improve health.' Identify the explanatory variable, the response variable, a plausible confounder, and say what can and cannot be concluded.
  1. Explanatory variable: whether a person takes a daily vitamin supplement (yes/no).
  2. Response variable: a measured health outcome (for example, rate of illness or a general health index).
  3. This is an observational study: nobody was randomly assigned to take vitamins or not — people self-selected into each group.
  4. Plausible confounder: general health-consciousness. People who choose to take supplements are, on average, also more likely to exercise, eat well, sleep enough, and see a doctor regularly.
  5. Health-consciousness is associated with BOTH the explanatory variable (supplement use) and the response (health outcome) — exactly the definition of a confounder. Its effect cannot be separated from any effect of the vitamins themselves.
✓ We can say vitamin use and health are associated. We cannot say vitamins cause better health, because health-consciousness is a plausible confounder that was never controlled. This exact pitfall reappears later in the course's regression module, where an uncontrolled confounder can make an unrelated predictor look statistically significant — there it is called a regression pitfall.
⚠️ Wrong: '"Association is not causation" is about the correlation number.'

Wrong: "'Association is not causation' is a statement about the correlation number — once it's high enough, causation follows."

Right: It is a statement about study design, not arithmetic. A correlation of 0.99 computed from an observational study still cannot prove causation, because no correlation coefficient — however large — can rule out a confounder. Only random assignment does that.

Why design, not math, settles causation

No formula fixes a design problem after the fact. Whether a causal claim is available was decided the moment the study was planned — specifically, the moment someone did or did not randomly assign subjects to treatments.

This is why the first question to ask about any headline claim is not 'how big was the sample?' or 'how strong was the correlation?' but 'was this observational or experimental?' The next lesson, Designing Experiments, covers how good experiments are built — control, randomization, replication, and blocking — so that a causal claim becomes defensible.

Check your understanding

1. A news story says: 'Study: people who take a daily multivitamin live longer.' The study observed people's existing habits without assigning who took vitamins. What can we correctly conclude?
Without random assignment, the study can only show association. A lurking variable, like general health-consciousness, could be the real explanation.
2. In an experiment testing whether a new fertilizer increases crop yield, what is the response variable?
Crop yield is the outcome being measured. Fertilizer type is the explanatory variable (and the treatment being imposed).
3. Which best distinguishes a lurking variable from a confounding variable?
A confounder is a specific, diagnosable kind of lurking variable: one tangled up with both the explanatory variable and the response.
4. A researcher reports a correlation of 0.99 between ice cream sales and drowning deaths, from an observational study, and claims ice cream sales cause drownings. What is the best critique?
No correlation value, however large, can rule out a confounder in an observational study. The problem is the design, not the size of the number.
✅ Key takeaways
  • An observational study measures subjects as they are; an experiment deliberately imposes a treatment, usually via random assignment.
  • The explanatory variable is believed to influence the response variable; the specific condition imposed in an experiment is the treatment.
  • A lurking variable is any unaccounted-for outside variable; it becomes a confounding variable when it's associated with both the explanatory and response variables.
  • No sample size rescues an observational study from confounding — only random assignment licenses a causal claim.
  • 'Association is not causation' is a fact about study design, not about how large a correlation is.
  • The vitamin-supplement / health-consciousness confound is a template case; the same pitfall reappears in regression as an uncontrolled confounder.