Statistics Cheat Sheet

1 Describe the middle & spread

Skewed data (income, prices)? Report the median, not the mean.

2 The normal distribution

The bell curve. Many averages tend toward it. Under it:

Z-score = how many σ a value is from the mean: z = (x − mean) / σ. A z beyond ±2 is unusual.

3 Population vs sample

• Population — everyone/everything you care about.
• Sample — the slice you actually measured.
• Statistics estimate population truths from samples.
• Random sampling is what makes the estimate fair — a biased sample can't be fixed by size.

A big sample that's badly drawn beats a small one at being confidently wrong.

4 Central Limit Theorem

Take repeated samples and average each: those averages form a normal distribution — even when the raw data isn't normal.

Why it matters: it's what lets us put confidence intervals and tests around a sample mean. Standard error of the mean:

SE = σ / √n

Quadruple the sample → halve the error. Precision gets expensive.

5 Confidence intervals

A range that likely contains the true value. A 95% CI for a mean:

mean ± 1.96 × SE

"95% confident the true value is in [a, b]" means: if we repeated the study many times, ~95% of such intervals would contain the truth.

Prefer a CI to a bare average — it shows the uncertainty, not just the point.

6 Hypothesis testing

• H₀ (null) — nothing's going on (no difference).
• H₁ (alternative) — there is an effect.
• Compute a p-value: the chance of data this extreme if H₀ were true.
• p < 0.05 → reject H₀ (conventional, not sacred).

p is not "probability H₀ is true", and it says nothing about effect size.

7 Which test?

Report the effect size and CI alongside the p-value — "how big" matters more than "is it significant".

8 Correlation ≠ causation

• r runs −1 → +1; 0 = no linear link.
• A strong r can be coincidence, or driven by a hidden confounder.
• Only a controlled experiment (randomised A/B) licenses "X caused Y".
• Correlation is linear-only — check a scatter plot for curves.

Ice-cream sales and drownings correlate. The cause of both is summer.

9 A/B tests, briefly

• Randomise who sees A vs B — that's the whole trick.
• Fix the sample size in advance (a power calculation).
• Don't peek and stop the moment it's significant — that manufactures false wins.
• Pick one primary metric before you start.
• Practical > statistical: a "significant" 0.1% lift may not be worth shipping.

10 Traps to dodge