Independent Events: Testing for Independence

Independence means the information is useless

Two events are independent when learning that one happened tells you nothing about the other.

Flip a coin and roll a die. Knowing the coin landed heads does not make a $6$ any more or less likely:

$$P(\text{six} \mid \text{heads}) = \frac{1}{6} = P(\text{six})$$

That equation is the definition written in symbols:

$$A \text{ and } B \text{ are independent} \iff P(A \mid B) = P(A)$$

The conditional probability equals the plain one, because the condition changed nothing.

The multiplication test

Put $P(A \mid B) = P(A)$ into the multiplication rule:

$$P(A \text{ and } B) = P(B) \times P(A \mid B) = P(B) \times P(A)$$

So for independent events:

$$P(A \text{ and } B) = P(A) \times P(B)$$

The argument also runs backwards: if $P(A \text{ and } B) = P(A)P(B)$, dividing by $P(B)$ gives $P(A \mid B) = P(A)$. The two tests are the same fact, so checking either one is enough.

Test	Independent when
conditional	$P(A \mid B) = P(A)$
multiplication	$P(A \text{ and } B) = P(A) \times P(B)$

Deciding from a two-way table

$200$ students: are playing a sport and being in the band independent?

	Band	No band	Total
Sport	$18$	$42$	$60$
No sport	$32$	$108$	$140$
Total	$50$	$150$	$200$

Conditional test.

$$P(\text{band}) = \frac{50}{200} = 0.25 \qquad P(\text{band} \mid \text{sport}) = \frac{18}{60} = 0.30$$

Multiplication test.

$$P(\text{sport}) \times P(\text{band}) = 0.30 \times 0.25 = 0.075 \qquad P(\text{sport and band}) = \frac{18}{200} = 0.09$$

Both tests disagree with independence, as they must — they are the same test. Sport players are more likely to be in the band than students overall, so the events are dependent.

Here is a table where they would be independent:

	Band	No band	Total
Sport	$15$	$45$	$60$
No sport	$35$	$105$	$140$
Total	$50$	$150$	$200$

$P(\text{band} \mid \text{sport}) = \tfrac{15}{60} = 0.25$, exactly $P(\text{band})$. The two rows split in the same proportion, and matching proportions across the rows is what independence looks like in a table.

Why independent is not mutually exclusive

The two words sound related and mean nearly opposite things.

	Independent	Mutually exclusive
can both happen?	yes	no
$P(A \text{ and } B)$	$P(A) \times P(B)$	$0$
does knowing one change the other?	no	yes — to $0$

If $A$ and $B$ are mutually exclusive and both possible, then learning that $A$ happened makes $B$ impossible. That is the strongest dependence there is.

One card: “a heart” and “a spade” are mutually exclusive.

$$P(\text{spade} \mid \text{heart}) = 0 \ne \frac{1}{4} = P(\text{spade})$$

So they are dependent. Meanwhile “a heart” and “a king” overlap and are independent:

$$P(\text{king} \mid \text{heart}) = \frac{1}{13} = \frac{4}{52} = P(\text{king})$$

Independence in real data

Probabilities computed from a sample rarely match to the last decimal, even when two things have nothing to do with each other. A difference of $0.25$ against $0.26$ in a survey is usually read as close to independent; a difference of $0.25$ against $0.45$ is not.

Deciding how big a difference has to be before it means something is the work of statistical inference. The tests here say exactly what independence would look like, and the data says how close it comes.

Worked examples

Common mistakes

Practice problems