Random Variables and Expected Value

A number for every outcome

Flip three coins. The outcomes are sequences like HHT or TTT, eight of them, all equally likely. Often the question is not which sequence came up but a number built from it: how many heads?

A random variable is that kind of rule. It assigns a number to every outcome of a chance process. Here $X$ is the number of heads:

Outcomes	$X$
TTT	$0$
HTT, THT, TTH	$1$
HHT, HTH, THH	$2$
HHH	$3$

The probability distribution

List each value of $X$ with its probability and the result is a probability distribution:

$x$	$0$	$1$	$2$	$3$
$P(X = x)$	$\tfrac{1}{8}$	$\tfrac{3}{8}$	$\tfrac{3}{8}$	$\tfrac{1}{8}$

The probabilities add to $1$, because every outcome lands on exactly one value.

A distribution can be graphed the same way as data. Take a five-question multiple-choice quiz with four choices per question, answered by pure guessing. There are $4^5 = 1{,}024$ equally likely ways to fill in the answers, and each has some number correct:

Each bar divided by $1{,}024$ is a probability. Getting exactly one right is the most likely result, at $\tfrac{405}{1024} \approx 0.40$, and a perfect paper turns up once in $1{,}024$ guesses.

Those counts come from combinations: $k$ correct answers can fall on $\binom{5}{k}$ sets of questions, and each of the other questions has $3$ wrong choices, so the count is $\binom{5}{k}\,3^{5-k}$.

Expected value

The expected value of a random variable weights each value by its probability and adds:

$$E[X] = \sum x \cdot P(X = x)$$

For three coins:

$$E[X] = 0\cdot\tfrac{1}{8} + 1\cdot\tfrac{3}{8} + 2\cdot\tfrac{3}{8} + 3\cdot\tfrac{1}{8} = \tfrac{12}{8} = 1.5$$

Why the expected value is a long-run average

No one ever flips three coins and gets $1.5$ heads. So what does $1.5$ mean?

Repeat the three flips many times. In the long run, each value turns up in about the share of trials its probability predicts: no heads about $\tfrac{1}{8}$ of the time, one head about $\tfrac{3}{8}$ of the time, and so on. The average of all the results is then about

$$0\cdot\tfrac{1}{8} + 1\cdot\tfrac{3}{8} + 2\cdot\tfrac{3}{8} + 3\cdot\tfrac{1}{8}$$

which is the expected value. The expected value is the mean of the distribution: the number the average of many results settles toward. It is computed exactly the way a mean is computed from a frequency table, with probabilities in place of frequencies.

Expected scores under different grading

The same guessing quiz, scored three ways, gives three expected scores per question:

Scoring	Expected points per guessed question
$+1$ right, $0$ wrong	$\tfrac{1}{4}(1) = 0.25$
$+1$ right, $-\tfrac{1}{4}$ wrong	$\tfrac{1}{4}(1) + \tfrac{3}{4}\left(-\tfrac{1}{4}\right) = 0.0625$
$+1$ right, $-\tfrac{1}{3}$ wrong	$\tfrac{1}{4}(1) + \tfrac{3}{4}\left(-\tfrac{1}{3}\right) = 0$

With the one-third penalty, blind guessing gains nothing on average. That penalty was chosen for exactly that reason. Scoring rules are designed around expected values.

Probabilities from data

Some distributions come from observation instead of theory. Suppose a survey of $200$ households records how many televisions each one has:

Televisions	$0$	$1$	$2$	$3$	$4$
Households	$8$	$50$	$72$	$44$	$26$
Probability	$0.04$	$0.25$	$0.36$	$0.22$	$0.13$

Each probability is a count divided by $200$. The expected number of televisions for a household chosen at random from this group is

$$0(0.04) + 1(0.25) + 2(0.36) + 3(0.22) + 4(0.13) = 2.15$$

These are empirical probabilities, and they describe the households surveyed. How well they describe a whole town depends on how the sample was chosen, which is the subject of surveys and experiments.

Worked examples

Common mistakes

Practice problems