Margin of Error: How Close Is an Estimate?

The same question, two different answers

Two pollsters ask the same question of the same population on the same day. One gets $52\%$, the other $49\%$. Neither made a mistake.

They measured different people. A sample is part of a population, and a different part gives a different number. The estimate moves, and the useful question is how far.

That question has an answer, and it can be watched happening.

Watching a thousand samples

Suppose a district really does have $60\%$ of its students walking or biking to school. That number is known here, which never happens in practice — it is known because this is a simulation, built so the scatter can be seen against a target.

Draw a random sample of $50$ students, record the percentage who walk, and repeat a thousand times.

Three things in that picture matter.

It is centered on the truth. The thousand sample percentages average $60.1\%$. Random sampling has no tilt, so the estimates miss high about as often as they miss low.

It is symmetric and mound-shaped. The pile has the normal shape, which is why the rules from that lesson apply here.

It is wide. The lowest sample came in at $38\%$ and the highest at $80\%$. A single sample of $50$ can land a long way from $60$.

Within how many points of $60$	Samples out of $1000$
$2$	$325$
$4$	$527$
$6$	$680$
$8$	$806$
$10$	$884$
$14$	$970$

Fourteen points catches $97\%$ of them. That number is about to get a name.

Why the spread shrinks like the square root of n

The scatter of those thousand percentages has a standard deviation of $0.070$. It is not an accident, and it does not need a simulation to find:

$$\text{SD of } \hat{p} = \sqrt{\frac{p(1-p)}{n}}$$ $$\sqrt{\frac{0.60 \times 0.40}{50}} = \sqrt{0.0048} = 0.069$$

The simulation gave $0.070$. The formula gives $0.069$.

The $n$ sits under a square root, and that single detail drives everything else. Doubling the sample does not halve the spread — it divides it by $\sqrt{2}$, about $1.41$. To halve the spread you need four times the data.

Run the same simulation with samples of $200$ and the pile tightens exactly that much:

Four times the sample, half the spread: the standard deviation drops from $0.070$ to $0.036$, and the extremes shrink from $38$–$80$ to $46$–$72$.

Margin of error

In a mound-shaped pile, about $95\%$ of the values sit within two standard deviations of the center. That distance is the margin of error.

$$\text{margin of error} = 2\sqrt{\frac{p(1-p)}{n}}$$

For the district, $2 \times 0.069 = 0.139$, or about $14$ percentage points — which is the $14$ that caught $97\%$ of the simulated samples.

In practice $p$ is unknown, so the sample’s own $\hat{p}$ goes in its place. The result is reported as an interval:

$$\hat{p} \pm \text{margin of error}$$

The quick rule

The margin is largest when $p = 0.5$, and there the formula collapses:

$$2\sqrt{\frac{0.5 \times 0.5}{n}} = 2 \cdot \frac{0.5}{\sqrt{n}} = \frac{1}{\sqrt{n}}$$

So $\tfrac{1}{\sqrt{n}}$ is both a shortcut and a worst case. It needs no knowledge of $p$ at all.

Sample size	$\tfrac{1}{\sqrt{n}}$
$50$	$14.1\%$
$100$	$10.0\%$
$200$	$7.1\%$
$400$	$5.0\%$
$1{,}000$	$3.2\%$
$2{,}000$	$2.2\%$

The $1{,}000$ row is why national polls report “plus or minus three points” so often. It is the sample size that buys that number.

The population size is absent from every formula on this page. A sample of $1{,}000$ measures a town of $20{,}000$ and a country of $300$ million with the same precision, which surprises nearly everyone the first time.

Is a claimed model consistent with the data?

The same machinery answers a different question: somebody claims a value, and data arrives. Does the data fit the claim?

A coin is claimed to be fair. It is flipped $100$ times and lands heads $62$ times.

Start by assuming the claim is true and asking what $100$ flips of a fair coin look like.

$$\text{SD} = \sqrt{\frac{0.5 \times 0.5}{100}} = 0.05$$

So fair coins produce head rates piled around $50\%$ with a standard deviation of $5$ points, and about $95\%$ of them land between $40\%$ and $60\%$.

The observed $62\%$ is $2.4$ standard deviations above $50\%$ — outside that range. Computing it exactly, a fair coin gives $62$ or more heads about $1.0\%$ of the time, and lands that far off in either direction about $2.1\%$ of the time.

A result that about one fair coin in fifty would produce is evidence against the claim. It is not proof; one coin in fifty is not zero.

Now change the data. Suppose the coin had landed heads $55$ times.

$$\frac{0.55 - 0.50}{0.05} = 1.0 \text{ standard deviation}$$

A fair coin does that or better about $18\%$ of the time, and misses by that much in either direction about $37\%$ of the time. That is an ordinary result, and it is consistent with a fair coin.

Heads in $100$	Distance from $50\%$	How often a fair coin does this	Verdict
$55$	$1.0$ SD	about $37\%$ of the time	consistent
$62$	$2.4$ SD	about $2\%$ of the time	evidence against

Worked examples

Common mistakes

Practice problems