Exam PApplicationsFree to read

Mixtures and the law of total probability in practice

A population made of sub-populations is a mixture: probabilities average with the mixing weights, but variances do not.

The formulas

Mixed density

$$f(x)=\sum _k{w}_k{f}_k(x),\sum _k{w}_k=1$$

Mixed mean

$$E[X]=\sum _k{w}_k{\mu }_k$$

Mixed second moment

$$E[X^2]=\sum _k{w}_k\left({\sigma }_k^2+{\mu }_k^2\right)$$

Variance

$$\mathrm{Var}(X)=\sum _k{w}_k\left({\sigma }_k^2+{\mu }_k^2\right)-{\left(\sum _k{w}_k{\mu }_k\right)}^2$$

Where it comes from

1. A mixture is a two-stage experiment: pick the sub-population, then draw from it — which is exactly the setting of the law of total probability.
2. Expectation is linear, so means mix linearly; the SQUARE of the mean does not, which is why the variance picks up a between-group term.
3. This is the conditional variance decomposition written for a discrete mixing variable.

Worked example

This answer is recomputed from the site’s own interest-theory and probability functions every time the test suite runs, so the page and the mathematics cannot drift apart.

Memory hooks

• Means mix; variances do not. The extra term is the spread of the group means themselves.
• A mixture of exponentials is NOT exponential — its tail is heavier than any of its components.

Traps

• Averaging the component variances and calling that the mixture's variance.
• Mixing densities but forgetting the weights must sum to 1.

Related

• Bayes' theorem and the law of total probability
• Conditional expectation and the variance decomposition
• Expectation, variance and moments

Drill this: the Exam P question bank has original questions on this topic, and today’s free round is open to everyone.