Exam PProbability foundationsFree to read

Bayes' theorem and the law of total probability

Reverse a conditional probability: weight each cause by its prior and its likelihood, then normalise. The denominator is always the law of total probability.

The formulas

Law of total probability

$$P(A)=\sum _{k}P(A\mid B_k)P(B_k)$$

Bayes

$$P(B_j\mid A)=\frac{P(A\mid B_j)P(B_j)}{{\sum }_{k}P(A\mid B_k)P(B_k)}$$

Two-event form

$$P(B\mid A)=\frac{P(A\mid B)P(B)}{P(A\mid B)P(B)+P(A\mid B^c)P(B^c)}$$

Odds form

$$\frac{P(B_1\mid A)}{P(B_2\mid A)}=\frac{P(A\mid B_1)}{P(A\mid B_2)}\cdot \frac{P(B_1)}{P(B_2)}$$

Where it comes from

1. The partition $\{B_k\}$ cuts $A$ into disjoint pieces $A\cap B_k$, so $P(A)={\sum }_{k}P(A\cap B_k)={\sum }_{k}P(A\mid B_k)P(B_k)$.
2. Bayes is then just $P(B_j\mid A)=P(A\cap B_j)/P(A)$ with both parts rewritten using the multiplication rule.
3. The odds form drops the shared denominator, which is why it is faster when only a ratio is asked for.

Worked example

This answer is recomputed from the site’s own interest-theory and probability functions every time the test suite runs, so the page and the mathematics cannot drift apart.

Memory hooks

• Numerator is one prior times one likelihood; denominator is the sum of all of them. The posteriors must sum to 1 — check it.
• The denominator IS the law of total probability. Compute it once and reuse it for every class.

Traps

• Forgetting a class in the denominator, so the posteriors do not sum to 1.
• Swapping prior and likelihood — the prior is the class probability, the likelihood is the event probability GIVEN the class.

Related

• Conditional probability and independence
• Probability axioms and set identities

Drill this: the Exam P question bank has original questions on this topic, and today’s free round is open to everyone.