Exam PMultivariate probabilityFree to read

Covariance, correlation and sums

Covariance measures joint movement, correlation rescales it to [−1, 1], and the variance of a sum is where both actually get used.

The formulas

Covariance

$$\mathrm{Cov}(X,Y)=E[XY]-E[X]E[Y]$$

Correlation

$$\rho =\frac{\mathrm{Cov}(X,Y)}{{\sigma }_X{\sigma }_Y},-1\le \rho \le 1$$

Variance of a sum

$$\mathrm{Var}(aX+bY)=a^2{\sigma }_X^2+b^2{\sigma }_Y^2+2ab\mathrm{Cov}(X,Y)$$

Bilinearity

$$\mathrm{Cov}(aX+b,cY+d)=ac\mathrm{Cov}(X,Y)$$

Independence

$$X\perp Y\Rightarrow \mathrm{Cov}(X,Y)=0$$

the converse is false

Where it comes from

1. Expanding $E[(X-{\mu }_X)(Y-{\mu }_Y)]$ and using linearity gives $E[XY]-{\mu }_X{\mu }_Y$.
2. Dividing by ${\sigma }_X{\sigma }_Y$ makes the measure scale-free, and Cauchy-Schwarz forces $|\rho |\le 1$.
3. $\mathrm{Var}(X+Y)=\mathrm{Cov}(X+Y,X+Y)$, and bilinearity expands it into the three-term formula.

Worked example

This answer is recomputed from the site’s own interest-theory and probability functions every time the test suite runs, so the page and the mathematics cannot drift apart.

Memory hooks

• For a DIFFERENCE the cross term is −2ab·Cov. Subtracting flips the sign, not the variance terms.
• Zero correlation does not mean independent — the standard counterexample is Y = X² with X symmetric about 0.

Traps

• Dividing the covariance by the variances instead of the standard deviations.
• Dropping the covariance term when the variables are merely uncorrelated in the question's wording but not stated independent.

Related

• Joint, marginal and conditional distributions
• Expectation, variance and moments
• Sums, the central limit theorem and normal approximation

Drill this: the Exam P question bank has original questions on this topic, and today’s free round is open to everyone.