Step 01 · Distributions
Moments and Expectation
A distribution is a whole picture, but a reader often needs only a few numbers from it. Moments are those numbers, arranged in a family, beginning with the centre and continuing outward through spread, tilt and weight.
Expectation as a weighted centre
Expectation is the long-run average of a quantity, computed by weighting each possible value by the density attached to it. In words, it is the balance point of the picture. If you cut the curve out of stiff card and balanced it on a knife edge, it would rest at the expectation. This physical reading keeps the symbol honest: the expectation is not a value the quantity must ever take, but the place where the whole mass would settle.
Expectation is linear, and that single property does enormous work. The expectation of a sum is the sum of the expectations, whether or not the quantities are independent. The expectation of a constant times a quantity is the constant times the expectation. Nothing else about expectation is this dependable, and much of the early algebra of statistics is simply the careful use of these two rules.
Variance and standard deviation
The second moment about the centre is the variance, and its square root is the standard deviation. Variance measures the average squared distance from the expectation, so it punishes large departures heavily and small ones lightly. The standard deviation returns the measure to the original units, which is why it, and not the variance, is the number usually quoted beside a mean. Adding a constant to every value leaves the variance untouched; multiplying every value by a constant multiplies the variance by the square of that constant.
| Moment | Name | What it reports |
|---|---|---|
| first | mean | where the mass balances |
| second (central) | variance | how far the mass spreads |
| third (standardised) | skewness | which tail is longer |
| fourth (standardised) | kurtosis | how heavy the tails are |
Tilt and weight in the tails
Skewness reports asymmetry. A positive skewness says the right tail is the longer one; a negative skewness says the left. Kurtosis compares the weight of the tails with the weight the normal curve would carry, using the value three as its benchmark. A curve with kurtosis above three is called heavy-tailed, and heavy tails are the reason a record can surprise a reader who trusted the normal law too literally.
A worked contrast
Two samples may share a mean and a variance and still differ sharply in their third and fourth moments. One may lean to the right with a long thin tail; the other may be nearly symmetric but carry more weight far from the centre than the curve predicts. Reading only the first two moments hides both differences, which is why the compendium introduces all four before the material grows harder.
Moment generating ideas
The moments of a distribution can be collected in a single auxiliary object whose derivatives at the origin reproduce them in turn. The first derivative gives the mean, the second gives the second moment, and so on. This device is not needed for arithmetic, but it shows that the moments are not a loose crowd; they are the unfolding of one compact statement about the distribution. When the object exists, it identifies the distribution as surely as a fingerprint identifies a hand.
Where this leads
Moments prepare the ground for the rest of the compendium. The covariance matrix of the second step is a square arrangement of second moments, and the spectral density of the fourth step can be read as the distribution of variance across frequencies. Learn to move between a picture and its moments, and the later material will feel like a change of language rather than a change of subject.