Variance measure “how far are the samples from average, on average.” It can be defined in term of expectation:

Definition

Given a random variable ,

The standard deviation of a distribution is the positive square root of its variance.

Notations

Here are various notations for standard deviations

  • or for the standard deviation of a distribution
  • (sigma) for population standard deviation
  • for sample standard deviation

and standard deviations:

  • and for population and sample variance, respectively

An Equivalent Definition of Variance

An equivalent way to define variance is the following equation

Properties of variance

Population Variance and Sample Variance

In statistics, the variance of a population of size is defined as

where

However, we often only have samples. Given a sample data size , the sample variance is

where

  • is the sample mean

Note the key difference in the denominator: for the sample variance versus for the population variance. This distinction is called the Bessel’s correction.

Variance for Discrete Random Variables

We can generalize the above definition into a discrete random variable with weight. For a discrete random variable in sample space with a probability mass function , the variance is

where the expected value can be seen as the generalization of population mean (but weighted by ):

Note that we still need to apply the Bessel’s correction when we sample from the random variable (e.g. in Monte Carlo methods).

Variance for Continuous Random Variables

Given a continuous random variable in sample space with a probability density function , the variance can be obtained in two ways:

where

See Also