Variance measure “how far are the samples from average, on average.” It can be defined in term of expectation:
Definition
Given a random variable
,
The standard deviation of a distribution is the positive square root of its variance.
Notations
Here are various notations for standard deviations
or for the standard deviation of a distribution (sigma) for population standard deviation for sample standard deviation
and standard deviations:
and for population and sample variance, respectively
An Equivalent Definition of Variance
An equivalent way to define variance is the following equation
Derivation
Properties of variance
- always non-negative (as it is the square of the standard deviation) (
) - variance is not linear (
) - law of total variance (
)
Population Variance and Sample Variance
In statistics, the variance of a population of size
where
is the value of the -th element in the population is the population mean
However, we often only have samples. Given a sample data size
where
is the sample mean
Note the key difference in the denominator:
Variance for Discrete Random Variables
We can generalize the above definition into a discrete random variable with weight. For a discrete random variable
where the expected value
Note that we still need to apply the Bessel’s correction when we sample from the random variable (e.g. in Monte Carlo methods).
Variance for Continuous Random Variables
Given a continuous random variable
where
See Also
- standard error
- variance reduction methods - methods to reduce variance in the sampling process