subnormal number

Subnormal numbers (or denormalized numbers) gradually approach zero and avoid a sudden underflow when dealing with very small numbers. [^1]

Motivation

IEEE floating point numbers are normalized with a leading 1. However, to deal with 0.0, it introduces the exception that if the exponent is 0 and mantissa is 0, then the number represent . If we consider only this rule, then the smallest non-zero positive number that can be represented is

However, this causes the problem of sudden underflow from this number to 0, which can cause numerical instability.

Subnormal numbers extend on the previous idea but introduces a new rule:

if the exponent is 0, then the leading bit becomes 0 (rather than 1), and the exponent is fixed to -126 rather than -127. Such numbers are called subnormal numbers.

The first thing to notice is that under this rule, a number with exponent 0 and mantissa 0 still represent 0, so we don’t need to specialize 0.0. Further,

• the smallest non-zero subnormal number is , which is much closer to than our previous
• the largest subnormal number is , which is quite close to the smallest non-subnormal number .

References

• c++ - What is a subnormal floating point number? - Stack Overflow
• Subnormal number - Wikipedia