p-value

In hypothesis testing, p-value is a number to weighing up whether the sample is consistent with the null hypothesis . It can be defined as the probability of observing the test statistic (or something more extreme) under the condition that is true.

A very small p-value indicates that the extreme observed outcome would be very unlikely under the null hypothesis. In practice, the p-value is often judged by a significant level (typical set as or much lower) where is considered statistical significant.

The misinterpretation and misuse of p-value and statistical significance is common in scientific literature that it is subjects to major debates in statistics and metascience communities. For example, in 2016, the American Statistical Association (ASA) mad a formal statement that discusses proper usage and outlines various misconception of the p-value ¹.

Subsections

• problems of the p-value
• p-hacking

Calculation

The calculation method of p-value depends on the type of statistical test being performed. The process involves a comparison of a test statistic against the CDF of a probability distribution (decided by the type of the statistical test).

• for a one-sided right-tail test-statistic distribution (when testing for a value greater than a threshold)
• for a one-sided left-tail test-statistic distribution (when testing for a value less than a threshold)
• for a two-sided test-statistic distribution. If the distribution of is symmetric about zero, then ²

Related

• confidence interval
• effect size

Footnotes

1. The ASA Statement on p-Values: Context, Process, and Purpose ↩

2. p-value - Wikipedia#Definition ↩