Q-value (statistics)

In statistics, testing multiple hypotheses simultaneously using methods appropriate for testing single hypotheses tends to yield many false positives: the so-called multiple comparisons problem. Since the 1950s, statisticians had been developing methods for multiple comparisons that reduced the number of false positives, such as controlling the family-wise error rate (FWER) using the Bonferroni correction, but these methods also increased the number of false negatives (i.e. reduced the statistical power). The pFDR and the q-value were introduced by John D. Storey in 2002. == Definition ==

Let there be a null hypothesis H_0 and an alternative hypothesis H_1. Perform m hypothesis tests; let the test statistics be i.i.d. random variables T_1, \ldots, T_m such that T_i \mid D_i \sim (1 - D_i) \cdot F_0 + D_i \cdot F_1. That is, if H_0 is true for test i (D_i = 0), then T_i follows the null distribution F_0; while if H_1 is true (D_i = 1), then T_i follows the alternative distribution F_1. Let D_i \sim \operatorname{Bernoulli}(\pi_1), that is, for each test, H_1 is true with probability \pi_1 and H_0 is true with probability \pi_0 = 1 - \pi_1. Denote the critical region (the values of T_i for which H_0 is rejected) at significance level \alpha by \Gamma_\alpha. Let an experiment yield a value t for the test statistic. The q-value of t is formally defined as : \inf_{\{\Gamma_\alpha : t \in \Gamma_\alpha\}} \operatorname{pFDR}(\Gamma_\alpha) That is, the q-value is the infimum of the pFDR if H_0 is rejected for test statistics with values \ge t. Equivalently, the q-value equals : \inf_{\{\Gamma_\alpha : t \in \Gamma_\alpha\}}\Pr(D = 0 \mid T \in \Gamma_\alpha) which is the infimum of the probability that H_0 is true given that H_0 is rejected (the false discovery rate). == Relationship to the p-value ==

The p-value is defined as : \inf_{\{\Gamma_\alpha : t \in \Gamma_\alpha\}} \Pr(T \in \Gamma_\alpha \mid D = 0) the infimum of the probability that H_0 is rejected given that H_0 is true (the false positive rate). Comparing the definitions of the p- and q-values, it can be seen that the q-value is the minimum posterior probability that H_0 is true. == Interpretation ==

The q-value can be interpreted as the false discovery rate (FDR): the proportion of false positives among all positive results. Given a set of test statistics and their associated q-values, rejecting the null hypothesis for all tests whose q-value is less than or equal to some threshold \alpha ensures that the expected value of the false discovery rate is \alpha. == Applications ==

Biology Gene expression Genome-wide analyses of differential gene expression involve simultaneously testing the expression of thousands of genes. Controlling the FWER (usually to 0.05) avoids excessive false positives (i.e. detecting differential expression in a gene that is not differentially expressed) but imposes a strict threshold for the p-value that results in many false negatives (many differentially expressed genes are overlooked). However, controlling the pFDR by selecting genes with significant q-values lowers the number of false negatives (increases the statistical power) while ensuring that the expected value of the proportion of false positives among all positive results is low (e.g. 5%). For example, suppose that among 10,000 genes tested, 1,000 are actually differentially expressed and 9,000 are not: • If we consider every gene with a p-value of less than 0.05 to be differentially expressed, we expect that 450 (5%) of the 9,000 genes that are not differentially expressed will appear to be differentially expressed (450 false positives). • If we control the FWER to 0.05, there is only a 5% probability of obtaining at least one false positive. However, this very strict criterion will reduce the power such that few of the 1,000 genes that are actually differentially expressed will appear to be differentially expressed (many false negatives). • If we control the pFDR to 0.05 by considering all genes with a q-value of less than 0.05 to be differentially expressed, then we expect 5% of the positive results to be false positives (e.g. 900 true positives, 45 false positives, 100 false negatives, 8,955 true negatives). This strategy enables one to obtain relatively low numbers of both false positives and false negatives. == Implementations ==

Note: the following is an incomplete list. R • The qvalue package in R estimates q-values from a list of p-values. == References ==