Statistical bias comes from all stages of data analysis. The following sources of bias will be listed in each stage separately.
Data selection Selection bias involves individuals being more likely to be selected for study than others,
biasing the sample. This can also be termed selection effect,
sampling bias and
Berksonian bias. •
Spectrum bias arises from evaluating diagnostic tests on biased patient samples, leading to an overestimate of the
sensitivity and specificity of the test. For example, a high prevalence of disease in a study population increases positive predictive values, which will cause a bias between the prediction values and the real ones. •
Observer selection bias occurs when the evidence presented has been pre-filtered by observers, which is so-called
anthropic principle. The data collected is not only filtered by the design of experiment, but also by the necessary precondition that there must be someone doing a study. An example is the impact of the Earth in the past. The impact event may cause the extinction of intelligent animals, or there were no intelligent animals at that time. Therefore, some impact events have not been observed, but they may have occurred in the past. • Volunteer bias occurs when volunteers have intrinsically different characteristics from the target population of the study. Research has shown that volunteers tend to come from families with higher socioeconomic status. Furthermore, another study shows that women are more probable to volunteer for studies than men. •
Funding bias may lead to the selection of outcomes, test samples, or test procedures that favor a study's financial sponsor. •
Attrition bias arises due to a loss of participants, e.g., loss of follow up during a study. •
Recall bias arises due to differences in the accuracy or completeness of participant recollections of past events; for example, patients cannot recall how many cigarettes they smoked last week exactly, leading to over-estimation or under-estimation.
Hypothesis testing In the Neyman–Pearson framework, the goodness of a hypothesis test is determined by its
type I and type II errors.
Type I error, or
false positive, happens when the null hypothesis is correct but is rejected. The false positive rate is written as \alpha.
Type II error, or
false negative, happens when the null hypothesis is not correct but is accepted. The false negative rate is written as \beta. For instance, suppose that speeding is defined as having average driving speed limit is below 85 km/h, and let the null hypothesis be "not speeding". If someone receives a ticket with an average driving speed of 70 km/h, the decision maker has committed a Type I error. Conversely, if someone does not receive a ticket with an average driving speed of 90 km/h, the decision maker has committed a Type II error. Generally, a statistical test may decrease \alpha, but possibly at the price of increasing \beta, and vice versa. For example, the test may be very sensitive to true positives, but at the price of creating many false positives, and vice versa. Furthermore, whereas \alpha depends on just the statistical test itself and the null hypothesis H_0, \beta depends on the statistical test and an unknown alternative hypothesis H_1. The
Neyman–Pearson framework bypasses the difficulty with having an unknown H_1 by imposing a kind of uniformity. That is, tests are good iff they work well on any H_1. Formally, define the following: • H_0, a null hypothesis. • \mathrm{T}, a test. Using the previous definitions, we have\Pr(\text{reject }H_0 | H_0, \mathrm{T}) = \alpha (H_0, \mathrm{T}), \quad \Pr(\text{reject }H_0 | H_1, \mathrm{T}) = 1-\beta (H_1, \mathrm{T})where H_1 is an unspecified alternative hypothesis. We say that the test is
unbiased (in the Neyman–Pearson sense) iff for any alternative hypothesis H_1,\Pr(\text{reject }H_0 | H_1, \mathrm{T}) \geq \alpha (H_0, \mathrm{T})Note that the right side of the equation cannot possibly be greater than \alpha (H_0, \mathrm{T}), because this formalism allows H_1 to be exactly the same as H_0, in which case\Pr(\text{reject }H_0 | H_1, \mathrm{T}) = \alpha (H_0, \mathrm{T})In short: an unbiased test is a test that minimizes the maximal false negative rate over all alternative hypotheses. Generally, one considers not a single test, but an entire family of tests, using a
test statistic. Let T be a test statistic. For each
significance level p \in [0, 1], the corresponding test is to check whether T > p. If so, then the null hypothesis is rejected at significance level p. Otherwise, it is accepted. We say that the test statistic (or the family of tests) is
unbiased (in the Neyman–Pearson sense) iff for any significance level p, it is unbiased.
Estimator selection The
bias of an estimator is the difference between an estimator's expected value and the true value of the parameter being estimated. Although an unbiased estimator is theoretically preferable to a biased estimator, in practice, biased estimators with small biases are frequently used. A biased estimator may be more useful for several reasons. First, an unbiased estimator may not exist without further assumptions. Second, sometimes an unbiased estimator is hard to compute. Third, a biased estimator may have a lower value of mean squared error. • A biased estimator is better than any unbiased estimator arising from the
Poisson distribution. The value of a biased estimator is always positive and the mean squared error of it is smaller than the unbiased one, which makes the biased estimator be more accurate. •
Omitted-variable bias is the bias that appears in estimates of parameters in regression analysis when the assumed specification omits an independent variable that should be in the model.
Analysis methods • Detection bias occurs when a phenomenon is more likely to be observed for a particular set of study subjects. For instance, the
syndemic involving
obesity and
diabetes may mean doctors are more likely to look for diabetes in obese patients than in thinner patients, leading to an inflation in diabetes among obese patients because of skewed detection efforts. • In
educational measurement, bias is defined as "Systematic errors in test content, test administration, and/or scoring procedures that can cause some test takers to get either lower or higher scores than their true ability would merit." The source of the bias is irrelevant to the trait the test is intended to measure. •
Observer bias arises when the researcher subconsciously influences the experiment due to
cognitive bias where judgment may alter how an experiment is carried out / how results are recorded.
Interpretation Reporting bias involves a skew in the availability of data, such that observations of a certain kind are more likely to be reported. == Addressing statistical bias ==