Missing data

Understanding the reasons why data are missing is important for handling the remaining data correctly. If values are missing completely at random, the data sample is likely still representative of the population. But if the values are missing systematically, analysis may be biased. For example, in a study of the relation between IQ and income, if participants with an above-average IQ tend to skip the question ‘What is your salary?’, analyses that do not take into account this missing at random (MAR pattern (see below)) may falsely fail to find a positive association between IQ and salary. Because of these problems, methodologists routinely advise researchers to design studies to minimize the occurrence of missing values. Missing completely at random Values in a data set are missing completely at random (MCAR) if the events that lead to any particular data-item being missing are independent both of observable variables and of unobservable parameters of interest, and occur entirely at random. When data are MCAR, the analysis performed on the data is unbiased; however, data are rarely MCAR. In the case of MCAR, the missingness of data is unrelated to any study variable: thus, the participants with completely observed data are in effect a random sample of all the participants assigned a particular intervention. With MCAR, the random assignment of treatments is assumed to be preserved, but that is usually an unrealistically strong assumption in practice. Missing at random Missing at random (MAR) occurs when the missingness is not random, but where missingness can be fully accounted for by variables where there is complete information. Since MAR is an assumption that is impossible to verify statistically, we must rely on its substantive reasonableness. An example is that males are less likely to fill in a depression survey but this has nothing to do with their level of depression, after accounting for maleness. Depending on the analysis method, these data can still induce parameter bias in analyses due to the contingent emptiness of cells (male, very high depression may have zero entries). However, if the parameter is estimated with Full Information Maximum Likelihood, MAR will provide asymptotically unbiased estimates. Missing not at random Missing not at random (MNAR) (also known as nonignorable nonresponse) is data that is neither MAR nor MCAR (i.e. the value of the variable that's missing is related to the reason it's missing). Structured missingness Missing data can also arise in subtle ways that are not well accounted for in classical theory. An increasingly encountered problem arises in which data may not be MAR but missing values exhibit an association or structure, either explicitly or implicitly. Such missingness has been described as ‘structured missingness’. Structured missingness commonly arises when combining information from multiple studies, each of which may vary in its design and measurement set and therefore only contain a subset of variables from the union of measurement modalities. In these situations, missing values may relate to the various sampling methodologies used to collect the data or reflect characteristics of the wider population of interest, and so may impart useful information. For instance, in a health context, structured missingness has been observed as a consequence of linking clinical, genomic and imaging data. Planned missingness Missing data can also be a deliberate part of study design. Specifically, planned missingness is a research design strategy, employed in survey research, in which data are intentionally left uncollected from individual respondents (typically by administering randomly sampled subsets of items to each participant) to reduce burden while preserving the ability to estimate parameters for the full item set across the sample. ==Techniques of dealing with missing data==

Missing data reduces the representativeness of the sample and can therefore distort inferences about the population. Generally speaking, there are three main approaches to handle missing data: (1) Imputation—where values are filled in the place of missing data, (2) omission—where samples with invalid data are discarded from further analysis and (3) analysis—by directly applying methods unaffected by the missing values. One systematic review addressing the prevention and handling of missing data for patient-centered outcomes research identified 10 standards as necessary for the prevention and handling of missing data. These include standards for study design, study conduct, analysis, and reporting. In some practical application, the experimenters can control the level of missingness, and prevent missing values before gathering the data. For example, in computer questionnaires, it is often not possible to skip a question. A question has to be answered, otherwise one cannot continue to the next. So missing values due to the participant are eliminated by this type of questionnaire, though this method may not be permitted by an ethics board overseeing the research. In survey research, it is common to make multiple efforts to contact each individual in the sample, often sending letters to attempt to persuade those who have decided not to participate to change their minds. However, such techniques can either help or hurt in terms of reducing the negative inferential effects of missing data, because the kind of people who are willing to be persuaded to participate after initially refusing or not being home are likely to be significantly different from the kinds of people who will still refuse or remain unreachable after additional effort. Any multiply-imputed data analysis must be repeated for each of the imputed data sets and, in some cases, the relevant statistics must be combined in a relatively complicated way. Methods such as listwise deletion have been used to impute data but it has been found to introduce additional bias. There is a beginner guide that provides a step-by-step instruction how to impute data. The expectation-maximization algorithm is an approach in which values of the statistics which would be computed if a complete dataset were available are estimated (imputed), taking into account the pattern of missing data. In this approach, values for individual missing data-items are not usually imputed. Interpolation In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points. In the comparison of two paired samples with missing data, a test statistic that uses all available data without the need for imputation is the partially overlapping samples t-test. This is valid under normality and assuming MCAR Partial deletion Methods which involve reducing the data available to a dataset having no missing values include: • Listwise deletion/casewise deletion • Pairwise deletion Full analysis Methods which take full account of all information available, without the distortion resulting from using imputed values as if they were actually observed: • Generative approaches: • The expectation-maximization algorithm • full information maximum likelihood estimation • Discriminative approaches: • Max-margin classification of data with absent features Partial identification methods may also be used. ==Model-based techniques==

Model based techniques, often using graphs, offer additional tools for testing missing data types (MCAR, MAR, MNAR) and for estimating parameters under missing data conditions. For example, a test for refuting MAR/MCAR reads as follows: For any three variables X,Y, and Z where Z is fully observed and X and Y partially observed, the data should satisfy: X \perp\!\!\!\perp R_y |(R_x,Z). In words, the observed portion of X should be independent on the missingness status of Y, conditional on every value of Z. Failure to satisfy this condition indicates that the problem belongs to the MNAR category. (Remark: These tests are necessary for variable-based MAR which is a slight variation of event-based MAR.) When data falls into MNAR category, techniques are available for consistently estimating parameters when certain conditions hold in the model. For example, if Y explains the reason for missingness in X, and Y itself has missing values, the joint probability distribution of X and Y can still be estimated if the missingness of Y is random. The estimand in this case will be: : \begin{align} P(X,Y)& =P(X|Y) P(Y) \\ & =P(X|Y,R_x=0,R_y=0) P(Y|R_y=0) \end{align} where R_x=0 and R_y=0 denote the observed portions of their respective variables. Different model structures may yield different estimands and different procedures of estimation whenever consistent estimation is possible. The preceding estimand calls for first estimating P(X|Y) from complete data and multiplying it by P(Y) estimated from cases in which Y is observed regardless of the status of X. Moreover, in order to obtain a consistent estimate it is crucial that the first term be P(X|Y) as opposed to P(Y|X). In many cases model based techniques permit the model structure to undergo refutation tests. A special class of problems appears when the probability of the missingness depends on time. For example, in the trauma databases the probability to lose data about the trauma outcome depends on the day after trauma. In these cases various non-stationary Markov chain models are applied. ==See also==