Arithmetic mean The design matrix for an
arithmetic mean is a
column vector of ones.
Simple linear regression This section gives an example of
simple linear regression—that is, regression with only a single explanatory variable—with seven observations. The seven data points are {
yi,
xi}, for
i = 1, 2, …, 7. The simple linear regression model is : y_i = \beta_0 + \beta_1 x_i +\varepsilon_i, \, where \beta_0 is the
y-intercept and \beta_1 is the slope of the regression line. This model can be represented in matrix form as : \begin{bmatrix}y_1 \\ y_2 \\ y_3 \\ y_4 \\ y_5 \\ y_6 \\ y_7 \end{bmatrix} = \begin{bmatrix}1 & x_1 \\1 & x_2 \\1 & x_3 \\1 & x_4 \\1 & x_5 \\1 & x_6 \\ 1 & x_7 \end{bmatrix} \begin{bmatrix} \beta_0 \\ \beta_1 \end{bmatrix} + \begin{bmatrix} \varepsilon_1 \\ \varepsilon_2 \\ \varepsilon_3 \\ \varepsilon_4 \\ \varepsilon_5 \\ \varepsilon_6 \\ \varepsilon_7 \end{bmatrix} where the first column of 1s in the design matrix allows estimation of the
y-intercept while the second column contains the
x-values associated with the corresponding
y-values. The matrix whose columns are 1's and
x's in this example is the design matrix.
Multiple regression This section contains an example of
multiple regression with two covariates (explanatory variables):
w and
x. Again suppose that the data consist of seven observations, and that for each observed value to be predicted (y_i), values
wi and
xi of the two covariates are also observed. The model to be considered is : y_i = \beta_0 + \beta_1 w_i + \beta_2 x_i + \varepsilon_i This model can be written in matrix terms as : \begin{bmatrix}y_1 \\ y_2 \\ y_3 \\ y_4 \\ y_5 \\ y_6 \\ y_7 \end{bmatrix} = \begin{bmatrix} 1 & w_1 & x_1 \\1 & w_2 & x_2 \\1 & w_3 & x_3 \\1 & w_4 & x_4 \\1 & w_5 & x_5 \\1 & w_6 & x_6 \\ 1& w_7 & x_7 \end{bmatrix} \begin{bmatrix} \beta_0 \\ \beta_1 \\ \beta_2 \end{bmatrix} + \begin{bmatrix} \varepsilon_1 \\ \varepsilon_2 \\ \varepsilon_3 \\ \varepsilon_4 \\ \varepsilon_5 \\ \varepsilon_6 \\ \varepsilon_7 \end{bmatrix} Here the 7×3 matrix on the right side is the design matrix.
One-way ANOVA (cell means model) This section contains an example with a one-way analysis of variance (
ANOVA) with three groups and seven observations. The given data set has the first three observations belonging to the first group, the following two observations belonging to the second group and the final two observations belonging to the third group. If the model to be fit is just the mean of each group, then the model is : y_{ij} = \mu_i + \varepsilon_{ij} which can be written : \begin{bmatrix}y_1 \\ y_2 \\ y_3 \\ y_4 \\ y_5 \\ y_6 \\ y_7 \end{bmatrix} = \begin{bmatrix}1 & 0 & 0 \\1 &0 &0 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 1\end{bmatrix} \begin{bmatrix}\mu_1 \\ \mu_2 \\ \mu_3 \end{bmatrix} + \begin{bmatrix} \varepsilon_1 \\ \varepsilon_2 \\ \varepsilon_3 \\ \varepsilon_4 \\ \varepsilon_5 \\ \varepsilon_6 \\ \varepsilon_7 \end{bmatrix} In this model \mu_i represents the mean of the ith group.
One-way ANOVA (offset from reference group) The ANOVA model could be equivalently written as each group parameter \tau_i being an offset from some overall reference. Typically this reference point is taken to be one of the groups under consideration. This makes sense in the context of comparing multiple treatment groups to a control group and the control group is considered the "reference". In this example, group 1 was chosen to be the reference group. As such the model to be fit is : y_{ij} = \mu + \tau_i + \varepsilon_{ij} with the constraint that \tau_1 is zero. : \begin{bmatrix}y_1 \\ y_2 \\ y_3 \\ y_4 \\ y_5 \\ y_6 \\ y_7 \end{bmatrix} = \begin{bmatrix}1 &0 &0 \\1 &0 &0 \\ 1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \\ 1 & 0 & 1\end{bmatrix} \begin{bmatrix}\mu \\ \tau_2 \\ \tau_3 \end{bmatrix} + \begin{bmatrix} \varepsilon_1 \\ \varepsilon_2 \\ \varepsilon_3 \\ \varepsilon_4 \\ \varepsilon_5 \\ \varepsilon_6 \\ \varepsilon_7 \end{bmatrix} In this model \mu is the mean of the reference group and \tau_i is the difference from group i to the reference group. \tau_1 is not included in the matrix because its difference from the reference group (itself) is necessarily zero. ==See also==