Here we describe the variance-covariance matrix adjustment of coefficients.

## Introduction

To estimate the covariance matrix of coefficients, there are many
ways. In `mmrm`

package, we implemented asymptotic,
empirical, Jackknife and Kenward-Roger methods. For simplicity, the
following derivation are all for unweighted mmrm. For weighted mmrm, we
can follow the details
of weighted least square estimator.

### Asymptotic Covariance

Asymptotic covariance are derived based on the estimate of \(\beta\).

Following the definition in details in model fitting, we have

\[ \hat\beta = (X^\top W X)^{-1} X^\top W Y \]

\[ cov(\hat\beta) = (X^\top W X)^{-1} X^\top W cov(\epsilon) W X (X^\top W X)^{-1} = (X^\top W X)^{-1} \]

Where \(W\) is the block diagonal matrix of inverse of covariance matrix of \(\epsilon\).

### Empirical Covariance

Empirical covariance, also known as the robust sandwich estimator, or “CR0”, is derived by replacing the covariance matrix of \(\epsilon\) by observed covariance matrix.

\[ cov(\hat\beta) = (X^\top W X)^{-1}(\sum_{i}{X_i^\top W_i \hat\epsilon_i\hat\epsilon_i^\top W_i X_i})(X^\top W X)^{-1} = (X^\top W X)^{-1}(\sum_{i}{X_i^\top L_{i} L_{i}^\top \hat\epsilon_i\hat\epsilon_i^\top L_{i} L_{i}^\top X_i})(X^\top W X)^{-1} \]

Where \(W_i\) is the block diagonal part for subject \(i\) of \(W\) matrix, \(\hat\epsilon_i\) is the observed residuals for subject i, \(L_i\) is the Cholesky factor of \(\Sigma_i^{-1}\) (\(W_i = L_i L_i^\top\)).

See the detailed explanation of these formulas in the Weighted Least Square Empirical Covariance vignette.

### Jackknife Covariance

Jackknife method in `mmrm`

is the “leave-one-cluster-out”
method. It is also known as “CR3”. Following McCaffrey and Bell (2003), we have

\[ cov(\hat\beta) = (X^\top W X)^{-1}(\sum_{i}{X_i^\top L_{i} (I_{i} - H_{ii})^{-1} L_{i}^\top \hat\epsilon_i\hat\epsilon_i^\top L_{i} (I_{i} - H_{ii})^{-1} L_{i}^\top X_i})(X^\top W X)^{-1} \]

where

\[H_{ii} = X_i(X^\top X)^{-1}X_i^\top\]

Please note that in the paper there is an additional scale parameter \(\frac{n-1}{n}\) where \(n\) is the number of subjects, here we do not include this parameter.

### Bias-Reduced Covariance

Bias-reduced method, also known as “CR2”, provides unbiased under correct working model. Following McCaffrey and Bell (2003), we have \[ cov(\hat\beta) = (X^\top W X)^{-1}(\sum_{i}{X_i^\top L_{i} (I_{i} - H_{ii})^{-1/2} L_{i}^\top \hat\epsilon_i\hat\epsilon_i^\top L_{i} (I_{i} - H_{ii})^{-1} L_{i}^\top X_i})(X^\top W X)^{-1} \]

where

\[H_{ii} = X_i(X^\top X)^{-1}X_i^\top\]

### Kenward-Roger Covariance

Kenward-Roger covariance is an adjusted covariance matrix for small sample size. Details can be found in Kenward-Roger

*Quality Control and Applied Statistics*48 (6): 677–82.