# Quantitative genetics using the sommer package

## Giovanny Covarrubias-Pazaran

### 2026-07-08

The sommer package was developed to provide R users with a powerful and reliable multivariate mixed model solver for different genetic and non-genetic analyses in diploid and polyploid organisms. This package allows the user to estimate variance components for a mixed model with the advantages of specifying the variance-covariance structure of the random effects, specifying heterogeneous variances, and obtaining other parameters such as BLUPs, BLUEs, residuals, fitted values, variances for fixed and random effects, etc. The core algorithms of the package are coded in C++ using the Armadillo library to optimize dense matrix operations common in the derect-inversion algorithms. Although the vignette shows examples using the mmes function with the default direct inversion algorithm (henderson=FALSE) the Henderson’s approach can be faster when the number of records surpasses the number of coefficients to estimate and setting the henderson argument to TRUE can bring significant speed ups.

The package is focused on problems of the type p > n related to genomic prediction (hybrid prediction & genomic selection) and GWAS analysis, although any general mixed model can be fitted as well. The package provides kernels to estimate additive (`A.mat`), dominance (`D.mat`), and epistatic (`E.mat`) relationship matrices that have been shown to increase prediction accuracy under certain scenarios or simply to estimate the variance components of such. The package provides flexibility to fit other genetic models such as full and half diallel models as well.

The vignettes aim to provide several examples in how to use the sommer package under different scenarios. We will spend the rest of the space providing examples for:

**SECTION 1: Introduction **

1. Background in linear algebra

**SECTION 2: Topics in quantitative genetics **

1. Heritability (\(h^2\)) calculation
2. Specifying heterogeneous variances in mixed models
3. Using the `vpredict()` calculator
4. Half and full diallel designs (using the overlay)
5. Genomic selection (predicting mendelian sampling)
   - GBLUP
   - rrBLUP
6. Indirect genetic effects
7. Single cross prediction (hybrid prediction)
8. Multivariate genetic models and genetic correlations

SECTION 3: Special topics in quantitative genetics

1. Partitioned model
2. UDU’ decomposition
3. Mating designs
4. GWAS by GBLUP
5. Reduced models

## SECTION 1: Introduction

### Backgrounds in linear algebra

The core of the package is the `mmes()` function which solve the mixed model equations. The functions are an interface to call the `NR` Direct-Inversion Newton-Raphson or Average Information or henderson-based Average Information (Tunnicliffe 1989; Gilmour et al. 1995; Lee et al. 2016). Since version 2.0, sommer can handle multivariate models. Following Maier et al. (2015), the multivariate (and by extension the univariate) mixed model implemented has the form:

\(y_1 = X_1\beta_1 + Z_1u_1 + \epsilon_1\)

\(y_2 = X_2\beta_2 + Z_2u_2 + \epsilon_2\)

…

\(y_i = X_i\beta_i + Z_iu_i + \epsilon_i\)

where \(y_i\) is a vector of trait phenotypes, \(\beta_i\) is a vector of fixed effects, \(u_i\) is a vector of random effects for individuals and \(e_i\) are residuals for trait `i` (i = 1, …, t). The random effects (\(u_1\) … \(u_i\) and \(e_i\)) are assumed to be normally distributed with mean zero. X and Z are incidence matrices for fixed and random effects respectively. The distributions of the multivariate response and the phenotypic variance covariance (V) are:

\(Y = X\beta + ZU + \epsilon_i\)

Y ~ MVN(\(X\beta\), V)

`M5$$\mathbf{Y} = \left[\begin{array} {r} y_1 \ y_2 \ … \ y_t \ \end{array}\right] $$

`M5$$\mathbf{X} = \left[\begin{array} {rrr} X_1 & … & … \ \vdots & \ddots & \vdots\ … & … & X_t \ \end{array}\right] $$

`M5$$\mathbf{V} = \left[\begin{array} {rrr} Z_1 K{\sigma^2_{g_{1}}} Z_1’ + H{\sigma^2_{\epsilon_{1}}} & … & Z_1 K{\sigma_{g_{1,t}}} Z_t’ + H{\sigma_{\epsilon_{1,t}}}\ \vdots & \ddots & \vdots\ Z_1 K{\sigma_{g_{1,t}}} Z_t’ + H{\sigma_{\epsilon_{1,t}}} & … & Z_t K{\sigma^2_{g_{t}}} Z_t’ + H{\sigma^2_{\epsilon_{t}}} \ \end{array}\right] $$

where K is the relationship or covariance matrix for the kth random effect (u=1,…,k), and R=I is an identity matrix for the residual term. The terms \(\sigma^2_{g_{i}}\) and \(\sigma^2_{\epsilon_{i}}\) denote the genetic (or any of the kth random terms) and residual variance of trait `i`, respectively and \(\sigma_{g_{_{ij}}}\) and \(\sigma_{\epsilon_{_{ij}}}\) the genetic (or any of the kth random terms) and residual covariance between traits `i` and `j` (i=1,…,t, and j=1,…,t). The algorithm implemented optimizes the log likelihood:

\(logL = &frac12; * ln(|V|) + ln(X'|V|X) + Y'PY\)

where || is the determinant of a matrix. The REML estimates are updated using a Newton optimization algorithm of the form:

\(\theta^{k+1} = \theta^{k} + (H^{k})^{-1}*\frac{dL}{d\sigma^2_i}|\theta^k\)

Where \(\theta\) is the vector of variance components for random effects and covariance components among traits, \(H^{-1}\) is the inverse of the Hessian matrix of second derivatives for the kth cycle, \(\frac{dL}{d\sigma^2_i}\) is the vector of first derivatives of the likelihood with respect to the variance-covariance components. The Eigen decomposition of the relationship matrix proposed by Lee and Van Der Werf (2016) was included in the Newton-Raphson algorithm to improve time efficiency. Additionally, the popular `vpredict()` function to estimate standard errors for linear combinations of variance components (i.e. heritabilities and genetic correlations) was added to the package as well.

Please refer to the canonical papers listed in the Literature section to check how the algorithms work. We have tested widely the methods to make sure they provide the same solution when the likelihood behaves well, but for complex problems they might lead to slightly different answers. If you have any concern please contact me at cova_ruber@live.com.mx.

In the following section we will go in detail over several examples on how to use mixed models in univariate and multivariate case and their use in quantitative genetics.

## SECTION 2: Topics in quantitative genetics

### 1) Marker and non-marker based heritability calculation

Heritability is one of the most popular parameters among the breeding and genetics communities because of the insight it provides in the inheritance of the trait and potential selection response. Heritability is usually estimated as narrow sense (\(h^2\); only additive variance in the numerator \(\sigma^2_A\)), and broad sense (\(H^2\); all genetic variance in the numerator \(\sigma^2_G\)).

In a classical breeding experiment with no molecular markers, special designs are performed to estimate and dissect the additive (\(\sigma^2_A\)) and non-additive (e.g., dominance \(\sigma^2_D\), and epistatic \(\sigma^2_E\)) variance along with environmental variability. Designs such as generation analysis, North Carolina designs are used to dissect \(\sigma^2_A\) and \(\sigma^2_D\) to estimate the narrow sense heritability (\(h^2\)) using only \(\sigma^2_A\) in the numerator. When no special design is available we can still disect the genetic variance (\(\sigma^2_G\)) and estimate the broad sense heritability. In this first example we will show the broad sense estimation which doesn’t use covariance matrices for the genotypic effect (e.g., genomic-additive relationship matrices). For big models with no relationship matrices, sommer’s direct inversion is a bad idea to use but we will still show how to do it, but keep in mind that for very sparse models with no relationship matrices or other special covariance structures we recommend using the `lmer()` function from the lme4 package or any other package using MME-based algorithms (e.g., asreml-R).

The following dataset has 41 potato lines evaluated in 5 locations across 3 years in an RCBD design. We show how to fit the model and extract the variance components to calculate the \(h^2\).

    library(sommer)
    data(DT_example, package="enhancer")
    DT <- DT_example
    A <- A_example

    ans1 <- mmes(Yield~1,
                 random= ~ Name + Env + Env:Name + Env:Block,
                 rcov= ~ units, nIters=10,
                 data=DT, verbose = FALSE)
    summary(ans1)$varcomp

    ##                   VarComp  VarCompSE    Zratio Constraint
    ## Name:mu:mu       3.718326  1.6962251 2.1921185   Positive
    ## Env:mu:mu       12.008785 12.2802490 0.9778943   Positive
    ## Env:Name:mu:mu   5.152831  1.4926126 3.4522225   Positive
    ## Env:Block:mu:mu  0.000000  0.1156493 0.0000000   Positive
    ## units:mu:mu      4.366107  0.6572043 6.6434544   Positive

    (n.env <- length(levels(DT$Env)))

    ## [1] 3

    vpredict(ans1, h2 ~ V1 / ( V1 + (V3/n.env) + (V5/(2*n.env)) ) )

    ##     Estimate        SE
    ## h2 0.6032698 0.1344766

That is an estimate of broad-sense heritability.

Recently with markers becoming cheaper, thousand of markers can be run in the breeding materials. When markers are available, a special design is not neccesary to dissect the additive genetic variance. The availability of the additive, dominance and epistatic relationship matrices allow us to estimate \(\sigma^2_A\), \(\sigma^2_D\) and \(\sigma^2_I\), although given that A, D and E are not orthogonal the interpretation of models that fit more than the A matrix at the same time becomes cumbersome.

Assume you have a population (even unreplicated) in the field but in addition we have genetic markers. Now we can fit the model and estimate the genomic heritability that explains a portion of the additive genetic variance (with high marker density \(\sigma^2_A\) = \(\sigma^2_{markers}\))

    data(DT_cpdata, package="enhancer")
    DT <- DT_cpdata
    GT <- GT_cpdata
    MP <- MP_cpdata
    DT$idd <-DT$id; DT$ide <-DT$id
    ### look at the data
    A <- A.mat(GT) # additive relationship matrix
    D <- D.mat(GT) # dominance relationship matrix
    E <- E.mat(GT) # epistatic relationship matrix
    ans.ADE <- mmes(color~1,
                     random=~vsm(ism(id),Gu=A) + vsm(ism(idd),Gu=D),
                     rcov=~units, nIters=10,
                     data=DT,verbose = FALSE)
    (summary(ans.ADE)$varcomp)

    ##                 VarComp    VarCompSE   Zratio Constraint
    ## id:A:mu:mu  0.003663741 0.0012202747 3.002390   Positive
    ## idd:D:mu:mu 0.001295002 0.0005269558 2.457515   Positive
    ## units:mu:mu 0.002106604 0.0002864009 7.355439   Positive

    vpredict(ans.ADE, h2 ~ (V1) / ( V1+V3) ) # narrow sense

    ##     Estimate         SE
    ## h2 0.6349258 0.08841872

    vpredict(ans.ADE, h2 ~ (V1+V2) / ( V1+V2+V3) ) # broad-sense

    ##    Estimate         SE
    ## h2  0.70184 0.06059124

In this example we showed how to estimate the additive (\(\sigma^2_A\)) and dominance (\(\sigma^2_D\)) variance components based on markers and estimate broad (\(H^2\)) and narrow-sense heritability (\(h^2\)). Notice that we used the `vsm()` function which indicates that the random effect inside the parenthesis (i.e. `id`, `idd` or `ide`) has a covariance matrix (A, D, or E), that will be specified in the `Gu` argument of the `vsm()` function. Please DO NOT provide the inverse, but rather the original covariance matrix when using henderson=FALSE.

### 2) Specifying heterogeneous variances in univariate models

Very often in multi-environment trials, the assumption that genetic variance is the same across locations may be too naive. Because of that, specifying a general genetic component and a location-specific genetic variance is the way to go.

We estimate variance components for \(GCA_2\) and \(SCA\) specifying the variance structure.

    # data(DT_cornhybrids, package="enhancer")
    # DT <- DT_cornhybrids
    # DTi <- DTi_cornhybrids
    # GT <- GT_cornhybrids
    # ### fit the model
    # modFD <- mmes(Yield~1,
    #               random=~ vsm(atr(Location,c("3","4")),ism(GCA2)),
    #               rcov= ~ vsm(dsm(Location),ism(units)), nIters=10,
    #               returnParam = F,
    #               data=DT, verbose = FALSE)
    # summary(modFD)

In the previous example we showed how the `atr()` function is used in the `mmes()` solver. By using the `atr()` function you can specify that i.e. the GCA2 has a different variance in different Locations, in this case locations 3 and 4, but also a main GCA variance. This is considered a CS + DIAG (compound symmetry + diagonal) model.

In addition, other functions can be added on top to fit models with covariance structures, i.e. the `Gu` argument from the `vsm()` function to indicate a covariance matrix (A, pedigree or genomic relationship matrix)

    # data(DT_cornhybrids, package="enhancer")
    # DT <- DT_cornhybrids
    # DTi <- DTi_cornhybrids
    # GT <- as(as(as( GT_cornhybrids,  "dMatrix"), "generalMatrix"), "CsparseMatrix")
    # GT[1:4,1:4]
    # DT=DT[with(DT, order(Location)), ]
    # ### fit the model
    # modFD <- mmes(Yield~1,
    #               random=~ vsm(atr(Location,c("3","4")),ism(GCA2),Gu=GT),
    #               rcov= ~ vsm(dsm(Location),ism(units)), nIters=10,
    #               data=DT, verbose = FALSE)
    # summary(modFD)

### 3) Using the vpredict calculator

Sometimes the user needs to calculate ratios or functions of specific variance-covariance components and obtain the standard errors for such parameters. Examples of these are the genetic correlations, heritabilities, etc. Using the CPdata we will show how to estimate the heritability and the standard error using the `vpredict()` function that uses the delta method to come up with these parameters. This can be extended for any linear combination of the variance components.

#### 3.1) Standar error for heritability

    data(DT_cpdata, package="enhancer")
    DT <- DT_cpdata
    GT <- GT_cpdata
    MP <- MP_cpdata
    ### look at the data
    A <- A.mat(GT) # additive relationship matrix
    ans <- mmes(color~1,
                    random=~vsm(ism(id),Gu=A),
                    rcov=~units, nIters=10,
                    data=DT, verbose = FALSE)
    (summary(ans.ADE)$varcomp)

    ##                 VarComp    VarCompSE   Zratio Constraint
    ## id:A:mu:mu  0.003663741 0.0012202747 3.002390   Positive
    ## idd:D:mu:mu 0.001295002 0.0005269558 2.457515   Positive
    ## units:mu:mu 0.002106604 0.0002864009 7.355439   Positive

    vpredict(ans, h2 ~ (V1) / ( V1+V2) )

    ##     Estimate        SE
    ## h2 0.6512605 0.0610886

The same can be used for multivariate models. Please check the documentation of the `vpredict` function to see more examples.

### 4) Half and full diallel designs (use of the overlay)

When breeders are looking for the best single-cross combinations, diallel designs have been by far the most used design in crops like maize. There are 4 types of diallel designs depending on whether reciprocal and self-crosses (omission of parents) are performed (full diallel with parents n^2; full diallel without parents n(n-1); half diallel with parents ½ * n(n+1); half diallel without parents ½ * n(n-1) ). In this example we will show a full diallel design (reciprocal crosses are performed) and half diallel designs (only one of the directions is performed).

In the first data set we show a full diallel among 40 lines from 2 heterotic groups, 20 in each. Therefore 400 possible hybrids are possible. We have pehnotypic data for 100 of them across 4 locations. We use the data available to fit a model of the form:

\(y = X\beta + Zu_1 + Zu_2 + Zu_S + \epsilon\)

We estimate variance components for \(GCA_1\), \(GCA_2\) and \(SCA\) and use them to estimate heritability. Additionally BLUPs for GCA and SCA effects can be used to predict crosses.

    # data(DT_cornhybrids, package="enhancer")
    # DT <- DT_cornhybrids
    # DTi <- DTi_cornhybrids
    # GT <- GT_cornhybrids
    #
    # modFD <- mmes(Yield~Location,
    #               random=~GCA1+GCA2+SCA,
    #               rcov=~units, nIters=10,
    #               data=DT, verbose = FALSE)
    # (suma <- summary(modFD)$varcomp)
    # Vgca <- sum(suma[1:2,1])
    # Vsca <- suma[3,1]
    # Ve <- suma[4,1]
    # Va = 4*Vgca
    # Vd = 4*Vsca
    # Vg <- Va + Vd
    # (H2 <- Vg / (Vg + (Ve)) )
    # (h2 <- Va / (Vg + (Ve)) )

Don’t worry too much about the `h2` value, the data was simulated to be mainly dominance variance, therefore the `Va` was simulated extremely small leading to such value of narrow sense `h2`.

In the second data set we show a small half diallel with 7 parents crossed in one direction. There are n(n-1)/2 possible crosses; 7(6)/2 = 21 unique crosses. Parents appear as males or females indistictly. Each with two replications in a CRD. For a half diallel design a single GCA variance component for both males and females can be estimated and an SCA as well (\(\sigma^2_GCA\) and \(\sigma^2_SCA\) respectively), and BLUPs for GCA and SCA of the parents can be extracted. We will show first how to do so with the `mmes()` function using the `overlay()` function. The specific model here is:

\(y = X\beta + Zu_g + Zu_s + \epsilon\)

    data("DT_halfdiallel", package="enhancer")
    DT <- DT_halfdiallel
    head(DT)

    ##   rep geno male female     sugar
    ## 1   1   12    1      2 13.950509
    ## 2   2   12    1      2  9.756918
    ## 3   1   13    1      3 13.906355
    ## 4   2   13    1      3  9.119455
    ## 5   1   14    1      4  5.174483
    ## 6   2   14    1      4  8.452221

    DT$femalef <- as.factor(DT$female)
    DT$malef <- as.factor(DT$male)
    DT$genof <- as.factor(DT$geno)
    #### model using overlay
    modh <- mmes(sugar~1,
                 random=~vsm(ism(overlay(femalef,malef)) )
                 + genof, nIters=10,
                 data=DT, verbose = FALSE)
    summary(modh)$varcomp

    ##                      VarComp VarCompSE   Zratio Constraint
    ## femalef:malef:mu:mu 5.508686 3.5787494 1.539277   Positive
    ## genof:mu:mu         1.815951 1.3631606 1.332162   Positive
    ## units:mu:mu         3.117370 0.9620929 3.240197   Positive

Notice how the `overlay()` argument makes the overlap of incidence matrices possible making sure that male and female are joint into a single random effect.

### 5) Genomic selection: predicting mendelian sampling

In this section we will use wheat data from CIMMYT to show how genomic selection is performed. This is the case of prediction of specific individuals within a population. It basically uses a similar model of the form:

\(y = X\beta + Zu + \epsilon\)

and takes advantage of the variance covariance matrix for the genotype effect known as the additive relationship matrix (A) and calculated using the `A.mat` function to establish connections among all individuals and predict the BLUPs for individuals that were not measured. The prediction accuracy depends on several factors such as the heritability (\(h^2\)), training population used (TP), size of TP, etc.

    data(DT_wheat, package="enhancer")
    DT <- DT_wheat
    GT <- apply(GT_wheat,2,as.numeric)
    rownames(GT) <- rownames(GT_wheat)

    colnames(DT) <- paste0("X",1:ncol(DT))
    DT <- as.data.frame(DT);DT$id <- as.factor(rownames(DT))
    # select environment 1
    K <- A.mat(GT) # additive relationship matrix
    colnames(K) <- rownames(K) <- rownames(DT)
    # GBLUP pedigree-based approach
    set.seed(12345)
    y.trn <- DT
    vv <- sample(rownames(DT),round(nrow(DT)/5))
    y.trn[vv,"X1"] <- NA
    head(y.trn)

    ##              X1          X2          X3         X4   id
    ## 775   1.6716295 -1.72746986 -1.89028479  0.0509159  775
    ## 2166 -0.2527028  0.40952243  0.30938553 -1.7387588 2166
    ## 2167         NA -0.64862633 -0.79955921 -1.0535691 2167
    ## 2465  0.7854395  0.09394919  0.57046773  0.5517574 2465
    ## 3881  0.9983176 -0.28248062  1.61868192 -0.1142848 3881
    ## 3889  2.3360969  0.62647587  0.07353311  0.7195856 3889

    ## GBLUP
    ans <- mmes(X1~1,
                random=~vsm(ism(id),Gu=K),
                rcov=~units,nIters=10,
                data=y.trn, verbose = FALSE) # kinship based
    cor(ans$u[vv,] ,DT[vv,"X1"], use="complete")

    ## [1] 0.5076055

    ## rrBLUP
    ans2 <- mmes(X1~1,
                 random=~vsm(ism(GT), buildGu = FALSE),
                 rcov=~units, getPEV = TRUE, nIters=10, # you do more iterations
                 data=y.trn, verbose = FALSE) # kinship based

    u <- GT %*% as.matrix(ans2$uList$`vsm(ism(GT), buildGu = FALSE`) # BLUPs for individuals
    rownames(u) <- rownames(GT)
    cor(u[vv,],DT[vv,"X1"]) # same correlation

    ## [1] 0.5737633

    # the same can be applied in multi-response models in GBLUP or rrBLUP

Please notice that when specifying the marker matrix as a random effect we used the argument ‘buildGu=FALSE’ to inform the ‘mmes’ function that a covariance matrix for the levels of the random effect shouldn’t be built. Imagine a model with 100,000 markers, that would imply a relationship matrix of 100,000 x 100,000. If that matrix is a diagonal it would only compromise the speed and memory of the function. By setting ‘buildGu=FALSE’ the mmes solver will avoid the matrix multiplications using that huge diagonal matrix. If you want to specify a relationship matrix for the marker matrix then you cannot use that ‘buildGu’ argument.

### 6) Indirect genetic effects

General variance structures can be used to fit indirect genetic effects. Here, we use an example dataset to show how we can fit the variance and covariance components between two or more different random effects.

We first fit a direct genetic effects model:

    # data(DT_ige, package="enhancer")
    # DT <- DT_ige
    # Af <- A_ige
    # An <- A_ige

    ## Direct genetic effects model
    # modDGE <- mmes(trait ~ block,
    #                random = ~ focal,
    #                rcov = ~ units, nIters=30,
    #                data = DT, verbose=FALSE)
    # summary(modDGE)$varcomp

We now fit the indirect genetic effects model without covariance between DGE and IGE:

    # data(DT_ige, package="enhancer")
    # DT <- DT_ige
    # A <- A_ige
    #
    # ## Indirect genetic effects model
    # modIGE <- mmes(trait ~ block, dateWarning = FALSE,
    #                random = ~ focal + neighbour, verbose = FALSE,
    #                rcov = ~ units, nIters=100,
    #               data = DT)
    # summary(modIGE)$varcomp

We now fit the indirect genetic effects model with covariance between DGE and IGE for which we will use the `gvsr()` function:

    # ### Indirect genetic effects model
    # modIGE <- mmes(trait ~ block, dateWarning = FALSE,
    #                random = ~ covm( vsm(ism(focal)), vsm(ism(neighbour)) ),
    #                rcov = ~ units, nIters=100, verbose = FALSE,
    #               data = DT)
    # summary(modIGE)$varcomp

On top of that we can include a relationship matrix for the two random effects that are being forced to co-vary

    ### Indirect genetic effects model
    # Ai <- solve(A_ige + diag(1e-5, nrow(A_ige),nrow(A_ige) ))
    # Ai <- as(as(as( Ai,  "dMatrix"), "generalMatrix"), "CsparseMatrix")
    # # Indirect genetic effects model with covariance between DGE and IGE using relationship matrices
    # modIGE <- mmes(trait ~ block, dateWarning = FALSE,
    #                random = ~ covm( vsm(ism(focal), Gu=Ai), vsm(ism(neighbour), Gu=Ai) ),
    #                rcov = ~ units, nIters=100, verbose = FALSE,
    #               data = DT)
    # summary(modIGE)$varcomp

### 7) Genomic selection: single cross prediction

When doing prediction of single cross performance the phenotype can be dissected in three main components, the general combining abilities (GCA) and specific combining abilities (SCA). This can be expressed with the same model analyzed in the diallel experiment mentioned before:

\(y = X\beta + Zu_1 + Zu_2 + Zu_S + \epsilon\)

with:

\(u_1\) ~ N(0, $K_1$$\sigma^2_u1$)

\(u_2\) ~ N(0, $K_2$$\sigma^2_u2$)

\(u_s\) ~ N(0, $K_3$$\sigma^2_us$)

And we can specify the K matrices. The main difference between this model and the full and half diallel designs is the fact that this model will include variance covariance structures in each of the three random effects (GCA1, GCA2 and SCA) to be able to predict the crosses that have not ocurred yet. We will use the data published by Technow et al. (2015) to show how to do prediction of single crosses.

    data(DT_technow, package="enhancer")
    DT <- DT_technow

    Md <- apply(Md_technow,2,as.numeric)
    rownames(Md) <- rownames(Md_technow)
    Mf <- apply(Mf_technow,2,as.numeric)
    rownames(Mf) <- rownames(Mf_technow)

    Md <- (Md*2) - 1
    Mf <- (Mf*2) - 1
    Ad <- A.mat(Md)
    Af <- A.mat(Mf)
    Adi <- solve(Ad + diag(1e-4,ncol(Ad),ncol(Ad)))
    Adi <- as(as(as( Adi,  "dMatrix"), "generalMatrix"), "CsparseMatrix")
    attr(Adi, 'inverse')=TRUE
    Afi <- solve(Af + diag(1e-4,ncol(Af),ncol(Af)))
    Afi <- as(as(as( Afi,  "dMatrix"), "generalMatrix"), "CsparseMatrix")
    attr(Afi, 'inverse')=TRUE
    # RUN THE PREDICTION MODEL
    y.trn <- DT
    vv1 <- which(!is.na(DT$GY))
    vv2 <- sample(vv1, 100)
    y.trn[vv2,"GY"] <- NA
    anss2 <- mmes(GY~1,  henderson=TRUE,
                  random=~vsm(ism(dent),Gu=Adi) + vsm(ism(flint),Gu=Afi),
                  rcov=~units, nIters=15,
                  data=y.trn, verbose = FALSE)
    summary(anss2)$varcomp

    ##                  VarComp VarCompSE    Zratio Constraint
    ## dent:Adi:mu:mu  16.06350 0.2111024  76.09342   Positive
    ## flint:Afi:mu:mu 11.42183 0.2128187  53.66927   Positive
    ## units:mu:mu     16.81645 0.1185587 141.84070   Positive

    # zu1 <- model.matrix(~dent-1,y.trn) %*% anss2$uList$`vsm(ism(dent), Gu = Adi)`
    # zu2 <- model.matrix(~flint-1,y.trn) %*% anss2$uList$`vsm(ism(flint), Gu = Afi)`
    # u <- zu1+zu2+as.vector(anss2$b)
    # cor(u[vv2,], DT$GY[vv2])

In the previous model we only used the GCA effects (GCA1 and GCA2) for practicity, altough it’s been shown that the SCA effect doesn’t actually help that much in increasing prediction accuracy, but does increase a lot the computation intensity required since the variance covariance matrix for SCA is the kronecker product of the variance covariance matrices for the GCA effects, resulting in a 10578 x 10578 matrix that increases in a very intensive manner the computation required.

A model without covariance structures would show that the SCA variance component is insignificant compared to the GCA effects. This is why including the third random effect doesn’t increase the prediction accuracy.

### 8) Multivariate genetic models and genetic correlations

Sometimes is important to estimate genetic variance-covariance among traits–multi-reponse models are very useful for such a task. Let see an example with 2 traits (`color` and `Yield`) and a single random effect (genotype; `id`) although multiple effects can be modeled as well. First, we need to create a dataset in the long format, that is having a single column for the trait value and the traits are row-binded one over the other (here we use reshape). Then the trait converts one more factor to model. We use the following variance covariance structure for the random effect id to be able to obtain the genetic covariance among traits.

    # data(DT_cpdata, package="enhancer")
    # DT <- DT_cpdata
    # GT <- GT_cpdata
    # MP <- MP_cpdata
    # traits <- c("color","Yield")
    # DT[,traits] <- apply(DT[,traits],2,scale)
    # DTL <- reshape(DT[,c("id", traits)],
    #                idvar = c("id"),
    #                varying = traits,
    #                v.names = "value", direction = "long",
    #                timevar = "trait", times = traits )
    # DTL <- DTL[with(DTL, order(trait)), ]
    # head(DTL)
    #
    # A <- A.mat(GT) # additive relationship matrix
    # # if using mmes=TRUE you need to provide the inverse
    # Ai <- solve(A + diag(1e-4,ncol(A),ncol(A)))
    # Ai <- as(as(as( Ai,  "dMatrix"), "generalMatrix"), "CsparseMatrix")
    # attr(Ai, 'inverse')=TRUE
    # #### be patient take some time
    # ansm <- mmes( value ~ trait, # henderson=TRUE,
    #                random=~ vsm(usm(trait), ism(id), Gu=A), # Ai if henderson
    #                rcov=~ vsm(dsm(trait), ism(units)),
    #                data=DTL)
    # cov2cor(ansm$theta[[1]])

Other examples of multi-trait models can be found in the documentation of the rice dataset (DT_rice) and in the example dataset (DT_example). The key here is to learn how to use the reshape function from base R.

## SECTION 3: Special topics in Quantitative genetics

### 1) Partitioned model

The partitioned model was popularized by () to show that marker effects can be obtained by fitting a GBLUP model to reduce the computational burden and then recover them by creating some special matrices MM’ for GBLUP and M’(M’M)- to recover marker effects. Here we show a very easy example using the DT_cpdata:

    library(sommer)
    data("DT_cpdata", package="enhancer")
    DT <- DT_cpdata
    M <- GT_cpdata

    ################
    # MARKER MODEL
    ################
    mix.marker <- mmes(color~1,
                       random=~Rowf+vsm(ism(M)),
                       rcov=~units,data=DT,
                       verbose = FALSE)

    me.marker <- mix.marker$uList$`vsm(ism(M`

    ################
    # PARTITIONED GBLUP MODEL
    ################

    MMT <-tcrossprod(M) ## MM' = additive relationship matrix
    MMTinv<-solve(MMT) ## inverse
    MTMMTinv<-t(M)%*%MMTinv # M' %*% (M'M)-

    mix.part <- mmes(color~1,
                     random=~Rowf+vsm(ism(id), Gu=MMT),
                     rcov=~units,data=DT,
                     verbose = FALSE)

    #convert BLUPs to marker effects me=M'(M'M)- u
    me.part<-MTMMTinv%*%matrix(mix.part$uList$`vsm(ism(id), Gu = MMT`,ncol=1)

    # compare marker effects between both models
    plot(me.marker,me.part)

As can be seen, these two models are equivalent with the exception that the partitioned model is more computationally efficient.

### 2) UDU’ decomposition

Lee and Van der Warf (2015) proposed a decomposition of the relationship matrix A=UDU’ together with a transformation of the response and fixed effects Uy = Ux + UZ + e, to fit a model where the phenotypic variance matrix V is a diagonal because the relationship matrix is the diagonal matrix D from the decomposition that can be inverted easily and make multitrait models more feasible.

    # data("DT_wheat", package="enhancer")
    # rownames(GT_wheat) <- rownames(DT_wheat)
    # GT <- apply(GT_wheat,2,as.numeric)
    # rownames(GT) <- rownames(GT_wheat)
    # G <- A.mat(GT)
    # Y <- data.frame(DT_wheat)
    #
    # # make the decomposition
    # UD<-eigen(G) # get the decomposition: G = UDU'
    # U<-UD$vectors
    # D<-diag(UD$values)# This will be our new 'relationship-matrix'
    # rownames(D) <- colnames(D) <- rownames(G)
    # X<-model.matrix(~1, data=Y) # here: only one fixed effect (intercept)
    # UX<-t(U)%*%X # premultiply X and y by U'
    # UY <- t(U) %*% as.matrix(Y) # multivariate
    #
    # # dataset for decomposed model
    # DTd<-data.frame(id = rownames(G) ,UY, UX =UX[,1])
    # DTd$id<-as.character(DTd$id)
    #
    # modeld <- mmes(cbind(X1,X2) ~ UX - 1,
    #               random = ~vsm(id,Gu=D),
    #               rcov = ~vsm(units),
    #               data=DTd, verbose = FALSE)
    #
    # # dataset for normal model
    # DTn<-data.frame(id = rownames(G) , DT_wheat)
    # DTn$id<-as.character(DTn$id)
    #
    # modeln <- mmes(cbind(X1,X2) ~ 1,
    #               random = ~vsm(id,Gu=G),
    #               rcov = ~vsm(units),
    #               data=DTn, verbose = FALSE)
    #
    # ## compare regular and transformed blups
    # plot(x=(solve(t(U)))%*%modeld$U$`u:id`$X2[colnames(D)],
    #      y=modeln$U$`u:id`$X2[colnames(D)], xlab="UDU blup",
    #      ylab="blup")

As can be seen, the two models are equivalent. Despite the fact that sommer doesn’t take a great advantage of this trick because it was built for dense matrices using the Armadillo library. Other software may be better using this trick.

### 3) Mating designs

Estimating variance components has been a topic of interest for the breeding community for a long time. Here we show how to calculate additive and dominance variance using the North Carolina Design I (Nested design) and North Carolina Design II (Factorial design) using the classical Expected Mean Squares method and the REML methods from sommer and how these two are equivalent.

#### North Carolina Design I (Nested design)

    data(DT_expdesigns, package="enhancer")
    DT <- DT_expdesigns$car1
    DT <- aggregate(yield~set+male+female+rep, data=DT, FUN = mean)
    DT$setf <- as.factor(DT$set)
    DT$repf <- as.factor(DT$rep)
    DT$malef <- as.factor(DT$male)
    DT$femalef <- as.factor(DT$female)
    # lattice::levelplot(yield~male * malef:femalef|set, data=DT, main="NC design I")
    ##############################
    ## Expected Mean Square method
    ##############################
    mix1 <- lm(yield~ setf + setf:repf + femalef:malef:setf + malef:setf, data=DT)
    MS <- anova(mix1); MS

    ## Analysis of Variance Table
    ##
    ## Response: yield
    ##                    Df Sum Sq Mean Sq F value   Pr(>F)
    ## setf                1 0.1780 0.17796  1.6646 0.226012
    ## setf:repf           2 0.9965 0.49824  4.6605 0.037141 *
    ## setf:malef          4 7.3904 1.84759 17.2822 0.000173 ***
    ## setf:femalef:malef  6 1.6083 0.26806  2.5074 0.095575 .
    ## Residuals          10 1.0691 0.10691
    ## ---
    ## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    ms1 <- MS["setf:malef","Mean Sq"]
    ms2 <- MS["setf:femalef:malef","Mean Sq"]
    mse <- MS["Residuals","Mean Sq"]
    nrep=2
    nfem=2
    Vfm <- (ms2-mse)/nrep
    Vm <- (ms1-ms2)/(nrep*nfem)

    ## Calculate Va and Vd
    Va=4*Vm # assuming no inbreeding (4/(1+F))
    Vd=4*(Vfm-Vm) # assuming no inbreeding(4/(1+F)^2)
    Vg=c(Va,Vd); names(Vg) <- c("Va","Vd"); Vg

    ##        Va        Vd
    ##  1.579537 -1.257241

    ##############################
    ## REML method
    ##############################
    mix2 <- mmes(yield~ setf + setf:repf,
                random=~femalef:malef:setf + malef:setf, nIters=10,
                data=DT, verbose = FALSE)
    vc <- summary(mix2)$varcomp; vc

    ##                            VarComp  VarCompSE   Zratio Constraint
    ## femalef:malef:setf:mu:mu 0.0805730 0.08098693 0.994889   Positive
    ## malef:setf:mu:mu         0.3948764 0.32883779 1.200824   Positive
    ## units:mu:mu              0.1069084 0.04781501 2.235876   Positive

    Vfm <- vc[1,"VarComp"]
    Vm <- vc[2,"VarComp"]

    ## Calculate Va and Vd
    Va=4*Vm # assuming no inbreeding (4/(1+F))
    Vd=4*(Vfm-Vm) # assuming no inbreeding(4/(1+F)^2)
    Vg=c(Va,Vd); names(Vg) <- c("Va","Vd"); Vg

    ##        Va        Vd
    ##  1.579506 -1.257214

As can be seen the REML method is easier than manipulating the MS and we arrive to the same results.

#### North Carolina Design II (Factorial design)

    DT <- DT_expdesigns$car2
    DT <- aggregate(yield~set+male+female+rep, data=DT, FUN = mean)
    DT$setf <- as.factor(DT$set)
    DT$repf <- as.factor(DT$rep)
    DT$malef <- as.factor(DT$male)
    DT$femalef <- as.factor(DT$female)
    #levelplot(yield~male*female|set, data=DT, main="NC desing II")
    head(DT)

    ##   set male female rep   yield setf repf malef femalef
    ## 1   1    1      1   1  831.03    1    1     1       1
    ## 2   1    2      1   1 1046.55    1    1     2       1
    ## 3   1    3      1   1  853.33    1    1     3       1
    ## 4   1    4      1   1  940.00    1    1     4       1
    ## 5   1    5      1   1  802.00    1    1     5       1
    ## 6   1    1      2   1  625.93    1    1     1       2

    N=with(DT,table(female, male, set))
    nmale=length(which(N[1,,1] > 0))
    nfemale=length(which(N[,1,1] > 0))
    nrep=table(N[,,1])
    nrep=as.numeric(names(nrep[which(names(nrep) !=0)]))

    ##############################
    ## Expected Mean Square method
    ##############################

    mix1 <- lm(yield~ setf + setf:repf +
                 femalef:malef:setf + malef:setf + femalef:setf, data=DT)
    MS <- anova(mix1); MS

    ## Analysis of Variance Table
    ##
    ## Response: yield
    ##                    Df  Sum Sq Mean Sq F value    Pr(>F)
    ## setf                1  847836  847836 45.6296 1.097e-09 ***
    ## setf:repf           4  144345   36086  1.9421  0.109652
    ## setf:malef          8  861053  107632  5.7926 5.032e-06 ***
    ## setf:femalef        8  527023   65878  3.5455  0.001227 **
    ## setf:femalef:malef 32  807267   25227  1.3577  0.129527
    ## Residuals          96 1783762   18581
    ## ---
    ## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    ms1 <- MS["setf:malef","Mean Sq"]
    ms2 <- MS["setf:femalef","Mean Sq"]
    ms3 <- MS["setf:femalef:malef","Mean Sq"]
    mse <- MS["Residuals","Mean Sq"]
    nrep=length(unique(DT$rep))
    nfem=length(unique(DT$female))
    nmal=length(unique(DT$male))
    Vfm <- (ms3-mse)/nrep;
    Vf <- (ms2-ms3)/(nrep*nmale);
    Vm <- (ms1-ms3)/(nrep*nfemale);

    Va=4*Vm; # assuming no inbreeding (4/(1+F))
    Va=4*Vf; # assuming no inbreeding (4/(1+F))
    Vd=4*(Vfm); # assuming no inbreeding(4/(1+F)^2)
    Vg=c(Va,Vd); names(Vg) <- c("Va","Vd"); Vg

    ##        Va        Vd
    ## 10840.192  8861.659

    ##############################
    ## REML method
    ##############################

    mix2 <- mmes(yield~ setf + setf:repf ,
                random=~femalef:malef:setf + malef:setf + femalef:setf,
                nIters=10,
                data=DT, verbose = FALSE)
    vc <- summary(mix2)$varcomp; vc

    ##                            VarComp VarCompSE    Zratio Constraint
    ## femalef:malef:setf:mu:mu  2215.618  2284.794 0.9697231   Positive
    ## malef:setf:mu:mu          5493.338  3610.989 1.5212836   Positive
    ## femalef:setf:mu:mu        2710.176  2236.621 1.2117280   Positive
    ## units:mu:mu              18580.739  2681.742 6.9286068   Positive

    Vfm <- vc[1,"VarComp"]
    Vm <- vc[2,"VarComp"]
    Vf <- vc[3,"VarComp"]

    Va=4*Vm; # assuming no inbreeding (4/(1+F))
    Va=4*Vf; # assuming no inbreeding (4/(1+F))
    Vd=4*(Vfm); # assuming no inbreeding(4/(1+F)^2)
    Vg=c(Va,Vd); names(Vg) <- c("Va","Vd"); Vg

    ##        Va        Vd
    ## 10840.704  8862.471

As can be seen, the REML method is easier than manipulating the MS and we arrive to the same results.

### 4) GWAS by GBLUP

Gualdron-Duarte et al. (2014) and Bernal-Rubio et al. (2016) proved that in (SingleStep)GBLUP or RRBLUP/SNP-BLUP, dividing the estimate of the marker effect by its standard error is mathematically equivalent to fixed regression EMMAX GWAS, even if markers are estimated as random effects in GBLUP and as fixed effects in EMMAX. That way fitting a GBLUP model is enough to perform GWAS for additive and on-additive effects.

Let us use the DT_cpdata dataset to explore the GWAS by GBLUP method

    data(DT_cpdata, package="enhancer")
    DT <- DT_cpdata
    GT <- GT_cpdata[,1:200]
    MP <- MP_cpdata
    #### create the variance-covariance matrix
    A <- A.mat(GT) # additive relationship matrix
    n <- nrow(DT) # to be used for degrees of freedom
    k <- 1 # to be used for degrees of freedom (number of levels in fixed effects)

First we fit a regular GWAS/EMMAX using the GWAS function available in sommer that first calculates variance components and then fits a regression marker by marker as a fixed effect.

    ###########################
    #### Regular GWAS/EMMAX approach
    ###########################
    # mix2 <- GWAS(color~1,
    #              random=~vsm(ism(id), Gu=A) + Rowf + Colf,
    #              rcov=~units, M=GT, gTerm = "u:id",
    #              verbose = FALSE, nIters=10,
    #              data=DT)

To compare EMMAX to the approach proposed by Gualdron-Duarte et al. (2014) and Bernal-Rubio et al. (2016) we will start fitting an RRBLUP/SNP-BLUP model to show that the estimate of the marker effect by its standard error is mathematically equivalent to fixed regression EMMAX GWAS.

    ###########################
    #### GWAS by RRBLUP approach
    ###########################
    # Z <- GT[as.character(DT$id),]
    # mixRRBLUP <- mmes(color~1,
    #               random=~vsm(ism(Z)) + Rowf + Colf,
    #               rcov=~units, nIters=10,
    #               verbose = FALSE,
    #               data=DT)
    #
    # a <- mixRRBLUP$uList$`vsm(ism(Z`# marker effects
    # start=mixRRBLUP$partitions[[1]][1]
    # end=mixRRBLUP$partitions[[1]][2]
    # se.a <- sqrt( diag(kronecker(diag(ncol(Z)),mixRRBLUP$theta[[1]]) - mixRRBLUP$Ci[start:end,start:end] ) ) # SE of marker effects
    # t.stat <- a/se.a # t-statistic
    # pvalRRBLUP <- dt(t.stat[,1],df=n-k-1) # -log10(pval)

Instead of fitting the RRBLUP/SNP-BLUP model we can fit a GBLUP model which is less computationally demanding and recover marker effects and their standard errors from the genotype effects.

    # ###########################
    # #### GWAS by GBLUP approach
    # ###########################
    # M<- GT
    # MMT <-tcrossprod(M) ## MM' = additive relationship matrix
    # MMTinv<-solve(MMT + diag(1e-6, ncol(MMT), ncol(MMT))) ## inverse of MM'
    # MTMMTinv<-t(M)%*%MMTinv # M' %*% (M'M)-
    # mixGBLUP <- mmes(color~1,
    #              random=~vsm(ism(id), Gu=MMT) + Rowf + Colf,
    #              rcov=~units, nIters=10,
    #              verbose = FALSE,
    #              data=DT)
    # a.from.g <-MTMMTinv%*%matrix(mixGBLUP$uList$`vsm(ism(id), Gu = MMT`,ncol=1)
    # start=mixGBLUP$partitions[[1]][1]
    # end=mixGBLUP$partitions[[1]][2]
    # var.g <- kronecker(MMT,mixGBLUP$theta[[1]]) - mixGBLUP$Ci[start:end,start:end]
    # var.a.from.g <- t(M)%*%MMTinv%*% (var.g) %*% t(MMTinv)%*%M
    # se.a.from.g <- sqrt(diag(var.a.from.g))
    # t.stat.from.g <- a.from.g/se.a.from.g # t-statistic
    # pvalGBLUP <- dt(t.stat.from.g,df=n-k-1) # -log10(pval)

Now we can look at the p-values coming from the 3 approaches to indeed show that results are equivalent.

    ###########################
    #### Compare results
    ###########################
    # plot(mix2$scores[,1], main="GWAS")
    # plot(-log(pvalRRBLUP), main="GWAS by RRBLUP/SNP-BLUP")
    # plot(-log(pvalGBLUP), main="GWAS by GBLUP")

### Final remarks

Keep in mind that mmes uses a direct inversion (DI) algorithm which can be very slow for large datasets with many records. When datasets have more records than coefficients to be estimated please shift to the use of the mmes function.

## Literature

Covarrubias-Pazaran G. 2016. Genome assisted prediction of quantitative traits using the R package sommer. PLoS ONE 11(6):1-15.

Covarrubias-Pazaran G. 2018. Software update: Moving the R package sommer to multivariate mixed models for genome-assisted prediction. doi: https://doi.org/10.1101/354639

Bernardo Rex. 2010. Breeding for quantitative traits in plants. Second edition. Stemma Press. 390 pp.

Gilmour et al. 1995. Average Information REML: An efficient algorithm for variance parameter estimation in linear mixed models. Biometrics 51(4):1440-1450.

Henderson C.R. 1975. Best Linear Unbiased Estimation and Prediction under a Selection Model. Biometrics vol. 31(2):423-447.

Kang et al. 2008. Efficient control of population structure in model organism association mapping. Genetics 178:1709-1723.

Lee, D.-J., Durban, M., and Eilers, P.H.C. (2013). Efficient two-dimensional smoothing with P-spline ANOVA mixed models and nested bases. Computational Statistics and Data Analysis, 61, 22 - 37.

Lee et al. 2015. MTG2: An efficient algorithm for multivariate linear mixed model analysis based on genomic information. Cold Spring Harbor. doi: http://dx.doi.org/10.1101/027201.

Maier et al. 2015. Joint analysis of psychiatric disorders increases accuracy of risk prediction for schizophrenia, bipolar disorder, and major depressive disorder. Am J Hum Genet; 96(2):283-294.

Rodriguez-Alvarez, Maria Xose, et al. Correcting for spatial heterogeneity in plant breeding experiments with P-splines. Spatial Statistics 23 (2018): 52-71.

Searle. 1993. Applying the EM algorithm to calculating ML and REML estimates of variance components. Paper invited for the 1993 American Statistical Association Meeting, San Francisco.

Yu et al. 2006. A unified mixed-model method for association mapping that accounts for multiple levels of relatedness. Genetics 38:203-208.

Tunnicliffe W. 1989. On the use of marginal likelihood in time series model estimation. JRSS 51(1):15-27.