Introduction to irtlink

irtlink estimates trends in the mean and standard deviation of latent ability distributions across the time points of educational assessments, and ability distribution differences across linked groups, with unidimensional item response theory (IRT) models. Under differential item functioning (DIF), the linked estimates depend on the selected common items. The package estimates this linking error together with the standard error.

Simulated data

sim_trend_data() generates dichotomous responses for several groups with a known trend. overlap sets the proportion of common items between successive groups.

set.seed(123)
d <- sim_trend_data(n_groups = 3, N = 1000, I = 20, overlap = 1,
                    mu = c(0, 0.3, 0.5), sigma = c(1, 1.2, 1.5),
                    dif = "random", dif_pct = 1, dif_sd = 0.3)
str(d, max.level = 1)
#> List of 3
#>  $ :'data.frame':    1000 obs. of  20 variables:
#>  $ :'data.frame':    1000 obs. of  20 variables:
#>  $ :'data.frame':    1000 obs. of  20 variables:
#>  - attr(*, "true_mu")= num [1:3] 0 0.3 0.5
#>  - attr(*, "true_sigma")= num [1:3] 1 1.2 1.5
#>  - attr(*, "true_ipars")='data.frame':   20 obs. of  3 variables:
#>  - attr(*, "design")= chr "all"
#>  - attr(*, "dif")= chr "random"
#>  - attr(*, "dif_items")= chr [1:20] "A01" "A02" "A03" "A04" ...
#>  - attr(*, "dif_effects")= num [1:20, 1:3] 0 0 0 0 0 0 0 0 0 0 ...
#>   ..- attr(*, "dimnames")=List of 2
#>  - attr(*, "dif_effects_a")= num [1:20, 1:3] 0 0 0 0 0 0 0 0 0 0 ...
#>   ..- attr(*, "dimnames")=List of 2
#>  - attr(*, "person_ids")=List of 3

Calibration

calibrate() fits one IRT model per group on the internal marginal maximum likelihood EM engine. The default is the 2PL with calibration = "separate". The alternatives are "concurrent", "fixed" (fixed item parameter calibration, FIPC), and "regularized" (regularized calibration with a smooth Bayesian information criterion, SBIC).

cal <- calibrate(d)
cal
#> irtlink calibration (2PL, separate)
#> Groups: 3 | Persons per group: 1000, 1000, 1000 
#> Items per group: 1:20  2:20  3:20

Linking

link_groups() places the separately calibrated groups on one common scale. approach selects the chain or the joint composition. method selects mean-mean, mean-geometric mean, Haberman, Haebara, or Stocking-Lord linking.

link <- link_groups(cal, approach = "chain", method = "mgm")
link$trend
#>   group        mu    sigma
#> 1     1 0.0000000 1.000000
#> 2     2 0.2180279 1.207128
#> 3     3 0.4178881 1.497740

ref selects the reference group of the reported trend.

link_groups(cal, approach = "chain", method = "mgm", ref = 2)$trend
#>   group         mu     sigma
#> 1     1 -0.1806170 0.8284123
#> 2     2  0.0000000 1.0000000
#> 3     3  0.1655666 1.2407458

Linking error

linking_error() estimates the linking error of the linked means and standard deviations. method selects the delete-one item jackknife, the approximate jackknife, or a sandwich estimator.

linking_error(link, method = "jackknife")$le
#>   group      le_mu   le_sigma    method n_reps
#> 1     1 0.00000000 0.00000000 jackknife     20
#> 2     2 0.08698816 0.04862817 jackknife     20
#> 3     3 0.07500620 0.04344685 jackknife     20
linking_error(link, method = "sandwich_osw")$le
#>   group      le_mu   le_sigma       method n_reps
#> 1     1 0.00000000 0.00000000 sandwich_osw     20
#> 2     2 0.07980828 0.04480819 sandwich_osw     20
#> 3     3 0.06884549 0.04019384 sandwich_osw     20

Full pipeline

estimate_trend() runs calibration, linking, and error estimation in one call. The bias-corrected methods decompose the uncertainty into standard error (se), linking error (le, le_bc), and total error (te, te_bc).

tr <- estimate_trend(d, calibration = "separate", approach = "chain",
                     link = "mgm", le = "sandwich_osw_bc")
#> le = "sandwich_osw_bc" uses the per-group covariance matrices from the calibration, which treat the person samples of the groups as independent. Pass `le_vcov` (for example from dependent_vcov()) when the samples are dependent.
round(tr$trend[, c("group", "mu", "sigma", "se", "le", "le_bc",
                   "te", "te_bc")], 3)
#>   group    mu sigma    se    le le_bc    te te_bc
#> 1     1 0.000 1.000 0.000 0.000 0.000 0.000 0.000
#> 2     2 0.218 1.207 0.059 0.080 0.072 0.099 0.093
#> 3     3 0.418 1.498 0.066 0.069 0.059 0.095 0.089

For dependent person samples, dependent_vcov() provides the joint item parameter covariance for the le_vcov argument.

DIF detection

detect_dif() flags items by root mean square deviation (RMSD) statistics or by per-item likelihood-ratio tests, optionally with iterative purification.

dif <- detect_dif(cal, method = "rmsd")
dif$flagged
#> [1] "A03" "A06" "A08" "A10" "A15" "A16" "A18"
head(dif$rmsd)
#>   item          g1          g2         g3        max
#> 1  A01 0.036711515 0.006244645 0.03076136 0.03671152
#> 2  A02 0.008517167 0.024201813 0.03073296 0.03073296
#> 3  A03 0.019940979 0.065028289 0.04435351 0.06502829
#> 4  A04 0.028778575 0.017423729 0.03491613 0.03491613
#> 5  A05 0.014261524 0.023482039 0.02799045 0.02799045
#> 6  A06 0.015227774 0.128367723 0.10182269 0.12836772

The function groups and the references are listed in ?irtlink.