Overview

The compare_waves() function is designed for a single metric and a single pair of waves. In most real applications we want to:

compare multiple wave pairs (e.g., 2023Q1–2023Q2, 2023Q2–2023Q3),
for multiple metrics,
and often by groups (e.g., business unit, level, critical talent).

This vignette shows how to move from individual comparisons to batch panel comparisons using compare_waves_batch().

We assume a long panel with:

one row per id–wave–metric,
optional metric identifiers and grouping columns.

Example panel

For illustration, we create a small synthetic panel with:

100 employees,
3 waves (0, 1, 2),
2 metrics:
- engagement_score
- pay_index
a simple group column.

set.seed(123)

n_id   <- 100
waves  <- 0:2
ids    <- sprintf("ee%03d", 1:n_id)

base_panel <- expand.grid(
  eeid  = ids,
  wave  = waves,
  KEEP.OUT.ATTRS = FALSE,
  stringsAsFactors = FALSE
)

# Simple group assignment (one group per id-wave, then reused)
base_panel$group <- sample(c("Group A", "Group B"),
                           nrow(base_panel), replace = TRUE)

# Engagement metric block
eng_panel <- base_panel
eng_panel$metric_id <- "engagement_score"
eng_panel$metric <-
  3.5 +
  0.2 * eng_panel$wave +
  ifelse(eng_panel$group == "Group A", 0.1, -0.1) +
  rnorm(nrow(eng_panel), sd = 0.4)

# Pay index metric block
pay_panel <- base_panel
pay_panel$metric_id <- "pay_index"
pay_panel$metric <-
  1.0 +
  0.05 * pay_panel$wave +
  ifelse(pay_panel$group == "Group A", 0.05, -0.05) +
  rnorm(nrow(pay_panel), sd = 0.1)

# Stack into one long panel
panel <- rbind(eng_panel, pay_panel)

head(panel)
#>    eeid wave   group        metric_id   metric
#> 1 ee001    0 Group A engagement_score 3.915096
#> 2 ee002    0 Group A engagement_score 3.907617
#> 3 ee003    0 Group A engagement_score 3.732881
#> 4 ee004    0 Group B engagement_score 2.996649
#> 5 ee005    0 Group A engagement_score 3.552219
#> 6 ee006    0 Group B engagement_score 3.287842
nrow(panel)  # should be 100 ids * 3 waves * 2 metrics = 600
#> [1] 600

Defining wave pairs

compare_waves_batch() expects an explicit set of wave pairs. A simple case is adjacent waves:

wave_pairs <- data.frame(
  wave_0 = c(0, 1),
  wave_1 = c(1, 2)
)

wave_pairs
#>   wave_0 wave_1
#> 1      0      1
#> 2      1      2

This will run comparisons for:

wave 0 vs 1, and
wave 1 vs 2.

Basic batch comparison

To run batch comparisons across all metrics and both groups:

batch_res <- compare_waves_batch(
  data          = panel,
  wave_pairs    = wave_pairs,
  id_col        = "eeid",
  wave_col      = "wave",
  metric_col    = "metric",
  metric_id_col = "metric_id",
  group_cols    = "group",
  method        = "auto",
  reliability   = 0.8
)

head(batch_res)
#>   wave_0 wave_1 status method error   group        metric_id   estimate
#> 1      0      1     ok linked  <NA> Group A engagement_score 0.21304817
#> 2      1      2     ok linked  <NA> Group A engagement_score 0.06255559
#> 3      0      1     ok linked  <NA> Group B engagement_score 0.28634627
#> 4      1      2     ok linked  <NA> Group B engagement_score 0.18291451
#> 5      0      1     ok linked  <NA> Group A        pay_index 0.02683106
#> 6      1      2     ok linked  <NA> Group A        pay_index 0.06924584
#>           se     ci_lower   ci_upper level N0 N1 nS   overlap       V0_bar
#> 1 0.07617753  0.063742956 0.36235338  0.95 57 46 26 0.5652174 0.0024755632
#> 2 0.08388545 -0.101856870 0.22696805  0.95 46 51 24 0.5217391 0.0034168503
#> 3 0.07356196  0.142167481 0.43052506  0.95 43 54 23 0.5348837 0.0032095131
#> 4 0.08508930  0.016142549 0.34968647  0.95 54 49 27 0.5510204 0.0029705039
#> 5 0.01661615 -0.005735988 0.05939811  0.95 57 46 26 0.5652174 0.0001584577
#> 6 0.01943184  0.031160127 0.10733154  0.95 46 51 24 0.5217391 0.0001677113
#>         V1_bar       C01_bar      V0_ind      V1_ind      C01_ind sigma2
#> 1 0.0034168503  4.469882e-05 0.141107104 0.157175115  0.004507704     NA
#> 2 0.0034075935 -1.061625e-04 0.157175115 0.173787267 -0.010377382     NA
#> 3 0.0029705039  3.843275e-04 0.138009063 0.160407209  0.038800371     NA
#> 4 0.0033866179 -4.415335e-04 0.160407209 0.165944278 -0.043270285     NA
#> 5 0.0001677113  2.503633e-05 0.009032090 0.007714718  0.002524817     NA
#> 6 0.0002333626  1.173873e-05 0.007714718 0.011901495  0.001147461     NA
#>   reliability
#> 1          NA
#> 2          NA
#> 3          NA
#> 4          NA
#> 5          NA
#> 6          NA

Each row corresponds to one combination of:

wave_0, wave_1
metric_id
group

with associated estimate, standard error, interval, and diagnostics.

Interpreting batch output

A typical minimal view is:

batch_res_min <- batch_res[, c(
  "wave_0", "wave_1",
  "metric_id", "group",
  "estimate", "se",
  "ci_lower", "ci_upper",
  "method"
)]

head(batch_res_min)
#>   wave_0 wave_1        metric_id   group   estimate         se     ci_lower
#> 1      0      1 engagement_score Group A 0.21304817 0.07617753  0.063742956
#> 2      1      2 engagement_score Group A 0.06255559 0.08388545 -0.101856870
#> 3      0      1 engagement_score Group B 0.28634627 0.07356196  0.142167481
#> 4      1      2 engagement_score Group B 0.18291451 0.08508930  0.016142549
#> 5      0      1        pay_index Group A 0.02683106 0.01661615 -0.005735988
#> 6      1      2        pay_index Group A 0.06924584 0.01943184  0.031160127
#>     ci_upper method
#> 1 0.36235338 linked
#> 2 0.22696805 linked
#> 3 0.43052506 linked
#> 4 0.34968647 linked
#> 5 0.05939811 linked
#> 6 0.10733154 linked

estimate: estimated change in the metric between wave_0 and wave_1.
se: standard error of that change.
ci_lower, ci_upper: normal-approximation confidence interval.
method: which estimator was actually used (linked, reliability, or bounds).

You can use standard dplyr / ggplot2 tooling to filter or visualize by metric, group, or wave pair.

Filtering by metric or group

To focus on engagement only:

eng_changes <- subset(batch_res_min, metric_id == "engagement_score")
eng_changes 
#>   wave_0 wave_1        metric_id   group   estimate         se    ci_lower
#> 1      0      1 engagement_score Group A 0.21304817 0.07617753  0.06374296
#> 2      1      2 engagement_score Group A 0.06255559 0.08388545 -0.10185687
#> 3      0      1 engagement_score Group B 0.28634627 0.07356196  0.14216748
#> 4      1      2 engagement_score Group B 0.18291451 0.08508930  0.01614255
#>    ci_upper method
#> 1 0.3623534 linked
#> 2 0.2269681 linked
#> 3 0.4305251 linked
#> 4 0.3496865 linked

To see differences for Group A vs Group B:

subset(batch_res_min,        metric_id == "engagement_score" &        wave_0 == 0 & wave_1 == 1)
#>   wave_0 wave_1        metric_id   group  estimate         se   ci_lower
#> 1      0      1 engagement_score Group A 0.2130482 0.07617753 0.06374296
#> 3      0      1 engagement_score Group B 0.2863463 0.07356196 0.14216748
#>    ci_upper method
#> 1 0.3623534 linked
#> 3 0.4305251 linked

When bounds are used

If there information on stayers is not available and no reliability estimate exists, method = "auto" may fall back to bounds. In that case, estimate / se are not available in the same way, and the result will use the bounds structure instead. In this example we supplied a reliability, so most comparisons will use linked or reliability.

For more detail on the scalar API and methods, see the Getting Started vignette

Summary

Use compare_waves() when you need a single, well-understood scalar comparison.
Use compare_waves_batch() when you need many comparisons across wave pairs, metrics, and groups, with a consistent, tabular result.

Both functions share the same underlying panel summary and variance logic, so batch results are method-consistent with the scalar API.

panel-comparisons