Skip to contents

Meta-Analysis

Source: vignettes/meta-analysis.Rmd
meta-analysis.Rmd

Schema and parameters from Shin, Hart, and Simmons (2024) — multi-study word-problem-solving meta-analysis with four nesting levels (study / cluster / case / session). See wp_data.csv at OSF 8WBTA.

Data 

library(scdbayes)

make_study <- function(study_id, n_cases, base_mean, gain, complexity) {
  do.call(rbind, lapply(seq_len(n_cases), function(i) {
    n_a <- 4 + 2 * (i - 1)
    n_b <- 16 - n_a
    data.frame(
      study             = study_id,
      cluster           = 1,
      case              = paste0(study_id, "_C", i),
      phase             = c(rep("A", n_a), rep("B", n_b)),
      time              = seq_len(16),
      complexityMeasure = complexity,
      outcome           = round(c(
        rnorm(n_a, base_mean,        4),
        rnorm(n_b, base_mean + gain, 5)
      ))
    )
  }))
}

wp_meta <- rbind(
  make_study("S1", 3, 32, 28, "singleType"),
  make_study("S2", 3, 28, 22, "mixedType"),
  make_study("S3", 3, 35, 18, "generalization"),
  make_study("S4", 3, 30, 30, "singleType")
)
head(wp_meta)
#>   study cluster  case phase time complexityMeasure outcome
#> 1    S1       1 S1_C1     A    1        singleType      34
#> 2    S1       1 S1_C1     A    2        singleType      28
#> 3    S1       1 S1_C1     A    3        singleType      33
#> 4    S1       1 S1_C1     A    4        singleType      32
#> 5    S1       1 S1_C1     B    5        singleType      57
#> 6    S1       1 S1_C1     B    6        singleType      47

Columns

The hierarchy is read directly from named columns — no detection step is needed: study, cluster, case, phase ("A"/"B"), time, outcome, and complexityMeasure (a study-level moderator). Pass these names directly to the functions below.

Hierarchy levels 

detect_hierarchy_levels(wp_meta)[c("structure", "n_level1",
                                   "n_level2", "n_level3")]
#> $structure
#> [1] "three_level_no_cluster"
#> 
#> $n_level1
#> [1] 192
#> 
#> $n_level2
#> [1] 12
#> 
#> $n_level3
#> [1] 4

Meta summary 

calculate_meta_summary(wp_meta,
                       structure = list(levels = list(study = "study")))
#> $n_studies
#> [1] 4
#> 
#> $n_effects
#> [1] 192
#> 
#> $total_participants
#> [1] 12
#> 
#> $effects_per_study
#> $effects_per_study$min
#> [1] 48
#> 
#> $effects_per_study$mean
#> [1] 48
#> 
#> $effects_per_study$median
#> [1] 48
#> 
#> $effects_per_study$max
#> [1] 48

Covariates 

analyze_covariates(wp_meta,
                   structure = list(levels = list(study = "study",
                                                  case  = "case")))$categorical$complexityMeasure
#> $levels
#> [1] "generalization" "mixedType"      "singleType"    
#> 
#> $frequencies
#> [1] 48 48 96
#> 
#> $proportions
#> 
#> generalization      mixedType     singleType 
#>             25             25             50

Phase (piecewise) coding 

wp_meta_p <- process_scd_phases(wp_meta, structure = list(),
                                scd_type = "multiple_baseline")
head(wp_meta_p[, c("study", "case", "time", "phase",
                   "level_AB", "time_A", "trend_AB", "outcome")], 4)
#> # A tibble: 4 × 8
#>   study case   time phase level_AB time_A trend_AB outcome
#>   <chr> <chr> <int> <chr>    <int>  <dbl>    <dbl>   <dbl>
#> 1 S1    S1_C1     1 A            0      0        0      34
#> 2 S1    S1_C2     1 A            0      0        0      34
#> 3 S1    S1_C3     1 A            0      0        0      31
#> 4 S2    S2_C1     1 A            0      0        0      31

Random-effects structure 

build_default_random_effects(wp_meta_p, scd_type = "ab",
                             case_var = "case", study_var = "study")
#> [1] "(1 + level_AB | study) + (1 + level_AB | case) + (1 + level_AB | case:complexityMeasure)"

The app’s Model > Random Effects tab exposes the same structure as four checkboxes (case/study intercept and slope). With all four on it emits the equivalent (1 + phase | case) + (1 + phase | study); the random coefficient is placed on the level-change term level_AB (the immediate phase shift) once phases are piecewise-coded, distinct from the trend-change term trend_AB.

Three-level model 

three_level <- outcome ~ time_A + level_AB + trend_AB +
                         (level_AB | study) +
                         (level_AB | case)
three_level
#> outcome ~ time_A + level_AB + trend_AB + (level_AB | study) + 
#>     (level_AB | case)

Four-level model 

four_level <- outcome ~ time_A + level_AB + trend_AB +
                        complexityMeasure +
                        complexityMeasure:level_AB +
                        complexityMeasure:trend_AB +
                        (level_AB | study/cluster/case)
four_level
#> outcome ~ time_A + level_AB + trend_AB + complexityMeasure + 
#>     complexityMeasure:level_AB + complexityMeasure:trend_AB + 
#>     (level_AB | study/cluster/case)

Fitting the Bayesian model 

opts <- list(formula = three_level, family = "gaussian",
             prior_type = "default")
if (interactive()) {
  result <- run_scd_analysis(wp_meta_p, options = opts)
  result$diagnostics$converged
  summarize_fixed_effect_decisions(result$model)
}

Per-case decisions

With case-level random effects, each case carries its own posterior. The app’s Per Study tab and summarize_per_case_effects() report these case-specific decisions.

if (interactive()) {
  summarize_per_case_effects(result$model,
                             intervention_goal = "increase")
}

Source study

  • Shin, M., Hart, S. L., & Simmons, M. (2024). Meta-analysis of single-case design research: Application of multilevel modeling. School Psychology, 39(6), 625-635. https://doi.org/10.1037/spq0000637