R v SAS McNemar’s test

Introduction

McNemar’s test is a test of marginal homogeneity. That is used with 2x2 contingency tables, when both x and y are binary factors.

General Comparison Table

The following table provides an overview of the support and results comparability between R and SAS for the new analysis point.

Analysis Supported in R Supported in SAS Results Match Notes
McNemar’s Chi-Squared test Yes Yes ✅ By default SAS doesn’t include the continuity correction.
Cohen’s Kappa CI Yes Yes ✅

In R,the {stats} or the {coin} package can be used to calculate McNemar. The {coin} package has the same defaults as SAS. But, using either of these packages, the first step is to calculate a frequency table, using the table function.

library(coin)
Loading required package: survival
colds <- 
  read.csv(
    file = "../data/colds.csv")
freq_tbl <- table("age12" = colds$age12, "age14" = colds$age14)
freq_tbl
     age14
age12  No Yes
  No  707 256
  Yes 144 212
coin::mh_test(freq_tbl)

    Asymptotic Marginal Homogeneity Test

data:  response by
     conditions (age12, age14) 
     stratified by block
chi-squared = 31.36, df = 1, p-value = 2.144e-08

In order to get Cohen’s Kappa an additional package is needed.

library(vcd)
Loading required package: grid
cohen_kappa <- Kappa(freq_tbl)
cohen_kappa
            value     ASE     z Pr(>|z|)
Unweighted 0.2999 0.02733 10.97 5.07e-28
Weighted   0.2999 0.02733 10.97 5.07e-28
confint(cohen_kappa, level = 0.95)
            
Kappa              lwr       upr
  Unweighted 0.2463654 0.3534966
  Weighted   0.2463654 0.3534966

The FREQ procedure can be used in SAS with the AGREE option to run the McNemar test, with OR, and RISKDIFF options stated for production of odds ratios and risk difference. These options were added as epibasix::mcNemar outputs the odds ratio and risk difference with confidence limits as default. In contrast to R, SAS outputs the Kappa coefficients with confident limits as default.

proc freq data=colds;
    tables age12*age14 / agree or riskdiff;
run;

Summary and Recommendation

When calculating the odds ratio and risk difference confidence limits, SAS is not treating the data as matched-pairs. There is advice on the SAS blog and SAS support page to amend this, which requires a lot of additional coding.

{stats} is using Edward’s continuity correction by default, but this can be removed. In contrast, there is no option to include Edward’s continuity correction in SAS, but this can be manually coded to agree with R. However, its use is controversial due to being seen as overly conservative.

There is another R package that is sometimes used to calculate McNemar’s, called epibasix. This package is no longer being maintained, and there was no documentation available for certain methods used. Therefore, the use of the epibasix package is advised against and other packages may be more suitable.

─ Session info ───────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.4.3 (2025-02-28)
 os       Ubuntu 24.04.2 LTS
 system   x86_64, linux-gnu
 ui       X11
 language (EN)
 collate  C.UTF-8
 ctype    C.UTF-8
 tz       Europe/London
 date     2025-03-13
 pandoc   NA (via rmarkdown)

─ Packages ───────────────────────────────────────────────────────────────────
 ! package * version date (UTC) lib source
 P coin    * 1.4-3   2023-09-27 [?] RSPM (R 4.4.0)

 [1] /home/michael/source/CAMIS/renv/library/linux-ubuntu-noble/R-4.4/x86_64-pc-linux-gnu
 [2] /opt/R/4.4.3/lib/R/library

 P ── Loaded and on-disk path mismatch.

─ External software ──────────────────────────────────────────────────────────
 setting value
 SAS     9.04.01M7P080520

──────────────────────────────────────────────────────────────────────────────