McNemar’s test in R

Performing McNemar’s test in R

The McNemar test is a statistical test used to determine if there are significant differences in the paired proportions of categorical data. It is particularly useful for testing the change in responses before and after an intervention on the same subjects, such as in pre-test/post-test study designs. By comparing the discordant pairs, where outcomes change in one direction versus the opposite, the McNemar test assesses if the intervention or treatment has a statistically significant effect on the outcome.

To demonstrate McNemar’s test, data was used concerning the presence or absence of cold symptoms reported by the same children at age 12 and age 14. A total of 2638 participants were involved, with the interest being in whether the prevalence was significantly different at either age. In order to use any of the packages to calculate first we need to convert our data into a frequency table.

Example Code using {stats}

McNemar’s test can also be performed using stats::mcnemar.test as shown below, using the same table X as in the previous section.

mcnemar.test(freq_tbl)


    McNemar's Chi-squared test with continuity correction

data:  freq_tbl
McNemar's chi-squared = 30.802, df = 1, p-value = 2.857e-08

The other possible input to mcnemar.test is correct = which is a true/false value to indicate if the continuity correction should be applied. To get the result without continuity correction by specify correct=FALSE.

mcnemar.test(freq_tbl, correct=FALSE)


    McNemar's Chi-squared test

data:  freq_tbl
McNemar's chi-squared = 31.36, df = 1, p-value = 2.144e-08

Example Code using {coin}

McNemar test is calculated without continuity correction which corresponds to SAS FREQ procedure (see R v SAS McNemar’s test for more details).

library(coin)

Loading required package: survival

## Asymptotic McNemar Test
# Corresponds to SAS FREQ procedure
coin::mh_test(freq_tbl)


    Asymptotic Marginal Homogeneity Test

data:  response by
     conditions (age12, age14) 
     stratified by block
chi-squared = 31.36, df = 1, p-value = 2.144e-08

## Approximative McNemar Test
coin::mh_test(
  freq_tbl, 
  distribution = coin::approximate(nresample = 10000)
)


    Approximative Marginal Homogeneity Test

data:  response by
     conditions (age12, age14) 
     stratified by block
chi-squared = 31.36, p-value < 1e-04

## Exact McNemar Test
coin::mh_test(freq_tbl, distribution = "exact")


    Exact Marginal Homogeneity Test

data:  response by
     conditions (age12, age14) 
     stratified by block
chi-squared = 31.36, p-value = 2.331e-08

Calculating Confidence Intervals with {vcd}

It is common when calculating McNemar, to also want to check the level agreement between the two rates. To do this we use Cohen’s Kappa and its associated confidence intervals. To do this, we use the vcd package as follows:

library(vcd)

Loading required package: grid

cohen_kappa <- Kappa(freq_tbl)
cohen_kappa

            value     ASE     z Pr(>|z|)
Unweighted 0.2999 0.02733 10.97 5.07e-28
Weighted   0.2999 0.02733 10.97 5.07e-28

confint(cohen_kappa, level = 0.95)

            
Kappa              lwr       upr
  Unweighted 0.2463654 0.3534966
  Weighted   0.2463654 0.3534966

Using the `epibasix::mcnemar` function

Another package that is sometimes used to calculate McNemar is {epibasix}. This package isn’t recommended by CAMIS. It was found that the author is no longer maintaining the package and there was no documentation available for certain methods used. Therefore, the use of the {epibasix} package is advised against and other packages may be more suitable.

Results

stats::mcnemar.test and coin::mh_test are both sutiable options for calculating McNemar. The {coin} package also has a variety of other marginal homogeneity test, for more information see here. stats::mcnemar.test uses a continuity correction as default but does allow for this to be removed. This function does not output any other coefficients for agreement or proportions but (if required) these can be achieved within other functions or packages in R.

Reference

Agresti, A. (2002). Categorical Data Analysis, Second Edition. Hoboken, New Jersey: John Wiley & Sons.

Cohen, J. (1960), A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46.

Everitt, B.S. (1968), Moments of statistics kappa and weighted kappa. The British Journal of Mathematical and Statistical Psychology, 21, 97–103.

Fleiss, J.L., Cohen, J., and Everitt, B.S. (1969), Large sample standard errors of kappa and weighted kappa. Psychological Bulletin, 72, 332–327.