Bootstrapping Krippendorff's Alpha

Question

I should like to calculate - by bootstrapping Krippendorff\'s Alpha outcomes - a 95% confidence interval for Krippendorff\'s Alpha coefficient of Raters Reliability using R

Answer 1

You can use the boot function from the boot package to bootstrap values. Here I'll bootstrap the set of subjects but keep the raters fixed:

library(boot)
library(irr)
ka <- function(data, indices) kripp.alpha(nmm[,indices], "ordinal")$value
b <- boot(seq(ncol(nmm)), ka, 1000)

Now you can use the boot.ci function to compute a 95% confidence interval for the bootstrapped value; I'll use the percentile confidence interval, but others are available (check out ?boot.ci):

boot.ci(b, type="perc")
# BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
# Based on 1000 bootstrap replicates
# 
# CALL : 
# boot.ci(boot.out = b, type = "perc")
# 
# Intervals : 
# Level     Percentile     
# 95%   ( 0.4297,  1.0000 )  
# Calculations and Intervals on Original Scale