Bootstrapping Krippendorff's Alpha

Question

I should like to calculate - by bootstrapping Krippendorff\'s Alpha outcomes - a 95% confidence interval for Krippendorff\'s Alpha coefficient of Raters Reliability using R

Answer 1

Probably too late to be helpful for Paul, but for future reference note that none of the proposed methods is consistent with the bootstrap algorithm described by Klaus Krippendorff (http://web.asc.upenn.edu/usr/krippendorff/boot.c-Alpha.pdf).

The repeated samples do neither draw raters nor units, but "hypothetical reliability data from the matrix of observed coincidences among pairs of values assigned to units in independent replications" (Krippendorff 2016). Thus, conventional bootstrap implementations will not give the answer intended by Krippendorff.

Best Daniel