112013-03-08 10:17:24localhostconverted to 1.6 markup102010-08-11 15:55:24PeterWatson92010-08-11 15:52:42PeterWatson82010-08-11 15:52:23PeterWatson72010-08-11 15:48:26PeterWatson62010-08-11 15:43:42PeterWatson52010-08-11 15:38:18PeterWatson42010-08-11 15:36:51PeterWatson32010-08-10 15:59:10PeterWatson22010-08-10 15:58:57PeterWatson12010-08-10 15:58:48PeterWatson

The code below can be pasted at a R prompt in R. Firstly read in the fields and foreign libraries. Next read in the raw data which can be read in from a SPSS spreadsheet with items as rows and raters as columns. An example input for 3 raters (r1, r2 and r3) rating each of 5 items (upto a maximum rating of 7) taken from section 5.2 of Kreuzpointner et al. (2010) is given here. Just paste this function text below into R. 0, dmax <- nitem * (b-a)*(b-a)*0.25*(nrate*nrate-1), dmax <- nitem * (b-a)*(b-a)*0.25*(nrate*nrate)) ]]>Then you are ready to run the function above using the maximum number of ratings and 1- the significance level as inputs. which should output something like the below So there is no evidence that there is an agreement between the raters. Note: we get a slightly different result to Kruezpointner et al. because we use a binomial probability of 0.15 instead of 0.2, as in their paper, to estimate the critical 5% threshold for statistical significance. Reference Kreuzpointner L, Simon P and Theis FJ (2010) The ad coefficient as a descriptive measure of the within-group agreement of ratings. British Journal of Mathematical and Statistical Psychology 63 341-360.