R
-1
archive,tag,tag-r,tag-14,bridge-core-1.0.5,ajax_fade,page_not_loaded,,qode-theme-ver-18.1,qode-theme-bridge,disabled_footer_top,disabled_footer_bottom,wpb-js-composer js-comp-ver-6.0.2,vc_responsive

R Tag

Inter rater reliability

If you want to obtain inter-rater reliability measures for dichotomous ratings, by more than two raters, but not all raters rated all items, Fleiss and Cuzick (1979) will be the referece you'll find. For example, we asked researchers which model they consider a process model (dichotomous rating), and we asked about 60 researchers (more than two rater), of whom not everyone was familiar with every model (not all raters rated all items). They proposed a measure between a minimum value and 1. Here is an function to calculate their kappa measure in R.

Today, Robert posted the nice idea to save seeds of simulations. He recommends storing seeds with results, alternatively, store seeds with starting parameter values if you run numerical optimization. It is also useful to create an external file which stores parameters and seeds with a unique identifier which one can use for naming results folders and file organization. Th is post contains a code to do the latter.