Contact
Inter-rater reliability
Data scientist in Basel
-1
archive,category,category-inter-rater-reliability,category-18,bridge-core-3.1.8,qode-page-transition-enabled,ajax_fade,page_not_loaded,,qode_grid_1300,side_area_uncovered_from_content,overlapping_content,qode-content-sidebar-responsive,qode-theme-ver-30.5,qode-theme-bridge,disabled_footer_top,qode_header_in_grid,wpb-js-composer js-comp-ver-7.6,vc_responsive

Inter-rater reliability

The survey Process Models in Judgment and Decision Making is an expert survey we conducted to find out about researchers' opinion about the term "process model" and whether it applies to several models in the field. The main objective was to find out whether researchers'...

25 December, 2014

Inter rater reliability

If you want to obtain inter-rater reliability measures for dichotomous ratings, by more than two raters, but not all raters rated all items, Fleiss and Cuzick (1979) will be the referece you'll find. For example, we asked researchers which model they consider a process model (dichotomous rating), and we asked about 60 researchers (more than two rater), of whom not everyone was familiar with every model (not all raters rated all items). They proposed a measure between a minimum value and 1. Here is an function to calculate their kappa measure in R.
24 December, 2014