Which reliability reflects agreement among different scorers assessing the same test?

• A. Intra-rater consistency
• B. Inter-rater consistency
• C. Test-retest reliability
• D. Internal consistency reliability

Answer: (B)

The key idea here is how similarly different evaluators score the same performance. When multiple scorers assess the same test and their ratings align, that shows inter-rater reliability. It tells you that the scoring is dependable across different observers. In practice, you measure this with statistics like Cohen’s kappa for two raters, Fleiss’ kappa for several raters, or the intraclass correlation coefficient for continuous ratings. This is distinct from intra-rater reliability (the same person scoring consistently over time), test-retest reliability (stability of scores across time), and internal consistency reliability (how well items on a test relate to each other). So agreement among different scorers assessing the same test is inter-rater reliability.