What is inter-observer reliability?

What is inter-observer reliability?

• A. Consistency of reports by the same observer across time.
• B. Consistency of reports across different observers.
• C. Consistency of a measure when repeated at two points in time.
• D. The relation to other measures as theory predicts.

Answer: (B)

Inter-observer reliability is about consistency across different observers. When more than one person is rating, coding, or observing the same event or data, you want them to arrive at similar conclusions. High inter-observer reliability means the measurement isn’t dependent on which observer did the rating, which strengthens the credibility of the data. This is typically assessed by comparing the observers’ scores or classifications and using statistics such as percent agreement, Cohen’s kappa, or intraclass correlation. This concept is different from intra-observer reliability, which focuses on the same person being consistent across time; test-retest reliability, which looks at stability of a measure over time; and validity considerations, which examine how well a measure relates to other related constructs or outcomes.

Inter-observer reliability is about consistency across different observers. When more than one person is rating, coding, or observing the same event or data, you want them to arrive at similar conclusions. High inter-observer reliability means the measurement isn’t dependent on which observer did the rating, which strengthens the credibility of the data. This is typically assessed by comparing the observers’ scores or classifications and using statistics such as percent agreement, Cohen’s kappa, or intraclass correlation.

This concept is different from intra-observer reliability, which focuses on the same person being consistent across time; test-retest reliability, which looks at stability of a measure over time; and validity considerations, which examine how well a measure relates to other related constructs or outcomes.