What is a Kappa coefficient? (Cohen's Kappa)

What is a Kappa coefficient? (Cohen's Kappa)

When two binary variables are attempts by two individuals to measure the same thing, you can use Cohen's Kappa (often simply called Kappa) as a measure of agreement between the two individuals.

Kappa measures the percentage of data values in the main diagonal of the table and then adjusts these values for the amount of agreement that could be expected due to chance alone.

Two raters are asked to classify objects into categories 1 and 2. The table below contains cell probabilities for a 2 by 2 table.

To compute Kappa, you first need to calculate the observed level of agreement

This value needs to be compared to the value that you would expect if the two raters were totally independent,

The value of Kappa is defined as

The numerator represents the discrepancy between the observed probability of success and the probability of success under the assumption of an extremely bad case. Independence implies that pair of raters agree about as often as two pairs of people who effectively flip coins to make their ratings.

The maximum value for kappa occurs when the observed level of agreement is 1, which makes the numerator as large as the denominator. As the observed probability of agreement declines, the numerator declines. It is possible for Kappa to be negative, but this does not occur too often. In such a case, you should interpret the value of Kappa to imply that there is no effective agreement between the two rates.

How to interpret Kappa

Kappa is always less than or equal to 1. A value of 1 implies perfect agreement and values less than 1 imply less than perfect agreement.

In rare situations, Kappa can be negative. This is a sign that the two observers agreed less than would be expected just by chance.

It is rare that we get perfect agreement. Different people have different interpretations as to what is a good level of agreement. At the bottom of this page is one interpretation, provided on page 404 of Altman DG. Practical Statistics for Medical Research. (1991) London England: Chapman and Hall.

Here is one possible interpretation of Kappa.

• Poor agreement = Less than 0.20
• Fair agreement = 0.20 to 0.40
• Moderate agreement = 0.40 to 0.60
• Good agreement = 0.60 to 0.80
• Very good agreement = 0.80 to 1.00

An example of Kappa

In an examination of self reported prescription use and prescription use estimated by electronic medical records

• http://www.biomedcentral.com/1472-6963/6/115

 the following table was observed.

4.5% 11.2%
10.6% 73.8%

 The value for Kappa is 0.16, indicating a poor level of agreement.