**Fleiss’ Kappa **is a way to measure the degree of agreement between three or more raters when the raters are assigning categorical ratings to a set of items.

Fleiss’ Kappa ranges from 0 to 1 where:

**0**indicates no agreement at all among the raters.**1**indicates perfect inter-rater agreement.

This tutorial provides an example of how to calculate Fleiss’ Kappa in Excel.

**Example: Fleiss’ Kappa in Excel**

Suppose 14 individuals rate 10 different products on a scale of Poor to Excellent.

The following screenshot displays the total ratings that each product received:

The following screenshot shows how to calculate Fleiss’ Kappa for this data in Excel:

The trickiest calculations in this screenshot are in column J. The formula used for these calculations is shown in the text box near the top of the screen.

Note that the Fleiss’ Kappa in this example turns out to be **0.2099**. The actual formula used to calculate this value in cell C18 is:

Fleiss’ Kappa = (0.37802 – 0.2128) / (1 – 0.2128) = 0.2099.

Although there is no formal way to interpret Fleiss’ Kappa, the following values show how to interpret Cohen’s Kappa, which is used to assess the level of inter-rater agreement between just two raters:

- < 0.20 | Poor
- .21 – .40 | Fair
- .41 – .60 | Moderate
- .61 – .80 | Good
- .81 – 1 | Very Good

Based on these values, Fleiss’ Kappa of **0.2099 **in our example would be interpreted as a “fair” level of inter-rater agreement.

Hi Zach

Can you explain column K? Why do you divide 14 with 13? What does 13 mean/represent?

Where did the 13 in cell K2 come from? Was it obtained by subtracting one from the number of individuals? (14-1)

I have a question. In cell K2, where you have sum sq.c/(14*13), I understand the 14 to be the number of people who scored/rated, however where does the 13 come from? Why are we multiplying by that number?

Thank you!

In row K, why did you multiply 14 by 13?

Hi Zach,

Im trying to follow your procedure in Excel. However, I dont understand where you get 13 from in your sum sq. c/(14*13). Is it number of observers -1? In that case: I have 5 observers testing 64 subjects. Would my equation be: sum sq. c/(5*4)?

Thanks in regards for your response.

Rasmus Gudmundsen