In statistical inference, non-parametric tests — also known as free distribution tests — are those where, despite being based on some hypotheses, observed data are assumed to follow a specific probability distribution, such as a normal distribution.

Non-parametric tests have the advantage of being versatile to apply, due to not having strict parameter requirements. They are generally based on simpler and easier-to-understand methods than parametric tests like the t-test. Furthermore, they can even be applied to non-numerical data, such as ordinal or binary (positive/negative) data, as well as to very small samples.

Meanwhile, these tests have the disadvantage that results are sometimes not conclusive enough, requiring additional sources of information or backups for their acceptance.

Some of the most popular types of non-parametric tests are Mann-Whitney U test, Wilcoxon signed-rank test, and Kruskal-Wallis test.

In the rest of the article, we will briefly describe the specific purpose of these three tests and illustrate their application using Python code.

Mann-Whitney U Test

One of the go-to solutions when our data doesn’t meet the assumption of normality is the Mann-Whitney U test. It is used to assess the existence (or absence) of significant differences between two samples of numerical data.

Python scipy library incorporates an implementation of this and other non-parametric test. The code below illustrates the application of the Mann-Whitney U test to compare the quality ratings between two groups of red wine samples associated with two Spanish regions: region A and region B.

import numpy as np
from scipy.stats import mannwhitneyu

# The two data samples, of size 30 each, are generated synthetically
np.random.seed(42)
region_a = np.random.randint(80, 95, size=30)
region_b = np.random.randint(85, 100, size=30)

# Apply Mann-Whitney U test
stat, p_value = mannwhitneyu(region_a, region_b, alternative='two-sided')

# Output the data
print(f"Region A ratings: {region_a}")
print(f"Region B ratings: {region_b}")
# Output the test results
print(f"Mann-Whitney U statistic: {stat}")
print(f"P-value: {p_value}")

# Interpretation
if p_value < 0.05:
    print("There is a significant difference in wine quality ratings between the two regions.")
else:
    print("There is no significant difference in wine quality ratings between the two regions.")

Some details about the previous code:

• The two samples of wine ratings are randomly generated within specified intervals: [80,95] for region A, and [85,100] for region B.
• The test produces a double output: the U statistic and a p-value. In the Mann-Whitney U test, the U statistic is a measure of overlap between the two groups when all values are merged and ranked together. A larger value indicates more overlap and therefore less statistical difference. Meanwhile, the p-value is a probability that, when low (below 0.05), indicates a statistical difference between the groups.

The current samples report a strong statistical difference between them according to this test. Try replacing the [85,100] interval for generating group B ratings by [81,96] and see how this affects the test results. Remember to remove the np.random.seed(42) if you want your test trials to be made upon truly randomly generated data between trials.

Wilcoxon Signed-Rank Test

Unlike the Mann-Whitney U test which compares two independent data groups to detect whether there is a statistical difference in their distributions or not, the Wilcoxon signed-rank test assumes both groups are related and the significant difference between their group medians is tested.

This test could be applied for instance if we want to analyze significant differences for two sets of ratings for wine produced in the same region: one before applying an innovative preservation technique, and one after applying it.

The application of Wilcoxon signed-rank test using Python is extremely similar to the previous example:

import numpy as np
from scipy.stats import wilcoxon

# Synthetic data generation: wine quality ratings before and after using a preservation technique
np.random.seed(42)
before = np.random.randint(85, 95, size=10)
after = before + np.random.randint(-2, 5, size=10)

# Apply Wilcoxon Signed-Rank Test
stat, p_value = wilcoxon(before, after)

# Output the data
print(f"Ratings before: {before}")
print(f"Ratings after: {after}")
#Output the test results
print(f"Wilcoxon statistic: {stat}")
print(f"P-value: {p_value}")

# Interpretation
if p_value < 0.05:
    print("There is a significant difference in wine quality ratings after applying the preservation technique.")
else:
    print("There is no significant difference in wine quality ratings after applying the preservation technique.")

The test statistic is calculated as the sum of the ranks of non-zero differences between paired samples, constituting an indicator of the magnitude and direction of changes between the two samples. Again, the Python implementation of this test applies these calculations internally for us.

Kruskal-Wallis Test

Last, the Kruskal-Wallis test is the go-to option when there are more than two data groups we want to compare in terms of statistically significant differences between their distributions. The below example is applied similarly as the previous two, this time evaluating the existence or absence of significant differences between wine ratings for producers in three different regions.

import numpy as np
from scipy.stats import kruskal

# Synthetic data generation
np.random.seed(42)
region_a = np.random.randint(80, 95, size=15)
region_b = np.random.randint(85, 100, size=15)
region_c = np.random.randint(75, 90, size=15)

# Apply Kruskal-Wallis test
stat, p_value = kruskal(region_a, region_b, region_c)

# Output the data
print(f"Region A ratings: {region_a}")
print(f"Region B ratings: {region_b}")
print(f"Region C ratings: {region_c}")
# Output the test results
print(f"Kruskal-Wallis statistic: {stat}")
print(f"P-value: {p_value}")

# Interpretation
if p_value < 0.05:
    print("There is a significant difference in wine quality ratings among the regions.")
else:
    print("There is no significant difference in wine quality ratings among the regions.")

The returned Kruskal-Wallis statistic compares the ranks of data values across groups. Higher values of the statistic are a stronger indicator of having significantly different distributions between groups. Therefore, unlike the Mann-Whitney U test, a higher statistic for Kruskal-Wallis test yields a lower p-value.

Wrapping Up

The three tests we have covered are interesting and valuable statistical inference tools to analyze and compare your data in the absence of normality assumptions.