When working with data, we want to arrive at an accurate conclusion from our data. To do so, we could rely on statistical hypothesis tests to infer whether the data we have already supported our conclusion (hypothesis).
However, there are times when our testing results provide errors, leading to wrong conclusions. This is what we call a statistical error.
This article will explore specific errors in hypothesis tests, especially the statistical error Type I and Type II.
Hypothesis Testing
Before we start learning about statistical error types, we need to understand what hypothesis testing is.
In general, hypothesis testing is a way to make inferences about the population based on the sample data by deciding if the sample data we have sufficiently supports a particular hypothesis about the population.
Let’s take an example of an Independent t-test. It’s a simple test where we try to assess if there is a significant difference between the means of two groups or if it happens by chance.
Imagine if you have a coffee shop and you introduce a new blend. Then, you want to see if the new blend satisfies the customer better. In this case, we can collect customer ratings from two groups: those who tried the new blend and those who tried the old blend.
Using a t-test, we set up a hypothesis where we have a null hypothesis (H₀) that the average satisfaction ratings for the new blend are equal to the old blend. We also set up the alternative hypothesis (H₁), where the average satisfaction ratings for the new blend is greater than the old blend.
By using a t-test, we can either reject the null hypothesis or fail to reject the null hypothesis.
You gain your result, but there is a risk that the statistical error will happen. But what are these error means?
Statistical Error Types
When we perform statistical tests and draw conclusions from the test, it always involve uncertainties, which means error is present.
Two types of errors could happen: Type I and Type II errors. A Type I error is where we have a false positive conclusion, while a Type II error is when we have a false negative conclusion.
Type I error or false positive happens when you reject the null hypothesis while the null hypothesis is actually true. If we are using the previous example, imagine if we conclude that the new blend is doing better than the old one while the reality isn’t.
The Type I error rate is denoted by the α (alpha) or significance level we set. Often, we set it as 0.05 (5%), but we can always change it depending on how strict we are with the criteria for rejecting the null hypothesis.
In contrast, a Type II error or false negative means you accept the null hypothesis, but the null hypothesis is actually false. With the previous example, there are no differences between the new and old blends, while the reality is that the new blend is better than the old one.
Type II error rate is denoted by the β (beta) and has an inverse relationship with the power analysis (1 – β). If we set the power analysis as 80%, then the risk of Type II error is 20%. We will not discuss the power analysis calculation further as it will take its article, but the 20% assumption is often the general standard.
That’s what you need to know about statistical error types.
