Image by Editor | Midjourney & Canva

Introduction

Statistics is a cornerstone discipline, indeed. In the era of data, its principles, tools, and applications are more important and omnipresent than ever before. Moreover, both machine learning and other subareas of artificial intelligence heavily rely on concepts and methods taken from statistics; hence, being familiar with its fundamentals is synonymous with being in an advantageous position to harness these revolutionary fields and all things data as a whole.

For a solid start or freshen-up, this article explores and demystifies 10 key statistical terms and concepts.

1. Probability Distribution

Definition: A central notion in the realm of statistics, a probability distribution is a function that describes the likelihood of different outcomes in a random experiment. Whilst this definition sounds somewhat “experimental”, there are different kinds of probability distributions, defined to model real-world data with diverse characteristics in terms of shape, spread, and central tendency. For instance, the normal distribution models data that is predominantly centered around a mean value and spread symmetrically, whereas the uniform distribution reflects situations where all outcomes are equally (or almost equally) likely.

Why it’s key: Probability distributions are extremely useful to model and manage data and uncertainty in a variety of scenarios: from assessing risk in medical studies to predicting stock market behavior, they enable quantifying likelihoods and make informed decisions.

2. Hypothesis Testing

Definition: Analyzing data and investigating their properties often involves hypothesis testing, which encompasses a set of methods to formally test assumptions (hypotheses) and validate statistical claims, for instance, whether a new drug is more effective than a placebo, whether there is significant difference between the average income of two demographic groups of people, etc. This is a rigorous and effective approach to determine if observed effects are statistically significant or rather due to chance.

Why it’s key: in processes like quality control, scientific research, and medical trials, hypothesis testing is key to guarantee that conclusions are evidence-based rather than subjective or arbitrary, helping to avoid undesired results or misleading insights due to random variations.

3. P-value

Definition: A key metric used in hypothesis testing is the P-value, which is a quantitative way to assess whether the observed data provides enough evidence to reject a so-called null hypothesis or not — a null hypothesis is a statement of “no significant effects or differences” assumed as true unless evidence given by the P-value indicates otherwise. In general, the P-value is interpreted as follows: if its value is smaller than 0.05, the result being analyzed (effect, difference, etc.) is considered statistically significant, and the null hypothesis is rejected. Otherwise, there is not sufficient evidence to do so.

Why it’s key: Making decisions or drawing reliable conclusions from a hypothesis testing process is made possible thanks to the interpretability and threshold-driven reasoning behind P-values.

4. Confidence Interval

Definition: A confidence interval is a range of values, e.g., [48.7, 52.3], likely containing the true value of something we’re trying to estimate based on a sample of data — for example, the average height of a population of cats. For instance, a 95% confidence interval of [48.7, 52.3] means we’re 95% confident that the true average height of a cat population falls somewhere within that range. The interval can be made wider or narrower depending on how much data we have available and how spread out the values are. Wider intervals are normally associated with more uncertainty, often because the sample is small or the data varies substantially.

Why it’s key: Thanks to confidence intervals, it is possible to measure uncertainty by assessing, for instance, how likely the true average income of a population is to fall within a specified range, based on a sample of people’s incomes.

5. Correlation vs. Causation

Definition: A common misconception in statistics is that of associating correlation with causation. While correlation indicates a statistical relationship or pattern between two data variables or groups, it does not imply that one causes the other. Take, for instance, the number of ice creams sold and heatstroke incidents — both rise in summer and may be positively correlated, but one is not the cause for the other. This article further explores the correlation vs. causation nuances.

Why it’s key: Having a clear understanding of these two similar phenomena is helpful to interpret data responsibly and avoid practical misconceptions, like assuming a medical treatment works simply because recovery rates are higher in its users.

6. Sampling Bias

Definition: Sampling data is often necessary when collecting information from an entire population is impractical or impossible. But sampling is not a process to do blindly — it demands a balanced approach to ensure representativeness, particularly to avoid sampling bias, which refers to systematic errors that arise when certain members of the population are more likely to be included than others.

Why it’s key: sampling bias can lead to undesirable consequences in analysis processes or in the process of building machine learning models, for instance, developing a credit scoring system trained on sampled data that accidentally excludes low-income applicants: this could yield unfair or even harmful decisions due to unrepresentative source data.

7. Descriptive Statistics

Definition: one of the main pillars or subareas of statistics, and often the starting points for anyone starting the study of the field, descriptive statistics encompasses the measures and methods to to summarize, organize, and describe the main features of a dataset, such as the mean, dispersion measures like standard deviation, skewness, and so on.

Why it’s key: among the building blocks of descriptive statistics, there are several metrics whose use is extremely extended in virtually every application domain and even certain aspects of our daily lives — such as using averages to compare product prices, or ranges and variability to evaluate academic or sports performance.

8. Inferential Statistics

Definition: Inferential statistics is another of the major subareas in statistics, and it focuses on making predictions or generalizations about a population based on data from a sample. Concepts explained earlier, like hypothesis testing, confidence intervals, and P-values are cornerstone elements in inferential statistics, along with specific testing methodologies like t-tests, chi-squared tests, and ANOVA.

Why it’s key: just like hypothesis testing and other related concepts, inferential statistics is behind many important real-world processes like like detecting defects in manufacturing in industry, or drawing conclusions from scientific clinical trials.

9. Central Limit Theorem

Definition: The central limit theorem states that the mean of many independent and identically distributed samples of data drawn from a population tend to follow a normal distribution, even for populations that are not normally distributed. This article further examines this concept. As a result, relevant underlying patterns and facts behind the original data can often be revealed without resorting to more mathematically complex methods.

Why it’s key: The central limit theorem makes it possible to apply various statistical methods that require normally distributed data inputs, for instance, confidence intervals and many hypothesis tests.

10. Bayesian Inference

Definition: A more advanced and “philosophically different” branch of statistics, Bayesian inference takes a step forward in incorporating prior knowledge and focuses on the reasoning process over probabilities (new or incoming knowledge) that are continuously updated as new data becomes available.

Why it’s key: Bayesian inference is widely used in applications like medical diagnosis, recommender systems in e-commerce, spam e-mail detection, and more. In essence, it is useful in decision-making scenarios under uncertainty where prior information plays a central role.

Wrapping Up

This article discussed 10 key terms associated to the field of statistics and several of its main subareas: from descriptive statistics to statistical inference to more advance subdomains like bayesian inference.