Image by Author

Propensity score matching (PSM) is a statistical technique that helps estimate treatment effects when you can’t randomly assign participants to groups. Instead of comparing groups that might be naturally different, it creates balance by matching similar participants from treatment and control groups.

The technique works by estimating, for each participant, a single score — the probability of receiving the treatment given observed characteristics. You then pair people with similar scores from different groups, creating a comparison that mimics what you’d get from a randomized experiment.

Whether you should use propensity score matching depends on your study design, your data, and what alternatives you have available.

When Propensity Score Matching Makes Sense

Propensity score matching works best in specific situations where traditional approaches struggle.

Your study uses observational data. When you’re analyzing data where treatment assignment wasn’t random, you face selection bias. People who received a treatment might differ from those who didn’t. Propensity score matching helps reduce bias from measured confounders by creating comparable groups based on observed characteristics.

You have many confounders relative to your sample size. When you need to balance groups on numerous characteristics, matching on each variable individually becomes impractical unless you have a large sample. Propensity score methods reduce dimensionality by collapsing multiple confounders into a single score, making the analysis more manageable.

Your outcome events are rare. When you have few outcome events relative to the number of confounding variables, fitting complex regression models can become problematic. Propensity score matching can provide a more stable approach in these situations.

Treatment groups are imbalanced. When one group is much larger than the other, standard regression approaches can produce unreliable estimates. Propensity score matching can handle this imbalance more robustly.

You need to verify comparability before analysis. Plotting propensity score distributions reveals when groups are inherently incomparable (which traditional methods might miss). If the distributions barely overlap (violating the overlap/positivity assumption), no statistical method can overcome this limitation.

When Propensity Score Matching Falls Short

Despite its advantages, propensity score matching has important limitations that might make other approaches better.

Randomized trials are possible. Nothing beats true randomization. It’s the only approach that reliably balances both known and unknown confounders between treatment groups. If you can conduct a randomized experiment, do it.

You can’t measure important confounders. Propensity score matching can only balance groups on variables you’ve measured. It can’t address unmeasured confounding. If important variables are missing (which you often can’t verify), your results may be biased. This is a fundamental limitation: unlike randomization, which balances both measured and unmeasured variables, matching only addresses what you’ve observed. This is especially problematic when working with administrative databases collected for purposes other than research.

Groups don’t overlap in propensity scores. When matches aren’t found, many treated participants may be excluded from the final matched sample. This can reduce generalizability and precision because you are estimating effects for a narrower subset of the data.

Your outcome is common and the sample is large. When you have many outcome events and a large sample, standard multivariable regression often performs as well or better. Propensity score matching can reduce statistical power by discarding unmatched observations.

Time or resources are limited. Propensity score analysis requires multiple steps: estimating scores, choosing a matching algorithm, checking balance, and analyzing outcomes. Traditional regression is faster and simpler when its assumptions hold.

Alternative Approaches to Consider

Before committing to propensity score matching, consider these alternatives.

Multivariable regression adjusts for confounders directly in the outcome model. This works well with large samples and when you can specify the correct functional form of relationships. It keeps all your data and runs quickly.

Inverse probability weighting uses propensity scores differently. It weights observations rather than matching them. This approach retains more data but can be unstable when weights become extreme.

Stratification divides participants into groups based on propensity scores and estimates effects within each stratum. This provides a middle ground between matching and regression approaches.

Regression discontinuity or instrumental variables might be options if your study design includes specific features like cutoff points for treatment assignment or natural experiments.

Making the Decision

Choose propensity score matching when you have observational data with many confounders, especially when events are rare or groups are imbalanced. The technique provides transparent comparison of similar participants and reveals when groups aren’t comparable.

Skip it when randomization is possible, when you suspect unmeasured confounding is substantial, or when standard regression assumptions hold and you have sufficient sample size. The loss of unmatched observations and added complexity may not justify the benefits.

The decision depends on your specific research context. Consider your data structure, whether you can measure all relevant confounders, how much overlap exists between groups, and whether you value the transparency of matched comparisons over keeping your full sample.

Propensity score matching is a tool, not a solution to all observational study problems. Use it when it addresses your specific challenges better than alternatives.