Image by Editor | Midjourney
Introduction
Factorial design, crossover design, the distinction between between-subjects and within-subject design, etc. They are all examples of experimental design practices, that is, approaches to structuring experiments and analyzing causal relationships among data managed in them. Experimental design is widely applied and necessary in disciplines as diverse as psychology, medicine, social sciences, engineering, and biology..
This article is intended to help you choose the right experimental design approach depending on your research aims, constraints underlying your data, and the nature of your data variables.
Decision Tree to Choose the Right Experiment Design
The diagram below depicts a visual guide to help you choose the right experimental design for your study, based on randomness considerations, subject assignment, and other practical constraints.
Choosing the right experimental design approach (click to enlarge)
Since the concepts of subject and treatment are two central elements in this classification, it is worthwhile to clarify their meanings in the context of experimental design. A subject is an observation unit to which (or to whom) treatments are applied. Meanwhile, a treatment is a specific condition or intervention that applies to the subject. For instance, in the field of engineering, subjects could be components being manufactured, whereas treatments could be different stress levels, temperatures, or other manufacturing methods used to produce them.
Having clarified these concepts, it’s time to navigate the visual decision tree. First, we ask ourselves whether subjects can be randomly assigned to treatments. If they cannot, we normally resort to quasi-experimental design, typically necessary when random assignment might be unethical, logistically unfeasible, or simply impossible. Otherwise, we move on to ask additional questions leading to the right approach, which won’t necessarily be a completely randomized design.
Repeated measures is the name of the approach used when each subject can receive more than one treatment. This is what happens, for instance, in clinical trials when patients are given multiple drugs or dosages over time, and it leads to the use of within-subjects approaches—like crossover design when carryover effects between treatments are a concern, or, on the contrary, to within-subjects factorial design when such effects are not a concern.
Back to the question concerning repeated measures: when they are not applicable — e.g., there is only one observation per subject or treatments cannot be repeated on the same unit — we move on to between-subjects approaches. Unlike within-subject approaches, between-subjects methods consider independent groups of subjects, each exposed to just one treatment.
The next question to ask is whether we expect major variability between subjects that we’d like to control for. For example, subjects might differ significantly in age, baseline health status, material type, or operating conditions. If the answer is affirmative (major variability expected), we must check if subjects can be grouped or matched in pairs. If they can be grouped into categories like age groups, gender, color, material used, etc., we resort to a randomized block design. If they cannot be grouped into categories but they can be matched by pairs (for instance, two machines of the same brand and model, or two patients with similar medical profiles), matched pairs design is the approach to abide with. Otherwise, we may be signaled to collect more data before proceeding to apply an experimental design method, be it by collecting more covariate data or increasing the sample size.
If we are not expecting major variability between subjects that shall be controlled for, there are three possible options:
- Factorial design is when we are investigating two or more independent variables or factors, and want to assess their main effects and interactions efficiently.
- Latin square design if two sources of nuisance variability (e.g., time and location) should be controlled for in a structured manner.
- A completely randomized design is used if none of the above applies and the subjects are sufficiently homogeneous to be randomly assigned to treatments without the need for additional control protocols.
The following table summarizes the fundamentals behind the application of one approach or another:
Summary of experimental design practices
Wrapping Up
This article made a distinction among different experimental design practices, helping practitioners choose one or another depending on research goals, the nature of the data, and other domain constraints.
