Image by Editor

A/B testing is a powerful method for comparing two versions of something (like a survey, quiz, or form) to determine which performs better based on specific metrics. Microsoft Forms, while not a purpose-built experimental platform, can be adapted to support simple A/B-style comparisons through careful use of branching logic, duplicate form strategies, and downstream analysis in Microsoft Excel. This often involves testing variations in question wording, layout, or response options to optimize user experience or data collection.

In this tutorial, we will show several practical techniques for approximating random assignment for A/B testing with Microsoft Forms. We will focus on using branching logic and duplicate forms to simulate assignment while also highlighting the statistical limitations of these approaches. 

Why Randomization Is Critical for Causal Inference

In A/B testing, you want to know: “Does version B really work better, or is something else causing the difference?” Without randomization, several problems can occur:

• Confounding: Other variables can bias results. One group might accidentally include more experienced, younger, or more motivated people, so they answer differently no matter which version they see.
• Selection Bias: People might choose which version to take, or only certain types of people might click your link, which means the groups are not comparable.
• Other Sneaky Patterns: Systematic differences (for example, time of day or demographics) can mimic treatment effects. If morning respondents end up in one group and evening respondents end up in another, it may look like your change caused the difference when it actually did not.

Randomization helps fix this by using a chance-based assignment mechanism to decide who gets version A or B. On average, both groups become more similar in age, experience, mood, time of day, and other observed and unobserved characteristics. So when you later see better results in version B, you can be more confident that the change you made (for example, different wording or a different scale) caused the difference.

Statistically, randomization helps balance covariates across groups as sample size increases. It supports unbiased estimation in hypothesis tests, such as t-tests or chi-square tests, when comparing group means or proportions.

Step 1: Setting Up Your A/B Test in Microsoft Forms

Microsoft Forms does not have built-in randomization for A/B assignments, but you can approximate assignment using branching logic (conditional routing based on responses) combined with pseudo-random inputs or duplicate forms. Branching allows forms to adapt dynamically, which you can leverage for approximate group assignment.

Method A: Using Branching Logic to Simulate Assignment

Branching in Microsoft Forms routes respondents to different questions or sections based on their answers. To simulate assignment, you can base branching on a pseudo-random response from the user (for example, something unpredictable like the last digit of their phone number). This can approximate randomness only if the input is not systematically related to the outcome and is not manipulated by respondents.

Creating the Form Basics:

• Open Microsoft Forms >> click New Form
• Give it a clear internal title (for example, Customer Satisfaction Survey – Randomizer)
• Click Add new >> select Section >> add four more sections:
  • Introduction & Demographics
  • Version A – 5-point Scale
  • Version B – 10-point Scale
  • Closing Questions

Adding Questions:

Section 1 (Introduction):

• Add demographics (for example, Choice: “What is your age group?” – 18-24, 25-34, etc.)
• Add a randomizer question (type Choice >> make it required):
  • Question: “What is the last digit of your mobile phone number? (0-9)”
  • Options: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
• Alternative randomizer (not truly random, but sometimes usable as a rough split):
  • Question type: Choice
  • Question: “Please select a range that comes to mind right now (think randomly!)”
  • Options: 1-50 (Group A) / 51-100 (Group B)

Section 2 (Version A):

• Navigate to Section 2 (“Version A – Satisfaction (5-point Scale)”)
  • Click Add new >> select Choice
  • Question: “How satisfied are you with our service?” 
  • Options: 1 = Very Dissatisfied, 2 = Dissatisfied, 3 = Neutral, 4 = Satisfied, 5 = Very Satisfied
  • Make it required

Section 3 (Version B):

• Navigate to Section 3 (“Version B – Satisfaction (10-point Scale)”)
  • Click Add new >> select Choice (or Rating for numeric scales)
  • Question: “Overall, how satisfied are you with our customer service?”
  • Options: 0 = Extremely Dissatisfied, 1, 2, …, 10 = Extremely Satisfied
  • Make it required

Section 4 (Closing):

• Navigate to Section 4 (“Closing Questions”)
  • Click Add new >> select Yes/No or Choice
  • Question: “Would you recommend us to a friend?” (Yes/No)
  • Add any other shared questions here

Setting Up Branching Logic for Assignment

Branching routes users based on the randomizer’s answer.

• Go back to the randomizer question in Section 1 (for example, the last-digit question)
• Click the ellipsis … >> select Add branching

• In the Branching options panel:
• For even digits (0, 2, 4, 6, 8):
  • Expand Go to >> select “Version A – Satisfaction (5-point Scale)” (Section 2)
• For odd digits (1, 3, 5, 7, 9):
  • Expand Go to >> select “Version B – Satisfaction (10-point Scale)” (Section 3)

• Set branching from the end of each variant:
  • Go to the satisfaction question in Section 2 (Version A)
  • Add branching:
  • Go to >> “Closing Questions” (Section 4)
  • Repeat for Section 3 (Version B):
  • Go to >> “Closing Questions”
• For the alternative randomizer (1-100 choice):
  • For “1-50 (Group A)”:
  • Click Go to >> select the Version A section (for example, the 5-point scale section)
  • For “51-100 (Group B)”:
  • Click Go to >> select the Version B section (for example, the 10-point scale section)

Previewing and Testing the Form:

• Click Preview at the top
• Test different inputs:
• Select an even digit >> click Next

• It should skip to Version A
• Select an option >> click Next

• The form should then move to the closing question
• Click Submit

• Enter an odd digit → it should skip to Version B, then Closing
• Ensure there are no loops or dead ends; users should reach the end seamlessly

Note: This is pseudo-random, not truly random, and respondents can potentially influence their assignment. For stricter randomization, use an external randomizer, Power Automate, or a pre-randomized distribution list outside Microsoft Forms.

Method B: Creating Duplicate Forms for Variants A and B

• Go to Microsoft Forms and sign in with your Microsoft account
• Click New Form
• Create your base form with common questions (for example, demographics)
• Add the variations:
  • For version A: Use one set of questions or options
  • For version B: Modify elements like wording, order, or visuals
• Duplicate the form (click the ellipsis … > Duplicate) to create separate A and B versions
  • Edit one as Form A (for example, add the 5-point question)
  • Edit the other as Form B (for example, add the 10-point question)
• Copy the share links for A and B, then distribute them using an external random assignment method (for example, randomly split an email list in Excel before sending links)

Choosing the Right Method

There are two primary methods: the Branching Logic method (single form) and the Duplicate Forms method (two separate forms). Each has distinct tradeoffs. Use the Branching Logic method when you want a single response dataset and your survey is short. Use the Duplicate Forms method when the two versions differ substantially or when you want a cleaner separation of conditions. However, duplicate forms require the random assignment to happen outside Microsoft Forms.

Step 2: Preventing Selection Bias in Voluntary Survey Environments

Voluntary surveys (for example, those shared via email or social media) are prone to selection bias, where respondents differ from non-respondents or self-select based on the invitation. Assignment helps, but implementation still matters:

• Use a Single Entry Point: Share one link to the “randomizer” form (with branching). This ensures all participants start the same way, reducing obvious self-selection.
• Avoid Descriptive Links: Do not label links as “Version A” or “Version B”; use neutral invitations like “Take our quick survey.”
• Randomize Distribution If Using Duplicates: If you email separate links, randomly split your contact list first (for example, in Excel, use =RAND() to assign groups).
• Monitor Response Rates: Track who responds to ensure there is no major group imbalance (for example, one link being shared more widely than the other).
• Incentivize Equally: Offer the same rewards across groups to avoid differential participation.

In voluntary settings, even good assignment does not eliminate all bias. For open surveys, results usually generalize only to respondents, so you should acknowledge that limitation.

Step 3: Exporting Data and Performing Balance Checks in Excel

Exporting Responses from Microsoft Forms

Once responses are collected:

• In Microsoft Forms, go to the Responses tab
• Click Open in Excel to export all data to an Excel workbook (responses.xlsx)

Tip: If you need the latest data later, return to Forms → Responses → Open in Excel again. For a static copy, use Download a copy if available, or save the opened file.

Preparing the Data in Excel

Identifying Key Columns:

• Randomizer column (for example, “What is the last digit…” or “Choose a range…”)
• Covariates for balance: age group, gender, and any other demographics collected early

Creating a “Group” Column (assign A or B):

• Insert a new column next to your data (for example, column “Group”)
• Use a formula based on your randomizer. For the last-digit version (even = A, odd = B): 

=IF(MOD([@[What is the last digit of your mobile phone number? (0-9)]],2)=0,"A","B")

• For the 1-50 / 51-100 version: 

=IF([@[Please select a range that comes to mind right now (think randomly!)]]="1-50 (Group A)","A",IF([@[Please select a range that comes to mind right now (think randomly!)]]="51-100 (Group B)","B","Invalid"))

Cleaning the Data:

• Filter out incomplete rows (for example, blank satisfaction answers)
• Remove or flag “Invalid” groups
• Freeze the top row (View → Freeze Panes) for easier scrolling

Performing Balance Checks

Now confirm that Groups A and B are similar on pre-treatment variables (demographics), which suggests the assignment process worked reasonably well.

Quick Visual Check with a PivotTable:

• Select your data range
• Go to the Insert tab >> select PivotTable >> select New worksheet
• In PivotTable Fields:
• Rows: Drag the Group field
• Values: Drag covariates one by one
  • Drag Group to the Rows area
  • Drag Age to the Rows area
  • Drag Age to the Values area >> set to Count
    • Show values as % of Grand Total
  • Drag Recommendation to the Values area >> set to Count

Review: Counts and proportions should look reasonably similar between A and B.

Statistical Tests:

• For numeric pre-treatment variables (for example, a continuous age field): use a t-test

=T.TEST(rangeA,rangeB,2,2)

Chi-square Test:

For categorical variables (for example, gender or age group), you can use the chi-square test. This tests whether the distribution of categories is similar between Group A and Group B. You can create a contingency table (crosstab) using a PivotTable or build one manually.

Using a PivotTable:

• Rows: “What is your age group?”
• Columns: “Group”
• Values: Count of any field (for example, ID)

=CHISQ.TEST(B5:C8,F5:G8)

Interpretation: Look at the p-values.

• p > 0.05 → the category distributions are similar enough to suggest good balance
• p < 0.05 → there is evidence of imbalance, which you should note as a limitation

If imbalances exist, you can control for those variables in later analysis or clearly note the imbalance as a limitation.

Step 4: Validating Group Equivalence Before Hypothesis Tests

Before analyzing outcomes:

• Confirm sample sizes are adequate (use power calculations; a common rule of thumb is at least 30 observations per group for simple comparisons, though the ideal sample size depends on the effect size you want to detect)
• Run balance checks (as above). If groups differ significantly on key covariates, investigate the assignment process (for example, faulty branching or biased self-selection) and consider restarting the test if needed.
• If groups look reasonably balanced, proceed to outcome analysis in Excel:
  • Null hypothesis: No difference between A and B
  • For a numeric outcome: use a two-sample t-test, such as =T.TEST(A_values,B_values,2,2)
  • If you are comparing a 5-point scale to a 10-point scale, rescale one version first so both outcomes are on the same metric before comparing means.
  • For a 95% confidence interval critical value under the normal approximation: use =NORM.S.INV(1-0.025)
• Interpret: If p < 0.05, reject the null hypothesis of no difference between A and B. Also report an effect size, such as Cohen’s d = (meanA – meanB)/SD_pooled, when comparing numeric outcomes.

Advanced Tips

• True Randomization with Power Automate: Store form links in SharePoint or another source, then use a flow to randomly assign and send Form A or Form B from a single entry process.
• Limitations: Pseudo-random methods assume the inputs are not systematically biased; test for bias where possible (for example, phone digits may not be perfectly uniform in some contexts).
• Ethics: Inform participants when appropriate and comply with data privacy requirements (for example, GDPR).
• Scaling Up: For larger tests, integrate with Microsoft Power BI or another analytics tool for more advanced reporting and statistical analysis.

This approach can make your A/B-style tests in Microsoft Forms more structured and informative, especially for lightweight internal experiments. Start small, iterate carefully, and always validate the assignment process before drawing conclusions.

Conclusion

By following the steps above, you can use branching logic and duplicate-form workflows to approximate A/B testing in Microsoft Forms. Using a single form with branching on a simple randomizer (such as a last-digit or range question) gives you a low-effort way to separate respondents into groups, but it is only an approximation of true random assignment. Always check group balance first with a PivotTable or chi-square test. If the groups look reasonably balanced, you can compare outcomes more confidently. Preview everything, validate your setup, and be explicit about the method’s limitations before making causal claims.