What Is A/B Testing?

What A/B testing means

Version A is usually your current page, called the control. Version B changes one element, such as the headline, the button text or the main image. Testing software splits visitors between the two at random and records how many complete the goal. Because both versions run at the same time, seasonal swings affect them equally.

Multivariate testing is a related method that changes several elements at once to find the best combination. It needs far more traffic than a simple A/B test, so most small and mid-sized business sites get more value from testing one clear change at a time.

Why A/B testing matters for a business website

A/B testing replaces opinions in design meetings with evidence from real customers. A change that looks better to the team can quietly reduce inquiries, and without a test you might never connect the drop to the change. Small, proven gains on a key page, repeated over a year, add up to more leads from the same traffic.

An A/B testing example

An HVAC company tests its emergency repair page. Version A has a button labeled Contact us. Version B says Get a technician today and adds the phone number beside it. Half of visitors see each version for four weeks. The team compares completed calls and forms, not button clicks, before deciding which version to keep.

A/B testing without hurting SEO

• Never show Googlebot a different version from the one people see, because Google treats that as cloaking under its spam policies.
• If each variant has its own URL, add rel=canonical pointing to the original page.
• Use 302 temporary redirects, not 301 permanent ones, when sending visitors to a test URL.
• End the test once you have enough data, then remove the variant URLs and testing scripts.

Copy this A/B test plan

A/B TEST PLAN
Page: https://www.example.com/emergency-repair
Owner: Marketing lead

Hypothesis: Changing the button from "Contact us" to "Get a technician today"
will raise completed calls and forms, because it states the outcome.

Version A (control): current page
Version B: new button text, with the phone number beside the button

Goal metric: completed calls and form submissions (GA4 key events)
Guardrail: cancellations and refund requests stay flat
Split: 50/50, all devices
Minimum run: 4 full weeks, ending on the same weekday it started
Winner rule: tool reports significance at ____ % or higher
SEO: rel=canonical to the original, 302 redirect if B has its own URL

Decision: keep B / keep A / test again
Notes:

How to check an A/B test result before you switch versions

1. Confirm the test ran for whole weeks, so weekday and weekend visitors are counted in both versions.
2. Compare visitor counts: with a 50/50 split, A and B should be close; a large gap points to a broken setup, not a winner.
3. Check the result is measured on the real goal, such as completed forms, calls or orders, not button clicks or time on page.
4. Read your testing tool's significance or probability figure, and do not stop early because one good day put B ahead.
5. Split the result by mobile and desktop, and make sure the winning version works on both before you make it permanent.

What good looks like: A result you can trust has a near even split, enough completed goals in each version for your tool to report a clear winner at the confidence level you set before launch, and the same direction on mobile and desktop.

Questions to ask your web developer

“Which testing tool will you use, now that Google Optimize has closed?”

A good answer: A named tool that works with our analytics and shows search engines the same content visitors see. Google Optimize stopped working on September 30, 2023, so any plan built on it is out of date.

“How will you decide when a test is finished?”

A good answer: The traffic needed, the minimum run time and the one metric that picks the winner are written down before launch, so the result is not chosen after the fact.

A/B testing: common questions

How long should an A/B test run?

Long enough to reach a reliable result, which depends on your traffic and how big the difference is. Running in whole weeks helps cover both weekday and weekend behavior. Google also warns against running tests unnecessarily long, so stop once your testing tool shows a clear answer.

Does A/B testing affect SEO?

Not if it is done properly. Google's guidance says testing is fine when you avoid cloaking, point variant URLs to the original with rel=canonical, use temporary 302 redirects and remove test pages when finished. Tests left running far longer than needed can be seen as an attempt to deceive search engines.

Sources

• Google Search Central: Minimize A/B testing impact in Google Search
• Google Optimize Help: Optimize sunset