Experiments

Experiment or Mark implemented?

Both record a change. Use whichever fits, or both.

	Mark implemented	Experiment
Starts from	An action on the Action plan	Any change you make
Covers	The action's page or query	One page, a list of pages, or the whole site
Measures	One review, 28 days later	Before and after, updated every morning while it runs
Shows	Positive, negative or inconclusive	A chart, before and after figures, and a verdict on whether the change is clear enough to trust

For a quick fix suggested by the Action plan, Mark implemented is enough. For a bigger change you want to watch closely, such as a new page template or a site move, start an experiment. See Working an action.

Start an experiment

Open Experiments and create one:

1. Choose the site and give the experiment a name.
2. Choose the scope:
   • Single page: one address. Copy it exactly as it appears on Pages, because we match it letter for letter.
   • URL group: up to 50 addresses, each matched exactly. To measure a saved URL group, start the experiment from the group's page instead.
   • Domain-wide: every page on the site.
3. Choose the type of change, such as Title Tag, Content Update or Site Redesign.
4. Set the start date: the day the change went live. An end date is optional.
5. Add a hypothesis (what you expect to happen) and any notes.

A new experiment starts as a draft. Open it and press Start to begin measuring.

How many experiments you can have depends on your plan: see Plans, trials and billing. Draft, running and completed experiments all count. Cancelled ones do not.

How it measures

The "after" period runs from the start date to the end date, or up to the latest data if you set no end date. The "before" period is the same number of days, ending the day before the start.

Without an end date, both periods keep growing as time passes.

The before period has to fit inside the history your plan keeps. If it reaches further back, the before figures only cover the days we still hold. See How long your data is kept.

How long to wait

For its first few days an experiment says Building baseline: too early to read anything into the figures. Then it says Settling while results come in and move around. After that, results are steadier.

We also flag results as insufficient until both periods have enough days and clicks to compare. On a quiet page that can take a while.

Google can take weeks to recrawl a page and settle its rankings, so do not judge a change on its first fortnight.

Read the results

The experiment's page shows:

• Clicks and impressions: average per day, before and after
• Position: average position, before and after
• CTR: clicks divided by impressions, before and after
• a chart of clicks, impressions and position with a line on the start date, and your annotations

We call a result positive when average daily clicks rose by enough to stand out from the usual ups and downs.

The CTR confidence says how likely it is that the change in CTR is real and not chance, using a standard statistical test. It needs a fair number of impressions in each period, and the page tells you when the change counts as significant.

This is before and after evidence, not proof. We do not adjust for seasonality, and there is no control group. Check the annotations, Google updates and the mix of queries before you credit the change.

End an experiment

If you set an end date, the experiment is marked Completed on the first morning on or after that date. You can also press Complete at any time. A completed experiment's results stop changing.

Edit or delete

You can change an experiment's name, hypothesis, status, end date and notes. The site, scope, pages, change type and start date are fixed: to change those, create a new experiment.

Set the status to Cancelled to stop an experiment counting towards your limit. Deleting removes it for good.

On the charts

Experiments show on the Overview chart on their start date. Turn them on or off with the Experiments toggle.