In this article
  1. On-page vs off-page, simply explained
  2. How an SEO split test works
  3. Seven published tests and what happened
  4. What the tests teach
  5. Why there are so few off-page tests
  6. What a small business should do
  7. Frequently asked questions

SEO advice is full of confident rules. Put the keyword first. Add numbers to titles. Build more links. But most of these rules come from experience and guesswork, not controlled tests. When a site changes something and traffic rises, it is hard to know whether the change caused it or whether something else, like a Google update or a busy season, did.

Split testing solves part of that problem. In this article we explain on-page and off-page SEO, then look at seven published split tests to see which fixes actually moved organic traffic.

Please note: These are tests published by the SEO testing company SearchPilot on its clients’ websites. They are not World Elev’s own experiments. Results on one site do not guarantee the same result on another.

Key facts

  • On-page SEO is anything you change on your own pages: titles, headings, text, structure and speed.
  • Off-page SEO is mostly about other sites linking to and talking about yours.
  • Small title changes moved organic traffic by up to 28% in published tests, in both directions.
  • Links still matter to Google, but buying them breaks Google’s spam policies.

On-page vs off-page, simply explained

On-page SEO covers everything under your control on your own website. Google’s SEO Starter Guide focuses on this: clear page titles, helpful content, descriptive headings, good link text and pages that are easy to crawl.

Off-page SEO covers signals from outside your site, mainly links from other websites. Google’s guide to its ranking systems confirms it uses link analysis systems, including PageRank, to understand what pages are about and which might be most helpful.

How an SEO split test works

In a normal A/B test, visitors are split into two groups. That does not work for SEO, because there is only one visitor that matters: Google’s crawler. So SEO split tests split pages instead. A site with hundreds of similar pages, such as product or location pages, changes half of them and leaves the other half alone. It then compares organic traffic between the two groups.

The result is reported with a confidence level. A result that is “positive at 95% confidence” is strong. One that is positive only at 80% or 90% is weaker and could be noise.

Seven published tests and what happened

Change testedResult on organic traffic
Removed the category name from product titles, keeping brand and product name+28% (on low-traffic pages)
Added “The Best” to title tags+10%
Added a question about cost to offer page titlesMore than +5%
Added age ranges to location page titles+4%, at 90% confidence
Removed meta descriptions so Google wrote the snippetNo clear effect at 95%; positive at 80%
Rounded exact listing counts in titlesNo detectable effect; likely negative
Added emojis to title tags-6%
Published SearchPilot split tests of on-page changes. Each was run on one client site.

What the tests teach

1. Titles matter more than most people think

Five of the seven tests changed only the title tag, and several moved traffic by 5% or more. Titles affect both how Google understands a page and whether a searcher clicks it. Even though Google now rewrites many titles in search results, SearchPilot found Google still used the changed titles in several of these tests.

2. Match the words people actually search

The biggest win, +28%, came from making product titles closer to what people type. The test that added age ranges worked for the same reason: it matched how people search for that service.

3. Some popular tricks backfire

Emojis in titles cut traffic by 6%. SearchPilot suggested two possible reasons: the main keyword was pushed away from the start of the title, or Google treated the page as less relevant. This is exactly why testing beats copying tricks you saw on social media.

4. “No effect” is a useful result too

Removing meta descriptions made no clear difference at the usual confidence level. That saves time: if a change does not help, you can stop spending hours on it.

Why there are so few off-page tests

Off-page SEO is much harder to test fairly. You cannot easily give half your pages new links and hold back the other half, because links pass value around your whole site. Links also take weeks or months to be found and counted.

There is also a rule problem. Google’s spam policies say that buying or selling links to manipulate rankings is link spam. A site that “tests” bought links risks a penalty, so honest public tests of link buying are rare.

So the evidence is lopsided. On-page changes have a growing body of controlled tests. Off-page SEO is supported mainly by what Google says about its systems and by observation, not split tests.

What a small business should do

  1. Fix titles first. Make sure each important page has a clear title that uses the words customers search for.
  2. Write useful pages that answer real questions, as Google’s starter guide recommends.
  3. Earn links naturally through local partners, suppliers, directories that real customers use and genuine press coverage.
  4. Measure changes one at a time, so you know what worked.

If you also run social media, see our break-even calculation for organic vs paid social. And for search ads, read our worked example of how Ad Rank picks the winner.

Frequently asked questions

Which is more important, on-page or off-page SEO?

Both matter. On-page SEO is fully under your control and has the most published test evidence. Off-page signals such as links still feed Google’s ranking systems, but they are harder to test and to influence honestly.

Do title tags still matter if Google rewrites them?

Yes. Several published split tests found that changing titles moved organic traffic, and Google often still uses the title you write.

Is it safe to buy backlinks?

No. Google’s spam policies class buying or selling links to manipulate rankings as link spam.

Can a small site run SEO split tests?

It is hard. Split tests need many similar pages and steady traffic. Small sites can still make one change at a time and track results in Google Search Console.

Browse more in our SEO section.

Sources

  1. Google Search Central: SEO Starter Guide
  2. Google Search Central: A guide to Google Search ranking systems
  3. Google Search Central: Spam policies for Google web search
  4. SearchPilot: case study, aligning keywords with search queries
  5. SearchPilot: case study, “The Best” in title tags
  6. SearchPilot: case study, emojis in title tags