Read on Twitter
SEO experiments are fundamentally different from typical A/B experiments because they need to test changes on a "page" level, vs. a "user" level. Below are five tips for running SEO experiments, courtesy of @fanfavorite_bta
1/ What kind of SEO experiments can I run?
Anything your heart desires, as long as it’s an on-page change. If you want to test your title tags, meta tags, or image alt – run wild. However, if you want to measure changes with internal linking, that’s a whole different beast.
Anything your heart desires, as long as it’s an on-page change. If you want to test your title tags, meta tags, or image alt – run wild. However, if you want to measure changes with internal linking, that’s a whole different beast.
2/ What should I measure with SEO experiments?
Organic traffic. Nothing else.
You’ll likely want to measure other things experimentally, but unfortunately, it will not work, including:
• Keyword rankings
• Average ranking
• Search Engine Result Page Clicks
• Conversions
Organic traffic. Nothing else.
You’ll likely want to measure other things experimentally, but unfortunately, it will not work, including:
• Keyword rankings
• Average ranking
• Search Engine Result Page Clicks
• Conversions
3/ How do I run an SEO experiment?
1. Bucket your treatment and control on the page level
2. Set up tracking to track all incoming organic traffic to your website
3. Roll out your experiment to the pages in the treatment group
4. Let the experiment run for 2-4 weeks...
1. Bucket your treatment and control on the page level
2. Set up tracking to track all incoming organic traffic to your website
3. Roll out your experiment to the pages in the treatment group
4. Let the experiment run for 2-4 weeks...
3b/ (continued)
5. Run an analysis on the expected differences in organic traffic
6. You can conclude the experiment after it has run for more than two weeks and the results are statistically significant
5. Run an analysis on the expected differences in organic traffic
6. You can conclude the experiment after it has run for more than two weeks and the results are statistically significant
4/ What are the limitations of SEO experiments?
1. Traffic: If you don’t have a consistently high amount of traffic to your website, running SEO experiments will be tough. As a general rule, you’ll want at least 5k organic visitors a day to the page type that you want to test.
1. Traffic: If you don’t have a consistently high amount of traffic to your website, running SEO experiments will be tough. As a general rule, you’ll want at least 5k organic visitors a day to the page type that you want to test.
4b/ (continued)
2. Number of pages: The number of pages that you have will limit the number of experiments you could run at a time. At Airbnb, we were able to run maybe 5 experiments at a time, while at Strava, we were limited to only one at a time.
2. Number of pages: The number of pages that you have will limit the number of experiments you could run at a time. At Airbnb, we were able to run maybe 5 experiments at a time, while at Strava, we were limited to only one at a time.
5/ What were the most impactful SEO experiments I’ve run?
In aggregate, title tag experiments led to a 15-20% increase in traffic at Airbnb. A minor title tag change resulted in the most impactful SEO experiment ever run at Airbnb.
In aggregate, title tag experiments led to a 15-20% increase in traffic at Airbnb. A minor title tag change resulted in the most impactful SEO experiment ever run at Airbnb.
5b/ (continued)
Tips:
1. Can you leverage content to provide more consistent title tags?
2. How do you provide more context to Google?
3. What information are people looking for when they’re searching for your high-performance keywords?
4. People really like clicking on numbers
Tips:
1. Can you leverage content to provide more consistent title tags?
2. How do you provide more context to Google?
3. What information are people looking for when they’re searching for your high-performance keywords?
4. People really like clicking on numbers
5c/ (continued)
Content always drives more traffic. There’s a good reason why SEO landing pages always have a ton of content and text on them, that’s because there’s a very strong correlation between the amount of unique, high-quality content on your page, ...
Content always drives more traffic. There’s a good reason why SEO landing pages always have a ton of content and text on them, that’s because there’s a very strong correlation between the amount of unique, high-quality content on your page, ...
5d/ (continued)
... and the amount of incoming organic traffic. Every time we added more content to our Airbnb landing pages, it always led to more incoming traffic. The most impactful single experiment we ever ran at Strava...
... and the amount of incoming organic traffic. Every time we added more content to our Airbnb landing pages, it always led to more incoming traffic. The most impactful single experiment we ever ran at Strava...
5e/ (continued)
...increased the amount of traffic to our pages by 20%, and the only thing that we did was add more routes to our “Where to run in {city} pages.”
...increased the amount of traffic to our pages by 20%, and the only thing that we did was add more routes to our “Where to run in {city} pages.”
6/ For much more on running SEO experiments, including a real-life SEO experiment looking at @figmadesign, check out the full post by @fanfavorite_bta https://www.lennyrachitsky.com/p/seo-experimentation
