How the Search Machine Messes with SEO Tests
Lest we forget
Hello Riddlers!
📌 Your support keeps this newsletter FREE —> [☕Buy Me a Coffee☕]💡 Want more from me? [SEO Strategy Course] | [Build Your Own SEO Tools (with Python & Agentic AI)] | [Premium SEO Mastermind]
The Mindset
As a LinkedIn’er myself 😂, I see tons of SEO tests and experiments shared. Some are really good and some are… just broken, and I’m sure you’ve seen your fair share too.
The problem I realized exists is, we need a better mindset to create SEO experiments and to read and assess their results.
Unless we want to build our strategies on broken assumptions, or even worse, do the right SEO tactics, but assume they don’t work because… we’re approaching the tests with a mindset that’s lacking!
Let me explain what I mean… buckle up, this is super FUN!
Google Search Console is a broken tool
There… I said it… again…
The first time I said GSC was buggy, it raised a lot of eyebrows… to my surprise.
Well, a few quick examples to establish an agreement on that, include the URL Inspection Tool showing a page as “not indexed” while that same page is literally appearing in the Performance Report collecting impressions… or impressions being inflated for almost a year 😂😂😂
There’s this unwritten rule, that if your SEO strategy is working, the first early signs would be an increase in impressions. I still believe that’s true. But as of today… I don’t think this is enough.
You maybe thinking rankings is a better measurement…. well yes and no (I’ll explain in the next section), but my conclusion is:
if you rely solely on GSC to see if your test/experiment is working, you may not be able to spot that accurately. There’s data lag, there’s seasonality, there’s inflated metrics, and more…
The best approach is imo:
Combine data from multiple sources (e.g: GA4)
Check user behavior on the page because that’s all that matters (use MS clarity for example)
Have a baseline of your website performance, and understand potential impact of seasonality (decide on % lift or decline)
Google plays games
I mentioned above using rankings to measure the success of an SEO strategy, right?
Well, imagine this…
You try an SEO tactic, and the next day your ranking jumps from position 50 to position 3.
Then you repeat the same tactic 2–3 more times… and the same thing happens again.
Wouldn’t that reveal a LOT about how Google works?
And wouldn’t that open the door to even more search manipulation? (As if backlinks spam, listicles, and everything else weren’t enough already 😂)
Google has two patents (Ranking documents & Changing a rank of a document by applying a rank transition function) describing how it intentionally slow down ranking changes instead of updating rankings instantly 😂
Google is using social engineering to fight spam.
For example, if you build backlinks to your website, you may notice:
nothing happens at all
a decline in rankings
rankings fluctuations
for a period of time (a transition period), before you get rewarded, that is if Google sees you deserve to be rewarded.
This on it’s own is enough
to mess up with a lot of SEO tests.
You can be doing the right thing, but seeing no rewards at all. I experienced that in my own Python for SEO page. I did everything right imo, and it was not until recently that it finally made it to the top of the SERPs.
The core goal of Google is spam detection and dampening manipulative SEO, such as aggressive link schemes or constant micro‑tuning based on daily SERP feedback.
By introducing some type of a ranking transition period with seemingly random or counter‑intuitive ranking changes, Google makes it harder for spammers to map “I did X yesterday, rankings went up today, so X works.”
The patent discusses observing how site owners react to these fluctuations; if someone rapidly reverses or overreacts to every movement, that behavior can itself be a spam signal
For link building specifically, the patent says:
“it might take approximately 70 days for a change in a document’s link-based information to change the rank of the document to its steady state”
The problem when you have such a transition period, a lot of other changes have already occurred on your website, so it can become a challenge to isolate the exact tactic that brought the positive impact.
Also, anyone saying SEO does not need a minimum of three months to work, I hope they read this…. things can be fast, but also can be slow, by design.
Best approach for analysis imo
Well if data from GSC is… inflated sometimes, and rankings are tricking you, what can we do?
The best analysis imo would be existing SERPs and citations, and finding patterns. That is if you’re looking for a quick analysis. Otherwise, long term SEO studies expanding over at least 3 months (I’d say 6 months is even better since we saw how mount AI takes some time before it goes down) are your second best alternative.
And That’s a Wrap (Almost 😄)
My goals in this edition was to shed the light on why so many SEO tests are broken.
GSC inflated impressions and Google’s social engineering tricks are not the only reason though.
More often than not, the experiment itself has design flows. But that’s probably a topic for another day.
That’s that for today folks and see you next newsletter.
Support the Riddler!
Sign up for my newsletter if you’re not already.
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.
Disclaimer: LLMs were used to assist in wording and phrasing this blog.