If you find value in my newsletter and want to support the work that goes into it consider —> [☕Buy Me a Coffee☕]💡 Want more from me? [SEO Strategy Course] | [Build Your Own SEO Tools (with Python & Agentic AI)] | [Premium SEO Mastermind]
Alternatively you can become a paid subscriber:
Intro
If you’re following me, you’ve probably came across the Sara & Nik podcast. It’s a chill podcast in a casual chat style and in our last episode we hosted Dan Petrovic. It was a cool chat, do check it out.
Any how, one of the things that was brought up the fact that there’s a lot of SEO research circulating and they’re not created equal! Some are really in-depth and some just touch the surface.
I’ve personally came across a lot of really good work, but then at times I saw faulty logic, bad data, or just the experiment was not setup correctly.
And Dan suggested to vet the good from well… not so good… just ask how:
Ask “how,” again and again, until you reach the first principles of what someone does. If they can’t explain the basic building blocks, it’s smoke and mirrors. He gives clients the same advice for vetting agencies and consultants.
I totally agree! So today we’re diving into vetting SEO research and separating SEO research gold from fool’s gold,
When it’s fool’s gold
Few things you’ll see in a broken SEO research or experimentation:
Assuming something, then treating it as a fact. Does broken schema code get ignored during parsing? I honestly don’t know. But if you assume it does and build an entire experiment around that assumption, shouldn’t you establish that it’s true first? Otherwise, your conclusions rest on a premise you haven’t actually verified. And if that premise is wrong, everything built on top of it becomes questionable.
Correlation is not causation. I did A, then B happened, therefore A causes B. I launched a new website, and it took so long for the URLs to get indexed, therefore all new websites need a long time for their URLs to be indexed. There’s definitely some correlation between being a new website, and having slower indexation, but that’s not the cause of this… low authority, low quality content, or small number of content, low social engagements, etc… all can be contributing factors.
The data… the data…. the data… a lot of time I see people treating SEO data as rigid solid actual numbers… and that’s far from reality. SEO data is directional, to help you make educated guesses… but as I always say we don’t work in healthcare or banking for example to need precise accurate data like our lives depend on it… question the data source, and what this data is measuring.
The conclusion is much bigger than the experiment. Small experiments on a small sample of pages or keywords don’t prove a lot, they’re still useful and inspire further investigation of the matter at hand, but let’s collectively agree not to jump to huge conclusions based on a one time off or a small quick experiment.
When SEO research is gold
On the other hand, these are some way that can help you assess (not guarantee) that an SEO research is really good work:
Methodology is transparent. A lot of times, I even link to the raw data I used because it can help people replicate this test themselves and maybe find my shortcomings!
There’s a clear question we’re trying to answer. A lot of research starts the other way around, with a lot of data and patterns analysis. That’s ok too, but the ones that start with a clear question, have clear methodology, transparency about the data used are more likely to have clear useful results and avoid the correlation vs causation pitfall.
The data source is reasonable. As I said, I don’t think we have super good data in SEO, data is either missing, skewed, or just not accurate, but understanding where the data came from, and the limitations of this data puts the results of a research in context.
Alternative explanations are considered. Unless we’re talking about facts, your test or research should consider different explanations and view points. For example finding a relation between having the keyword in the title and getting cited does not ONLY mean that if you put the keyword in the title you may potentially get cited more… what other explanations is this study offering.
I can really keep going… good research is a lot of things, and here’s one last thing (that should not be last)…. the person behind the research matters the most, their research history and overall work. That’s the only shortcut to flagging whether a piece of research is good or not.
Popular research != good research
I’ve been around this industry for enough time to see “popular research” that is simply faulty. And even when you clarify it is, no one really cares….
Popularity is the main reason a research gets shared and widespread… business collaborations, personal gains and connections… that’s what makes a research good or bad in the industry unfortunately…
The research that gets the most attention isn’t necessarily the research that deserves the most trust.
but know you know how to find the real value.
And back to Dan’s advice…
Always ask how.
Then ask how again.
Keep going until you understand what was actually tested, what the evidence supports, and what remains uncertain.
That’s where you’ll find the real gold.
And That’s a Wrap (Almost 😄)
One of the things I deeply love about SEO is how dynamic it is as a field. There’s always something new to learn, question, and test.
But if there’s one thing I’d change about this industry, it would be making it more objective, facts and evidence-based, and less biased by “but my friend wrote this research it must be true” 😄
That’s that for today folks and see you in the next newsletter!
Support the Riddler!
Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups on Substack by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.
Disclaimer: LLMs were used to assist in wording and phrasing this blog.