SEO Riddler by Sara Taher

What Critical Thinking Can Teach Us About SEO

Sara Taher — Tue, 03 Nov 2026 14:54:09 GMT

If you find value in my newsletter and want to support the work that goes into it consider —> [☕Buy Me a Coffee☕]💡 Want more from me? [SEO Strategy Course] | [Build Your Own SEO Tools (with Python & Agentic AI)] | [Premium SEO Mastermind]

Alternatively you can become a paid subscriber:

Intro

I read a post recently how just few years ago Wikipedia was considered pseudoscience, yet today, somehow, we’re perfectly comfortable consuming content from AI Overviews and AI Mode systems that, by their very nature, can hallucinate. Ironic, isn’t it?

That’s why I believe the right mindset to thinking can take you places nothing else can. How often do we push back on or question the sheer volume of information thrown at us every time we scroll on social media for example?

That’s also why last week I wrote about spotting good SEO research from, well the not so good :)

This week I want to talk about a concept very often confused, and how a symptom is not the same as the disease.

Wait a minute… is this an SEO newsletter or a medical journal?

Don’t worry, it’s just an analogy. But it’s one worth exploring, so buckle up!

The symptom vs. the root cause

You get a headache, you focus on “treating the headache”… but the headache is just a symptom of an underlying problem. If you identify the actual problem and fix it, the headache and any other related symptoms will be gone!

The thing is, a headache can be a symptom to various conditions varying in severity:

and it needs medical understanding, and good diagnosis to narrow down the root cause. You cannot jump to conclusions, or just attempt to treat every possible root cause. That would be madness 😄

The SEO root cause

Any SEO who saw a traffic decline in the past year+ will understand this: stakeholders will always say it’s because of AI Overviews and AI Mode in search (only). Before there was AI in search, people would attribute any traffic decline to “google updates”.

I can list a lot of such “diagnosis” that I saw people use throughout the years of my work. The problem is this mindset, is not helpful in actually fixing the issue.

Another time, we become fixated on the symptom itself. Your new website not getting indexed? we throw everything we know at it, every tactic we can think of. But understanding the root cause would help you narrow down and prioritize what may actually work, and if you’re working with a team you’re saving their time.

So we need to focus on finding the root cause, and understand that an SEO issue, can be just a symptom of something else.

The 5 whys

One of the simplest frameworks of getting down to the root cause of something is asking why 5 times. And the best thing, finding and addressing a root cause, can resolve the issue at hand, plus other issues you didn’t even realize you have!

Example: Google is not indexing website URLs

Initial symptom: A website has 500 URLs that are not being indexed by Google.

The 5 why step by step:

As you can see in the scenario above, the problem wasn’t really an indexation issue. That was the symptom. The actual underlying cause was the faceted navigation.

And by fixing this one thing, I’m pretty sure a lot of other positive outcomes could follow.

Beyond improved indexation, for example, you could also see improvements in rankings, as you’d have better control over canonical URLs and a more intentional approach to URL generation. You could also improve the user experience by ensuring that only a carefully selected set of useful URLs is created, rather than generating countless variations that offer little additional value.

In other words, find the root cause, and fix it, and potentially multiple positive outcomes can come.

Here’s another way this could’ve gone:

Again, in this scenario, the indexation issue is not the actual problem it’s a symptom. The actual root cause was the website information architecture. While this is a big fix, it will address a lot of other fundamental SEO issues and improve overall website performance.

That’s the magic of finding the root cause, and that’s also how to build your strategy. Find the root causes and prioritize them instead of going after small tactics and a fix here and there.

Flashback: how to find the root cause

I wrote before about The Cause and Effect: How to Diagnose SEO Problems and presented how the fishbone analysis model can help narrow down the potential causes. You may want to check it out.

And That’s a Wrap (Almost 😄)

It’s always a good idea to sit back and reflect on the way we think or our thinking patterns.

Usually, we’re focused on the output of our thinking (the conclusions we reach) and rarely on how we actually arrive at those conclusions.

But the process matters just as much, if not more.

Learning different frameworks like this helps develop the thinking muscles we need to deal with the new state of search.. AI search that is…

We have more information than ever before, all just one click away and at our fingertips. Access to information is no longer the problem. It’s how we process, question, interpret, and think about that information that really matters.

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups on Substack by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Not All SEO Research Is Created Equal

Sara Taher — Tue, 27 Oct 2026 13:14:43 GMT

Alternatively you can become a paid subscriber:

Intro

If you’re following me, you’ve probably came across the Sara & Nik podcast. It’s a chill podcast in a casual chat style and in our last episode we hosted Dan Petrovic. It was a cool chat, do check it out.

Any how, one of the things that was brought up the fact that there’s a lot of SEO research circulating and they’re not created equal! Some are really in-depth and some just touch the surface.

I’ve personally came across a lot of really good work, but then at times I saw faulty logic, bad data, or just the experiment was not setup correctly.

And Dan suggested to vet the good from well… not so good… just ask how:

Ask “how,” again and again, until you reach the first principles of what someone does. If they can’t explain the basic building blocks, it’s smoke and mirrors. He gives clients the same advice for vetting agencies and consultants.

I totally agree! So today we’re diving into vetting SEO research and separating SEO research gold from fool’s gold,

When it’s fool’s gold

Few things you’ll see in a broken SEO research or experimentation:

Assuming something, then treating it as a fact. Does broken schema code get ignored during parsing? I honestly don’t know. But if you assume it does and build an entire experiment around that assumption, shouldn’t you establish that it’s true first? Otherwise, your conclusions rest on a premise you haven’t actually verified. And if that premise is wrong, everything built on top of it becomes questionable.
Correlation is not causation. I did A, then B happened, therefore A causes B. I launched a new website, and it took so long for the URLs to get indexed, therefore all new websites need a long time for their URLs to be indexed. There’s definitely some correlation between being a new website, and having slower indexation, but that’s not the cause of this… low authority, low quality content, or small number of content, low social engagements, etc… all can be contributing factors.
The data… the data…. the data… a lot of time I see people treating SEO data as rigid solid actual numbers… and that’s far from reality. SEO data is directional, to help you make educated guesses… but as I always say we don’t work in healthcare or banking for example to need precise accurate data like our lives depend on it… question the data source, and what this data is measuring.
The conclusion is much bigger than the experiment. Small experiments on a small sample of pages or keywords don’t prove a lot, they’re still useful and inspire further investigation of the matter at hand, but let’s collectively agree not to jump to huge conclusions based on a one time off or a small quick experiment.

When SEO research is gold

On the other hand, these are some way that can help you assess (not guarantee) that an SEO research is really good work:

Methodology is transparent. A lot of times, I even link to the raw data I used because it can help people replicate this test themselves and maybe find my shortcomings!
There’s a clear question we’re trying to answer. A lot of research starts the other way around, with a lot of data and patterns analysis. That’s ok too, but the ones that start with a clear question, have clear methodology, transparency about the data used are more likely to have clear useful results and avoid the correlation vs causation pitfall.
The data source is reasonable. As I said, I don’t think we have super good data in SEO, data is either missing, skewed, or just not accurate, but understanding where the data came from, and the limitations of this data puts the results of a research in context.
Alternative explanations are considered. Unless we’re talking about facts, your test or research should consider different explanations and view points. For example finding a relation between having the keyword in the title and getting cited does not ONLY mean that if you put the keyword in the title you may potentially get cited more… what other explanations is this study offering.

I can really keep going… good research is a lot of things, and here’s one last thing (that should not be last)…. the person behind the research matters the most, their research history and overall work. That’s the only shortcut to flagging whether a piece of research is good or not.

Popular research != good research

I’ve been around this industry for enough time to see “popular research” that is simply faulty. And even when you clarify it is, no one really cares….

Popularity is the main reason a research gets shared and widespread… business collaborations, personal gains and connections… that’s what makes a research good or bad in the industry unfortunately…

The research that gets the most attention isn’t necessarily the research that deserves the most trust.

but know you know how to find the real value.

And back to Dan’s advice…

Always ask how.

Then ask how again.

Keep going until you understand what was actually tested, what the evidence supports, and what remains uncertain.

That’s where you’ll find the real gold.

And That’s a Wrap (Almost 😄)

One of the things I deeply love about SEO is how dynamic it is as a field. There’s always something new to learn, question, and test.

But if there’s one thing I’d change about this industry, it would be making it more objective, facts and evidence-based, and less biased by “but my friend wrote this research it must be true” 😄

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups on Substack by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

SEO: A Game With No Clear Rules

Sara Taher — Tue, 20 Oct 2026 13:14:12 GMT

Alternatively you can become a paid subscriber:

Intro

Google is the largest search engine on the planet and everyone wants to rank high in SERPs and get cited and mentioned in AI overviews and AI mode to get clicks and ultimately conversions. To do that, we need to follow Google guidelines, very simple, right?

Not really… that’s the whole problem as well. When John Muller says things like “create helpful content” as part of guidance on what content Google considers to be valuable and high quality, that’s not really clear… it’s still very vague.

Well that’s not the only challenge in understanding SEO. Google also does not provide the complete data in GSC, even if using the API. Google also does not provide any metric or way to quantify the “clicks” we get from it’s AI features, but they did tell us they have a very super busy backlog and have finally launched the feature we all wanted “dark mode in GSC” 😂

It also seems that Google works based on feedback. After publishers have been very vocal about traffic drops, study after study, and post after post… Google finally decided to add more links in their AI features. Makes you wonder why was this not in the plan from the start?

These things make SEO challenging and really like a game with no clear rules… but that’s not it… Google has more for SEOs 😂

Google poisoning search results

A recent study by Nozzle found that Google is purposefully serving up false data to AI scrapers. In simple words, Google is serving AI scrapers different search results than the ones actually being served to users.

It has been fighting scrapers for a while now, first with requiring JavaScript execution to view its search engine results pages (SERPs) which is a challenge to some of the scrapers.

Then by trying the google.com/goto as an intermediary hop, so that when you click an organic search result, your browser briefly passes through a Google-owned redirect (google.com/goto) before landing on the target page.

But since September 13, 8.4% of SERPs came back poisoned, against 0.17% before. Google is simply serving “different” results to scrapers than users… Google is Cloacking… the irony 😂

For transparency, I’ve always seen some kind of discrepancy between what SEO tools returns as SERPs and what I can see actually in SERPs, but it looks like there’s a huge difference now.

As I always say, it’s always fun in SEO!

Historically speaking

Manipulation from Google is not something new.

Take this for example….

You follow the guidelines you get results right? well no because Google also has a “spam” fighting system that intentionally delays any improvement in performance when you apply SEO improvements to your website.

Instead of immediately improving search rankings when optimization tactics are applied, the system may use time-based delays, dampened responses, or even initially negative responses before rankings gradually change over time.

Did you catch that? You can do everything right, follow SEO best practices, and still see your rankings drop before they improve.

Why? Google may observe how websites and SEO professionals react to unexpected delays or temporary ranking drops, then use statistical correlations to help identify potential spam.

In other words, doing good work doesn’t always produce immediate positive results—and an initial drop doesn’t necessarily mean you’ve done something wrong.

So tell me again, how is SEO not a game? 😄

What does that mean for SEOs?

when you operate under uncertainty and ambiguity, you gotta have the right mindset to SEO. Here are few things to always keep in mind:

“It depends” is a perfectly valid answer.
Trust your gut and understand the limitations of your data. What you can measure isn’t always the whole picture. Data is either missing, incomplete, skewed or manipulated by Google.
Ask yourself, am I doing a really good job? did I really do my due diligence in creating this piece of content? could it have been any better? and that’s actually your best bet.
Learn what’s working from others, but create your own thing. Don’t copy, innovate.
Test, and perform data studies, but combine that with gut feeling and understanding that the data we have in this industry is a lot of times directional.

And That’s a Wrap (Almost 😄)

SEO is the least boring field there is in my opinion, every day something new, and every website is a new and different challenge or may I say Puzzle that we need to solve.

The important think is accepting the uncertainty and adapting.

The right mindset can help you navigate that… you know what won’t? being rigid like a checklist… but that’s a chat for another day.

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups on Substack by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Google Knowledge Graph API Script [FREE]

Sara Taher — Tue, 13 Oct 2026 11:32:18 GMT

Alternatively you can become a paid subscriber:

Intro

You entity matter for your visibility in search. It also matters in your visibility in LLMs not necessarily directly but also as a secondary impact of how prominent and clear your entity is in search engines and online in general.

Because LLMs generate their answers based on their training data, and their grounding searches, your entity prominence and authority is very important.

Entities have always been important for Google search. Any one remembers when Google moved from dealing with “strings” to “things” and they built their knowledge graph specifically for this.

Well the good news is that, there’s an API for Google Knowledge Graph that you can use for free and that’s what I’m sharing today, a python script to analyze your entity!

So buckle up!

First get the API key

Here’s how:

Go to https://console.cloud.google.com/
Click the project drop-down menu at the top of the page and select a New Project to create a new one.
Then, open the left navigation menu and navigate to APIs & Services > Library.

In the search bar, look for Knowledge Graph Search API.
Click on the API from the search results and click the Enable button
Once enabled, go back to the left menu and select APIs & Services > Credentials.
Click the Create Credentials button at the top of the screen.
Select API key from the drop-down list.
A dialog box will appear displaying your newly created API key. Copy this key and save it securely.

The script

Now this is the part where you don’t have to do anything! I built the code for you using Python and Google collab, all you need to do is to make a copy of this script.

Once you click on the run button, and scroll down you’ll be asked to enter the API key.

Done? awesome, now you’ll be asked to enter the name of your entity, it can be anything really!

The Output

The output will be printed directly on your screen in Google collab, but also you’ll notice that there’s an HTML report downloaded! I ran this few times.

Here are the HTML report for “avengers” entity. What I find interesting is two items, the relevance score and the URL associated with an entity. I tell ya, did not anticipate that “age of ultron” specifically would be the highest associated entity with the term “avengers” 🤷

Also note how some entities are not associated with a URL.

what’s cool is that, at the end of this report, there’s also an entity search summary table.

Cool stuff, eh?

And That’s a Wrap (Almost 😄)

2026 is the SEO DIY era, come on board and enjoy this ride. I know a lot of people are not very technical but you don’t need to be that technical to use a simple script!

There’s definitely a small (tiny tiny) learning curve in the beginning but within 1-2 hours you can learn how to get an API key and use a script and the best part? you now have an unlimited choice of things you can do!

If you liked this script and looking for more? maybe check out my Build Your Own SEO Tools course?

I hope this script is helpful, and if you have not used a python script before, I hope this was enough to capture your interest and gave you the motivating to try it out!

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups on Substack by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

I’m Building a Search Engine (Part 2): Content Extraction

Sara Taher — Tue, 06 Oct 2026 07:28:56 GMT

Alternatively you can become a paid subscriber:

Intro

I shared the other week my new project, building a search engine. You can read about part 1 of this journey here. I’ve also started other projects since then 😄 but that’s a chat for another day! Today I’m sharing the next step in building the search engine:

Main Content Extraction

The term “main content” has come to light again recently with Google’s update to their helpful content documentation. You see, your content quality is assessed based on the “main content” of the page. Things like top navigation, footer, sidebars, popups, etc… are not part of your main content, and would not influence your content quality score.

So the next step after crawling and collection the URLs, is extracting the main content. This is part of the indexation process. The first step basically.

Because in indexation, what you want to do is build a table mapping each page and a set of keywords. You’re basically saying for these keywords, this page provides a relevant answer.

The table will be used to find and serve pages when a user searches.

Ya… so let’s dive in!

Main Content Extraction

Using my dear friend Claude Code, I decided to extract text from pages in the following way:

I decided to ignore all non 200 pages. [Only 541 pages returned 200, and those are the ones worth extracting.]
Also, I decided to ignore all pages that require JS for their content, to simplify my process.
I build a different set of extraction rules. One is tailored specifically for my substack pages, one for my website pages (and body did I find space for technical optimizations there 😄), one for YouTube and an open source pre-existing tool for other pages called trafilatur.

Here’s a summary of the extraction methods:

As a general rule, make sure to keep headings and paragraphs, as simple Markdown (# headings, paragraphs, lists), not one flat block of text.
Finally I generated a report to review the extraction process:

looks good right?

Limitations

Few things I had to set aside at this stage to focus on delivering vs perfecting:

I would’ve loved to build a better extraction mechanism for YouTube that pulls the transcript.

The goal is not a perfect search engine, but rather a good enough one. I think I was able to do that at this point.

Next Steps

Clearly indexation is not done yet. I still need to normalize, and tokenize, and decide on the mechanism to assign keywords to pages. I like what I’m doing and that all that matters. Will keep you updated!

And That’s a Wrap (Almost 😄)

Building tools is great, but building a search engine is something else. It’s definitely a passion project. I guess that’s why we do SEO 😄

I hope you found this helpful and inspirational.

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups on Substack by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

I Built an MCP for SEO. Then I Analyzed My Own Entity.

Sara Taher — Tue, 29 Sep 2026 13:39:18 GMT

Alternatively you can become a paid subscriber:

Intro

I have a lot of WIP SEO projects, and I think it’s a nice problem to have. Because you can get bored of one, and simply switch to another, or just start another one 😄

One of those projects that have been waiting for a while is building an MCP for SEO.

MCPs (or Model Context Protocol) are easy ways to connect your AI model of choice (e.g claude) to your favorite SEO tools so that you can have a conversation with the AI while it retrieves and uses real data from those tools to answer your questions.

In my case, I decided to build an MCP for Google Knowledge Panel! so buckle up here’s what and how I did it!

Building an MCP for SEO

I wanted to build an MCP for Google trends and somehow pivoted to Google knowledge Panel MCP.

First things first, I defined 4 functions I want this MCP to be able to do:

Find every entity behind a name.
Work out which entity is strongest.
Benchmark competitors.
Audit one entity in-depth.
Generate schema for an entity.

To achieve this, I needed two things:

1# Google Knowledge Graph Search API: what Google itself knows about an entity. This is the graph behind knowledge panels.

2# Wikidata: the free, open knowledge base Google has long drawn on. It holds how the entity connects to the real world.

Google tells you that an entity exists and how Google describes it. Wikidata tells you who it is connected to, what it’s linked to and how well-documented it is.

After working with claude code a bit, some testing, etc… the MCP came to life and now it was time to actually use it.

I downloaded claude for desktop (I usually use the terminal) and connected the MCP. All the steps can be found here. You can also check the github repo directly here. You will need to have an API key for Google Knowledge Graph API.

Auditing my entity with the new Knowledge Panel MCP

I think this was the most fun part. I asked claude to “analyze my knowledge panel “sara taher” and how strong it is compared to other entities under the same name”

Using the MCP, it found that there are 20 entities sharing the same name, and to my surprise the most prominent one is NOT me! Claude use the tools in the MCP, did the research and came back with a summary table.

WOW! I always thought the problem was the noise around my entity having other “sara taher” people, but apparently the other Sara has a some strong signals that I have to compete with 😄

Answers Vs. Data

If you’ve been following my work, I’m a big fan of python scripts. Could this task been done with one? absolutely and I’m sure there are cases where a python script would have been more ideal. But what using an MCP did here is help me have a conversation about the data, and draw conclusions or possible areas to explore.

A python script using the API will help you get the data, you will be in full control, you will do the analysis all by your own. I love that and I think that this is very powerful.

But an MCP comes in handy if you don’t want to build a new python script every time you have a different use case for this API, if you want to have a conversation or a brainstorming session about the data, or looking for a quick check rather than invest time in building and testing a python script.

Overall I highly recommend you play around with MCPs, and this MCP of course 😄😄😄

And That’s a Wrap (Almost 😄)

I loved every bit of this and I think MCPs will have a special place in my workflows, but also python scripts are still in my heart 😄

I think in this era, we should try everything at least once (ideally more to get a good taste of the experience) before deciding if it works for us or not and for what use cases.

If you try this MCP and have any questions, my LinkedIn DMs are wide open!

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups on Substack by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

I’m Building a Search Engine (Part 1): Crawling

Sara Taher — Tue, 22 Sep 2026 13:18:15 GMT

Alternatively you can become a paid subscriber:

Intro

So you’ve heard… I’m building a search engine from scratch. Here’s the tea: I want to build a small search engine for all my content specifically, so you can use it to search for my input about a specific topic.

I’ll be sharing my journey of building this in my newsletter. Every time a new milestone is done, you’ll read about it here first!

I think is is a fun project but also building a search engine is far from simple. One way to simplify the process is by being very specific like in my case, the goal is my content specifically, although there’s still details that go into this.

So how to build a search engine, eh?

Components of a search engine

We are not re-investing the wheel here by any means, but there are design decisions that allow for creativity.

A search engine is primarily made of 4 components:

A crawler: of course we need some time a of a system to go find the URLs for our search engine and define the rules this crawler will follow.
Indexer: we also need a system to process the content on those URLs and produce something that’s simpler and searchable and store it in a database that we may call our “search index”
Query processor and ranking system: these are two sides of the same coin. What happens to the query when a user searches? how do we interpret it? for example finding the user intent behind the query, so we can match this query against our search index, that will be sorted and filtered based on the “ranking system” we have built.
Search Interface (SERP): our search results page, where user enters their search and get back results.
So this is generally speaking the 4 components you need to build a search engine. Now let’s talk about my search engine! I don’t have a name for it yet but I’m open for ideas.

My search engine: the crawler

I started planning with Claude Code. There are few design decisions and the steps I chose to build my crawler:

The crawl depth: this is a small search engine, that will rely on two sitemaps for seoriddler.com and sara-taher.com. It will also crawl any specific URLs (not the domains) I have linked to in my content. But that’s as deep as it will go. So the maximum crawl depth is 1.
Pre-fetch URL cleanup: before a URL is fetched, it get normalized. This means the URL gets reduced to it’s canonical format by removing tracking parameters for example.
Link filtering: I have decided to exclude links to .pdf, .jpg, .zip, .css, .js, etc. from my crawls and only fetch html pages. However, those non-html links are stored and labelled as such.
Storage architecture: For each URL we decide to fetch, we store two things: information about the URL and the actual HTML of the page.
Information about the URL includes the URL itself, page title, description, crawl depth, crawling errors, status code, and other metadata. This information is stored in an SQLite database.
The actual HTML of the page is stored separately as a compressed gzip file. The name and location of that gzip file are stored in the SQLite database entry for the corresponding URL, so we can always connect the URL’s metadata to its raw HTML.

Link Graph: I’m also storing which page links to which other page, the anchor text used for the link, and, if we didn’t fetch the target URL, the reason it was skipped. Together, these rows form a graph of how the pages are connected. Right now that’s 2,830 rows from 744 pages. Here’s how this looks in the DB. I’m using
https://sqliteviewer.app/ to view the database.

Crawl controls: I also decided, with the help of Claude Code, that we would not crawl pages sequentially, because that could take a long time. Instead, we crawl multiple URLs concurrently, with a maximum of 8 requests running at the same time. We also make sure that we do not send multiple requests to the same domain at the same time, and we maintain at least a 1-second gap between requests to the same domain.
Respecting robots.txt: because I’m polite 😂 so before crawling a website, we check its robots.txt file to see which pages the site allows crawlers to access. We fetch and cache robots.txt once per domain, then check it before each page request. If a URL is blocked by robots.txt, we don’t request it and record it as blocked_by_robots.
If we can’t access a site’s robots.txt, our crawler currently continues rather than blocking the site. This is a deliberate choice for my small, low-volume personal crawler. The crawler also identifies itself with a User-Agent and provides a contact email. If you see sara-taher-seo-crawler as user-agent, that’s me 😉
Errors: because every crawl will have errors. My idea is a failed page should not be left out from the dataset, nor it should interrupt the crawl. Every URL we attempt to crawl gets a record in the database, whether the crawl succeeds or fails. If something goes wrong, such as a robots.txt block, HTTP error, network error, or extraction error, we record the reason in the error field instead of silently dropping the URL.
Stack choices: I chose a relatively simple stack that gives me everything I need for this project. Here’s everything:

Limitations

Ok so I built my crawler and I ran it. I have the database ready for the next system “indexation” but there is something that I want to include but won’t right now as it will complicate things:

I want to include my video content in this database such that when you use the search engine to search for a topic, you can get videos back in the results. While video pages are included in the crawl, I didn’t download the transcripts for example.
I also want to include content I have written on third part websites, this is not included in the current version of this crawler.
Another thing is, how often should this crawler re-crawl the sitemaps to discover URLs? right now, it’s a one and done thing and to update the database with new URLs I have to manually run a crawl.

So these are few things that come to mind when working on this.

And That’s a Wrap (Almost 😄)

I really think this is a fun project and I will be recommending it moving forward to anyone who wants to learn SEO. We’ve always recommended things like build your own website, or audit online websites, but building a search engine should definitely come at the top of SEO learning tactics!

I know this is way technical that some of my readers would like, but it’s also a good change of pace, and it may open up ideas and inspire you to build things of your own. If you ever want to vibe code, reach out I’m happy to help!

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups on Substack by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

How Much Thinking Goes Into Your Content?

Sara Taher — Tue, 15 Sep 2026 12:46:47 GMT

Alternatively you can become a paid subscriber

Intro

Creating good content is not easy for many reasons, one of them is that we don’t know what good content looks like 😄

For example, some SEO news comes out, my entire timeline is talking about the exact same thing… sometimes I just hold off… do I need to say something just because everyone is saying something? would that be good content really?

And honestly, Googlers have been trying to help clarify what is good content. Just earlier this year I attended Google Search Central Toronto, and out of this event came the famous slide shared by Danny Sullivan about commodity vs non-commodity content:

And there’s also this checklist from Google that you can use to measure the quality of your content:

But still, I feel something is missing. That’s why today I want to introduce you to a new concept… new in SEO but definitely not new in general:

Bloom’s Taxonomy

what’s that? read below!

What’s Bloom’s Taxonomy?

In simple words, Bloom’s Taxonomy is a way of organizing different levels of thinking and learning, from simple to more complex.

It’s used by teachers and educators, but we’re not going to dig too deep into it today.

So what does this have to do with content creation?

Well, there’s a quote by Paul Graham that I will probably quote for life (quote to quote... sorry 😄):

“Good writing is just good thinking.”

And I think this is where Bloom’s Taxonomy and the concept of organizing different levels of thinking, from simple to more complex, becomes interesting for content creators.

Understanding the level of thinking required for different types of writing can help us create better content.

You know what? Let’s not talk about Bloom’s Taxonomy.

Let’s see it in action.

The Cognitive Domain Pyramid

Bloom’s taxonomy defines six levels of thinking. We can use those levels to define our very own six levels of writing! Also note the triangle shape, it’s because level one of thinking is easy, so it is more common, same with content: if it’s easy to produce, there’s probably a lot of it out there.

Level 1: Remember

If you’re writing at level 1, it’s basically recalling facts. You are retrieving established information whether from another source or from your own knowledge and writing about it.

I’d say this is similar to the skyscraper technique in SEO: you find what is already known, gather the information, and put it together in one place.

Example of writing at this level: “Meta descriptions are snippets that summarize the content of a webpage and can appear in search results.”

Level 2: Understand

If you’re writing at Level 2, you’re making sense of information you already know. You’re interpreting it, explaining it, comparing it, or translating it into a different way of thinking.

You’re not just repeating the fact. You’re showing that you understand what it means.

Example of writing at this level: “think of internal links like signposts on a highway: they help search engines discover and navigate between the different pages on your website.”

Level 3: Apply

If you’re writing at Level 3, you’re taking a concept, rule, method, or process you already know and using it in a new situation to solve a specific problem.

Example of writing at this level: "For this local plumber’s landing page, we can apply buyers journey principles by focusing the page on emergency plumbing services to attract conversions rather than writing a general article about plumbing."

Level 4: Analyze

If you’re writing at Level 4, you’re breaking something down into its parts and looking at how those parts relate to each other.

You’re not simply reporting what happened. You’re looking for patterns, relationships, differences, and possible reasons behind them.

Example of writing at this level: "This website lost traffic not because of AI Overviews. A YoY comparison shows a reduction in the number of pages and content created."

Level 5: Evaluate

If you’re writing at Level 5, you’re making a judgment based on data and specific criteria or standards.

This is where you move from “here’s what the data says” to “here’s what I think we should conclude from it, and why.”

Example of writing at this level: "Based on our latest Search Console data, our current content strategy is failing because, while overall traffic increased, the average clicks/page is still very low”

Level 6: Create

If you’re writing at Level 6, you’re combining knowledge, ideas, and information to produce something new.

This is the highest cognitive level, so it generally requires more original thinking and effort, which aligns with the content-effort measure Google uses to assess content quality.

Example of writing at this level: "After studying how LLMs retrieve and cite information, we built an interactive SEO tool that lets users analyze the sources behind an LLM answer."

Note: I asked chatgpt to assess this article and assign a bloom taxonomy level, here’s what it came back with:

There are elements of Understand, Apply, Analyze, and Evaluate throughout, but the overall intellectual work is synthesis: you took concepts from education, SEO, Google’s content guidance, and your own experience and combined them into a framework that didn’t previously exist in this form. That makes the article itself a strong Create-level piece, which is also a nice real-world example of the argument you’re making.

And That’s a Wrap (Almost 😄)

Ya so when writing content, it’s good to see what level of thinking you are putting into your content. Here’s a short summary that you can save somewhere to always assess yourself!

Remember: I know this.
Understand: I can explain this.
Apply: I can use this.
Analyze: I can break this down and find relationships.
Evaluate: I can make a judgment about this.
Create: I can use what I know to make something new.

I hope I gave you some inspiration on what is good content, you can also integrate this framework into your content automation workflows if you’re automating your content production.

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

I Opened the Network Tab. Here’s What ChatGPT Was Doing.

Sara Taher — Tue, 08 Sep 2026 12:38:31 GMT

Alternatively you can become a paid subscriber

Intro

SEO is not dead, but AI is definitely a big part of it now. Clicks are not dead either, but they do not really fully reflect/capture the value both traditional search and AI platforms bring to your businesses.

I mean look at brand mentions. Your brand can be recommended, users can see it, later go to google and perform a navigational search to find your website. Or if they’re already familiar with your website (e.g amazon) and realize there’s no better place to get what they need, they can just go back via direct traffic. You may also end up getting traffic as a referral from one of the cited sources.

For example:

I monitor my newsletter and brand status on different AI platforms. I recently checked ChatGPT for “top indie seo newsletters” and I came out on top with two cited sources, none of them is my website. People will see my website name and either go to one of those sources and look for a link to my newsletter, or manually search Google for “SEO Riddler Sara Taher” for example to find my website. So people will come to your website via organic channels in new ways that mere clicks may not account for.

I think this is a big reason why we have not yet seen many people talking about how much ChatGPT is sending traffic to their websites.

So yeah, definitely fun times in SEO! And because of how important getting mentioned/cited in ChatGPT is becoming, this brings us to today’s newsletter topic: what happens when you prompt ChatGPT?

Let’s dissect that… not in a theoretical way, but what I actually saw happening!

The Network Tab

I really want everyone to play around with this, so here’s what to do. Before prompting chatgpt, right click and select “inspect”.

Next you write your prompt, it can be anything really for me I typed “best crochet yarn for beginners in canada”

You’ll see things starting to populate in the inspect window that you opened, click on network, find the conversation, then click on EventStream.

You can always copy and paste this to chatgpt or claude to explain to you everything! This object captures the moment chatgpt decides it needs to search the web for your query.

The Conversation Anatomy

When you look at the text in EventStream, you get something like this:

{
  "v": {
    "message": {
      "id": "2cc91a19-092d-4735-a0d4-3c6581736af9",
      "author": {
        "role": "tool",
        "name": "web.run",
        "metadata": { "real_author": "tool:web.run" }
      },
      "create_time": 1788831562.9684362,
      "content": {
        "content_type": "text",
        "parts": [""]
      },
      "status": "finished_successfully",
      "metadata": {
        "search_model_queries": {
          "type": "search_model_queries",
          "queries": ["best yarn for crochet beginners Canada worsted weight acrylic cotton Bernat Caron"]
        },
        "reasoning_titles": ["Searching best yarn for crochet beginners Canada worsted weight acrylic cotton Bernat Caron"],
        "reasoning_status": "is_reasoning",
        "tool_icons": ["globe"],
        "reasoning_start_time": 1788831561.9985478,
        "model_slug": "gpt-5-6-mini",
        "model_adjustments": ["auto:smaller_model:reached_message_cap"],
        "parent_id": "d5410723-6ee6-4435-938e-1558808447f0",
        "turn_exchange_id": "6a85u2bb-37cc-1155-a3b4-1625056f1268"
      },
      "recipient": "all"
    },
    "conversation_id": "6a9f673e-e3b8-83e9-8d7d-5b48e139ae34"
  },
  "c": 9
}

so what is happening here? This conversation happens everytime chatgpt decides it needs to search the web during an interaction with the user.

This conversation shows:

The author: can be the user (you), the AI, or a helper tool that the AI can ask to do things for it like searching. This shows that the AI doesn’t search the web itself. It basically says “hey search tool, go look this up for me” and then waits for the answer. Why is this important? it shows that information retrieval (IR) is a separate algorithm than the LLM, optimizing for IR can be different and Google/Bing-like compared to LLMs.
search_model_queries: THIS is the actual useful information, because it’s the exact words the AI typed into the search engine tool it is using. In our example, the AI typed “best yarn for crochet beginners Canada worsted weight acrylic cotton Bernat Caron” so it included most popular brands and yarn types as well.

Interesting note: I tried prompting “best icecream stores near me” and do the same analysis, turns out chatgpt has a separate search tool for local queries called “map_search_model_queries” instead of the “search_model_queries” I mentioned above.

And That’s a Wrap (Almost 😄)

I really hope this newsletter edition helps you explore ChatGPT and how it works closer. And it’s not just about the query fan outs.

As you can see, the query sent by ChatGPT to search the web can include pre-decided brand names, or specific product types for example.

I always knew that search is a separate functionality/algorithm than the LLM itself. Getting cited in an LLM relies on whether this system can find you. Definitely something we should keep in mind when thinking of optimizing for this new era!

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2026 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Is Your SEO Strategy Agile?

Sara Taher — Tue, 01 Sep 2026 13:52:18 GMT

If you find value in my newsletter and want to support the work that goes into it consider becoming a paid subscriber!

Intro

I’ve been super vocal about what a good SEO strategy should look like. And one of the things you’ll see often is a 12 months SEO checklist shared with stakeholders as an SEO strategy.

[checkout my SEO strategy course]

I guess stakeholders want some stability in what to expect is coming next, but also, it needs to be super clear that SEO is dynamic with new things happening and Google updates to search itself and to websites ranking criteria happen all the time.

Not to mention a lot of experimentations are happening to improve visibility in LLMs and we’re basically learning new things every day.

So whatever you present as an SEO strategy, there needs to be an understanding that this strategy is agile, and will adapt to whatever new thing comes our way. Or else, we will be living in the past.

A new report

I was reading a recent report saying that B2B buyers have started to embrace LLMs. But not just any B2B buyers, small and medium size buyers. The use LLMs to create a vendor list, and then take this list to do their own further research on other platforms [source].

What does that look like?

Bob, VP of Operations at a 50-person SaaS company, asks ChatGPT: “What are the best project management platforms for a growing B2B software company?” The LLM recommends five vendors, and Bob narrows his shortlist to three. He then visits their websites, checks G2 reviews, and watches YouTube demos before booking a few sales calls.

What does that mean for SEO?

If you’re optimizing your blog and landing pages pages content hoping this would help new users discover you and learn about your products, you maybe missing out a lot. Because users at the awareness (or discovery stage) may not be starting at google search at all, but rather on LLMs.

I mean here’s another simple example to demonstrate a real life scenario:

I was playing around with ChatGPT’s “fan-out” searches and noticed something interesting: the brands it recommends are already part of the search—not something it discovers from the websites it finds.

So, if you ask ChatGPT for “top SEO newsletters,” it already has a set of brands in mind. The additional searches are mostly gathering information about those brands or validating its choices.

In other words, the shortlist comes first; the search helps justify or verify it. That shortlist is influenced by what the model has already learned from its training data.

Adjust your strategy to the new funnel

The buyer journey funnel for b2b buyers changed a bit 😄

The buyers in the study were evaluating solutions with an average annual value of approximately $250,000, and 94% of those buyers reported using LLMs during their buying journey. [source]

So this means you need to yes continue thinking of search, but optimizing for LLMs visibility should take most of your attention. Ranking in bing is actually more important for you than ranking in Google, since chatgpt uses Bing.

The good news is Bing gives us a bit more data in it’s Bing webmaster tool dashboard, so that should be fun!

The next thing that should be a solid part of your strategy is how you get mentioned on third party websites, your social presence and if you’re doing enough of those to begin with. Because that’s were b2b buyers go to verify and do their own research, so you gotta be ready for them.

While these two things seem simple to understand, execution is where you separate the men from boys. It is a complete directional shift, and a good SEO will come up with a good list of tactics to execute on those two new directions.

If your SEO is still doing SEO for your B2B thinking only about Google, they’re still using the old playbook, in this new new world (yes new new 😄)

And That’s a Wrap (Almost 😄)

Agility is essential to success as much as doing the SEO right tactics. The last thing you want to be doing is sailing your ship at the best speed in the best possible way, only in the wrong direction.

I always say understand who the customer is, in-depth. Only then you would be able to create a tailored SEO strategy for them.

Ya… a mix of agility, and knowing your customer well would take your SEO strategy a very long way!

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Is Meta Description Important for AI Search?

Sara Taher — Tue, 25 Aug 2026 12:15:56 GMT

If you find value in my newsletter and want to support the work that goes into it consider becoming a paid subscriber!

Intro

I had plans to talk about a research paper today, but got excited for an SEO experiment and I think you will too!

A lot of SEO research is being done right now which I really appreciate, and some of it is around meta descriptions. So I decided to contribute to the discussion and run my own quick test.

So bring your coffee and lock in!

The setup

I asked chatgpt (of course) to create content for a fictional coffee shop called “the open book”. Thanks chatgpt for saving my time into writing copy!

Then I tool this content, went to my other dear friend “claude code” and asked it to create a landing page with this text and push it live - done✅

Then, I asked claude code again to add a fictional name, address in Japan, and a random phone number information in the meta description. This is the only place this information is available on the page - done✅

I also added a title tag and a canonical.

You can check out the fictional bookstore here: https://www.sara-taher.com/the-open-book

Now let’s see if different LLMs can read the NAP (name, address, phone number) information in the meta description!

What Chatgpt said

I asked chatgpt “where is this book store located and who owns it: https://www.sara-taher.com/the-open-book” but of course it had to act out like smarty pants and say this is a fictional bookstore, bla bla bla because it created the content and I was logged in.

So I logged out and asked it again, here’s what it said:

“I checked the page you linked, and there’s an important wrinkle: I can’t verify that “The Open Book” is an actual physical bookstore with a publicly disclosed location or owner.”

It also said that:

“Location: Not disclosed on the page, and I couldn’t find a reliable external listing establishing a physical address.

Owner: Not identified as the bookstore’s owner. The website is Sara Taher’s personal site, but that alone isn’t evidence that she owns an actual bookstore. “

Verdict: Chatgpt seems to not read or consider content in meta descriptions.

Note: apparently chatgpt was trying to load an image or a map but couldn’t

What Claude said

I asked claude the same question, and claude said it could not find this information on the page:

“I fetched the page, but I want to flag something: the visible page content for The Open Book doesn’t actually mention an address or an owner’s name anywhere — it’s just marketing copy about the bookstore/café concept (menu, events, FAQs, etc.).”

However, it said that this information is available in the meta description!

Verdict: Claude seems to read/consider content in meta descriptions.

Note: this is an interesting finding since just recently I came across research that Claude - while being perceived as more trustworthy - has the second highest level of inaccuracies. Well, it definitely is thorough though.

What Gemini said

I asked Gemini the same question, and here’s what is said:

Basically Gemini did not check the meta description for this information, AND it seems like it’s trying to connect the “open book” bookshop to existing entities it already knows about. I think this is super interesting!

Verdict: Gemini seems to not read or consider content in meta descriptions.

What Google search & AI Mode said

I searched for the same query on Google (try it yourself and let me know if you get any different response): where is this bookshop located and who owns it: https://www.sara-taher.com/the-open-book

I- of course - got an AI overviews, when I expanded it, it took me to AI mode and overall I think this is the worst and more irrelevant answer! It started looking for books I mentioned in another irrelevant article 🤷‍♀️.

Verdict: Google search/AIO/AI Mode seems to not read or consider content in meta descriptions.

What Perplexity said

I asked perplexity the same question, and I’m very concerned at the confidence level it responded with 😂😂😂 It just assumed this is the same bookshop as another bookshop it has in it’s statistical prior or something.

and you can see it referenced 27 sources, when I checked, none of them is the URL I provided, although it did check my website homepage 🤷‍♀️

Verdict: Perplexity seems to not read or consider content in meta descriptions.

And That’s a Wrap (Almost 😄)

I really had so much fun working on this small text. It seems that different models have different approaches to how they handle meta descriptions and reading the HTML. Something I have been saying alone in this industry (or so it feels).

Back to our question, is meta description important for AI search? overall no, maybe if you’re optimizing for claude. But It’s not a matter of importance, it’s really a question of do ai search bots read the meta description tag?

So that also brings us to another interesting thought, some models maybe considering the entire html of the page and some may not, some extract specific elements (e.g schema or meta descriptions) and others may not be doing that at all.

Would that work if the html is broken? that’s a totally different question that cannot be answered by this simple test but I have an answer and so many related ideas that I’ll probably discuss in one of my upcoming newsletter editions (ya! more to come).

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Built to Collapse!

Sara Taher — Tue, 18 Aug 2026 12:50:35 GMT

If you find value in my newsletter and want to support the work that goes into it consider becoming a paid subscriber!

Intro

Publishers are fighting back in a battle of humans against bots...

They have been complaining extensively about how AI platforms and Google’s own AI features are using their content and, ultimately, taking their traffic.

There is also a clear divide among publishers on whether they should allow AI bots to access their content at all. Some are blocking them, while others are allowing access, and so far, there is no clear strategy or agreed-upon path for how publishers should tackle this.

And if you haven’t heard, just last week, more than 300 French newspapers filed a complaint against Google over AI Overviews, less than a month after the feature was rolled out in France.

Seems like publishers are hoping for governments to protect the industry. But things are not that simple, are they?

The challenge isn’t just about LLMs stealing content and serving it to users. Journalists and reporters are also competing against AI-powered newsrooms, some of which are completely human-free and fully operated by AI.

An AI news publication called RuntimeWire beat WIRED and other popular news publications by posting stories THREE HOURS faster! And in the world of news, time is of the essence… so you can imagine how big a deal that is.

That should be a fun and interesting one to look into, so buckle up!

The Tea

At the recent Black Hat security conference in Las Vegas, OpenAI shared new details about a hacking incident involving its own rogue AI agents. They have been talking about those incidents very proudly, by the way 😄

Anyways, as usual, after the event, reporters raced to publish, and WIRED was among the fastest. But an outlet called RuntimeWire beat it by more than three hours.

THREE WHOLE HOURS.

RuntimeWire had no reporter at the event. Actually, they have no reporter at all. It’s an AI newsroom, where AI bots scout the internet to find stories they can write about.

In around 3 months, RuntimeWire posted more than 2,000 stories by scanning court databases, forums, company filings, and social media for tech news. This AI newsroom costs $100/day to run.

On the day of the Black Hat security conference, Ryan Merket, the founder of RuntimeWire, spotted an OpenAI executive posting about the talk on X. Merket fed a transcript of the livestream to his AI agents while the session was still underway, and RuntimeWire published about six minutes later.

Voila! Or is it? Is this the death of real journalism? Should we be worried?

The Truth

It’s easy to see news like this and say, “Oh great, one more job category AI is going to take over”…. but that’s far from the truth.

While this is a successful experiment to some extent, let’s not forget the facts:

The quality is not comparable to real journalism.
There will be incidents where real journalism that is present live covering an event cannot be simply replaced by an AI reporter.

Then there’s also the data… as a curious SEO, I decided to have a look at how RuntimeWire performs in search. It all comes down to two questions:

1# Will people like AI reporting?

The data says there’s just momentary interest in the brand, probably related to news about the brand rather than stories reported by the brand itself!

2# Will bots like this content?

Will fellow bots like the content generated by one of their own? 😄 The irony…

We know that Google isn’t against AI-generated content per se. However, we also know that this content is low effort to generate with minimal human intervention and real-world experiences (bots cannot live life 😄)… it’s the definition of commodity content… you know, the type of content that is just canned and mass-produced, that looks good at face value, but offers no actual value over any other page on the internet.

A simple search for the brand name shows that there aren’t even sitelinks for this website. If anything, it goes to show that none of the pages come off to Google as important or popular to users.

Compare that to WIRED in search? There’s a Knowledge Panel, there are sitelinks, there’s “Latest from WIRED,” and more…

If you also look at the traffic, the website received around 100k visitors in the last month, despite having over 2,000 articles. That’s 50 visitors/article. Also, it does not seem to be successful in search. Most of its traffic is from organic social (Reddit and Facebook). All data is from Similarweb.

The top referring website is news.ycombinator.com, which is a platform where you can submit your own stories, really, so no magical tricks here or anything that cannot be replicated or, even better, outperformed…

My 2 cents is, the website idea is cool, and is good enough for people looking for a very high-level surface read into the news. It’s a battle of brand and distribution, not quality at all.

While these kinds of websites can see some success, I don’t see them winning big. They’re definitely an annoyance to big news publishers, but nothing that cannot be addressed.

And That’s a Wrap (Almost 😄)

I think the biggest lesson here is to always pause and think, and understand the play you’re up against.

Creating awesome content is great, but relying just on that is like saying, “Create great products and people will come.”

In news, I don’t see AI reporters outperforming real, experienced reporters or publishers that have been in the game for a long time. Yeah, sure, AI reporting can be faster, so they have a few tricks up their sleeves. However, that is something all publishers can learn from and use.

Any traffic AI newsrooms are capturing today is because of gaps big publishers have left. Are you fast enough? Are you distributing on the same platforms? Are you branded right for your audience?

Nope. AI is still far from replacing humans in journalism.

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

The Simple Way to Check AI Overview Visibility in Bulk

Sara Taher — Tue, 11 Aug 2026 13:20:57 GMT

Alternatively you can become a paid subscriber

Intro

We talk a lot about optimizing for better visibility in AI, and there’s tons of content being shared, but one of the questions I get asked in my community calls is how to understand where you stand today in AI?

Personally, every once in a while I have someone reach out to me, and they say they found me from an LLM platform.

I just don’t share those very often so not to jinx it 😄😄😄

Anyways, for today I decided to put together a simple process for you to understand where you stand today in your AI visibility in AI Overviews.

Note: this is not an audit to find fixes, it’s an audit to observe how and when do LLMs cite you, and recommend you.

I did that before

I recently created the data study “Who Does Google Trust in AI Overviews for SEO Advice?”

and it all started with a simple data export of a bunch of related keywords (thousands)

Then I did some cleanup, used an API to get AIO for each of the keywords, and analyzed those answers in bulk.

I really recommend you check out this study so you can imagine what you can do if you follow my process. Works wonders when you begin with the end in mind.

Let’s get to it

So there are two thing you want to do:

how often, and in what situations is your brand cited and/or mentioned
who does AIO cites and mentions in general (competitor analysis)

The steps are simple.

The keywords list

First, pull a list of keywords that you want to analyze your visibility for. You can get those keywords from GSC, or you can get them from any of the third-party SEO tools.

Once you have your list looking something like this:

The cleanup

If you pulled a large list of keywords, it’s always a good idea to sift through it to remove any noise. For example, you may want to remove competitor brand names (or not depending on what keywords are important to you to analyze), keywords with Non-Latin letters for example. API calls can be expensive if you’re doing them for a large number of keywords, so cleanup is important to reduce the cost, but also, bad or noise keywords will reduce the quality of your report.

The best way to do this cleanup is a mix of both manual checks and using claude or chatgpt for more large scale automated changes. But always ask them to show you examples of any keywords they removed. Double check, double check, double check.

The API calls

There are a lot of SEO APIs out there, and I have been using SERPapi and the good news is that you can create a free account and get 250 free search per month (not sponsored)!

250 searches are not a lot, but can be a good place if you’re working on a small audit for your website.

I created this free SEO python script for you, once you run it, it will ask you for your API key, which you can get from SERPapi as I mentioned above. Then it will ask you for a CSV that has your keywords. It needs to be called “sample_keywords” and the keywords are in the first column that’s called “keywords” all lowercase. here’s the sample input and output file that I used for the purpose of this blog.

For each of the keywords on your list, the output will look like this

You have the keyword, the AI Overviews, and all the URLs in serps as well. Looks good right?

Note: before running the API on thousands of urls, make sure you run the code on a small set of keywords to test it’s doing what you would like it to do.

The analysis

This is what separates men from boys, or great SEOs from just SEOs. Using chatgpt or claude (I personally used claude code) I would ask it to analyze this output data. For every analysis it should save the output, as we will need to create an HTML report in the end (similar to the report I created).

Note: If you think this is a quick and easy task, you’re probably doing it wrong. You need to double- and triple-check the output for every question before moving on to the next one.

I also recommend asking Claude or ChatGPT to write code to analyze the data, rather than having them analyze the dataset directly. This will give you more consistent results, save you money, and, perhaps most importantly, make it much easier to audit the analysis when you understand exactly how the results were generated in the first place.

Some of the analysis questions you can perform:

How often was my brand cited compared to my competitors (provide the list of competitors and of course your brand name)
How often does being cited or mentioned overlaps with ranking and in what positions.
When your brand is cited, what was the sentiment? perform a sentiment analysis for each time your brand was cited or mentioned in AI Overviews to understand was is cited in positive, negative or neutral tone.
What type of queries trigger my visibility (is it informational? is it when a specific feature is mentioned, etc…)
What type of queries are we lagging in visibility (same question as previous one but looking to the other half of the glass)

the sky is really the limit… and this is where you shine!

Finally combine all this data into an hmtl report because it’s easier to format and you have a very cool report that you can use to drive optimization initiatives in your organization.

And That’s a Wrap (Almost 😄)

I may have made this task sound simpler than it actually is, I know. But that’s not necessarily a bad thing.

Well… I hope I made it simple enough for you to test out and have fun with!

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

How to Learn SEO in 2026?

Sara Taher — Tue, 04 Aug 2026 13:23:03 GMT

Hello Riddlers!

📌 If you find value in my newsletter and want to support the work that goes into it consider —> [☕Buy Me a Coffee☕]💡 Want more from me? [SEO Strategy Course] | [Build Your Own SEO Tools (with Python & Agentic AI)] | [Premium SEO Mastermind]

Alternatively you can become a paid subscriber

Intro

I run monthly live SEO townhalls to chat about anything and everything SEO, you can also bring your own questions or tips that you’d like to share. You can watch recording of previous sessions here, or signup and join us for the next one!

These sessions always inspire me, and last time someone asked, “How do you learn SEO in 2026?” That really got me thinking...

We’ve come a long way. SEO has evolved into a vast ocean that requires continuous learning if you want to keep sailing (such a dramatic analogy, right? That's totally me, not the AI helping me write this post 😂).

AI is also a big part of this evolution, so I thought it was a great question to answer in this newsletter. So, hold tight!

What is SEO today?

First, I want to set the stage to what learning SEO includes today and the areas of knowledge you should pick up. I, by no means intend to overwhelm you, we are all still learning. But it’s important to begin with the end in mind.

SEO imo covers a lot more area than we used to. Auditing reddit and finding brand interaction opportunities for your social team, or auditing your YouTube Channel and optimizing the content and descriptions, not just to rank on YouTube but also to rank those videos better in search, and get your brand cited in AI answers using those videos (YouTube gets cited 25% of the time in AI answers).

You may be wondering at this point… OMG do I need to learn all of this? not right away… you’ll build your knowledge overtime, with every audit and every client and every new SEO article you read… but you gotta cover the basis.

1# Cover the basis

People in this industry would eat me alive if I said anything before saying this: the fundamentals are still the fundamentals. If you want to learn SEO in 2026, you still need to master many of the same concepts you had to learn years ago.

Now that I’ve got that out of the way, here’s what I recommend:

If you can build your own test website, please do. It costs little more than a domain name and about $4/month for GitHub, and you can build your own website from scratch using Claude Code. This is the best learning environment—better than any course I can recommend.

If you don't want to build a website, find a small website you'd like to audit instead.

Next I recommend you go through Google’s SEO Starter Guide. As you read and learn, keep going back to your website and apply what you've learned.

For example, the documentation talks about the site: search operator (e.g., site:wikipedia.org). Go try it in Google Search and see what happens.

Don’t understand something? The good news in 2026 is that you can ask ChatGPT, or join one of the many SEO communities and ask your questions there. I also run an SEO community, apply to join here.

In my opinion, Google's documentation is the best place to start. If you're looking for something more comprehensive that will set you up well, I'd also recommend the SEO specialization certificate on Coursera. By the end of it, you'll have the knowledge expected of a junior SEO.

2# Quality content

Google recommends creating “helpful content”... but what exactly is helpful content?

One of the best places to start is Google’s own “Search Quality Rater Guidelines: An Overview”

I highly recommend reading it and creating your own content audit checklist as you go. It’s a great way to turn what you’re learning into something practical that you can reuse.

You can also upload the document to NotebookLM. It can generate a podcast based on the file, and you can ask questions about the document and get answers grounded in its content.

3# Uncle Tom

What about doing SEO for platforms like ChatGPT and Claude?

Well, some people call this GEO...

And if you haven’t heard, using the term GEO is very inflammatory for some people in the SEO industry. Is SEO the same as GEO? Is that even the right question? The debate goes on and on...

I don’t really get into that debate anymore. Call it whatever you want—you can call it Uncle Tom for all I care. 😂

I highly recommend you devour this resources to get on top of this field:

The iPullRank AI Search Manual

That’s it. That’s my advice!

4# Signup for the right resources

LinkedIn is great, but newsletters are even better. Here are a few newsletters I recommend. I’m not affiliated with any of them (except, of course, my own 😉).

SEORiddler.com: a newsletter by yours truly, it’s not really a beginner level newsletter, but it opens your eyes to new ideas, and shared actionable tips as well.
seroundtable.com: that’s where I go for news, you’ll find every single thing that ever happened any day in SEO there, that’s a given.
Mike King’s iPullRank newsletter: big topics, in-depth AI/SEO research, and so many gems in there.
Tech SEO Tips Newsletter: this one is by Nikki Halliwell, I had her on my podcast one time and she’s awesome and her newsletter is no less if you want to build a good technical SEO foundation.
SEO Notebook by Steve Toth: short actionable tips on SEO and GEO, definitely worth a spot in your inbox.

5# Wash, rinse, repeat

If you feel the resources I recommended above are not enough, you’re right—you need practice. That’s where you truly grow and learn.

Audit more websites. Audit big websites too: Amazon, Wikipedia, CNN, and others.

Learn to find patterns. Every time you use Google, ask yourself: Why is this URL here?

And That’s a Wrap (Almost 😄)

I hope I didn’t overwhelm you—please don’t be! We’re all learning, and anyone who says otherwise is simply not being honest.

Take baby steps, learn as you go, and try to find a mentor who can guide you if you can.

Good luck!

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already. (Pssst, you can also become a paid subscriber)
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Google Might Be Learning the Facebook Lesson

Sara Taher — Tue, 28 Jul 2026 13:30:11 GMT

Hello Riddlers!

Alternatively you can become a paid subscriber

Intro

Google has been facing a lot of criticism and, in many cases, deservedly so.

First came AI Overviews. Then AI Mode.

Features that few people were asking for, but Google had to prove it wasn't falling behind in the AI race after OpenAI launched first using the Transformer architecture that originated from Google's own research.

And the data reflects the sentiment.

Since May 2024, searches from people looking for ways to turn off AI Overviews have been steadily increasing. Just facts. The trend peaked around April–May 2026.

Maybe the people actively searching for ways to remove AI Overviews are a minority. But I suspect many more are simply quietly dissatisfied.

You know who else is dissatisfied? Publishers.

I don't think I've ever seen this level of audacity toward publishers. AI platforms are using content they didn't create to train their models and perform live searches, then assembling answers that compete directly with the very sites that produced that content, taking away the clicks those publishers depend on.

Google says it’s putting users first. I’m not convinced.

To me, the goal isn’t just to help users find information really. It’s to keep users inside Google’s own ecosystem for as long as possible. That’s what this is really about.

And let’s not forget...

Google spent years lecturing all of us about E-E-A-T.

Publishers, SEOs, and content creators have been told to create “helpful content” that demonstrates Experience, Expertise, Authoritativeness, and Trustworthiness. We audited pages, rewrote articles, added author bios, cited sources, and jumped through every hoop because that’s what Google said good information should look like.

Then Google turned around and gave us AI Overviews.

A system that has repeatedly produced inaccurate or misleading information.

I could pause here and start talking about OpenAI and Anthropic. They aren’t exactly better when it comes to how they use other people’s content either.

But to keep this conversation focused, I’ll stick with Google today.

I think Google is up for a wakeup call.

Even tech giants can stumble, and history has a funny way of repeating itself in unexpected ways.

A challenging echo system

A lot of businesses have been impacted over the years with Google updates and a lot of websites lost traffic but none of it is equivalent to what we are seeing today.

Reddit is reportedly considering not renewing its $60 million deal with Google, which allowed Google to access Reddit content to train its AI models.

Reddit’s stated concern is that Google isn’t sending enough traffic back to the platform.

That argument is a little hard to take at face value, considering Reddit has become one of the most visible domains in Google Search results. Reddit threads appear everywhere now, often ranking prominently for product reviews, recommendations, and real user experiences.

And then we have publishers. Google said that it “sends billions of clicks to websites every week through AI search.” but also does not provide the data to verify this in GSC, and publishers are seeing declines in their traffic, the traffic they rely on to generate revenue through ads, subscriptions, and other business models.

Hence, a lot of publishers are considering blocking Google.

And just as Google is facing all of these challenges, SerpApi out of nowhere gets a Google lawsuit dismissed.

You see, Google had the audacity to sue SerpApi for copyright infringement because SerpApi was crawling Google.

Yes, you read that right… do not laugh, please. 😂

Google literally crawls every single website on the internet.

At least when SerpApi crawled Google, they weren’t taking clicks and revenue away from publishers.

Justice was served, and the court ruled that there was no copyrighted material to begin with. 😂😂😂

So some users are not happy, publishers are definitely not happy, Reddit is pushing back, and Google SERPs have been declared as non-copyrighted.

My 2 cents: as these incidents continue and more pressure builds on Google, something has to give. Google will eventually have to find ways to compensate publishers… right?

You may say, “Sara, you’re a small content creator. What do you know?”

Fair enough.

But I do know a thing or two from my startup days.

Hear me out.

Once upon a time

Do you remember when Facebook reduced the organic reach of pages from around 25% to somewhere between 1%–5%?

The goal was clear: push more businesses toward paid advertising and increase ad revenue.

But what happened over time?

Content creators slowed down. Pages became less active. The platform became far less engaging and, at times, almost felt like a digital desert.

I remember back in 2015 suggesting that Facebook should allow people to make money from creating great content on pages.

Without active publishers and creators, platforms risk becoming less engaging and less valuable because the content that keeps users coming back depends on those creators.

and turns out, that wasn’t such a crazy idea.

It took Facebook years to realize it, but eventually they started paying publishers and creators to keep producing content.

Facebook paid out nearly $3 billion to creators in 2025. Of those $3 billion in payouts, 60% went to Reels while the rest went to Stories, photos and text posts.

The lesson? Platforms need creators just as much as creators need platforms.

Maybe Google isn’t there yet, maybe they need to go through the same journey as Facebook to realize that. But they will eventually get there, if publishers and creators are not happy, the platform slowly starts to die.

And That’s a Wrap (Almost 😄)

If you’re a publisher, try to navigate this challenging period.

Because there will come a point when Google and other platforms realize that publishers are not just content suppliers, they are essential partners in their success.

And at some point, they will have to find a way to give back as much as they take.

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already.
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

First They Came for the Clicks. Now They Want the Checkout.

Sara Taher — Tue, 21 Jul 2026 13:14:09 GMT

Hello Riddlers!

Alternatively you can become a paid subscriber

Intro

Everyone seems to be angry at Google right now. Just yesterday, I came across the latest piece from The New York Times titled “Google Is Building an A.I. Fence Around the Internet It Once Championed.”

And, of course, even before this article was released, we had already heard statements from Google representatives emphasizing that Google continues to send significant amounts of traffic to the web. (yes Google we believe you 😂 and that’s exactly why you don’t want to show us the proof in GSC 😂)

Just last Friday, Google’s Head of Search, Nick Fox, wrote on LinkedIn that Google is “sending billions of clicks to websites every week through AI features in Search alone.” He added that these AI features were designed with this goal in mind: “We’ve designed our AI features in Search to connect people to websites.” Overall, he stated that Google “continues to send billions of clicks to the web every day through Search” and that people are using Google Search “more than ever.”

Similarly, last year, Google Search VP Liz Reid said that Search was still sending a “relatively stable” number of clicks—again, in the billions—and argued that reports showing declining search traffic relied on flawed methodologies.

The interesting part is that the conversation is no longer just about how many clicks Google sends. A billion clicks can be a very reassuring number until you start asking for the missing clicks data in GSC.

Well, in this sea of pessimism, I might actually have something potentially good news for ecommerce websites.

So buckle up!

First they came for the clicks, now they’re coming for the conversions

Publishers are upset because Google is taking away clicks from their websites, clicks they earned by creating great content and playing by the rules, clicks they need to generate ad revenue.

Ecommerce businesses, on the other hand, may have different concerns: how their products get recommended across AI surfaces and Google’s recent efforts to capture purchase clicks by offering users the ability to complete checkout directly within Search.

Just last year, ChatGPT announced that users could buy directly from U.S. Etsy sellers within chat. And in January, Google announced tools through the Universal Commerce Protocol that allow shoppers to complete purchases directly within AI Mode and the Gemini app.

I can imagine many concerns around this. One of them is that when people purchase directly from your website, they may discover other products they weren’t originally looking for and add them to their cart. Moving the purchase experience to a third-party platform — whether ChatGPT, Gemini, or AI Mode — gives merchants less control over the overall shopping experience.

For many retailers, this could be a significant challenge. The traditional ecommerce experience allows brands to cross-promote products, recommend complementary items, and encourage customers to explore more of their catalog. In a third-party checkout experience, products may be listed and purchased individually, without the same opportunities or control to guide customers toward other items or offers from the same seller.

This could also be seen as another example of what the New York Times article I shared earlier described: “Tech companies have built empires that locked users in and were focused on maximizing profit instead of enhancing public utility.”

So, as you can see, the concern has evolved. First, they came for the clicks. Now, they’re moving toward controlling the conversions by keeping users within their own platforms which raises an important question: how much control will merchants have over the customer journey in the future that Google and other platforms are building?

I can hear you saying… Sara, where’s the Good news you promised?

Quick historical analysis

I’ve been digging into the history of innovation in ecommerce trying to find patterns for an innovation strategy. Here’s one thing I noticed and I’m sure glad I did:

Does anyone remember Pinterest’s “Buyable Pins”?

Pinterest originally launched in-app purchasing through “Buyable Pins” in July 2015. However, the feature was officially retired globally in October 2018. Instead of keeping purchases within the platform, Pinterest shifted toward Product Pins, which redirect users to the retailer’s website to complete their purchase.

Then there was “Checkout on Instagram.”

Instagram launched its in-app checkout feature in March 2019, allowing users to complete purchases without leaving the app. However, Meta retired the feature globally in April 2024, moving back toward product tags that send users to the retailer’s external ecommerce store to complete their purchases.

So historically, we have two examples of major tech companies attempting to own the checkout experience and both eventually moved away from it.

Why?

Because users don’t just want to buy a product; they want to experience the brand they are buying from. They want to understand shipping times and costs, learn the brand’s story, decide whether they trust the company, explore other products, discover promotions, and so much more.

If there’s one thing this history teaches us, it’s that when users have an established way of doing things on a platform - like shopping on the web- changing that behavior is incredibly difficult.

The better opportunity may not be replacing existing habits, but building solutions around them: reducing friction, improving discovery, and making the existing customer journey easier rather than trying to completely reinvent it.

If you’re an ecommerce website, congratulations it seems you are safe for now. Your focus should be on optimizing your platform for conversions, removing any friction and simply making every click to your website count!

And That’s a Wrap (Almost 😄)

History is not boring! It’s actually one of the most fascinating places to look for insights, if you’re open to learning from it.

There’s a lot of negative news circulating as we navigate this AI boom: layoffs, declining clicks, concerns about the future of the web, and even broader conversations around how technology is affecting the way we think.

But we shouldn’t simply give in, passively consume, and accept everything we read as fact… right?

Speaking of being “open to learning”…. that’s a much bigger discussion for another day. There’s so much I want to say about what it takes to build a better SEO industry.

Hopefully. 🙏

That’s that for today folks and see you in the next newsletter!

Support the Riddler!

Sign up for my newsletter if you’re not already.
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Where Did the Clicks Go?

Sara Taher — Tue, 14 Jul 2026 13:04:45 GMT

Hello Riddlers!

📌 Join my in my next webinar on Thursday July 16 · 1 PM UTC / 6:30 PM IST where I will be discussing how to navigate the zero click era. Sign up here.

Intro

Google is getting billions of searches every month, but we’re also learning that only 32% of all searches end with any type of click and of those, 66.6% go to the open web (non-google non PPC ads).

We’ve been blaming Google’s AI experiences, and rightfully so. Google, on the other hand, has been trying to soften the blow by saying websites are only losing low-quality clicks from users who would’ve bounced anyway. Just last year, Google’s Elizabeth Reid said AI Overviews drive “higher-quality clicks.”

This year, there’s a different story.

Perhaps more data is now available (like Google needs more data 😂) but Liz said in a recent interview that “the decline [of traffic to publishers] isn’t solely caused by AI Overviews, but rather by broader shifts in user behavior”

So I decided to analyze this statement.

Is it true? because if so, we maybe missing out on so much opportunity, and holding on to optimizing for the wrong type of content.

So buckle up!

As always, I'm going to build the case step by step before getting to my final conclusion.

Platformisation

A recent Reuters study has some very interesting findings.

When it comes to news consumption, the study found that “social media and video networks are, on average across the markets covered, more popular than both TV and owned news websites and apps as sources of news,” and that “people increasingly like to watch rather than read the news, often drawing on a wider range of sources and voices.”

And video content isn’t synonymous with YouTube.

The study shows that:

YouTube is used by 34% of people for news.
Instagram follows at 26% and continues to grow.
TikTok, while starting from a smaller base, is growing the fastest and is now used by 20% of people for news.
Facebook remains the largest platform overall for news consumption, with 43% of respondents using it, reversing recent declines.

But here’s another interesting question...

Who’s creating all this content?

Not necessarily large, trusted news organizations.

The study found that:

27% of respondents globally get some of their news from news-focused creators or influencers.
46% get some news from creators of any type.

Why?

Respondents said creators are more entertaining, easier to understand, and more relatable than traditional news outlets.

Ironically, those same respondents also believe creators are less trustworthy and less impartial.

So despite recognizing that creators are generally less trustworthy, people still choose to consume their content because it’s more engaging, easier to digest (likely because of simpler language), and more relatable.

The following chart shows how trust varies across different news sources.

So... is traditional media dead?

Actually, no.

The study found that people are consuming these newer formats alongside traditional news rather than completely replacing them.

Another interesting finding is that more than half (52%) of respondents aged 18–24 now say that social media, video platforms, and AI chatbots are their primary way of getting news.

To me, this tells us three things.

First, user preferences are shifting toward consuming information on social media and video platforms rather than traditional text-based content that ranks in search.

Second, the age of your audience matters. Millennials—and especially Gen Z—are increasingly choosing these platforms as their preferred way to discover and consume information.

Third, people want to hear from people, not necessarily large brands.

Interesting Expertise Content

For so long, our focus has been on creating “helpful content.”

And naturally, helpful content is not always built to be entertaining.

But it seems that to be successful with your content strategy in 2026, and appeal to wider audiences, you need to think about the entertainment factor as well.

Now, Liz Reid didn’t say that exactly. What she said was:

“To the extent that you produce really interesting expertise content, we still see that people are interested in that.”

May I second that?

I’ve been seeing a lot of misinformation in our industry being amplified by the same voices over and over again. And somehow, those voices become very popular.

What’s interesting is that, almost every time I see this content being shared, because it’s entertaining. It’s funny, provocative, or creates a strong reaction.

And that’s the point: attention and trust are not always correlated.

So if your content is not getting the attention or user engagement it deserves, it may be missing that element.

I’m not suggesting you become a clown 🤡. I’m simply pointing out an observation, and it’s up to you to decide whether you want to adapt your content strategy.

But keep in mind: the more intellectually sophisticated your content and concepts are, the narrower your audience will naturally become. And you can be ok with that by the way.

This is not a new finding. It has always been the case.

However, it may be more visible than ever today because users have an endless number of sources competing for their attention.

Content Length

Liz is also arguing that:

"To the extent that you might just read an AI review and you wouldn’t click, you might have previously just read the headline and not clicked through… You were going to go spend 10 minutes reading the article, you’re probably not going to decide that an AI Overview is a substitute for the 10-minute article."

She attributes this to the fact that sometimes people just want a quick answer—a sports score, for example—and they were never going to read the full article anyway.

So, according to this argument, AI Overviews are not necessarily taking clicks away from publishers. People will still click when they want something beyond basic information: an in-depth analysis, a unique perspective, an opinion, or expert commentary.

Ok Liz, but here’s the thing...

AI Overviews can also summarize those unique viewpoints and opinionated analyses. So that argument isn’t necessarily a strong defense for Google either 😂

However, this does point us toward a bigger question.

At some point, we have to face reality.

And reality says: people are consuming less long-form content.

As dramatic as this title from The Atlantic sounds “The End of Reading Is Here”,it doesn’t feel entirely disconnected from what we’re seeing.

Research at the University of California, Irvine, highlights how our attention patterns have changed over time. By tracking digital activity, researchers found that the average time spent on a single screen before switching tasks dropped from around 150 seconds in 2004 to approximately 47 seconds today.

The reality is that people’s attention is becoming more fragmented. With constant notifications, infinite scrolling, and an endless supply of content competing for attention, users are increasingly looking for faster and easier ways to consume information.

That’s why short-form answers like AI Overviews, as well as alternative discovery platforms like social media and video networks, are becoming increasingly attractive for quick and passive information consumption.

AIO and Few More Things

Yes AIO is taking clicks away, but also we need to acknowledge that people are preferring video content, shorter content (a lot of the times), content from creators and social media, content that makes them smile or laugh 🤷‍♀️

That’s the reality of where we are today.

And while I will remain a strong supporter of long-form content, I don’t think the answer is to ignore these patterns.

My recommendation is to branch out.

Take the valuable insights, research, and expertise from your long-form content and repurpose it into different formats and distribute it across different platforms.

And That’s a Wrap (Almost 😄)

Next time someone asks where the clicks go (I asked Google ppl that in search central live 😄) I guess now you know the answer…. Google took a lot of it, users want different things, and we have not adjusted to this new era yet.

That’s that for today folks and see you in the next newsletter!

References

Support the Riddler!

Sign up for my newsletter if you’re not already.
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Why Content Chunking Was Never Really Up for Debate

Sara Taher — Tue, 07 Jul 2026 11:52:10 GMT

Hello Riddlers!

Alternatively you can become a paid subscriber

Intro

You’ve probably seen the term content chunking circulating across SEO social media, with some people recommending it as a tactic to improve AI visibility while others dismiss it as unnecessary.

And, of course, Google had to weigh in. On multiple occasions, they have made it clear that you don’t need to chunk your content for AI. They even added this in their official “Optimizing your website for generative AI features on Google Search” guide.

But… there are a couple of issues at play here.

First, a lot of SEOs are still relatively new to LLMs and AI—myself included—and I don’t think we always acknowledge that. If we did, we’d probably be a little more cautious about immediately rejecting new ideas.

Second, Google’s guidance is, to some extent, self-serving. Google doesn’t want site owners restructuring their content because of every new trend that popsup through the industry. From Google’s perspective, that’s a reasonable position to take. They said verbatim “We don’t want people to have to be crafting anything for Search specifically.” So their concern or objection is not the concept of chunking itself; rather, it’s trying to change your content specifically for search and visibility with disregard to users.

Chunking is also misunderstood in my opinion.

Of course it is, as I said we’re all new to this.

So when people try to interpret chunking into actionable approach, they may get it wrong, but then the nay-sayers would latch to those misinterpretations and label the entire concept as flawed.

So…

What is chunking, actually? And why was it never up for debate to begin with?

Buckle up!

What is chunking?

Chunking is the process of breaking a large text document into smaller, coherent sections called chunks. Each chunk represents a complete idea or segment of the text while preserving the original meaning and context.

Chunking is essential for large language models because they cannot process an unlimited amount of text all at once.

Makes sense right?

LLMs need to go through vast amounts of unstructured texts, and by dividing the content into smaller chunks in a size that LLMs can process (this size is called the context window) LLMs can process text more efficiently and therefore search for and retrieve information better.

Without chunking, LLMs are searching for a needle in a haystack.

If you ask an LLM, “What is an atom?”, it does not need to process every chemistry and physics resource ever created. Instead, a retrieval system can identify and provide the LLM with the specific sections of content that are most relevant to the question, such as passages explaining the definition and properties of atoms. This approach improves efficiency by reducing the amount of information the model needs to process, while also improving accuracy by allowing it to focus on the most relevant context.

This isn't just theoretical. Here's a simple example showing how splitting text into coherent paragraphs changes retrieval relevance.

Totally copying iPullRank here, I created this chunking simulator to show the relevance difference of keywords and text, before and after splitting the text into 2 paragraphs:

When all the text was one paragraph the relevance scores were 45.3% and 55.3%:

When the text was split into 2 paragraphs the relevance scores went up to 58.4% and 68.6% and that’s just by splitting the text into 2 proper paragraphs:

I’m still working on this tool and will probably launch it soon on my website for you to play around, stay tuned!

So….

We’ve established that chunking content is a fact of life used by RAG systems to feed relevant information for LLMs instead of giving LLMs the entire haystack…

But this raises an important question: Should you manually chunk your content, or is chunking already handled by the LLM’s retrieval system?

The answer is that you still should structure your content into clear, logical sections. While retrieval systems automatically chunk documents before passing them to an LLM, they can only work with the content they’re given.

So having well-structured content with clear headings, focused paragraphs, and distinct ideas, makes it easier for retrieval systems to identify accurate chunk boundaries and isolate the most relevant information.

And the result is, the chunks presented to the LLM are more likely to contain complete, self-contained answers, increasing the chances that your content is retrieved and used correctly.

But things like using good headers has always been part of what we do, right?

Absolutely, but that’s not the chunking process I’m proposing 😊

Introducing… the proper paragraph 🤷‍♀️

As I said in the title of this blog “Content Chunking Was Never Really Up for Debate” not only because that is actually what RAG systems already use when feeding information to LLMs, but because chunking happens to be how proper writing is!!!

Let me introduce you to….. 🥁🥁🥁

THE PROPER PARAGRAPH

According to the English language, there's such a thing as a 'proper paragraph if you’re writing formally, academically or professionally.

So unless your content is in conversational slang format like I’m writing right now - which we all know is not optimal for search - you should chunk your content using the proper paragraph method.

I can hear you in my mind saying “What do you mean Sara? what is that?”

A proper paragraph in English has the following structure:

--> Topic Sentence (The Opening): The first sentence that introduces the main point or controlling idea. It tells the reader exactly what to expect in this paragraph.

--> Supporting Sentences (The Middle): The body of the paragraph. These sentences provide explanations, facts, data, statistics, or concrete examples to back up your topic sentence

--> Transitions (The Connectors): Words or phrases (e.g., however, furthermore, for example, in contrast) used to link your sentences together smoothly.

--> Concluding Sentence (The Closing): The final sentence that summarizes the main idea, analyzes the evidence, or provides a logical transition into the next paragraph

writing your content in this format, makes each paragraph an individual entity that covers a specific point.

Here’s an example of this concept in action:

This has always been the proper way to write a paragraph. It’s not a new concept I’m introducing. And it’s very beneficial for users because just by reading the first sentence of a paragraph they can tell what the rest of the paragraph is about, making the content skimmable no matter how long it is!

Are we already using this format? I can argue that most of the content online is not written like that, nor do the average copywriters structure their content this way.

I tested the proper paragraph format using my simulator, and it did increase the relevance score.

Why? because it helps you group relevant information in one paragraph and have logically coherent content. So when RAG chunking comes in, it already has a solid foundation to work on.

Now I must be transparent, that I’m still testing more and more of this to provide solid numbers, but the concept still holds, this is how you should write your content. It is good writing which would naturally mean good chunking.

And That’s a Wrap (Almost 😄)

My argument is pretty straightforward, chunking is an established process that’s used by RAG systems to find and feed LLMs the most relevant information in a way an LLM can process.

But the more I read about chunking and what it’s trying to achieve, the more I found it similar to just proper English writing structure that has been established for a very long time.

I ran a few tests with my simulator tool, and early results showed improved relevance scores when using the proper paragraph format. So it’s a approach worth exploring.

Chunking is not just about the headers as some think and I hope by discussing the concept from both an LLM standpoint, and the English/user standpoint I showed you that, there’s really nothing to talk about… chunking your content is a fact of life 😄

That’s that for today folks and see you next newsletter!

Sources Worth Checking

Support the Riddler!

Sign up for my newsletter if you’re not already.
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Agencies Need to Rethink the SEO Project Manager

Sara Taher — Tue, 30 Jun 2026 12:39:46 GMT

Hello Riddlers!

Alternatively you can become a paid subscriber

Agency Life

“AI won’t replace project managers, because there’s nothing to replace” 😂

I saw this joke on twitter the other day and it inspired this post…!

Throughout my career, I’ve worked at several agencies and experienced different models of working with SEO project managers. They also go by other titles—SEO account managers, SEO coordinators, client success managers, and more.

In one setup, the project manager was the primary client contact. They managed timelines, assigned tasks, followed up on deliverables, removed blockers, and handled most client communication. The SEO specialists focused on strategy and execution.

Many of these project managers had little or no hands-on SEO experience. They were excellent project managers, but not necessarily SEO practitioners.

A large portion of the role involved coordination: chasing updates, writing meeting notes, updating project boards, preparing status reports, scheduling meetings, documenting action items, and making sure nothing slipped through the cracks.

In other words… SEO clerking.

But that was years ago… today a lot of things have changed, my perspective for one as well as AI tools becoming integral part of our work.

So buckle up, as in this blog I intend to share my 2 cents on the role of SPM in 2026!

SEO depth

I really think we need to reconsider how we select people for the SPM role. After being in this industry for over 12 years, I believe an SEO project manager should have some hands-on SEO experience.

Some of the they should have done before:

Build or contribute to building a strategy
performed technical audits, even simple ones
collaborated with developers to get things done
created content recommendations for SEO
worked directly with content writers to update/create new content
worked with external third party agencies on top of an internal SEO team

….. and more just to name a few!

I’m not undermining people with pure project management backgrounds. Traditional PM skills are still important.

But SEO is one of those fields where you can’t fully “coordinate” your way through complexity.

Without hands-on exposure, there’s a real risk of slowing things down, misinterpreting priorities, or becoming an unnecessary layer between execution and decision-making.

If you’re currently in an SEO project manager role without SEO experience, the fix is simple: start getting closer to the work.

Play around with the tools, learn, try things even minor things here and there….

You’re not trying to convert to a full stack SEO, you are trying to build enough fluency to not be a blocker, and ideally to be able to step in when needed.

AI leverage

Not all agencies even have a dedicated SPM role. In many cases, SEO managers inherit the coordination and “clerical” side of delivery. That works too.

So whether you have an SPM or not, the person handling delivery just got a massive unlock.

Because AI is already handling a big chunk of the administrative workload:

Meeting summaries
Turning conversations into structured tasks
Drafting client updates and follow-ups
Organizing project documentation
Rewriting or refining status reports
Converting notes into actionable briefs

And more….

A lot of this admin work was never the “value” part of the job to begin with, it was just necessary overhead.

If you’re an SPM or doing SPM as part of your role, here are two additional areas I recommend you explore:

Optimize the project plan. Once you have a strategy in place, use AI tools to stress test it. AI can help you take it even to the next level, make it better, suggest changes here and there in priorities to drive more value to your clients.
Agility management. Changes happen as you go in any project. SEO is far from being a static field. Instead of maintaining static documents that quickly become outdated, use AI to continuously reframe plans based on new inputs.

Not to mention how I think Claude code should be your best friend.

Familiarize yourself with the idea of “Claude skills” or reusable AI workflows. For example, I recently built a simple SEO Claude skill that turns a single issue name into a structured SEO user story.

For example, here’s the output using this skill for the issue “organization schema missing on the homepage”

Small input → structured output → faster execution. Right?

And That’s a Wrap (Almost 😄)

Basically what I’m trying to say is that, in 2026 the role/skills for SEO project management have changed.

and this blog was my attempt to change the joke “AI won’t replace project managers, because there’s nothing to replace” to a no-joke.

If you pickup some hands-on SEO experience, and upskill yourself with necessary AI tools for project management, AI cannot replace you because you are down to the bones… you’re down to the real value that AI cannot replace!

That’s that for today folks and see you in the next newsletter!

Sources Worth Checking

SEO User Story Claude Skill

Support the Riddler!

Sign up for my newsletter if you’re not already.
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.

Adapting to the Zero Click Era

Sara Taher — Tue, 23 Jun 2026 12:23:35 GMT

Hello Riddlers!

Alternatively you can become a paid subscriber

SEO has changed/is changing…. ya ya ya

You’ve probably heard it a thousand times by now: “SEO is changing”, “SEO is evolving”, and every other variation of the same message. And we’ve all been bombarded with GEO advice on how to optimize for the new world of LLMs.

Here’s the thing: LLMs still aren’t sending much traffic, so for now, we’re still focused on figuring out how to get the most out of Google.

But Google is now eating up most of the clicks… we’re in the zero click ERA 🤷‍♀️

“In the first four months of 2026, a whopping 68.01% of Google searches ended without a click,” according to a new study by Similarweb and SparkToro.

So where do the clicks go, what happens after the user searches?

There are 3 possible scenarios:

32% of the time people click to an organic result, to another Google property (Youtube, AI Mode, Maps, Images, etc…)
39% they do nothing after the search.
29% they do another search.

The study found that for every 1000 us Google searches, only 232 convert to clicks to a non-Google property.

That’s not good…

And so far every solution I see proposed to this dilemma misses the biggest opportunity out there…

We forgot that Google has different surfaces

I was listening to a webinar recently about “Google Discover” and it was during this webinar that I got my Aha moment.

Google Discover is a Google surface that shows users content based on their interests. Google also said they’re rolling the feature out on desktop last year—not sure where that went, but they said it’s coming.

Google discover is a huge traffic source, for a lot of website owners, it makes the biggest source of traffic, even more than traditional search. and that’s why we should consider optimizing for discover.

But here’s the thing…

the content that works in Google discover is slightly different than the content that works in traditional SERPs, although there’s an overlap of course.

So what type of content works well in Google Discover?

Content for Google Discover

Your best ideas are going to come from browsing your discover feed and seeing what made it there, however here are some pointers:

Sensational titles. But you need to avoid click bait, lean more on people’s natural curiosity vs been purely click-baity. An example is “New Zealand reaction to Mo Salah’s goal”.
Quoting people in titles. This is something that would not work in traditional SERPs, but for Google discover, having a title like “Mbappe on PSG’s season: “for me, a team that wins will always be an example of football”.
Questions in titles. Some have reported that this type of titles perform really well in discover. Worth testing out. An example is “why men keep dropping out of the workforce: it starts in childhood”
Timely fresh content. Yes you can still get visibility in discover with your evergreen content, but timely content has an edge. One thing you can do is to keep an eye on Google trends for keywords related to your niche, and once there’s something relevant to your audience, you can blog about it.
Images are very important for Google discover. Having the right featured image can improve your CTR there, this is something that is not relevant for traditional search results since you rarely see the featured image in search.
Localized content. Google discover surfaces content that is very relevant to the user and one of the factors that determine relevancy is location. If your customers are in a specific geographical area e.g Ontario, content like “Spring in Ontario. Uncover colors, rhythms and flavors that make this province a true cultural canvas” may work well.

Cheesy: understand your audience

Yes I know saying “understand your audience” is as lame as the Google advice “create helpful content” but it’s a step we rarely do in SEO.

Sometimes when I meet with business owners, I’m pleasantly surprised when they share things like:

“Our target audience are SEO agencies managing multiple clients who need to standardise workflows, speed up repetitive tasks, and maintain consistency without increasing headcount.”

Because then you can tailor interest-based content that would perform well in discover and on socials for your audience.

Topics you may write about for example:

Sensational: What actually breaks first when an SEO agency starts scaling
Quotes: “SEO manager at a 30-client agency: ‘Reporting is where we lose most of our time”
“Why do SEO agencies still do so much work manually in 2026?”
“Inside Canada’s SEO agencies: why reporting is still the biggest time drain”

You are not a ship anchored to the past

Our main focus has always been the 10 blue links, only today there’s rarely 10 blue links anymore… they’re usually less, with a SERPs designed to send less clicks to your website with Google’s AI experiences.

I’m not suggesting you abandon the 10 blue links, I just think you should widen your horizons a bit, and explore other surfaces available to you.

Creating content for Google Discover, can be the answer to your traffic challenges. And you probably have everything already in place:

Google technical foundations for discover highly are practically almost the same as the requirements for traditional serps.
User experience and bounce rate matters a lot for how your content would perform in discover, but these are things we should’ve always taken into account when creating content for SEO, right?
You probably have a social team in place, they can work with SEO to brainstorm content ideas that can work better for both social and discover!

The name of the game for SEO in 2026 is testing every thing, and this is one thing I think you should experiment with.

And That’s a Wrap (Almost 😄)

It’s time we tackle the zero-click reality with a different mindset and play the hand we’ve been dealt. The landscape has changed but there are still real opportunities for those who adapt to how distribution actually works today.

At the same time, we should keep an honest conversation going about the relationship between platforms and publishers.

Google continues to say “create helpful content,” and publishers have done exactly that, investing significant time, money, and effort. Yet the return in the form of clicks and owned audience is becoming harder to measure and less predictable.

As the web shifts toward more zero-click experiences, the question of fairness, attribution, and sustainability for publishers becomes harder to ignore.

We should continue pushing for a web where value flows both ways, not just one where content is consumed but not always returned to its source.

That’s that for today folks and see you in the next newsletter!

Sources Worth Checking

Google Discover Webinar with Barry Adams

Support the Riddler!

Sign up for my newsletter if you’re not already.
Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂
Provide feedback on how I can make this newsletter better!!!
Buy me coffee.
If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.