Why Content Chunking Was Never Really Up for Debate

Meh...

Hello Riddlers!

📌 If you find value in my newsletter and want to support the work that goes into it consider —> [☕Buy Me a Coffee☕]💡 Want more from me? [SEO Strategy Course] | [Build Your Own SEO Tools (with Python & Agentic AI)] | [Premium SEO Mastermind]

Alternatively you can become a paid subscriber

Intro

You’ve probably seen the term content chunking circulating across SEO social media, with some people recommending it as a tactic to improve AI visibility while others dismiss it as unnecessary.

And, of course, Google had to weigh in. On multiple occasions, they have made it clear that you don’t need to chunk your content for AI. They even added this in their official “Optimizing your website for generative AI features on Google Search” guide.

But… there are a couple of issues at play here.

First, a lot of SEOs are still relatively new to LLMs and AI—myself included—and I don’t think we always acknowledge that. If we did, we’d probably be a little more cautious about immediately rejecting new ideas.

Second, Google’s guidance is, to some extent, self-serving. Google doesn’t want site owners restructuring their content because of every new trend that popsup through the industry. From Google’s perspective, that’s a reasonable position to take. They said verbatim “We don’t want people to have to be crafting anything for Search specifically.” So their concern or objection is not the concept of chunking itself; rather, it’s trying to change your content specifically for search and visibility with disregard to users.

Chunking is also misunderstood in my opinion.

Of course it is, as I said we’re all new to this.

So when people try to interpret chunking into actionable approach, they may get it wrong, but then the nay-sayers would latch to those misinterpretations and label the entire concept as flawed.

So…

What is chunking, actually? And why was it never up for debate to begin with?

Buckle up!

What is chunking?

Chunking is the process of breaking a large text document into smaller, coherent sections called chunks. Each chunk represents a complete idea or segment of the text while preserving the original meaning and context.

Chunking is essential for large language models because they cannot process an unlimited amount of text all at once.

Makes sense right?

LLMs need to go through vast amounts of unstructured texts, and by dividing the content into smaller chunks in a size that LLMs can process (this size is called the context window) LLMs can process text more efficiently and therefore search for and retrieve information better.

Without chunking, LLMs are searching for a needle in a haystack.

If you ask an LLM, “What is an atom?”, it does not need to process every chemistry and physics resource ever created. Instead, a retrieval system can identify and provide the LLM with the specific sections of content that are most relevant to the question, such as passages explaining the definition and properties of atoms. This approach improves efficiency by reducing the amount of information the model needs to process, while also improving accuracy by allowing it to focus on the most relevant context.

This isn't just theoretical. Here's a simple example showing how splitting text into coherent paragraphs changes retrieval relevance.

Totally copying iPullRank here, I created this chunking simulator to show the relevance difference of keywords and text, before and after splitting the text into 2 paragraphs:

When all the text was one paragraph the relevance scores were 45.3% and 55.3%:

When the text was split into 2 paragraphs the relevance scores went up to 58.4% and 68.6% and that’s just by splitting the text into 2 proper paragraphs:

I’m still working on this tool and will probably launch it soon on my website for you to play around, stay tuned!

So….

We’ve established that chunking content is a fact of life used by RAG systems to feed relevant information for LLMs instead of giving LLMs the entire haystack…

But this raises an important question: Should you manually chunk your content, or is chunking already handled by the LLM’s retrieval system?

The answer is that you still should structure your content into clear, logical sections. While retrieval systems automatically chunk documents before passing them to an LLM, they can only work with the content they’re given.

So having well-structured content with clear headings, focused paragraphs, and distinct ideas, makes it easier for retrieval systems to identify accurate chunk boundaries and isolate the most relevant information.

And the result is, the chunks presented to the LLM are more likely to contain complete, self-contained answers, increasing the chances that your content is retrieved and used correctly.

But things like using good headers has always been part of what we do, right?

Absolutely, but that’s not the chunking process I’m proposing 😊

Introducing… the proper paragraph 🤷‍♀️

As I said in the title of this blog “Content Chunking Was Never Really Up for Debate” not only because that is actually what RAG systems already use when feeding information to LLMs, but because chunking happens to be how proper writing is!!!

Let me introduce you to….. 🥁🥁🥁

THE PROPER PARAGRAPH

According to the English language, there's such a thing as a 'proper paragraph if you’re writing formally, academically or professionally.

So unless your content is in conversational slang format like I’m writing right now - which we all know is not optimal for search - you should chunk your content using the proper paragraph method.

I can hear you in my mind saying “What do you mean Sara? what is that?”

A proper paragraph in English has the following structure:

--> Topic Sentence (The Opening): The first sentence that introduces the main point or controlling idea. It tells the reader exactly what to expect in this paragraph.

--> Supporting Sentences (The Middle): The body of the paragraph. These sentences provide explanations, facts, data, statistics, or concrete examples to back up your topic sentence

--> Transitions (The Connectors): Words or phrases (e.g., however, furthermore, for example, in contrast) used to link your sentences together smoothly.

--> Concluding Sentence (The Closing): The final sentence that summarizes the main idea, analyzes the evidence, or provides a logical transition into the next paragraph

writing your content in this format, makes each paragraph an individual entity that covers a specific point.

Here’s an example of this concept in action:

This has always been the proper way to write a paragraph. It’s not a new concept I’m introducing. And it’s very beneficial for users because just by reading the first sentence of a paragraph they can tell what the rest of the paragraph is about, making the content skimmable no matter how long it is!

Are we already using this format? I can argue that most of the content online is not written like that, nor do the average copywriters structure their content this way.

I tested the proper paragraph format using my simulator, and it did increase the relevance score.

Why? because it helps you group relevant information in one paragraph and have logically coherent content. So when RAG chunking comes in, it already has a solid foundation to work on.

Now I must be transparent, that I’m still testing more and more of this to provide solid numbers, but the concept still holds, this is how you should write your content. It is good writing which would naturally mean good chunking.

And That’s a Wrap (Almost 😄)

My argument is pretty straightforward, chunking is an established process that’s used by RAG systems to find and feed LLMs the most relevant information in a way an LLM can process.

But the more I read about chunking and what it’s trying to achieve, the more I found it similar to just proper English writing structure that has been established for a very long time.

I ran a few tests with my simulator tool, and early results showed improved relevance scores when using the proper paragraph format. So it’s a approach worth exploring.

Chunking is not just about the headers as some think and I hope by discussing the concept from both an LLM standpoint, and the English/user standpoint I showed you that, there’s really nothing to talk about… chunking your content is a fact of life 😄

That’s that for today folks and see you next newsletter!

Sources Worth Checking

• Moving from a Google-shaped Web to an Agent-shaped Web: A Refutation of Misinformation about Chunking

• Semchunk: a semantic chunking technique on github

Support the Riddler!

• Sign up for my newsletter if you’re not already.

• Share the newsletter and invite your friends to signup. Help me reach 2k signups by end of 2025 please 🙂

• Provide feedback on how I can make this newsletter better!!!

• Buy me coffee.

• If you’re an SEO tool or an SEO service provider, consider sponsoring my newsletter. I’m also open to other partnership ideas as well.

Disclaimer: LLMs were used to assist in wording and phrasing this blog.