✂️ Chunking Techniques

A practical guide to 6 document chunking strategies for RAG and LLM applications — with working code and plain-English explanations.

Chunking · Document Chunking · RAG Chunking · Text Splitting · Semantic Chunking · LlamaIndex · LLM Chunking

---

Why chunking matters

Before any RAG system can retrieve relevant context, your documents have to be split into chunks. The way you chunk directly affects what gets retrieved — and therefore what the LLM answers with.

Too large: chunks carry noise, LLM gets confused. Too small: chunks lose context, answers feel disconnected. Wrong boundaries: semantic meaning gets cut mid-thought.

This repo covers 6 strategies from the simplest (no splitting at all) to the most intelligent (LLM decides the boundaries).

Related: If you want to see what happens after chunking — retrieval strategies — check out Advanced RAG Cookbook.

---

Techniques

#	Technique	API Key	Speed	Best For
01	Document Chunking	No	Fastest	Baseline, very short docs
02	Fixed-Size Chunking	No	Very Fast	Uniform text, quick baseline
03	Recursive Chunking	No	Fast	Most prose documents — default choice
04	Sentence Chunking	No	Fast	Fact-dense docs, high-precision retrieval
05	Semantic Chunking	Yes	Medium	Multi-topic documents, coherent chunks
06	Agentic Chunking	Yes	Slowest	Complex docs, logic-preserving splits

---

Quick Decision Guide

Default for most documents → Recursive Chunking — sentence-aware, fast, no API needed.

Need exact sentence-level retrieval → Sentence Chunking — 100-token fine-grained chunks.

Document has multiple distinct topics → Semantic Chunking — cuts where meaning changes.

Complex document structure (legal, research, specs) → Agentic Chunking — LLM understands what belongs together.

Just want to understand the baseline → Document Chunking → then Fixed-Size.

---

Setup

1. Clone

git clone https://github.com/Jeevav62/chunking-techniques.git
cd chunking-techniques

2. Install

pip install -r requirements.txt

3. API key (only for techniques 05 and 06)

cp .env.example .env
# Add your OPENAI_API_KEY to .env

---

Running a Technique

Each techniques/XX/ folder has a standalone script. Update SAMPLE_FILE at the top to point to your document, then:

cd techniques/03_recursive_chunking
python recursive_chunking.py

Techniques 01–04 need no API key. Techniques 05–06 need OPENAI_API_KEY.

---

Utility: Token Counter

Before choosing a chunk size, count your document's tokens:

cd utils
python token_counter.py

Tells you token count, word count, and estimated chunks at 500 and 100 token sizes. Useful for setting chunk_size before running any technique.

---

Chunking Comparison

All techniques run on the same sample document (data/sample.txt — an emergency evacuation policy, ~400 tokens).

Technique	Chunks Produced	Needs API	Boundary Type
Document	1	No	None
Fixed-Size	~2	No	Token count
Recursive	~2-3	No	Sentence boundary
Sentence	~8-12	No	Sentence boundary (fine)
Semantic	~4-6	Yes	Meaning shift
Agentic	~3-5	Yes	LLM judgment

---

Stack

Component	Library
RAG Framework	LlamaIndex 0.10+
Embeddings	OpenAI text-embedding-3-small
LLM	OpenAI GPT-4o-mini
Token counting	tiktoken

---

Repo Structure

chunking-techniques/
├── data/
│   └── sample.txt              # Sample document to run scripts on
├── utils/
│   └── token_counter.py        # Count tokens before choosing chunk size
├── requirements.txt
├── .env.example
└── techniques/
    ├── 01_document_chunking/
    ├── 02_fixed_size_chunking/
    ├── 03_recursive_chunking/
    ├── 04_sentence_chunking/
    ├── 05_semantic_chunking/
    └── 06_agentic_chunking/

---

Contributing

Want to add a technique (sliding window, hierarchical, markdown-aware)? PRs welcome.

1. Add folder techniques/07_your_technique/
2. Include README.md + standalone script
3. Open a PR

---

License

MIT — use it, share it, build on it.

---

If this helped you understand chunking, give it a ⭐

See also: Advanced RAG Cookbook — what to do with chunks once you have them.