# RAG Engineering Playbook

> [View this product on lib-pro](https://lib-pro-3f8437.gitlab.io/libs/rag-engineering) · Stop building naive RAG. Start building RAG that doesn't hallucinate.

**Outcome:** Ship a RAG pipeline that doesn't hallucinate.

5 skills for shipping RAG pipelines that work in production: the 5-stage RAG pipeline (parse → chunk → embed → retrieve → generate), chunking strategies, retrieval patterns (vector + hybrid + rerankers), evaluation metrics, and advanced patterns (agentic, graph, multimodal). Methodology, not framework — works with LangChain, LlamaIndex, Haystack, or custom code.

**Price:** £35 — lifetime · **Category:** Production AI Skills · **Tech:** pi-agent, Claude Code, Cursor, Cline, Markdown, Vector DBs (Pinecone, Weaviate, Qdrant, pgvector — vendor-agnostic)

| Metric | Value |
|--------|-------|
| Skills | 5 |
| Stages covered | 5 |
| Patterns documented | 15+ |

## Quick start

```bash
tar -xzf rag-engineering-0.1.0.tgz && cd rag-engineering-0.1.0
```

- # Pick your agent host
- ls adapters/
- # pi-agent: cp -r skills/* ~/.pi/agent/skills/

## Code preview

```
# rag-engineering — The RAG Decision Tree

Naive RAG fails in production. The fix is not "better embeddings" — it's a deliberate architecture.

## When to use RAG

- Your data is too large for the context window → RAG
- Your data changes frequently → RAG (vs fine-tuning)
- You need citations in answers → RAG
- Your data is highly structured → maybe graph RAG

When NOT to use RAG:
- The answer fits in the prompt → just put it in the prompt
- You can fit the data in context → just use long-context
- You need the model to *learn* patterns → fine-tuning

## The 5-Stage Pipeline

1. **Parse** — extract text from PDFs, docs, code, tables
2. **Chunk** — split into retrieval units (multiple strategies)
3. **Embed** — turn chunks into vectors
4. **Retrieve** — find relevant chunks (vector + hybrid + rerankers)
5. **Generate** — prompt the LLM with retrieved context

Each stage has 2-3 patterns. The architecture is your choice of patterns. The pack teaches the tradeoffs.
```

## Use cases

### 1. Your first RAG pipeline

You want to ground an LLM in your company's docs. You picked a vector DB, embedded naively, and the answers are 'okay-ish' but full of hallucinations.

**Outcome:** Walk the 5-stage pipeline, pick the right patterns for your data, ship a RAG that doesn't hallucinate.

### 2. Your RAG is good, not great

Your RAG works for some queries. Others miss. You don't know which patterns to change. Tuning is hit-or-miss.

**Outcome:** Add a golden set. Measure retrieval quality. Find the weak stage. Apply the right pattern. Iteratively improve.

### 3. Naive RAG isn't enough

Single-step retrieval misses multi-hop questions. Your users need answers that combine 3+ documents. Time for advanced patterns.

**Outcome:** Move to agentic RAG (multi-step retrieval) or graph RAG (knowledge graph). The advanced-rag skill teaches when each is worth the complexity.

## Built for

- AI engineers building their first RAG pipeline
- AI builders with naive RAG that hallucinates
- Senior engineers evaluating vector DBs and chunking strategies
- AI product teams who need to defend retrieval quality with data

## Highlights

### 5-stage pipeline, 2-3 patterns per stage

Parse → chunk → embed → retrieve → generate. Each stage has patterns with explicit tradeoffs. The architecture is your choice. The pack teaches the choices.

### Retrieval that actually retrieves

Vector alone misses. BM25 alone misses. Hybrid (vector + BM25) with a reranker catches 40% more relevant chunks. The pack teaches the patterns.

### Measure retrieval quality

NDCG, MRR, recall@k. Without metrics, you're guessing. The rag-evaluation skill teaches you to measure, then improve.

## What's in the box

- `5 skills` — decision tree + 4 stage skills
- `5 tool adapters` — pi-agent, Claude Code, Cursor, Cline, generic
- `2 worked examples` — chunking comparison + reranker eval
- `Pattern selection` — tradeoffs for each stage
- `Vendor-agnostic` — works with any stack

## Adapters included

These skills were authored for pi-agent, but we include drop-in adapters for the major AI coding tools.

| Tool | Adapter location | What you get |
|------|------------------|--------------|
| pi-agent | `adapters/pi-agent/` | SKILL.md drop-in |
| Claude Code | `adapters/claude-code/` | .claude/commands/*.md + CLAUDE.md |
| Cursor | `adapters/cursor/` | .cursorrules rule sets |
| Cline / Continue | `adapters/cline/` | System-prompt rules |
| Aider / generic | `adapters/generic/` | INSTRUCTIONS.md snippets |

**Same playbook, different packaging.** The adapters translate the methodology into the format each tool expects. Install once, swap agents without re-learning the method.

## Features

- rag-engineering skill — master orchestrator + RAG decision tree
- chunking-strategies skill — fixed-size, semantic, hierarchical, late-chunking, table-aware
- retrieval-patterns skill — vector, BM25, hybrid, rerankers, multi-query, HyDE, query rewriting
- rag-evaluation skill — retrieval metrics (NDCG, MRR, recall@k) + end-to-end metrics + golden sets
- advanced-rag skill — agentic RAG, graph RAG, multimodal RAG, when to escalate
- 5 tool adapters — pi-agent, Claude Code, Cursor, Cline, generic
- 2 worked examples — chunking comparison + reranker evaluation

## FAQ

### Does this work with my vector DB?

Yes. The pack is methodology, not infrastructure. It teaches you to pick the right chunking, embedding, and retrieval patterns. Your vector DB (Pinecone, Weaviate, Qdrant, pgvector, Chroma) doesn't change the patterns.

### How is this different from LangChain / LlamaIndex docs?

Framework docs tell you *what* the framework does. This pack tells you *which pattern to pick and why*. The methodology is portable; the framework isn't.

### I have naive RAG. Will this fix it?

Probably. The first step is rag-evaluation — build a golden set, measure retrieval quality. Then apply the right pattern (often hybrid search + reranker). Most naive RAGs improve 30-50% with one or two pattern swaps.