Agent Reliability Playbook

Agent Reliability Playbook

The 7-layer reliability stack for shipping AI agents that work in production.

Built and battle-tested. 7 skills covering orchestration, tool design, state management, error recovery, observability, stopping conditions, and the catalog of 15 anti-patterns. Adapters for pi-agent, Claude Code, Cursor, Cline, and any tool that reads a system-prompt file.

What's Inside

Quick Start

# 1. Extract
tar -xzf agent-reliability-playbook-0.1.0.tgz
cd agent-reliability-playbook-0.1.0

# 2. Pick your agent host
ls adapters/

# pi-agent (recommended)
mkdir -p ~/.pi/agent/skills
cp -r skills/* ~/.pi/agent/skills/
# /reload in pi to pick them up

# Claude Code
mkdir -p .claude/commands
cp -r adapters/claude-code/commands/* .claude/commands/
cat adapters/claude-code/CLAUDE.md >> CLAUDE.md

# Cursor / Cline / Aider
# See INSTALL.md for per-tool setup

The Core Idea

The single sentence that ties this whole pack together:

A reliable agent has 7 layers. Skip one and it breaks in production.

The Pre-Launch Checklist

Before shipping any agent to real users, walk this list. Bold items are non-negotiable.

• Orchestration chosen deliberately — workflow vs. agent vs. hybrid
• Tool definitions have descriptions the model can read — what/when/when-not/examples
• Tool calls validated before execution — schema, permissions, rate limits
• State management strategy defined — in-context, working, persistent
• Error recovery policy for every tool call — retry, fallback, escalate
• Observability is logging — LLM calls, tool calls, state changes
• Stopping conditions in code — MAX_ITERATIONS, MAX_COST_USD, MAX_DURATION_MS
• Eval set covers happy path + 3 adversarial cases
• Cost estimate per session — token cost × expected sessions
• Latency budget per turn — p50 and p95 targets
• Rollback plan — what to do if the agent misbehaves in prod
• Sample of 5 prod transcripts reviewed end-to-end

The Architectures

Workflow = orchestrated via code paths. The LLM is called from specific places. The control flow is yours.

• ✅ Predictable, testable, deterministic, cheaper
• ❌ Rigid — can't handle novel inputs
• Use for: customer support routing, structured data extraction, multi-step forms

Agent = LLM-driven process. The model decides what to do next.

• ✅ Flexible, natural for tool-using tasks
• ❌ Unpredictable, expensive, hard to debug
• Use for: research tasks, open-ended exploration, non-deterministic steps

Hybrid = workflow at the outer layer, agent at the inner layer.

• ✅ Best of both
• ❌ More design work upfront
• Default for production

Cost Discipline (Real Numbers)

A loop with no cost cap on a max-thinking model = $12+/session. Cost caps are not optional. They live in code.

Adapters Included

Works in every major AI coding tool:

Updates

Free for life. Re-download for new anti-patterns, new vendor patterns, new LLM tier cost data.

What This Isn't

• Not a framework — works with LangChain, LangGraph, CrewAI, raw OpenAI SDK, anything
• Not prompt-only — the patterns live in code, not in the system prompt
• Not academic — every pattern comes from a real production failure

What's in the download

agent-reliability-playbook-0.1.0/
├── README.md              ← setup guide
├── OVERVIEW.md            ← positioning
├── METHODOLOGY.md         ← the longer write-up
├── INSTALL.md             ← per-tool setup walkthrough
├── LICENSE.md
├── package.json
├── skills/                ← 7 SKILL.md files (one per layer)
│   ├── agent-reliability/
│   ├── tool-design/
│   ├── agent-state/
│   ├── error-recovery/
│   ├── agent-observability/
│   ├── stopping-conditions/
│   └── anti-patterns/
├── adapters/              ← drop-in for AI coders
│   ├── pi-agent/
│   ├── claude-code/
│   ├── cursor/
│   ├── cline/
│   └── generic/
├── examples/              ← 2 real worked examples (tool rewrite, runaway loop)
└── docs/                  ← diagrams + decision trees