Concepts behind reliable AI automation
Short notes on the ideas I build on: agents and workflows, context and retrieval, and the operations discipline that keeps automation from failing silently. Links between notes show which concepts depend on or differ from each other.
Drag to explore. Hover to trace a concept's neighborhood. Click a node to open it.
/
Build the first reliable agent. What a workflow is, when a loop earns its keep, and how to bound it.
Choose what enters the model's context, preserve its source, and test whether retrieval returned the right material.
Record every run, test outputs against expected results, and send failures to a separate error path.
Graph engineering models the work of building software, not the data the software uses: what must be true of the product, written as testable statements linked by dependency. Build only what is unblocked; count a result as progress only when its test passes and the outcome is recorded.
Agents Agent Anatomy Every agent is three parts: instructions, a model, and tools. Everything else is packaging. Agents Agent Hosts The place an agent runs: a session you watch, a gateway that listens, or a job runtime that survives crashes. Agents Agent Loop An LLM in a loop where the model itself decides when the loop stops. The runtime around it is plain code. Agents Agent Wiring Two ways to build an agent: a definition file a host runs, or code written against a framework. The cut is who owns the loop. Agents Augmented LLM A single model call extended with tools, retrieval, and memory. The atomic unit every agentic system is built from. Agents Autonomy Ladder Five rungs between manual work and an autonomous agent. Pick the lowest rung that does the job. Agents Compounding Error The reliability tax of chaining. High per-step success still collapses across a long chain. Agents Guardrails Policy checks at the points where the loop touches the outside world: input, tool call, output. Agents Loop Freedom How much of the route the model chooses: from a path you designed to a goal it pursues its own way. Agents Stop Condition A loop ends one of two ways: done is reached, or a stop condition fires. Agents Sub-agent A child agent with its own fresh window that does the work and returns only the conclusion. Agents Task Decomposition Cutting one large task into steps whose output you can check on its own. Agents Tool Calling The model asking for a function to run, and your code deciding whether to run it. Agents Workflow A system where the developer fixes the steps before the model runs. Predictable, and usually cheaper than an agent. Agents · comparative Workflow vs Agent Both use LLMs. The difference is who owns the control flow. Context & Retrieval Chunking Splitting source text into the pieces that get stored and retrieved as single units. Context & Retrieval Context Engineering Curating and maintaining the right set of tokens in the window during a task, not just wording one prompt. Context & Retrieval Context Rot The decay of a long session: constraints forgotten, work redone, a task declared done that is not. Context & Retrieval Context Window The bounded slab of text a model reads on each call. Everything outside it is invisible. Context & Retrieval Description as Trigger The model decides what to use from descriptions alone, so the description does the selecting. Context & Retrieval Embeddings Text turned into a list of numbers positioned by meaning, so similar text lands close together. Context & Retrieval Hallucination A model stating something false with full confidence. Context & Retrieval LLM Wiki A knowledge base the model itself maintains: interlinked pages it revises as sources arrive, not an index it searches. Context & Retrieval Lost in the Middle Models recall the start and end of a long prompt reliably, and the middle worst. Position is part of the prompt. Context & Retrieval Progressive Disclosure Capability loads in tiers: a permanent trigger, full instructions on match, references on demand. Context & Retrieval Prompt Structure Assembling a prompt from named parts (identity, task, context, constraints, output) so failures are diagnosable. Context & Retrieval RAG Fetching relevant context before generation. Vector search is the common way, not the only one. Context & Retrieval Reranking A second, slower scoring pass that re-orders the retriever's candidates by actual relevance before the model sees them. Context & Retrieval Retrieval Choosing, out of everything stored, which pieces go in front of the model this turn. Context & Retrieval · comparative LLM Wiki vs RAG Both feed outside knowledge to the model. RAG searches raw sources at query time; a wiki digests them at ingest and gets navigated. Context & Retrieval · comparative Prompt vs Context Engineering Prompt engineering shapes one message. Context engineering curates the whole window across a session. Context & Retrieval · comparative RAG vs Context Window Both get information to the model, but a bigger window does not replace retrieval. Production Ops Durable Execution A runtime that records a job's progress at each point it waits, so a crash resumes and a long pause survives. Production Ops Error Workflow A separate path that catches failures, logs them with context, and alerts a human. Production Ops Evals A repeatable test suite that scores output against expected results, so quality is measured, not felt. Production Ops Human in the Loop Explicit approval gates that keep a human as the closing edge on high-stakes actions. Production Ops Idempotency Designing an action so running it twice has the same effect as running it once. Production Ops LLM-as-Judge A separate model call that scores an output against a written rubric, where no exact answer exists to compare against. Production Ops Model Routing Choosing a model per task by cost per successful outcome, not by price per token. Production Ops Observability Traces, tool calls, and inputs captured per run, so behaviour is visible after the fact. Production Ops Prompt Injection An attack where text the model reads carries instructions, and the model follows them. Production Ops Structured Output Constraining the model's answer to a declared schema, so the next step can parse it instead of guessing. Graph Engineering Claim Graph The durable record of what must be true about the product being built, written as testable statements with dependencies, each marked true only when its test passes. Graph Engineering Evidence Reconciliation The step that turns a finished result into progress: check it against the claim's test, then write the outcome into every record that owns a piece of it. Graph Engineering Graph Climbing The loop that works a claim graph to done: find the claims that are unblocked, build toward one, try to break the result, record what survives, repeat. Graph Engineering Graph Engineering Shaping the work of building software as a dependency graph, where nodes are small self-contained jobs and an edge exists only when one job's output feeds another. Graph Engineering Work Graph The optional record of who is building what, which gate is still shut, and what is owed. Also called an execution graph or ledger.