When we started building AIssistant we faced the classic decision: build the AI agent's orchestration on an off-the-shelf framework like LangGraph, or write our own? We chose our own state machine. Here's why, and what we learned along the way.

What orchestration actually solves for an AI agent

An AI agent is not "one LLM call." Every inbound message in our system passes through a pipeline: input sanitization → classification → agent → output validation → deterministic action execution → audit log. Orchestration decides how those steps compose: what happens when a step fails, when to retry, when to time out, when to escalate to a human, and how the whole pass gets recorded.

Add real-world traffic on top: concurrent messages from the same contact, idempotency (a telephony webhook can arrive twice), and the requirement that an in-flight conversation must survive an application restart. That's an orchestration problem, not an AI problem.

What LangGraph offers

LangGraph models an agent as a graph of nodes with shared state. You get checkpointing (state persisted between steps), streaming, human-in-the-loop interrupts and graph visualization. For rapidly prototyping multi-agent flows it's a great tool, and the ecosystem around LangChain is huge.

We're not disputing any of that. Our problem was elsewhere.

Why we chose custom orchestration

  • Security layers must be out of the framework's reach. Our architecture rests on the LLM generating only intents while deterministic code validates and executes. We didn't want the boundary between the "LLM world" and the "deterministic world" to run through third-party abstractions that shift with every major version.
  • Our graph is small and stable. The pipeline has a handful of well-defined states and transitions. An explicit enum + switch is enough. A framework pays off when the graph is large, dynamic, or changes often. Ours isn't.
  • Upgrade churn. The LLM framework ecosystem was changing week to week while we were building. Every upgrade means re-auditing a security-critical path. With our own code we only audit our own changes.
  • Debugging and observability. A stack trace from our own state machine leads straight into our code. Debugging a production incident through several layers of framework abstraction is significantly more expensive.
  • Testability. The transition function is a pure function: (state, event) → new state + action. Unit tests without framework mocks, including error paths, which is exactly what security review demands from us.

What our state machine looks like

Nothing exotic, and that is the point:

  • An explicit enumeration of conversation states and allowed transitions; an invalid transition is an error that gets logged and alerted.
  • Every step is idempotent: redelivering the same event never causes a second action.
  • State is persisted after every transition; the process can crash at any point and resume.
  • Timeouts and retry limits are data (per-step configuration), not constants scattered around.
  • Every transition writes to the audit log: input hash, state before/after, latency, outcome.

When we would recommend LangGraph

Honestly: our choice is not a universal rule. LangGraph (or a similar framework) makes sense when you're prototyping and need results within a week, when your flow genuinely is a dynamic multi-agent graph, or when you don't have the capacity to maintain your own orchestration long-term. A framework also gives you for free the things that are tedious to write yourself, streaming and checkpointing above all.

Lessons learned

  • Decide along security boundaries, not hype. The question wasn't "is LangGraph good?" but "where will the boundary between probabilistic and deterministic code run?".
  • A small number of states is a feature. If your agent needs dozens of states, simplify the product, not the orchestration.
  • Custom code isn't free. We pay in maintenance. It's only worth it because the logic is small and critical.

A year from now the answer may differ, because frameworks mature. But the principles stay: security invariants belong in code you fully control.