Three months ago my team needed to automate a code review pipeline — pull a PR, analyze it across security, performance, and readability dimensions, then generate a structured report. Classic multi-agent problem. I figured I’d pick a framework and ship it in a week.
Six weeks later, I’d rebuilt it three times across three different frameworks, burned through more API credits than I care to admit, and learned a lot about what these tools are actually good for versus what the marketing says they’re good for.
This is what I found.
What I Was Actually Building (And Why It Matters)
The pipeline had four agents: a Fetcher that pulled PR diffs from GitHub, a Security Reviewer, a Performance Reviewer, and a Summarizer that synthesized everything into a final report. Agents needed to coordinate — the reviewers ran in parallel when possible, but the Summarizer had to wait for both. Occasionally reviewers needed to ask follow-up questions, which meant some back-and-forth with a pseudo-user context.
Not a toy example. Not a “search the web and write a poem” demo. A real workflow with conditional logic, parallelism requirements, and structured output.
I ran each framework for roughly two weeks on this same task, using gpt-4o as the backbone (with claude-sonnet-4-6 for some comparisons). My setup: Python 3.12, running locally during development, targeting eventual deployment on a small AWS Lambda + SQS setup.
AutoGen: Brilliant Conversations, Painful Determinism
AutoGen (Microsoft, currently on v0.4.x as of early 2026) is built around the idea that agents talk to each other like coworkers in a Slack thread. You define agents, give them personas and tools, put them in a GroupChat, and let the conversation unfold.
Here’s a simplified version of how the reviewer agents looked:
import autogen
config_list = [{"model": "gpt-4o", "api_key": os.environ["OPENAI_API_KEY"]}]
security_agent = autogen.AssistantAgent(
name="SecurityReviewer",
system_message="""You are a security-focused code reviewer.
Analyze the provided diff for vulnerabilities: injection risks,
secrets exposure, auth issues. Return findings as JSON.""",
llm_config={"config_list": config_list},
)
perf_agent = autogen.AssistantAgent(
name="PerfReviewer",
system_message="""You review code for performance issues:
N+1 queries, unnecessary allocations, blocking I/O.
Return structured JSON findings.""",
llm_config={"config_list": config_list},
)
# GroupChat orchestrates the conversation
groupchat = autogen.GroupChat(
agents=[security_agent, perf_agent, summarizer, user_proxy],
messages=[],
max_round=12,
speaker_selection_method="auto", # LLM decides who speaks next
)
That speaker_selection_method="auto" line is where things get interesting — and not always in a good way.
AutoGen’s conversational model is genuinely impressive when the problem is open-ended. The agents reason about what needs to happen next, delegate naturally, and the GroupChat manager (which is itself an LLM call) decides who should speak. For exploratory tasks — “research this topic and synthesize findings” — it feels almost magical.
For my pipeline? It was a nightmare.
The problem: I needed the two reviewers to run, then the Summarizer to run. In AutoGen, enforcing that order reliably requires either a custom speaker_selection_method function (which takes real work to get right) or careful prompt engineering that breaks down the moment the LLM decides to do something “helpful.” I had the Security agent spontaneously offering to summarize on round 8 of a 12-round chat at least four times during testing.
I also ran into a gnarly bug where UserProxyAgent would sometimes inject a human input request mid-pipeline in ways I didn’t expect — even with human_input_mode="NEVER". Turns out there’s a known issue (tracked in the AutoGen repo, GitHub issue #3847 or thereabouts) around how human_input_mode interacts with nested chats introduced in the 0.4 refactor. Your mileage may vary depending on exactly which 0.4.x release you’re on.
The actual gotcha: AutoGen’s conversation history blows up in cost. Every agent in a GroupChat gets the full conversation history injected into their context. With 12 rounds and four agents, my token usage per PR review was about 4x what I’d estimated. For high-volume use this adds up fast.
Where AutoGen genuinely shines: research agents, anything where you want LLM-driven collaboration to handle ambiguity. If I were building a “help me debug this mysterious production issue” agent that might need to explore different hypotheses — AutoGen’s conversational model is the right fit.
LangGraph: The Framework for People Who Want Control
LangGraph (LangChain’s graph-based agent framework) treats agent workflows as state machines. You define nodes (functions or LLM calls), edges (transitions between nodes), and a shared state object that flows through the graph. It’s more code. It’s also much more predictable.
My pipeline in LangGraph looked like this:
from langgraph.graph import StateGraph, END
from typing import TypedDict, Annotated
import operator
class ReviewState(TypedDict):
pr_diff: str
security_findings: list[dict]
perf_findings: list[dict]
final_report: str
def fetch_pr(state: ReviewState) -> ReviewState:
# Pull diff from GitHub API
diff = github_client.get_diff(state["pr_url"])
return {"pr_diff": diff}
def run_security_review(state: ReviewState) -> ReviewState:
findings = security_chain.invoke({"diff": state["pr_diff"]})
return {"security_findings": findings}
def run_perf_review(state: ReviewState) -> ReviewState:
findings = perf_chain.invoke({"diff": state["pr_diff"]})
return {"perf_findings": findings}
def summarize(state: ReviewState) -> ReviewState:
report = summarizer_chain.invoke({
"security": state["security_findings"],
"perf": state["perf_findings"],
})
return {"final_report": report}
# Build the graph
workflow = StateGraph(ReviewState)
workflow.add_node("fetch", fetch_pr)
workflow.add_node("security_review", run_security_review)
workflow.add_node("perf_review", run_perf_review)
workflow.add_node("summarize", summarize)
# Parallel execution: both reviews happen after fetch
workflow.set_entry_point("fetch")
workflow.add_edge("fetch", "security_review")
workflow.add_edge("fetch", "perf_review")
workflow.add_edge("security_review", "summarize")
workflow.add_edge("perf_review", "summarize")
workflow.add_edge("summarize", END)
app = workflow.compile()
This is exactly the parallelism I wanted — both reviewers fire after the fetch, Summarizer waits for both. LangGraph handles the fan-out/fan-in natively.
What I loved: the ReviewState TypedDict made debugging way easier. When something broke, I could inspect exactly what state each node received and returned. No mysterious conversation history to unpick. Conditional edges (for cases where security findings trigger a deeper scan) are first-class, not hacked in via prompt tricks.
The tradeoff is verbosity. Setting this up took significantly more boilerplate than AutoGen. And if you’re not already in the LangChain ecosystem, you’re pulling in a big dependency tree. I had a version conflict with langchain-core and langchain-openai that took the better part of an afternoon to resolve — they move fast and breaking changes are common between minor versions.
One thing I noticed: LangGraph’s persistence layer (using SqliteSaver or PostgresSaver) is excellent for production workflows where you need resumability. A PR review that fails midway can pick up from the last checkpoint. That’s not something you get easily in the other frameworks without building it yourself.
I’m not 100% sure LangGraph scales gracefully to very large graphs (50+ nodes) — I’ve heard reports of the compilation step getting slow, but I haven’t hit that personally.
CrewAI: Fast to Start, Rigid to Extend
CrewAI has a clever mental model: you define Agents with roles, backstories, and goals, then group them into a Crew that tackles a set of Tasks. It reads almost like a job description doc. The first time I got a working pipeline, it took maybe 45 minutes including reading the docs.
from crewai import Agent, Task, Crew, Process
security_agent = Agent(
role="Security Reviewer",
goal="Identify security vulnerabilities in code changes",
backstory="Senior appsec engineer with 10 years in vulnerability research",
verbose=True,
allow_delegation=False, # important: keep agents focused
)
perf_agent = Agent(
role="Performance Reviewer",
goal="Flag performance regressions and inefficiencies",
backstory="Backend engineer obsessed with latency and resource usage",
verbose=True,
allow_delegation=False,
)
security_task = Task(
description="Review this PR diff for security issues: {diff}",
expected_output="JSON list of security findings with severity ratings",
agent=security_agent,
)
perf_task = Task(
description="Review this PR diff for performance issues: {diff}",
expected_output="JSON list of performance findings",
agent=perf_agent,
)
crew = Crew(
agents=[security_agent, perf_agent, summarizer_agent],
tasks=[security_task, perf_task, summary_task],
process=Process.sequential, # or hierarchical
verbose=True,
)
The role-and-backstory approach actually works better than I expected for keeping agents on-task. Giving the security agent a persona (“senior appsec engineer”) genuinely reduced hallucination rates compared to a bare system prompt in my testing — though I’d want more data before making a strong claim there.
The frustration: CrewAI’s parallel execution is clunky. Process.sequential runs tasks in order. Process.hierarchical adds a manager agent that delegates, which adds latency and cost. True fan-out parallelism (my security + perf reviewers running simultaneously) requires workarounds that feel like going against the grain of the framework.
Honestly, CrewAI disappointed me on the customization front. When I needed the security agent to conditionally trigger a deeper tool call based on its initial findings, I ended up fighting the framework. CrewAI is opinionated in ways that work great for straightforward pipelines and start to hurt when your workflow has nuance.
The debugging experience is also the weakest of the three. verbose=True dumps a lot of output, but it’s hard to trace exactly what prompt each agent received or what state the crew is in at a given point.
What I’d Actually Ship
Here’s where I land after six weeks of this:
For production workflows with defined structure: LangGraph. It’s the most work upfront, but it’s the one I trust. The explicit state, the graph topology, the persistence layer — these are things that matter when you’re shipping something real that needs to be debugged at 2am. My code review pipeline runs on LangGraph in production now. It’s been stable.
For exploratory or research-style agents: AutoGen. If I were building an internal tool where an agent needs to explore a problem space, AutoGen’s conversational model fits that better than a rigid graph. Just watch your token spend and test your speaker selection logic thoroughly before trusting it.
For prototyping or small teams that want something working fast: CrewAI. If your workflow is three to five sequential tasks with clear handoffs, CrewAI gets you there with minimal friction. For anything more complex, you’ll hit walls.
The honest answer is that LangGraph wins for serious production use right now — but it’s the most demanding to work with. AutoGen and CrewAI both have stronger “quick start” experiences, and that matters for teams evaluating whether agent workflows are even worth the investment.
My recommendation: build your first version in CrewAI to validate the concept, then migrate to LangGraph when you need reliability and control. It’s not exciting advice, but it’s what I’d actually tell a teammate.