A Multi-Agent Data Analysis System: Why We Chose DSPy

We built a system that lets operations staff query, analyze and attribute data in natural language. We went back and forth between LangGraph and DSPy at the selection stage, and chose the latter.

The deciding factor: evaluation

Multi-agent systems have a deeply misleading property: they get worse slowly, and nobody knows when it happened.

You tweak a prompt, the model gets a version upgrade, a tool changes its return format — any of these can silently degrade some query you never test. By the time users complain, two weeks have usually passed.

What DSPy actually buys you is not orchestration. It turns the optimization target into a measurable metric:

import dspy

class Analyze(dspy.Module):
    def forward(self, question: str, schema: str) -> dspy.Prediction:
        plan = self.planner(question=question, schema=schema)
        query = self.query_writer(plan=plan.reasoning, schema=schema)
        return self.executor(sql=query.sql)

# With a real set of historical questions, you can run a regression.
metric = dspy.ExampleBasedMetric(
    examples=[
        dspy.Example(
            question="Which links drove packet loss during Friday evening peak?",
            schema="network_metrics",
            reasoning="Filter high-loss links first, then join traffic to check congestion",
            sql="SELECT ...",
        ).with_inputs("question", "schema")
    ]
)

Orchestration turns out to be the least important part

This runs contrary to most people's expectations.

LangGraph gives you an explicit state machine, loops and conditional branches — which are an advantage when the flow is deterministic. But data analysis is the opposite: how many tables to touch, how many iterations to run, whether to backtrack — those are exactly the decisions the model should be making. Wrapping that in a hardcoded state machine means taking over the one judgment the model is best at.

DSPy's bet is the opposite: keep the flow flat, and hide the complexity inside "can the model pick the right tool."

The cost

Honestly, DSPy had a price:

• A steep learning curve — the Signature mental model takes days to click
• A long debugging chain, where it's hard to tell a prompt problem from a model problem
• A smaller ecosystem; niche scenarios require building your own

When I'd pick LangGraph instead

If the flow is deterministic, needs strict approval gates, or involves irreversible external actions, an explicit state machine wins outright. There is no universally best answer — only a better or worse fit for the shape of the task.