AI Agents#Agents#LangGraph#RAG#Production AI

Building a Multi-Agent ESG Copilot

How I turned a brittle single-prompt ESG analyzer into a LangGraph multi-agent system with tools, validation loops, and human review.

May 18, 20263 min read

The first version of our ESG copilot was a single prompt. It worked in the demo and fell apart in production. This is the story of rebuilding it as a multi-agent system that a compliance team could actually trust.

The problem with one big prompt

A single prompt asked the model to read a company's sustainability report, extract claims, cross-check them against a rubric, and produce a scored assessment. It failed in three predictable ways:

No separation of concerns. Retrieval, reasoning, and formatting were entangled, so a failure anywhere corrupted everything.
No place to intervene. A human reviewer couldn't inspect or correct an intermediate step — only accept or reject the final answer.
No measurable stages. When output quality dropped, there was nothing to point at.

If you can't say which step produced a bad answer, you don't have a system — you have a wish. Decomposition is what makes AI debuggable.

The multi-agent decomposition

I split the workflow into discrete LangGraph nodes, each with one job and a typed contract:

from langgraph.graph import StateGraph
 
def build_graph() -> StateGraph:
    g = StateGraph(ESGState)
    g.add_node("classify", classify_query)      # what kind of assessment?
    g.add_node("retrieve", retrieve_evidence)    # hybrid RAG over the report
    g.add_node("assess", score_against_rubric)   # tool-using agent
    g.add_node("validate", validate_schema)      # guardrail + repair loop
    g.add_node("review", human_checkpoint)       # optional human-in-the-loop
    g.set_entry_point("classify")
    g.add_conditional_edges("validate", needs_review, {
        "review": "review",
        "done": END,
    })
    return g

Each node reads and writes a shared, typed state object. That state is the audit trail — you can replay exactly what each agent saw and produced.

Tools, not freeform reasoning

The assessment agent doesn't "think about" numbers — it calls tools: lookup_rubric_criterion, fetch_prior_year_metric, compute_delta. Tools turn vague reasoning into checkable operations.

The validation loop

The most valuable node is the least glamorous. validate runs the model's output through a schema and, on failure, feeds the errors back for one repair attempt before escalating to a human.

Give repair loops a hard budget. An agent that retries forever is an outage with a progress bar. One repair attempt, then escalate.

Human review as a first-class edge

High-stakes ESG scores route through a review checkpoint. The graph pauses, surfaces the evidence and the draft score, and waits. This isn't a fallback for when the model fails — it's a designed control point for decisions that carry regulatory weight.

What changed in production

Metric	Single prompt	Multi-agent
Traceable failures	No	Yes
Schema-valid outputs	~72%	~98%
Human-correctable steps	0	4

The accuracy gain mattered, but the real win was operability. When something went wrong, we could see where — and fix that node without touching the rest.

Takeaways

Decompose until each agent has one testable job.
Prefer tools over freeform reasoning for anything checkable.
Make human review a designed edge, not an afterthought.
Budget every loop. Escalation beats infinite retry.

Multi-agent isn't about more models. It's about turning an opaque prompt into a system you can reason about.

← Back to writing