The problem
The client held a large body of regulated reference data behind a query API. Getting an answer out of it meant knowing the right dataset, the right parameters, and the right pagination — which meant every question went through someone technical. Their users wanted to just ask. And a follow-up like "what about last year?" should mean what a person means by it, not restart the conversation.
The obvious approach — hand a language model an API and let it improvise — was a non-starter. In a regulated data environment, an invented number is worse than no answer at all.
What was built
A three-stage agentic pipeline that separates understanding the question from fetching the data from writing the answer. Each stage is independently testable, and each has one job.
Rewrites the user’s message into a standalone question using the conversation so far, returned against a strict schema — the dataset to query, the parameters, and a stable hash identifying the query. This is what makes "what about last year?" work.
Sends the resolved parameters to the client’s data API, server-side, with a recovery ladder: if a query returns nothing, the question is re-resolved once before giving up.
Streams the answer back word by word, narrating the table that was actually returned — under one hard rule: use only this table, invent no numbers.
The separation is the whole design. Conversation history reaches Resolve, so references get resolved. The fetched table reaches Ground, so the answer stays grounded. The two never mix — history cannot leak into the answer, and the model is never in a position to fabricate a figure. Every turn is persisted with its resolved question, parameters, and a summary of what came back, so any answer in the system can be traced to the exact query that produced it.
Outcome
A working conversational interface over the client’s regulated dataset, delivered end to end — frontend, backend, pipeline, and deployment — by one engineer.