Don’t let the LLM do the maths
My first CRM insights agent asked an LLM to summarise the numbers, and it got them wrong. Every data agent I’ve built since computes in SQL and lets the model narrate, if it needs a model at all.
The problem
An LLM summarising CRM data dropped digits and mislabelled fields.
The solution
SQL computes every figure; the model only narrates them.
The first version of my CRM insights agent did what most tutorials suggest. It fetched the data, put it in the prompt and asked the model for a summary of the day.
The output read beautifully, and it was wrong. In one test run, the model dropped a digit from a rupee figure and described a lead status as a marketing channel. Nobody would have noticed until a manager quoted the number in a meeting.
Why this keeps happening
Language models predict text. Adding up 400 rows, or noticing that a field means something subtle, is arithmetic and schema knowledge: precise work that fails unpredictably. Prompting harder doesn’t fix a tool that’s the wrong shape for the job.
Split the work
Postgres does the maths. I deleted about 255 lines of Python analytics and replaced them with 327 lines of SQL functions: funnel counts, conversion rates, pipeline by stage, and anomaly checks. Anomalies are fixed rules, not vibes. Examples:
- a salesperson holding a pile of active leads with almost no pipeline is flagged as overloaded;
- a lead source where at least 30% of leads are junk is flagged as toxic.
The model only narrates. It receives one compact JSON payload of already-computed figures and writes two sections from one prompt: a dashboard report and a WhatsApp summary. The summary is capped at 1,500 characters and cut at a line break, because WhatsApp’s limit is 1,600.
Then I took even that away for the numbers people forward. The WhatsApp message became a fixed template filled with the computed values. The model still writes the commentary, but it never retypes a figure.
The same idea at a bigger scale
The growth-intelligence platform has a chat where managers ask questions. Most questions are numbers: “demos last week?”, “leads from Meta this month?”. Those go to a deterministic router, about 14 SQL “skills” that run read-only queries on allow-listed views with an 8-second timeout and a row cap. They answer in under a second, for nothing, and they’re correct by construction.
Only “why” questions (why did demos drop?) reach Claude, along with the relevant computed numbers. The model does what it’s good at: connecting facts and explaining them.
Rules I now build every data agent with
- If it can be computed, compute it. Never ask a model for a sum.
- Give the model a payload, not the database. Small, typed and precomputed.
- Templates for anything people will forward. Numbers in fixed slots; prose around them.
- Route first. Most “AI” questions are SQL questions in disguise.