Putting Claude in production at a beverage plant

The MES/ERP platform I built and run at Kingsley Beverage in Dubai has the Claude API and the OpenAI API inside it, in production, doing real work on document processing and reporting. This note is about the design rule that made that safe enough to ship, and about the things I have not built yet - because the second list is the one that tells you whether someone has actually run an LLM feature in front of users.

1. The model advises, deterministic code acts

This is the whole architecture in one sentence, and everything else follows from it.

The AI path holds no write credentials to the ERP. It cannot create a job order, post a GRN, adjust stock, or touch a Sage batch. When a model output leads to a change, that change goes through the same validated, typed, permission-checked code path a human form submission goes through - the model just got there first with a suggestion. If the model produces nonsense, the worst case is that a person reads nonsense and ignores it. The worst case is not a corrupted inventory ledger.

That constraint is unglamorous and it costs you features. It is also the reason I was willing to put a non-deterministic component into a system that the finance team reconciles against Sage every day. A plant does not care that your extraction is 95% accurate. It cares which 5% moved stock.

2. What I was actually on the hook for

Nobody handed me an accuracy KPI, and I am not going to invent one retrospectively. The standard I was measured against was much blunter, and much more useful:

• the system is up during shifts;
• the stock figure on screen matches the physical store;
• the Sage postings reconcile.

None of those three are AI metrics. That is the point. The LLM features are judged by whether they make those three easier to hold, not by a benchmark score. An AI feature that improves extraction accuracy but makes the evening reconciliation harder to explain is a net loss, and would be switched off within a week - by the people using it, not by me.

3. The integration underneath is the hard part

The part of this system I would defend hardest in a review has no model in it at all: the bridge between Sage Evolution on SQL Server and the application's own MongoDB state.

Inventory syncs live from SQL Server, with an automatic offline fallback when the Sage box is unreachable - the floor keeps working, it just knows the number is stale. Goods receipts post back into Sage rather than being retyped by the accounts team. And when a posting fails, it fails visibly: it surfaces to a human instead of disappearing into a silent retry loop that nobody discovers until month end.

That last decision is the same instinct as the one in section 1. In an operations system the expensive failure is not the error - it is the error that looks like success.

4. RAG is retrieval first, and mostly retrieval

The copilot pattern I keep coming back to answers from a document corpus and cites the source it used, as a chip you can click to open the underlying document. Two reasons, and neither is polish.

First, a citation converts "trust the model" into "check the model", which is the only version an auditor or a QC manager will accept. Second, it makes failures diagnosable. When an answer is wrong and you can see which chunk it came from, you can tell within seconds whether retrieval pulled the wrong document or generation mangled the right one. Those are completely different bugs with completely different fixes, and without citations you cannot tell them apart - you just have a vague sense that "the AI is bad today".

In my experience the answer is usually retrieval. Chunking, and what you attach as metadata, moves answer quality far more than prompt wording does.

5. What I have not built - stated plainly

This is the section I would want to read on someone else's site.

There is no automated evaluation harness. No golden dataset, no regression suite over a fixed question set, no scored CI gate. What I do instead is manual and unglamorous: I run real user questions through it, including deliberately out-of-scope ones to see whether it declines or confabulates; I check every answer against the citation it produced; when something is wrong I classify it as retrieval or generation before touching anything; and I watch latency and per-conversation API cost, because a feature that is correct and unaffordable is not shipped, it is just expensive.

That is enough to catch obvious regressions and nowhere near enough to catch subtle ones. What I would build next, in order: a fixed question set with expected source documents, so retrieval can be scored separately from generation; then an assertion suite over the answers; then that in CI with a cost and latency budget.

There is no formal escalation or authority-tier framework. "The model advises, deterministic code acts" is a hard architectural boundary, not a graded system of what the model may do unsupervised at each confidence level. For the scope I have - one plant, a handful of workflows, a user base I see in person - the hard boundary is the right trade. It would not survive being scaled to a system where the AI is expected to act autonomously, and I would want to design that properly rather than loosen this one gradually.

6. What this changed about how I build

Adding a language model to an operations system did not change my priorities so much as sharpen them. The questions that turned out to matter were the ones I was already asking about Sage postings and QC records: who is accountable for this number, how would we notice if it were wrong, and what happens on the floor while it is wrong.

An LLM is unusually good at producing output that looks accountable and is not. Most of the engineering is in refusing to let it be the last step before something irreversible - and in making the checking cheap enough that people actually do it.

Related

The RAG-with-citations pattern is running end-to-end in Sanad, the support-copilot demo - agent inbox, streaming chat widget, citation chips, and a deterministic mock fallback so it works with no key configured. The Nabta policy assistant is the same shape over HR policy and UAE Labour Law. Both are portfolio builds with fabricated data; the production work described above is private.