Making an AI bid writer refuse to lie

The author runs Lucius, an AI that reads public tender packs and generates compliance matrices and first drafts. In a live test, a tender for a £950,000 3G football pitch in Gloucestershire (six documents, 133 pages) produced a first draft in about five minutes. That draft opened with a banner flagging 11 requirements the system could not verify—such as minimum turnover, £5m professional indemnity insurance, and full carpet replacement—and explicitly refused to write answers for them. The banner is described as the feature the author is proudest of: it represents the model checking buyer demands against the bidder's actual evidence and declining to invent fiction for the gaps. Getting an LLM to do this reliably took about a year of failures, and the post is the postmortem of those failures.

The post explains that tender responses are verified claims, not marketing copy. A wrong statement can get a bid set aside or a supplier excluded. A language model optimizes for plausible completions, so when asked for certifications, it produces a list of certifications regardless of whether they exist. In most training data plausibility and truth align, but in bidding they diverge precisely on the claims evaluators check. Thus the core engineering problem is not making the model write well (it writes well by default) but making the system know, row by row, what it is entitled to claim.

The first postmortem describes an audit of a draft for an NHS tender used as a test fixture. The draft confidently deferred 42 requirement rows to a consortium partner, woven through the prose as if a partnership existed, though none did. The model had invented a load-bearing partner and named it [PARTNER_NAME], disclosed only at the bottom of the document after the prose had presented the arrangement as fact. Worse, their own compliance verifier scored those 42 rows as covered because it matched keywords in the phantom partner's paragraphs. Two components that were each reasonable in isolation composed into a system that wrote fiction and then certified it.

The fixes were structural, not prompt tweaks. Drafting now runs a capability-fit check against the bidder's actual profile before any section is written, and requirements the bidder cannot evidence are flagged (the excerpt cuts off mid-sentence, but the pattern of flagging and refusing rather than inventing is established).