Patterns
Make the model choose, not invent
· 6 min read
There is a comment in our chat pipeline that took a while to earn: the model selects supplied text, never invents a query. It is the difference between asking a model what the shopper's budget is and handing it every number in the sentence and asking which one is the budget. The second question cannot be answered with a hallucinated number, because a hallucinated number is not one of the options.
The failure you are trying to remove
Free-text extraction fails in a specific, irritating way. Ask a model to pull a budget out of "my son is 8, I'm looking for something around 1500 lira, model 4032 if you have it" and it will usually return 1500. Usually. Some fraction of the time it returns 8, or 4032, or 1500.00 formatted differently than last time, or a helpful 1450 because that felt like a better price point.
You can validate the output, and you should, but validation only catches values that are obviously malformed. It cannot catch a plausible wrong number. The age and the model number are both plausible budgets.
Turn extraction into selection
The fix is to do the extraction yourself, badly, and let the model do the part it is good at. We pull every numeric span out of the message with a regular expression, cap it at six, and hand those spans to Jev as the options of a Choice question.
numeric = re.findall(r'\d[\d.,]*', query)
budgets = {str(i): v for i, v in enumerate(numeric[:6])}
budgets['KEEP'] = str(prior.get('budget') or 'No budget')
budgets['NONE'] = 'No maximum budget or remove earlier budget.'The regular expression is deliberately dumb. It does not know which number is a budget, and it does not need to; it only needs to not miss one. Precision is the model's job and recall is the regex's job, which is the opposite of how people usually split that work.
Two options are not spans from the message. KEEP means the previous budget still applies, which matters because a follow-up message often contains no number at all and silence is not the same as removal. NONE means there is no maximum, or an earlier budget has just been revoked. Both are states the conversation can genuinely be in, and neither can be expressed by picking a number.
The same trick on the query itself
The harder case is deciding what the shopper is actually looking for now. Conversations drift. Someone starts with a birthday gift for a nephew, corrects the age, rejects a suggestion, then mentions their spouse. Ask a model to write a search query for that and you get a sentence that blends all four turns, weighted by nothing in particular.
So we do not ask it to write anything. We split the message into clauses and offer each clause as a candidate, alongside the previous query and the whole current message.
focus = {
'PREVIOUS': prior.get('query', 'No previous query'),
'LATEST': query,
}
# every clause of the message becomes its own candidate
for i, part in enumerate(re.split(r'[;.!?]|\b(?:ama|fakat|ancak)\b', query)):
if len(part.strip()) > 4:
focus['PART_' + str(i)] = part.strip()The split points are sentence punctuation and the Turkish contrastive conjunctions: ama, fakat, ancak. Those words are where a shopper changes their mind mid-sentence, and the clause after one of them is usually the one that matters now. The model picks a clause. Whatever it picks is text the shopper typed, so the retrieval step never searches for something nobody said.
PREVIOUS earns its place for the same reason KEEP does. When the latest message is only an exclusion, a budget change or a request for alternatives, the right query is still the previous one, and the instruction says so explicitly.
Negation has to leave the query and stay in the state
Exclusions are where this gets interesting, because they have to be removed from one place and preserved in another. A shopper who says they do not want plush toys has given you two things: a constraint to remember, and a phrase that will poison an embedding search if it stays in the query text.
We find the exclusion clauses by their Turkish markers, olmasın, istemiyorum, hariç, dışında, and each surviving term becomes its own Choice question asking whether it still applies in the current search. Then we strip those clauses out of the text that goes to retrieval. The constraint lives in structured state; the phrase does not live in the query.
Old exclusions expire on their own terms. A new independent search drops them unless they are repeated, and a later message can revoke one, which the model reports as a NO on that term's question. None of this is reliable if you keep the exclusions as free text and hope the query embedding handles them, because embeddings have no NOT.
Where the parsing still lands on you
Selection removes the invention problem. It does not remove the parsing problem, and it is worth being clear about that. Once the model has picked the span "1.299", something still has to decide whether that is one thousand two hundred ninety nine in Turkish thousand separators or one point two nine nine. We handle that with an explicit check for the grouped-digits pattern before falling back to a decimal comma swap.
That code is unglamorous and it is exactly where it belongs. A regular expression that mis-parses a number is a bug you can find, reproduce and fix. A model that occasionally invents a number is a behaviour you can only sample.
The general shape
Whenever you are about to ask a typed decision model for a value, check whether you can ask it for a pointer instead. Enumerate the candidates from data you already hold, add the states that are not candidates, and let the model choose. You lose the ability to handle values that never appear in the input, which is usually a feature, and you gain a class of failure that becomes impossible rather than merely unlikely.
Navlu is a conversational product discovery assistant for ecommerce stores. Budget, focus, currency and every exclusion in a turn are Choice questions over candidates extracted from the shopper's own message.