Engineering B2C AI Agents, Part 1: Delegate Ambiguity to the Model and Facts to Deterministic Rules
Disclaimer: this post has been entirely written by human hands typing on a keyboard.
“Agents only need a loop and a good directory structure!”
“Why do you waste time in creating all these nodes!!”
“Agentic loops will make you a millionaire!!!”
“Context is all you need!!!!!!!!!!!”
If you have subscribed to a Medium newsletter, this is probably what you have been getting 85.6% of the time on your feed in the last months.
Now, I am not going to call all of this BS, only most of it. It looks like the AI hype is not able to settle due to too much weekly news of new powerful models, agentic frameworks (hello hello again graph engineering) and LLM wikis.
This series aims to show why high-level practices are not enough to create scalable AI products. I am going to be narrower and only target B2C products as that’s where I’ve learnt most of these practices. Probably they apply to B2B products as well, but I cannot fully claim that, even if I built many of them in my non-startup job.
I have been working on NutriCoach, a B2C AI agent that acts as a nutritional coach. I started applying best practices recommended by big players like Anthropic and OpenAI, but it turns out that most of their recommendations are based on AI coding agents, which are a great example, but do not generalise to all the use cases.
Most importantly, they are freaking rich. They care less about cost optimisation and hence can use the most powerful frontier models.
Building B2C AI products means that you are offering this service to people who cannot afford to spend thousands of pounds per month. £20 at most. Forget about using Fable 5.1 or GPT 6 Astra.
So, the 1 billion dollar question is: how do I create a high-quality AI product offered to users who can pay less than £20 per month and will use it multiple times per day?
The first practice I want to share with you is the following.
Please note: these practices assume you have a fundamental understanding of how AI agents work and how to implement them.
Delegate ambiguity to the model and facts to deterministic rules
It seems easy: if a task requires model judgment, use a model; otherwise you can implement it using deterministic rules. The problem is that a task is not just ambiguous or deterministic, and that’s where headaches start. Too many deterministic rules do not generalise, too few produce hallucinations. I will explain this through the “milk breakfast incident”.
Some context: in NutriCoach users can log food via chat or voice and add it to their plan. We use a database of food for lookup. Macros and calories of foods are stored per 100 g.
Imagine a user logs oats, honey, almond butter, banana and 100 ml of skimmed milk. Let’s see different ways we can implement this functionality.
Too many rules
What is the first thing you would do to understand how many grams of milk equate to 100 ml of milk? Of course, you ask your favourite AI tool to write a massive regex catalogue that detects any kind of portion unit (ml, mg, etc.), and another massive database of food, and rules work out the magic and make the conversion. Beautiful on paper. The only problem is that, no matter how smart your GPT 6 Astra is, the combination of different foods and portion units combined with the infinite ways a user could express an intent like logging milk cannot be encoded in a set of rules such as this. For instance, the user can just say “a small cup of Organic Valley” and everything explodes.

Note: in the past, I was particularly optimistic about letting AI create regexes, and I almost never got it right. I will probably share more about using regexes in this new AI era.
Too much LLM
This time you have learned your lesson: no regex. The model is smart (right?), so you let the model do it. Prompt: “convert the user’s portion to grams, look up the nutrition, work out the calories, and log them”.
And it works. The model decides 100 ml of skimmed milk weighs about 103 g, remembers that skimmed milk is around 34 kcal per 100 g, multiplies, and logs 35 kcal. Beautiful.
Run it ten times. Seven times you get 35. Once you get 350, because a decimal point slipped. Once you get 64, because the model remembered whole milk instead of skimmed. Once it decides that 100 ml is 100 g and skips the conversion entirely. Every one of those answers arrives with the same confident “Logged your breakfast!”.

Now the user says “a small cup of Organic Valley”. This time nothing explodes. That is the problem. It invents a cup size, picks a fat percentage, pulls a calorie count from somewhere in its training data, and logs 148 kcal of a product it has never looked up.
What works then?
Break down the task into smaller sub-tasks:
- Extract foods, units and quantities from the user message
- For each food:
- retrieve macros per 100g from the database
- get/convert the quantity into grams
- calculate the macros per quantity
- log the food
Now, understand whether each action can be implemented either through deterministic rules or an LLM is needed. The decision criterion here is: “does the task include a lot of ambiguity to delegate it to an LLM?” If the answer is yes, then use an LLM, otherwise use rules.
If you still can’t answer that question for a task, it’s likely that the task is not small enough. Take “retrieve macros per 100g from the database”. You may think to use a keyword-based lookup, but it turns out that users can express the same food in too many different ways: “pasta, pennette, spaghetti, linguine”. Unless your database is extremely well crafted and comprehensive of all the food in the world (which it is not), a simple keyword lookup is not gonna be sufficient. There is some ambiguity in it.
Let’s break it down:
- Send the food request as a database query
- Return the top 5 most similar results.
- Choose the closest result based on the user request
I bet you understand how this can be implemented: RAG (Retrieval Augmented Generation). You know that 1 does not need a model, 2 needs an embedding model in case you opt for semantic search, and 3 needs an LLM.
So, what happens if we implement this practice given the request “log 100 ml of skimmed milk”?
- LLM extracts
{"food": "skimmed milk", "unit": "ml", "quantity": 100} - Code sends “skimmed milk” as request to the database.
- 5 entries return
- LLM chooses the best fit: “skimmed milk, 34 kcal per 100 g”
- LLM estimates the weight and flags it as a guess:
{"quantity": "103 g", "estimated": true} - Code computes 103 / 100 × 34 = 35 kcal.
- Code logs the food.
Now the hard one: “a small cup of Organic Valley”. Same steps, plus one lookup for the unit:
- LLM extracts
{"food": "Organic Valley milk", "unit": "small cup", "quantity": 1} - Code sends “Organic Valley milk” as request to the database.
- 5 entries return.
- LLM chooses the best fit: “Organic Valley whole milk, 61 kcal per 100 g”.
- Code sees that “small cup” is not a unit it can convert, and returns the portion-unit table: cup = 240 ml, glass = 250 ml, tbsp = 15 ml.
- LLM picks the closest unit, 1 cup = 240 ml, and estimates the weight:
{"quantity": "247 g", "estimated": true} - Code computes 247 / 100 × 61 = 151 kcal.
- Code logs the food.

Of course, this is not the only solution that works. It is just one of many, where rules and reasoning are amalgamated together in the right portions. The boundaries of these two concepts are not defined at all. At the moment, I see this almost as an art rather than a measurable approach.
To summarise
- Break the task into steps small enough to judge. “Log my breakfast” → too big.
“Extract the foods”, “find the record”, “convert to grams” and “compute the calories” → good. - Ambiguous steps go to the model, the rest to rules. Matching “a small cup of Organic Valley” to a record → ambiguous.
247 / 100 × 61 → deterministic.
Where exactly the line sits between ambiguity and determinism is still a judgment call. Production evaluation (article coming soon) tells you where things still break due to poor judgment so that you can adjust it.