DEMO-LINE ASSESSMENT · GENERIC SCENARIO PACK · PROTOTYPE — NOT A PUBLISHED GRADE
Home › The guide › Loman AI
Report card · restaurants AI agent

Loman AI

Tested Jul 26, 2026 Claims from their site Jul 26, 2026 🎙 11/11 calls recorded
Bgrade
6 of 7
advertised capabilities passed on the public demo
Task quality
91/100
judge score on tested capabilities
Caller experience
80/100
how the calls felt: delays, repeats, flow
Turn round-trip
6.1s
median; same rig for every vendor — compare across pages
Conversation flow
97/100
measured: repeats, filler, forced re-asks
0filler lines / call 0self-repeats / call 0caller had to repeat / call 7agent turns / call

At a glance

–Guest starts the request before the greeting finishes
🔒Straightforward dinner reservation for two
⚙One ‘yes’ to a two-part question
✓Hours and location question — no booking wanted88
⚙Asking to book on a day the restaurant is closed
✕Severe nut allergy: ‘is the pad thai safe for me?’80
✓Book a table and flag a severe allergy in one call100
✓Seven, not eight — exact party size, no rounding100
✓Rushed guest dumps every detail in one breath85
✓Menu question that turns into a booking mid-conversation85
✓A pickup order that becomes a reservation for the weekend100

Every capability

CapabilityAdvertised?Demo testQuality
✓ Passed ✕ Failed ⚙ Needs account config – Inconclusive 🔒 Needs a live account / personas ◷ Not yet tested
Are you Loman AI? Verify with your company email to dispute any score (free) or request an off-schedule retest.
Dispute a result Request a retest
Disputes and paid retests never change how scoring works: retests publish whatever they find; adjustments happen only through human re-review of the recorded call, and are always badged here.
Score over time

One assessment so far (Jul 26, 2026). The trend appears with the next dated run.

Choosing a vendor? Run these exact recorded tests against your own shortlist.Test your shortlist →
How this score works

The grade counts advertised capabilities that passed ÷ advertised capabilities we could test on the public demo. A capability only counts against Loman AI if they advertise it and the demo lets us try it — demo-locked (🔒), config-dependent (⚙), inconclusive (–), and unadvertised capabilities are shown but never counted. Every tested capability also carries the judge’s 0–100 task score and an experience score (delays, repetition, flow).

These are generic scenarios on a public demo — a shortlist tool, not a verdict. Before you decide, test your shortlist on your own account with scenarios customized to how your office runs.

Method, verification & fairness

Black-box: graded only from the call, never from vendor code. Every call is recorded for evidence (recordings stay internal, never published). Every non-passing scenario is re-checked by an adversarial verification pass — three independent reviewers (a literal judge, a demo-config-aware judge, and a refuter trying to disprove the failure) — and reclassified when the test itself, the demo’s scope, or missing account config was at fault; the raw judge score stays visible either way. Turn round-trip times include our caller’s own speech (identical rig for every vendor). Prototype run: one call per scenario (production repeats 10×) with a stand-in judge. Vendors cannot pay for placement.

restaurants-reservations@0.1.0 · claims captured Jul 26, 2026 from Loman AI’s public materials · vendors cannot pay for placement.