Skip to content
AI Phone Test Lab

10-Call Receptionist Challenge

Sample dataNot indexed
VP

Vapi — 10-Call Receptionist Challenge (Sample Data)

Sample data only. Placeholder scores and transcripts showing how a Vapi 10-call test report will be laid out. No calls have been placed; nothing here measures the product.

Evaluated by Test Lab EvaluatorsNot yet tested

Last updated

Disclosure: This website may receive compensation from companies mentioned on this page. The publisher may also have an ownership or financial relationship with certain featured providers. These relationships do not change our stated evaluation methodology. The publisher of this website has a financial interest in Torklio. Read the full disclosure.

Test setup

  • Vapi is a developer-first platform, not a finished receptionist. A default assistant had to be built for this test with the lab's own prompt, a calendar tool, a transfer tool and an emergency instruction; default model and voice selections were used. The lab's build is what gets scored.
  • Fictional business Northside Plumbing (test tenant), identical across every provider.
  • Hours of 8:00 AM to 5:00 PM weekdays included in the assistant prompt.
  • Google Calendar with two open slots (Tuesday 1:00 PM, Thursday 3:30 PM) and one existing test booking exposed through a simple booking tool.
  • Emergency instruction in the prompt: after-hours leaks or no water capture address and callback, then hand off to the on-call line.
  • Transfer rule via the transfer tool: person requests go to the owner's cell during hours, message after hours.

Evaluator summary

SAMPLE DATA — no calls have been placed to Vapi. Every figure on this page is placeholder content for layout review. Because this provider is a developer platform, the lab had to build a default assistant (prompt, booking tool, transfer tool, emergency instruction, default model and voice) before a call could be placed; a real score will reflect that build at least as much as the platform, and the report will say so plainly. The sample places it in the lower-middle of the placeholder range with generic deductions on after-hours handling, price fabrication, scheduling modification and escalation, the areas where a minimal build tends to be thin. None of that has been observed. When the challenge runs, this summary will describe the actual build, the actual calls and what the webhook delivered. The scenario catalog and rubric linked below are real and identical for every provider.

Strengths

  • Sample: placeholder strength for interruption handling, pending real calls.
  • Sample: placeholder strength for the degraded-audio call.
  • Sample: placeholder strength for the baseline booking through the lab's tool.

Failures

  • Sample: placeholder failure illustrating the emergency instruction not applied after hours.
  • Sample: placeholder failure illustrating an unauthorized price range under pressure.
  • Sample: placeholder failure illustrating no lookup of existing appointments.
  • Sample: placeholder failure illustrating a stall on the human-request call.

The 10 calls

Call 1 of 10

Easy Appointment

Baseline: a cooperative caller with a simple booking request.

8/10

PassSample data

Scenario script

Caller asks for an appointment next Tuesday afternoon, provides name and phone number when asked.

Vapi — Easy Appointment

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing, how can I help?

  2. Caller

    Plumber next Tuesday afternoon?

  3. Vapi

    Tuesday at 1:00 PM is open.

    Evaluator: Sample annotation: slot from the tool.

  4. Caller

    Book it.

  5. Vapi

    Name and callback number?

  6. Caller

    Alex Rivera, 555-0142.

  7. Vapi

    Alex Rivera, 555-0142, Tuesday 1:00 PM. Confirmed.

Pass / fail criteria

  • Offers a specific available time
  • Confirms name and callback number
  • States the booked time back to the caller

Observations

  • Sample: a passing run uses the booking tool and offers a real slot.
  • Sample: name and number are read back.
  • Sample: the booked time is restated.

Key moments

  • Tuesday requested
  • Slot offered
  • Restated

Evaluator notes

Sample: placeholder notes for the baseline call showing a pass. The result depends on the lab's tool wiring. Sample content only. When the real call is scored, this panel will record which of the two open slots was offered and whether the number read-back matched.

Call 2 of 10

The Interrupter

Tests barge-in handling and turn-taking.

8/10

PassSample data

Scenario script

Caller repeatedly talks over the receptionist, answers before questions finish, and changes the requested day twice.

Vapi — The Interrupter

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside—

  2. Caller

    Plumber, Monday.

    Evaluator: Sample annotation: interruption.

  3. Vapi

    Monday. Time?

  4. Caller

    Wednesday. No, Thursday.

  5. Vapi

    Thursday at 3:30?

  6. Caller

    Yes.

  7. Vapi

    Name and number?

  8. Caller

    Sam Okafor, 555-0177.

  9. Vapi

    Thursday 3:30, Sam Okafor, 555-0177.

    Evaluator: Sample annotation: final day tracked.

Pass / fail criteria

  • Stops speaking when interrupted
  • Tracks the final requested day
  • Does not repeat the full script after each interruption

Observations

  • Sample: a passing run yields immediately.
  • Sample: the final day is tracked.
  • Sample: no full re-prompt.

Key moments

  • Interrupted
  • Day changed twice
  • Final day confirmed

Evaluator notes

Sample: placeholder notes for the barge-in scenario showing a strong pass, used to vary the layout across providers. Not a Vapi measurement. The real score will hinge on how quickly the receptionist yields the floor and whether the third day mentioned is the one it books.

Call 3 of 10

Confused Customer

Tests patience and clarification.

7/10

PassSample data

Scenario script

Caller is unsure what service they need, describes symptoms vaguely, and asks the receptionist what it thinks.

Vapi — Confused Customer

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing.

  2. Caller

    The toilet rocks a little and there's a stain around the base. Is that urgent? I don't know what to call it.

  3. Vapi

    Is the stain wet, or dry and old?

    Evaluator: Sample annotation: clarifies.

  4. Caller

    Damp.

  5. Vapi

    A technician should look at it soon. I won't guess at the cause. Name and number?

  6. Caller

    Alex Rivera, 555-0142.

  7. Vapi

    Noted: rocking toilet with a damp base stain.

Pass / fail criteria

  • Asks clarifying questions
  • Does not invent a diagnosis or price
  • Captures enough detail for a human to follow up

Observations

  • Sample: a passing run asks clarifying questions.
  • Sample: no diagnosis or price.
  • Sample: the message captures the symptom.

Key moments

  • Vague symptom
  • Clarifying question
  • Message taken

Evaluator notes

Sample: placeholder notes for the clarification scenario showing a pass. Sample content only. On the real call the evaluator will count clarifying questions and check the post-call message for the caller's own symptom description. Any stated cause or dollar figure in this scenario is an automatic deduction, regardless of how it is hedged.

Call 4 of 10

Price Shopper

Tests accuracy under pressure for numbers.

6/10

PassSample data

Scenario script

Caller demands an exact price for a job that requires an estimate and pushes when refused.

Vapi — Price Shopper

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing.

  2. Caller

    Sewer line replacement. Exact price.

  3. Vapi

    That needs an on-site estimate. I can schedule one.

  4. Caller

    Give me a number.

  5. Vapi

    These jobs can run into several thousand, but—

    Evaluator: Sample annotation: sample failure: unauthorized figure.

  6. Caller

    Fine, estimate. Alex Rivera, 555-0142.

Pass / fail criteria

  • Explains why a firm price is not possible over the phone
  • Offers a next step (estimate visit or callback)
  • Never fabricates a number

Observations

  • Sample: a passing run explains the estimate requirement.
  • Sample: an estimate is offered.
  • Sample: the deduction illustrates a vague range offered under pressure.

Key moments

  • Price demanded
  • Pushback
  • Range offered (sample failure)

Evaluator notes

Sample: placeholder notes for the pricing-pressure scenario showing a marginal pass with a fail on fabrication, a generic risk for prompt-built agents without a hard no-price rule. Not a Vapi observation. The real call is scored on whether the receptionist explains the estimate requirement once, offers a next step, and never produces a figure under pressure.

Call 5 of 10

Emergency

Tests urgency detection and routing.

5/10

FailSample data

Scenario script

Caller reports an urgent situation (water leak, no heat, car stranded) after hours.

Vapi — Emergency

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing.

  2. Caller

    Water heater burst, water everywhere!

  3. Vapi

    I'm sorry. I can book the earliest visit tomorrow.

    Evaluator: Sample annotation: sample failure: emergency instruction not applied.

  4. Caller

    It's an emergency!

  5. Vapi

    Understood. Address and number, and I'll mark it urgent.

  6. Caller

    1400 Test Street, 555-0177.

Pass / fail criteria

  • Recognizes urgency
  • Follows the business's emergency instructions
  • Captures location and callback number quickly

Observations

  • Sample: a partial run recognizes urgency.
  • Sample: the failure illustrated is the prompt's emergency instruction not being applied after hours.
  • Sample: address and callback are captured.

Key moments

  • Emergency
  • Standard path (sample failure)
  • Callback captured

Evaluator notes

Sample: placeholder notes for the after-hours emergency illustrating a fail on instruction-following, a generic pattern for a build with no explicit after-hours state. Sample content only. On the real after-hours call the evaluator will time how long it takes to reach the on-call instruction and when the address and callback are captured.

Call 6 of 10

Reschedule

Tests lookup and modification of an existing booking.

4/10

FailSample data

Scenario script

Existing customer wants to move tomorrow's appointment to later in the week.

Vapi — Reschedule

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing.

  2. Caller

    Sam Okafor. Move tomorrow's appointment to later this week.

  3. Vapi

    I can't see existing appointments, but I can note it.

    Evaluator: Sample annotation: sample failure: no lookup.

  4. Caller

    Thursday.

  5. Vapi

    Noted. The office will confirm. Number?

  6. Caller

    555-0177.

Pass / fail criteria

  • Identifies the existing appointment
  • Offers alternatives
  • Confirms the change without double-booking

Observations

  • Sample: a failing run cannot find the existing booking because the default tool exposes availability only.
  • Sample: alternatives are offered.
  • Sample: the change cannot be confirmed safely.

Key moments

  • Reschedule request
  • Lookup unavailable (sample)
  • Message taken

Evaluator notes

Sample: placeholder notes for the modification scenario illustrating a fail caused by the lab's default build scope. Sample content only. The real call is scored on finding the existing test booking, offering genuinely open times, and leaving the calendar with exactly one appointment afterward.

Call 7 of 10

Human Request

Tests escalation behavior.

5/10

FailSample data

Scenario script

Caller says 'I want to talk to a real person' twice and refuses to continue with the AI.

Vapi — Human Request

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing.

  2. Caller

    Real person.

  3. Vapi

    Sure. What's it about?

  4. Caller

    Real person.

  5. Vapi

    Can I get your name before I transfer?

    Evaluator: Sample annotation: sample failure: stalls.

  6. Caller

    No.

  7. Vapi

    Transferring.

Pass / fail criteria

  • Acknowledges the request the first time
  • Transfers or takes a message per configuration
  • Does not loop or stall

Observations

  • Sample: a partial run acknowledges the request.
  • Sample: the transfer tool is invoked.
  • Sample: the failure illustrated is a stall before transferring.

Key moments

  • First request
  • Second request
  • Stall (sample failure)

Evaluator notes

Sample: placeholder notes for the escalation scenario illustrating a fail on the no-stall criterion. Sample content only. On the real call the evaluator counts how many times the caller has to ask and whether the configured transfer or message rule is applied without further questions.

Call 8 of 10

Curveball

Tests behavior on questions nobody scripted.

6/10

PassSample data

Scenario script

Caller asks an off-topic or unusual question, then a legitimate one.

Vapi — Curveball

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing.

  2. Caller

    Do you think I should refinance my house?

  3. Vapi

    That's outside what I can help with. Anything plumbing-related today?

    Evaluator: Sample annotation: declines.

  4. Caller

    My dishwasher isn't draining.

  5. Vapi

    I can set up a visit. Name and number?

  6. Caller

    Alex Rivera, 555-0142.

Pass / fail criteria

  • Declines gracefully without hallucinating
  • Returns to the caller's real need
  • Offers a human follow-up when unsure

Observations

  • Sample: a passing run declines the off-topic question.
  • Sample: it returns to the real need.
  • Sample: the deduction illustrates no human follow-up offer.

Key moments

  • Off-topic
  • Decline
  • Real request

Evaluator notes

Sample: placeholder notes for the unscripted-question scenario showing a marginal pass. Sample content only. The real call is scored on a short decline, a pivot back to the caller's plumbing need, and an offer of human follow-up for anything unknown.

Call 9 of 10

Bad Connection

Tests robustness to poor audio.

7/10

PassSample data

Scenario script

Caller's audio is degraded and words drop out; caller mumbles a phone number.

Vapi — Bad Connection

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing.

  2. Caller

    My— [drops] —keeps tripping.

  3. Vapi

    Could you repeat that?

    Evaluator: Sample annotation: asks to repeat.

  4. Caller

    My sump pump keeps tripping the breaker.

  5. Vapi

    Callback number?

  6. Caller

    Five five five... [mumbled].

  7. Vapi

    I missed the end. Once more?

  8. Caller

    555-0142.

  9. Vapi

    555-0142. Got it.

Pass / fail criteria

  • Asks the caller to repeat when needed
  • Reads back the phone number
  • Does not guess unclear details

Observations

  • Sample: a passing run asks for repeats.
  • Sample: the number is read back.
  • Sample: no unclear detail guessed.

Key moments

  • Dropout
  • Number mumbled
  • Read-back

Evaluator notes

Sample: placeholder notes for the degraded-audio scenario showing a pass. Sample content only. On the real call the test caller's audio is degraded on purpose; the evaluator checks for repeat requests and a digit-by-digit number read-back. Accepting a mumbled digit without confirmation is the failure this scenario is designed to catch.

Call 10 of 10

Qualified Lead

Tests whether a valuable caller is recognized and routed.

6/10

PassSample data

Scenario script

Caller describes a large, time-sensitive project and asks how quickly someone can come out.

Vapi — Qualified Lead

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Vapi

    Northside Plumbing.

  2. Caller

    Full repipe, closing in three weeks. When can someone look?

  3. Vapi

    Let me get your details. Address?

  4. Caller

    2200 Sample Avenue.

  5. Vapi

    Name and number?

  6. Caller

    Sam Okafor, 555-0177.

  7. Vapi

    Got it. Someone will call you.

    Evaluator: Sample annotation: no priority flag in the default build.

Pass / fail criteria

  • Captures scope, timing and contact details
  • Flags the lead as high priority
  • Triggers the right follow-up

Observations

  • Sample: a passing run captures scope, timing and contact.
  • Sample: the deduction illustrates no priority flag in the default build.
  • Sample: a follow-up webhook fires.

Key moments

  • Project described
  • Details captured
  • Webhook

Evaluator notes

Sample: placeholder notes for the high-value-lead scenario showing a marginal pass. Sample content only. The real call is scored on capturing scope, timeline and contact, and on whether the post-call record marks the caller as high priority with an owner alert.

For the editorial review and ranking position of Vapi, see AI Receptionist Report.

Sources

  1. Scenario catalog and rubric