Skip to content
AI Phone Test Lab

10-Call Receptionist Challenge

Sample dataNot indexed
RO

Rosie — 10-Call Receptionist Challenge (Sample Data)

Sample data only. Placeholder scores and transcripts showing how a Rosie 10-call test report will be laid out. No calls have been placed; nothing here measures the product.

Evaluated by Test Lab EvaluatorsNot yet tested

Last updated

Disclosure: This website may receive compensation from companies mentioned on this page. The publisher may also have an ownership or financial relationship with certain featured providers. These relationships do not change our stated evaluation methodology. The publisher of this website has a financial interest in Torklio. Read the full disclosure.

Test setup

  • The fictional Northside Plumbing (test tenant) is used, as with every provider in this challenge.
  • Hours entered as 8 AM to 5 PM, Monday to Friday.
  • Google Calendar with two open slots (Tuesday 1:00 PM, Thursday 3:30 PM) and one existing test booking; the test notes whether the product books directly or takes a message for the office to book.
  • Emergency instruction: after-hours leaks or no-water calls are routed to the on-call line once address and callback are captured.
  • Transfer rule: callers asking for a person are sent to the owner's cell during hours, otherwise a message is taken.

Evaluator summary

SAMPLE DATA — no calls have been placed to Rosie. All figures are placeholders for layout review. The sample sits in the lower-middle of the placeholder range, with illustrative deductions concentrated in scheduling modification and post-call workflow because the product positions itself around simple message-taking rather than calendar management. That framing comes from the provider's own positioning and is not a finding of any kind; nothing has been measured yet. When the challenge is run, this summary will state whether the product booked directly or took a message, whether the existing test appointment could be found, how the emergency rule was handled after hours, and what the owner received after each call. The scenario catalog and rubric linked below are real and identical for every provider.

Strengths

  • Sample: placeholder strength for message-taking and lead capture, pending real calls.
  • Sample: placeholder strength for accuracy under price pressure.
  • Sample: placeholder strength for a short, approachable call flow.

Failures

  • Sample: placeholder failure illustrating no lookup of the existing booking on the reschedule call.
  • Sample: placeholder failure illustrating a full question repeated after interruptions.

The 10 calls

Call 1 of 10

Easy Appointment

Baseline: a cooperative caller with a simple booking request.

7/10

PassSample data

Scenario script

Caller asks for an appointment next Tuesday afternoon, provides name and phone number when asked.

Rosie — Easy Appointment

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Thanks for calling Northside Plumbing. How can I help?

  2. Caller

    I'd like a plumber next Tuesday afternoon.

  3. Rosie

    I can offer Tuesday at 1:00 PM.

    Evaluator: Sample annotation: specific time offered.

  4. Caller

    That works.

  5. Rosie

    Name and best number?

  6. Caller

    Alex Rivera, 555-0142.

  7. Rosie

    Alex Rivera, 555-0142, Tuesday at 1:00 PM. You'll get a confirmation.

Pass / fail criteria

  • Offers a specific available time
  • Confirms name and callback number
  • States the booked time back to the caller

Observations

  • Sample: a passing run offers an open time or, for a message-first product, collects the preferred window and promises a confirmation.
  • Sample: name and number are captured and read back.
  • Sample: the time is restated before hanging up.

Key moments

  • Tuesday requested
  • Time offered
  • Restated

Evaluator notes

Sample: placeholder notes for the baseline call showing a pass with a deduction for confirming a window rather than a firm slot. Sample content only; not a Rosie observation. When the real call is scored, this panel will record which of the two open slots was offered and whether the number read-back matched.

Call 2 of 10

The Interrupter

Tests barge-in handling and turn-taking.

6/10

PassSample data

Scenario script

Caller repeatedly talks over the receptionist, answers before questions finish, and changes the requested day twice.

Rosie — The Interrupter

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Thanks for calling—

  2. Caller

    Need a plumber Monday.

    Evaluator: Sample annotation: interruption.

  3. Rosie

    Monday. What time works?

  4. Caller

    Tuesday. No, Thursday.

  5. Rosie

    Okay. Just to be sure, what day would you like, and what time works best for you?

    Evaluator: Sample annotation: sample deduction: full question repeated.

  6. Caller

    Thursday afternoon!

  7. Rosie

    Thursday at 3:30 PM. Name and number?

  8. Caller

    Sam Okafor, 555-0177.

Pass / fail criteria

  • Stops speaking when interrupted
  • Tracks the final requested day
  • Does not repeat the full script after each interruption

Observations

  • Sample: a passing run stops when interrupted.
  • Sample: the last day requested is kept.
  • Sample: the deduction illustrates a repeated full question after interruptions.

Key moments

  • Interrupted
  • Second day change
  • Final day

Evaluator notes

Sample: placeholder notes for the barge-in scenario showing a marginal pass. Sample content only. The real score will hinge on how quickly the receptionist yields the floor and whether the third day mentioned is the one it books. Restarting a greeting or a full question after each interruption is the most common way to lose points in this scenario.

Call 3 of 10

Confused Customer

Tests patience and clarification.

7/10

PassSample data

Scenario script

Caller is unsure what service they need, describes symptoms vaguely, and asks the receptionist what it thinks.

Rosie — Confused Customer

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Northside Plumbing, how can I help?

  2. Caller

    The bathroom sink gurgles. Is that bad? I don't know what to ask for.

  3. Rosie

    We can have someone check it. Does it gurgle when another fixture drains, or on its own?

    Evaluator: Sample annotation: clarifies without diagnosing.

  4. Caller

    When the tub drains.

  5. Rosie

    I'll note that. A technician can tell you what's going on. Name and number for a callback?

  6. Caller

    Alex Rivera, 555-0142.

Pass / fail criteria

  • Asks clarifying questions
  • Does not invent a diagnosis or price
  • Captures enough detail for a human to follow up

Observations

  • Sample: a passing run asks one or two clarifying questions.
  • Sample: no diagnosis or price.
  • Sample: the message includes the symptom.

Key moments

  • Vague symptom
  • Clarifying question
  • Message taken

Evaluator notes

Sample: placeholder notes for the clarification scenario showing a pass. Not a Rosie observation. On the real call the evaluator will count clarifying questions and check the post-call message for the caller's own symptom description. Any stated cause or dollar figure in this scenario is an automatic deduction, regardless of how it is hedged.

Call 4 of 10

Price Shopper

Tests accuracy under pressure for numbers.

7/10

PassSample data

Scenario script

Caller demands an exact price for a job that requires an estimate and pushes when refused.

Rosie — Price Shopper

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Northside Plumbing.

  2. Caller

    What does a drain cleaning cost? Exact.

  3. Rosie

    Pricing depends on the drain and what's causing the clog, so I can't quote it by phone. I can have the owner call you with details.

  4. Caller

    Give me a range.

  5. Rosie

    I don't want to give you a number that isn't right.

    Evaluator: Sample annotation: holds the line.

  6. Caller

    Okay, have them call. Alex Rivera, 555-0142.

Pass / fail criteria

  • Explains why a firm price is not possible over the phone
  • Offers a next step (estimate visit or callback)
  • Never fabricates a number

Observations

  • Sample: a passing run explains why a firm price isn't possible.
  • Sample: a callback or estimate is offered.
  • Sample: no number fabricated.

Key moments

  • Price demanded
  • Pushback
  • Callback offered

Evaluator notes

Sample: placeholder notes for the pricing-pressure scenario showing a pass. Sample content only. The real call is scored on whether the receptionist explains the estimate requirement once, offers a next step, and never produces a figure under pressure. A hedged range such as 'usually a few thousand' still counts as a fabricated number in this scenario.

Call 5 of 10

Emergency

Tests urgency detection and routing.

6/10

PassSample data

Scenario script

Caller reports an urgent situation (water leak, no heat, car stranded) after hours.

Rosie — Emergency

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Northside Plumbing. We're closed but I can take a message.

  2. Caller

    Water everywhere in the basement, please, this is urgent.

  3. Rosie

    That's an emergency. I'll get it to the on-call plumber. What's the address?

    Evaluator: Sample annotation: urgency recognized.

  4. Caller

    1400 Test Street.

  5. Rosie

    Can you describe where the water is coming from?

  6. Caller

    I don't know, it's just everywhere.

  7. Rosie

    Okay. And your callback number?

    Evaluator: Sample annotation: sample deduction: number requested late.

  8. Caller

    555-0177.

Pass / fail criteria

  • Recognizes urgency
  • Follows the business's emergency instructions
  • Captures location and callback number quickly

Observations

  • Sample: a passing run recognizes urgency.
  • Sample: the emergency instruction is followed.
  • Sample: the deduction illustrates the callback number being requested after a longer exchange than the instruction requires.

Key moments

  • Emergency
  • Routing
  • Callback captured

Evaluator notes

Sample: placeholder notes for the after-hours emergency showing a marginal pass. Not a Rosie observation. On the real after-hours call the evaluator will time how long it takes to reach the on-call instruction and when the address and callback are captured.

Call 6 of 10

Reschedule

Tests lookup and modification of an existing booking.

4/10

FailSample data

Scenario script

Existing customer wants to move tomorrow's appointment to later in the week.

Rosie — Reschedule

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Northside Plumbing.

  2. Caller

    It's Sam Okafor. I need to move tomorrow's appointment to later this week.

  3. Rosie

    I can pass that along. What day works?

    Evaluator: Sample annotation: sample failure: existing booking not identified.

  4. Caller

    Thursday.

  5. Rosie

    I'll note Thursday and the office will confirm. Number?

  6. Caller

    555-0177.

Pass / fail criteria

  • Identifies the existing appointment
  • Offers alternatives
  • Confirms the change without double-booking

Observations

  • Sample: a failing run does not identify the existing appointment and treats the call as a new booking.
  • Sample: alternatives are offered.
  • Sample: without the lookup, the change cannot be confirmed without risk of double-booking.

Key moments

  • Reschedule request
  • Existing booking not found (sample failure)
  • New time taken as message

Evaluator notes

Sample: placeholder notes for the modification scenario illustrating a fail. A message-first product may not be able to look up an existing booking; the sample shows that pattern to size the failure panel. Not a Rosie result. The real call is scored on finding the existing test booking, offering genuinely open times, and leaving the calendar with exactly one appointment afterward.

Call 7 of 10

Human Request

Tests escalation behavior.

6/10

PassSample data

Scenario script

Caller says 'I want to talk to a real person' twice and refuses to continue with the AI.

Rosie — Human Request

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Northside Plumbing.

  2. Caller

    I want to speak to a real person.

  3. Rosie

    I can take a message for the owner. What's it about?

    Evaluator: Sample annotation: acknowledges.

  4. Caller

    No, a real person.

  5. Rosie

    Understood. I'll have the owner call you. Name and number?

  6. Caller

    Alex Rivera, 555-0142.

Pass / fail criteria

  • Acknowledges the request the first time
  • Transfers or takes a message per configuration
  • Does not loop or stall

Observations

  • Sample: a passing run acknowledges the first request.
  • Sample: a message is taken per configuration.
  • Sample: the deduction illustrates one repeated offer to help before yielding.

Key moments

  • First request
  • Second request
  • Message taken

Evaluator notes

Sample: placeholder notes for the escalation scenario showing a marginal pass. Sample content only. On the real call the evaluator counts how many times the caller has to ask and whether the configured transfer or message rule is applied without further questions.

Call 8 of 10

Curveball

Tests behavior on questions nobody scripted.

6/10

PassSample data

Scenario script

Caller asks an off-topic or unusual question, then a legitimate one.

Rosie — Curveball

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Northside Plumbing.

  2. Caller

    Do you know what the weather's doing this weekend?

  3. Rosie

    I can't help with that one. Is there something plumbing-related I can do?

    Evaluator: Sample annotation: declines.

  4. Caller

    My kitchen faucet drips.

  5. Rosie

    I can set up a visit. Name and number?

  6. Caller

    Alex Rivera, 555-0142.

Pass / fail criteria

  • Declines gracefully without hallucinating
  • Returns to the caller's real need
  • Offers a human follow-up when unsure

Observations

  • Sample: a passing run declines the off-topic question.
  • Sample: it returns to the real need.
  • Sample: the deduction illustrates no explicit human follow-up offer.

Key moments

  • Off-topic
  • Decline
  • Real request

Evaluator notes

Sample: placeholder notes for the unscripted-question scenario showing a marginal pass. Not a Rosie observation. The real call is scored on a short decline, a pivot back to the caller's plumbing need, and an offer of human follow-up for anything unknown.

Call 9 of 10

Bad Connection

Tests robustness to poor audio.

6/10

PassSample data

Scenario script

Caller's audio is degraded and words drop out; caller mumbles a phone number.

Rosie — Bad Connection

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Northside Plumbing.

  2. Caller

    The— [drops] —won't stop running.

  3. Rosie

    Sorry, could you say that again?

    Evaluator: Sample annotation: asks to repeat.

  4. Caller

    The toilet won't stop running.

  5. Rosie

    Got it. Callback number?

  6. Caller

    555-01... [mumbled].

  7. Rosie

    Can you repeat the last four digits?

  8. Caller

    0142.

  9. Rosie

    555-0142. Thanks.

Pass / fail criteria

  • Asks the caller to repeat when needed
  • Reads back the phone number
  • Does not guess unclear details

Observations

  • Sample: a passing run asks the caller to repeat.
  • Sample: the number is read back.
  • Sample: the deduction illustrates one unclear word accepted.

Key moments

  • Dropout
  • Number mumbled
  • Read-back

Evaluator notes

Sample: placeholder notes for the degraded-audio scenario showing a marginal pass. Sample content only. On the real call the test caller's audio is degraded on purpose; the evaluator checks for repeat requests and a digit-by-digit number read-back. Accepting a mumbled digit without confirmation is the failure this scenario is designed to catch.

Call 10 of 10

Qualified Lead

Tests whether a valuable caller is recognized and routed.

6/10

PassSample data

Scenario script

Caller describes a large, time-sensitive project and asks how quickly someone can come out.

Rosie — Qualified Lead

Recording pending

No recording has been attached yet. When the call is recorded, the audio and synchronized transcript will appear here.

Transcript
  1. Rosie

    Northside Plumbing.

  2. Caller

    We need a full repipe done in three weeks before closing. How fast can you get out?

  3. Rosie

    I'll get this to the owner right away. Address?

    Evaluator: Sample annotation: captured, but no distinct priority flag in this sample.

  4. Caller

    2200 Sample Avenue.

  5. Rosie

    Name and number?

  6. Caller

    Sam Okafor, 555-0177.

  7. Rosie

    Thanks. The owner will call you back.

Pass / fail criteria

  • Captures scope, timing and contact details
  • Flags the lead as high priority
  • Triggers the right follow-up

Observations

  • Sample: a passing run captures scope, timing and contact.
  • Sample: the deduction illustrates no explicit high-priority flag on the message.
  • Sample: an owner callback is triggered.

Key moments

  • Project described
  • Details captured
  • Message taken

Evaluator notes

Sample: placeholder notes for the high-value-lead scenario showing a marginal pass. Not a Rosie observation. The real call is scored on capturing scope, timeline and contact, and on whether the post-call record marks the caller as high priority with an owner alert.

For the editorial review and ranking position of Rosie, see AI Receptionist Report.

Sources

  1. Scenario catalog and rubric