Demos are built to impress. These are the questions that tell you what you'd actually be living with — including the ones vendors would rather you skip. We've answered each one for ourselves, including the one where our answer is no.
Every AI phone demo sounds good. That's what a demo is for: a clean script, a cooperative caller, no background noise, no edge cases. It tells you almost nothing about the Tuesday in month four when a patient with an accent calls from a parking lot with a real problem.
These twelve questions are the ones we'd want asked of us. Ask them of every vendor you're considering, including us. Where our own answer is worth stating plainly, we've said so — including question ten, where the honest answer is "no, not today."
This is the fastest disqualifier in the whole list. If a vendor can hear, transcribe, or store your patients' health information, HIPAA generally treats them as a business associate. A healthcare-ready company answers this in one word. Hesitation, "we're working on it," or a pivot to how secure their infrastructure is means the answer is no.
"In the cloud" is not an answer. Ask which systems hold the data, whether their staff can read your patients' calls, whether that access is logged, and how long recordings are retained by default. Then ask whether you can get your data out — and delete it — on request.
Some vendors treat "our AI is indistinguishable from a human" as the headline feature. For a medical practice, that's a liability dressed as a benefit. Recording-consent law varies by state — several, including Massachusetts, require all parties to be aware — and patients deciding whether to describe a health problem deserve to know what they're describing it to.
This is the question that separates products. Every system handles the clean call. Ask specifically: what does it do with an unclear request, an unusual accent, a caller who interrupts, or a question it has no answer for? The good answer is that it gracefully captures the caller's details and flags the call for a human — not that it guesses, and not that it loops the caller through the same prompt.
Ask them to walk you through a caller describing swelling and difficulty swallowing. You want to hear two things: that the system does not attempt clinical judgment, and that a true emergency is pointed to 911 or an emergency room rather than added to a callback list. A vendor whose AI offers reassurance or advice here is a vendor to walk away from.
Your problem isn't the average Tuesday, it's Monday at 9am and the first day back after a holiday. If simultaneous calls are capped, or cost extra, you'll discover it on exactly the day you can least afford to.
Ask to see the actual screen your front desk would use, not a marketing slide. Can they scan the day in five seconds? Are new and urgent calls visually distinct? Do notifications go out in real time, and can each team member control what they get? A system that produces a daily email digest is a system that quietly recreates the voicemail problem.
Everything has outages. What matters is the designed behavior when one happens. The answer you want is that calls automatically fall back to your existing main line, so the phone still rings at the practice. A vendor who has never considered this question has told you something important.
Your hours change. A provider joins. You stop taking a plan. Ask who edits what the AI says, how long a change takes, and whether edits cost money. "Submit a ticket, allow 5–7 business days" is a real answer some vendors give, and it's disqualifying for a practice that changes anything.
Ask this one bluntly, because it's where the most optimistic claims live. If a vendor says yes, ask which systems specifically, what happens when the integration breaks, and what stops a double-booking at 2am. A shaky write-integration into your live schedule can be worse than none at all.
Per-minute pricing is where surprises live. Ask what's included, what a minute costs beyond that, whether the AI's talk time is billed differently from a transfer, and what a genuinely busy month looks like on the invoice. Then ask whether features are gated by tier — some vendors reserve the useful parts for the expensive plan.
Ask about contract length, what happens to your data if you leave, and whether you can trial the thing on a real phone line before committing. A vendor confident in the product doesn't need to lock you in for a year.
Call the vendor's own main number, after hours, as if you were a patient. Whatever you hear is what they actually believe about answering the phone.
Don't send all twelve as an email questionnaire — you'll get marketing copy back. Ask them live, in the demo, and listen for the shape of the answer rather than the content. Vendors who've done this seriously answer quickly and specifically, and volunteer their own limitations. Vendors who haven't will reframe every question as a feature.
The one that tells you the most is question ten. Not because the answer needs to be yes, but because it's the easiest place to overclaim — and how a vendor handles a question they can't fully win is a reasonable preview of how they'll handle a problem in month six.
Book a short demo and put this list in front of us. We'll show you the actual dashboard, play a real call, and tell you plainly where we fit and where we don't.
Get a free demo →
