Three different things called the same thing
Hospitals deploying a messaging front door are usually offered a single product that spans three quite different functions. The first is transactional: booking appointments, sharing reports, sending reminders, answering questions about timings and location. The second is navigational: helping a patient work out which department or specialty is appropriate. The third is clinical: assessing described symptoms and advising on urgency or action.
These carry very different risk and very different obligations, and conflating them is the source of most of the trouble in this area. A booking bot that malfunctions inconveniences someone. A symptom assessment that tells a patient with serious illness that they can wait is a clinical event, and the fact that it was delivered by a chat interface does not change that.
Establish which of the three you are deploying, in writing, before evaluating any product. A hospital that intends the first and buys something that also does the third has acquired obligations it has not considered, and the discovery usually happens after go-live.
When a tool becomes something more than convenience
The line that matters is whether the tool is intended to inform a clinical decision. A system that asks about symptoms and outputs a suggested urgency or a recommended action is functioning clinically regardless of how carefully its disclaimers are worded, and the regulatory framework that applies to software intended for a medical purpose is a genuine consideration rather than a formality.
Where a tool falls in that framework depends on its intended purpose and claimed function, and it is a question to put to the vendor directly and in writing: what is the stated intended use, is the product registered or licensed in India, and at what classification. A vendor who answers vaguely, or who asserts that disclaimers place the product outside the framework, is telling you to verify independently.
There is also the separate question of whether an automated interaction that advises a patient sits comfortably alongside the professional framework governing telemedicine, which contemplates a registered practitioner exercising judgement. A hospital deploying automated advice should be clear about whose clinical judgement is being exercised and how, and should take its own advice on that rather than relying on the vendor's characterisation.
Questions to put to a vendor in writing
- What is the product's stated intended use, precisely worded
- Is it registered or licensed in India, and at what classification
- What clinical validation exists, on what population
- How are outputs generated, and can that be explained to a clinician
- What is the vendor's position when an output contributes to harm
Safety netting, which is the design that matters most
Whatever the tool does, the design question that most affects patient safety is what happens when something serious might be present. Any patient-facing automated interaction needs an unmissable route to urgent care and a bias toward escalation when the situation is unclear, and that route has to be present at every step rather than at the end of a flow.
Design the failure direction deliberately. A tool that errs toward advising in-person assessment generates unnecessary attendances, which is a capacity cost. A tool that errs toward reassurance generates the other kind of error. Those are not symmetrical and the design should reflect that plainly, even though the cautious direction is the one that makes the tool look less impressive in a demonstration.
Handle the abandoned conversation, which is the case nobody designs for. A patient who begins describing symptoms and stops mid-flow has left the interaction in an undefined state, and the tool has no idea whether they went to hospital, felt better, or deteriorated. At minimum, an interaction that surfaced concerning content and was not completed should produce a clear closing message with the urgent route, and arguably should be visible to a human.
“We tested it by abandoning conversations halfway through. Nothing happened at all — no message, no flag, nothing. That was the finding that stopped the deployment until it was fixed.”
Handover to a human, which has to actually work
Every automated front door needs a route to a person, and the quality of that route determines whether the whole thing is acceptable. It has to be available at any point, obvious rather than buried, and it has to reach someone within a stated time during stated hours, with an explicit and honest statement of what happens outside them.
The commonest failure is a handover that reaches a queue nobody staffs. A patient who asks for a person, is told someone will respond, and hears nothing until the following morning has been given a false assurance, and the false assurance is worse than an honest message stating that the service operates during defined hours and directing them elsewhere meanwhile.
The handover must carry the conversation with it. A patient who has described their situation once should not have to repeat it, both because repetition is a poor experience and because the detail may be clinically relevant. Transcript, structured summary and any flags raised should be visible to whoever picks it up.
Consent, channels and what these platforms retain
Messaging channels raise their own questions distinct from the tool's clinical function. Establish the basis on which you are messaging the patient, capture it properly, and honour opt-out across every channel. Distinguish care communication, which a patient generally expects, from promotional messaging, which requires separate agreement and separate withdrawal.
Be conscious of who else may see the message. Phones are shared, particularly in family contexts, and a message disclosing a department name or a condition may disclose more than the patient intended. Keeping message content minimal and holding detail behind an authenticated step is a straightforward design choice that avoids a foreseeable harm.
Understand what the messaging platform itself retains and where. Conversations on a third-party channel exist in that provider's infrastructure under their terms, and content that includes symptom descriptions is sensitive personal data. Decide deliberately how much clinical content is permitted to exist on the channel at all — a good default is that the channel handles transactions and directs anything clinical into an authenticated environment.
Channel decisions to take before launch
- The recorded basis for messaging, and how opt-out is honoured
- Care communication kept separate from promotional consent
- Message content minimised against the shared-phone risk
- What the platform provider retains, where, and under whose terms
- How much clinical content may exist on the channel at all
Monitoring what the thing actually says
Automated front doors are deployed and then rarely examined, which is unwise for anything that talks to patients unsupervised, and particularly so where generative components mean the output is not fully enumerable in advance. Review a sample of real conversations regularly, with a clinician involved, looking at what the tool said rather than only at whether users completed flows.
Track the outcome-relevant signals rather than engagement metrics: how often the urgent route was surfaced, how often handover was requested and how quickly it was met, abandonment at points where concerning content had been raised, and any complaint or incident referencing the tool. Engagement metrics tell you the product is being used, which is not the question.
Establish a fast route to withdraw or restrict it. If a pattern of unsafe or inappropriate responses emerges, someone needs the authority and the mechanism to switch it off within hours rather than raising a change request. Decide who that is before launch, because deciding during an incident is how a tool stays live for a week longer than it should have.
A defensible starting scope
The scope most hospitals should start with is the transactional one: appointments, reminders, report availability, directions, timings, and simple administrative questions. It carries modest risk, delivers real value to patients and to the call centre, and builds the operational competence to run a channel well before anything clinical is contemplated.
Navigational assistance is a reasonable extension, provided it is genuinely navigational — helping someone find the right department — and stops short of assessing symptoms. The boundary between the two is easier to state than to hold, and it needs enforcing in the design rather than trusted to the model, because a conversational system will follow a patient describing symptoms unless something prevents it.
If you do go further, do it as a governed clinical deployment with the validation, oversight, monitoring and ownership that implies, not as an enhancement to a chatbot project. Record the decision and its basis in your AI governance register alongside your other use cases, so that the scope you chose is documented and the next person to propose extending it starts from what was already decided and why.
