Leveraging AI and ML to Identify Urgent Patient Portal Messages

Baystate Health, a not-for-profit, integrated healthcare system serving over 800,000 people throughout western New England, uses an AI/ML-based methodology to automatically detect urgent incoming messages sent via its MyBaystate patient portal. The approach improves patient care, reduces risk, and boosts staff productivity by enabling contact center agents, nursing and clinicians to quickly, easily, and reliably identify potentially time-sensitive patient communications.

Baystate partnered with InterSystems to make their AI/ML vision a reality. InterSystems developed an AI model using large language model (LLM) to support the project, based on Baystate’s input. A multidisciplinary team comprised of Baystate professionals and InterSystems subject matter experts worked closely together to refine and train the model.

InterSystems also supplied data science and AI expertise and engaged with Baystate physicians and executive stakeholders to review the benefits of the technology and alleviate potential misgivings and misconceptions.

Baystate uses InterSystems Personal Community as the foundation for its MyBaystate portal. The portal improves patient experiences and satisfaction by allowing patients to view their care records and test results, request appointments and prescriptions, and communicate with providers—easily and securely.

The patient portal is intended strictly for non-urgent purposes. It includes prominent notifications instructing patients not to use the portal for medical emergencies. Still, patients occasionally do just that. In fact, about 6% of patient-portal messages are deemed clinically urgent by Baystate staff.

Historically, these time-sensitive messages were not prioritized for nurses at Baystate medical practices and health centers, or for contact center agents who schedule appointments and handle other routine inquiries. Recipients had no way to determine which of their many messages required prompt attention by the clinical team without reviewing each message individually.

“The portal is very popular with our primary care patients,” explains Jennifer Barrett, Contact Center Director, Baystate Health. “It’s not unusual to show up on a Monday morning and have 120 appointment requests that came in over the weekend. Our agents had to look at each one individually to make sure it wasn’t urgent. We make it really clear that patients should not use the portal to schedule emergency appointments, but it still happens.”

Nurses in ambulatory offices faced similar challenges triaging incoming messages to determine if they needed to be sent to a clinician for prompt attention. And even once a nurse identified and forwarded an urgent message, there was still a chance it could go unnoticed in the busy clinician’s crowded Oracle Health¹ inbox given clinicians can be time-constrained and often receive a substantial number of EHR messages.

With the help of InterSystems, Baystate implemented an AI/ML-based system to automatically identify urgent messages in the EHR and CRM platforms. Time-sensitive messages are now tagged with an asterisk and are easily spotted by contact center agents, medical assistants, nurses, doctors, and other team members.

An interdepartmental Baystate team reviewed a collection of historical patient communications to label messages that should be classified as urgent in the AI model. Because the AI model is based on real-world patient interactions it accurately reflects the local language characteristics of Baystate patients and can be readily extended to other applications in the future.

“The InterSystems people and partnership were fantastic,” says Ken Riley, Director of Patient Technologies, Baystate Health. “We didn’t have a lot of internal AI expertise or any prior experience using large language models. The InterSystems team developed the model and met with our stakeholders to explain how it worked and why it was scientifically sound.”

Baystate conducted a pilot program to validate and tune the model prior to full-scale production deployment. This helped reduce misses and false positives and resolve difficulties the model initially had understanding context. (e.g., the model might falsely flag the message “several months ago I went to the ER with severe stomach pain” by keying in on the words “stomach pain.”)

“I was amazed at how well the technology worked and how it was able to accurately recognize urgent symptoms,” recalls committee member Michelle Phillips, RN, Director Ambulatory Nursing, Baystate Health. “It’s pretty incredible how the model learns and gets even more accurate over time.”

The AI module, which runs on an InterSystems technology stack, is now live in production, supporting four health centers and twelve primary-care practices. The solution helps improve care quality by enabling Baystate to identify and respond to urgent patient communications much more quickly and efficiently.

“The project has been a big success for us. It is like an extra pair of eyes, which is extremely helpful, particularly on really busy days,” explains Barrett. “We recently had a situation where a patient reported chest pain via the portal, and we were able to get them the right medical care very quickly, because the message was flagged as urgent.”

“The clinicians have been very happy with it as well,” adds Phillips. “It helps optimize workflows at the practice and makes it easy for us to identify portal messages that need a quick call-back.”

Going forward, Riley and his team plan to use AI to improve care quality and staff productivity even further. “The next step is to automatically forward appropriate messages to the right department or clinician to reduce human latency, free up resources, and speed up responses,” explains Riley. “Beyond that we’re looking at using AI and LLM for a number of different use cases like automatically responding to non-urgent messages and automatically flagging quality measure compliance status for reporting.”

---

1 - Formerly Cerner