Skip to content


Cyan circle with white stethoscope icon inside and cyan swooshes on either side

Responding to Issues: Chapter 11

Cover of PROTEUS Guide

In this excerpt (Chapter 11) from the PROTEUS-Practice Guide, you’ll consider options for alerting patients and providers to potentially concerning patient-reported outcome measure (PROM) scores.

This webpage contains the entire contents of Chapter 11. You can also download the PROTEUS-Practice Guide by clicking here.

Key Points

  • Clinical teams must be equipped to respond to alerts or refer to other support services prior to integrating patient-reported outcome measures (PROMs) into clinical care
  • Evidence-based cut points are already established for some measures, and these cut-points can be used to inform when providers are notified about possibly concerning scores
  • Scores that may be critical in some contexts might be normal or expected in other contexts, and this context is important when determining how to respond


It is important to establish a plan for responding to PROM data prior to integrating PROMs into clinical care. Groups must first consider how PROM data will be integrated into the patient record, and specifically whether providers need to actively accept the PROM information, or whether that information can be passively added to records or pushed to patient portals without additional provider vetting. It is also important to consider what notifications a provider should receive about new PROM data, including no notification at all, notification for everything, or notification only for specific alerts. These alert systems should be co-designed with the patient and clinical team.

It is important to differentiate what is typically considered a critical score from that which is critical (or normal) for a particular patient when determining a response. Determining what threshold constitutes an alert is a significant challenge, and both patients and providers want to maximize the chance that providers see possibly concerning results while also minimizing the number of alerts they receive.

Evidence-based cut points are already established for some measures, and these cut points can be used to inform when providers are notified about possibly concerning scores. For the many PROMs that do not have established cut points, there are rigorous quantitative and qualitative approaches that can be used to establish cut points, or more rule-of-thumb values that might be used.

The clinical team must be equipped to respond to alerts. Discussing possibly concerning scores with patients is advised. Some validated thresholds may be mapped onto a need for specific clinical actions. Establishing clear pathways can reduce uncertainty in clinical decision-making. When clear pathways do not exist, clinical team members should explore the identified issue(s) with the patient and can engage multidisciplinary care team members to inform next steps. In some instances, reassurance and acknowledgement of difficult-to-manage symptoms may be helpful.

PROM data can be communicated to patients, their proxies, primary care providers, the provider “ordering” the PROM, patient navigators, or any combination of these groups. Who receives the PROM data has implications for how the information will be included in care. Modalities for communicating about PROM data include email, secure clinical notes, and text/page.

Questions and Considerations

A. Should providers have to “accept” PROM data into the record?


  • In this configuration, PROM data can be moved into the health record without it being actively “accepted” by a provider
  • This approach allows for the easy integration of data into a patient-reported outcome (PRO) system
  • This approach reduces burden on the healthcare provider as they do not need to accept potentially large volumes of data that do not require a response
  • This configuration may lead to more regular and meaningful use of the data
  • The medical record should reflect what information was provider-input in comparison to what data was patient-input
  • Concerning results need to be flagged for provider review


  • In this configuration, providers are required to review and accept PROM data when it is included in the record
  • This approach increases the likelihood that PROM data is correct and ensures that providers see all the PROM data
  • Reviewing and incorporating PROM data into the record is time consuming, and may inadvertently lessen the utility of this information

B. What kinds of notifications should providers receive about PROM data?

No notification

  • One approach is to consider PROM data like vital sign or other information that gets imported into the system without notification
  • Potentially concerning data could be flagged, but the provider would not get a pushed notification of the flag
  • This approach does not incur any added workload and does not interfere with workflow, but runs the risk of providers missing important clinical information
  • There are potential liability concerns if the patient believes that data will be reviewed when this might not be the case

Notification of all PROM information

  • Providers receive a notification every time new PROM information is entered into the system
  • Notification modality is usually not flagged as urgent, and is delivered as an email or clinical message
  • Ideally notification should include the PROM results themselves
  • As compared to receiving no notification at all, receiving notifications makes it less likely that PROM data will be missed. However, at high volumes it may be possible that results are ignored
  • This approach may be most effective in a low-volume setting

Notification of “alert” PROM information

  • Alert notifications can be used to indicate to a clinician that a patient requires follow-up, such as if the alert indicates they are experiencing chest pain or severe depression
  • Systems should carefully consider whether they want to assess symptoms that would require immediate action
  • Open-ended alerts will notify the provider but do not require that the provider electronically close the alert
  • Closed-loop alert notifications require that the provider electronically close or address the alert
  • Alerts to providers can be sent at the same time as notifications are sent to patients advising them on next steps for their care; it is also recommended that patients be warned not to report urgent issues via the PRO system and to contact their provider directly for urgent issues
  • Notifications to patients should be co-designed with patients and follow user-centered design practices
  • As compared to receiving a notification of all PROM information, only notifying providers with alerts may be more likely to bring attention to a given PROM finding
  • Notification of an alert requires that some sort of action be taken to address the alert. It is not always clear what this action should be

C. What approaches can be used to select alert and other clinical thresholds?

Select measures with existing, validated severity thresholds

  • PROMs that have validated severity thresholds can be purposively selected
  • Depression and anxiety measures are more likely to have thresholds compared to other PROMs

Develop severity thresholds for measures without them

  • Validated severity thresholds have not been established for many PROMs
  • PRO-bookmarking, Rasch measurement theory, anchor-based methods, and other approaches can be used to identify severity thresholds
  • Consensus-based approaches, such as Delphi panels, are increasingly used to identify appropriate cut points

Use rule-of-thumb cut points

  • Re-scaled PROs with scores that can be mapped onto a 0-10 scoring system can be very generally grouped as: 0 = none, 1-3 mild, 4-7 moderate, and 8-10 severe
  • >=4 or >=5 are often used as a cut point for general alert to a provider

D. What modality is used to communicate a result?


  • Non-secure email is not typically recommended, but may be acceptable when using a 3rd party PRO system that is not integrated with the electronic health record or other clinical messaging systems
  • In cases when email can be used securely, email can be beneficial as it does not need to be integrated with the electronic health record and can be accessed with routine login information

Secure clinical messages within the electronic health record

  • Clinical messages within the electronic health record can be easily used if the PRO system is embedded within the electronic health record
  • Clinical messages are secure, automatically documented, non-interruptive, and are easily accessible within the electronic health record platform
  • Clinical messages require access to a specific network or login through a virtual private network

Texts, secure texts, page

  • Both 3rd party and electronic health record systems can often be integrated with a text/paging platform
  • These options are secure, and present information in real time
  • The real-time delivery of results can be either useful or interruptive

E. What methods can be used to respond to alerts identified by PROMs?

Discuss PROM scores with patients to better understand their experiences

  • Generally recommended to better understand patients’ PROM scores. In some cases PROM scores may be related to issues outside of the specific clinical context (e.g., limited physical function related to a broken ankle rather than a cancer diagnosis is more relevant to the orthopedist than the oncologist)
  • This approach can be time-consuming but may also help patients holistically

Follow a disease management pathway

  • Established disease management pathways can be used to easily identify clinical next steps
  • Having established pathways reduces uncertainty and creates uniform responses to address patient needs
  • Creating new pathways is a time consuming and challenging process

Engage multidisciplinary team members

  • Healthcare professionals from different disciplines, including different medical specialties as well as from groups such as social workers, nutritionists, and counselors, can be engaged to troubleshoot patient problems or identify next steps for care
  • It can be time-consuming and difficult to coordinate multidisciplinary team members to address patient problems

F. Who should be notified of PROM results?

The patient, their proxy, or their choice of providers

  • As the ones experiencing the outcome, patients are the ultimate owners of their data and may be interested in receiving their PROM results
  • It can be difficult to provide the full context of the results to patients when not accompanied by contextual information
  • Patients can also select who they would like the PROM results to be shared with, including who within their clinical care team they would like to inform

Ordering provider

  • The provider who requested the patient complete the PROM could be the individual alerted in the event of a problem
  • Typically, the problem detected would be relevant to the ordering provider, and ordering providers would like to see the results
  • It can at times be difficult to identify the ordering provider within the PRO system

Primary care providers

  • Primary care providers are often considered responsible for a patient’s overall health and therefore may be more likely to do something when presented with a potentially concerning result
  • Patients have preexisting relationships with their primary care providers, which may increase primary care providers’ willingness to address the concerning health problem
  • It may be overwhelming for primary care providers to be tasked with addressing results from PROMs for all patients

Navigator or administrator

  • Patient navigators or administrators sometimes take on the role of organizing information across many patients and ensuring that information is relayed to the correct clinical party
  • Navigators or administrators can be notified of the results and can then route this information to the appropriate member of a patient’s care team
  • This approach can be administratively challenging

G. Are PROMs ready to drive clinical decision-making?

Yes, if PROM scores can be meaningfully interpreted and tied to an action

  • Translating PROM responses into actions is valuable
  • Requires that that there is sufficient evidence supporting linking a clinical action to a PROM score
  • When used correctly, PROM scores can be used to improve clinical decision support and can increase likelihood of completed PROMs in the future

Not yet, if PROM scores are unclear or cannot be tied to a clinical action

  • Most PROMs have not been used widely enough to be associated with clearly defined clinical actions
  • More experience with using a given PROM to inform clinical care may be required before using the measure to inform clinical action
  • Urgent alerts should not be sent for PROM data that does not have an associated action
  • If a PROM is directly tied to a clinical action, it may be considered a medical device in some countries and subject to regulatory requirements
  • PROMs can still be used to inform discussions, even if not tied to a clinical action

Relevant Primary Resources

The information presented here is an overview of responding to PROMs. For more detailed information please see the following sources:

Background And Citing The Proteus-Practice Guide

Nothing in this Guide should be construed to represent or warrant that persons using this Guide have complied with all applicable laws and regulations. All individuals and organizations using this template have the responsibility for complying with the applicable laws and regulations or regulatory requirements for the relevant jurisdiction.

Each chapter of the Guide lists the key foundational resources that informed its content. To appropriately recognize the foundational resources, we encourage you to cite both the Guide and the relevant foundational resource(s). Recommended citations are provided here.

Suggested Citation

The PROTEUS Guide to Implementing Patient-reported Outcomes in Clinical Practice
A synthesis of resources. Prepared by Crossnohere N, Brundage M, Snyder C, and the Advisory Group, 2023. Available at:

Further Reading

The Guide draws primarily from the foundational resources cited in each chapter. Please click here to find a selection of other relevant references.

Back To Top