PROTEUS — PRACTICE
Identifying, Engaging, and Training End Users and Stakeholders: Chapter 3
In this excerpt (Chapter 3) from the PROTEUS-Practice Guide, you’ll see how integrating the feedback from relevant stakeholders can improve the design, development, and implementation of your patient-reported outcome (PRO) system.
This webpage contains the entire contents of Chapter 3. You can also download the PROTEUS-Practice Guide by clicking here.
Key Points
- Numerous relevant perspectives should be engaged in the design, development, and implementation of a patient-reported outcome (PRO) system
- Training activities can build capacity for robust engagement with PRO systems
- The content and modality of training might vary based on the goals of the training and the role of the stakeholder
- Participation in PRO systems can be motivated by demonstrating the value of PROs to clinical care, and reducing barriers to effective system use
Overview
There are numerous relevant perspectives to consider when developing a PRO system: those who provide and may directly benefit from the use of PROs in clinical care, such as patients and their clinical teams; those who design PRO systems, such as electronic health records (EHR) and PRO specialists; and those who use PRO data to inform organizational or research initiatives, such as patients, clinical team members, and administrators. Including the perspectives of these stakeholders contributes to improved PRO systems. Stakeholders can advise on topics ranging from what outcome data to collect and what measures to use to collect them, to the development and design of the PRO system itself, including how data is collected, stored, and integrated into care.
Achieving the benefits of diverse stakeholder participation often requires training to build capacity so that their perspectives can be most meaningfully integrated. If patients are included in the co-design it is important to reflect on how they are similar to or different from the representative population. Common goals of training include educating stakeholders on the value of PROs in clinical care, teaching specific functionalities of a PRO system, and determining how and when to collect and integrate PRO data into clinical care. Training can occur through predeveloped materials and/or live training sessions, depending on the goals of the training and the level of familiarity and engagement stakeholders have with the PRO system. In conjunction with training, stakeholders can also be engaged in the process of designing PRO systems.
Motivating stakeholders to participate in PRO systems is essential to system sustainability. The optimal motivational approach to encourage use of a PRO system may vary based on the type of stakeholder. For instance, for clinical team members, motivators might include identifying local champions of the system, and building a system that seeks to reduce, rather than increase, burden. For patients, motivation to use the PRO system may come from clear communication around how and why PRO data will be used to inform care, prompting patients to use the PRO system, making the PRO system easy-to-use, and providing patients with incentives for doing so.
Questions and Considerations
A. Whose perspectives should be included in the development of a PRO system?
A range of stakeholders
- Relevant perspectives on PRO systems can include anyone involved in the development, collection, or use in clinical settings
- Including individuals with a range of perspectives will facilitate better PRO systems that meet the needs of diverse groups
- Consider guidelines for promoting diversity, equity, inclusion, and accessibility when identifying and engaging with stakeholders
- Examples of potentially relevant perspectives to include in developing PRO systems include:
- Patients
- Caregivers
- Administrators
- Administrative support staff
- Providers
- Clinical champions
- Clinical staff
- Electronic health record designers
- Informaticians
- Operation leads
- PROM specialists
- Researchers
B. What topics can stakeholders advise on?
- Identifying how data can be used to support care
- Selecting symptoms/experiences to assess by PRO system
- Determining appropriate frequency of assessment
- Selecting specific PROMs
- Identifying clinically concerning PROM scores
- Identifying appropriate alert thresholds
- Developing recommendations for acting on PRO results
- Creating a ‘wish list’ of attributes for a PRO system
- Identifying barriers and proposing solutions to overcome them
- Optimizing presentation/visualization of PRO data
- Co-designing the system/interface
- Using PRO data for clinical decision-making
- Developing guidelines for PRO system design
- Addressing accessibility issues for the target population
- Considering the cultural acceptability of PROMs
C. What are common goals of PRO system training?
To understand the value of PROs
- Collective understanding of why PROs are being collected at a clinical/organizational/or other level may foster buy-in
To understand how to use a PRO system
- Provide the technical knowledge to operate a PRO system, such as how to collect and input data, retrieve data records, visualize results
To understand how to interpret and integrate PROs into clinical decision-making
- Using PRO data to inform shared decision-making conversations between patients and providers
- Providing notifications/alerts for concerning results
- Referring patients/clinicians to decision support tools within the electronic health record
- Integrating score interpretation into the PRO report/electronic health record
To specify the context for PRO use
- Determining, for instance, which patients, what setting, and what timepoints PRO data are being collected
- Considering whether and how current practices within a specific context might need to be adapted to accommodate a PRO system, or vice versa
D. What modalities are available for training?
Live training modalities
- Group sessions may be appropriate when initiating a PRO system as it may not yet be clear what questions users typically have
- One-on-one sessions may be useful to teach specific or advanced PRO system features
Pre-developed modalities
- Pre-developed modalities can be useful if there is high staff turnover or if the training program is resource intensive. Pre-developed materials can be easily shared without the need for one-on-one training
- These modalities may include videos, decision-trees, and information summary sheets, for example
E. How do you motivate provider and staff end-users to use PRO systems?
Identify local champions
- Identify a champion who has a long-term interest in the success of PRO use and who will help sustain the PRO strategy over time
- Champions may be individuals with experience using PROs and PRO data
- Champions may also become trainers who can administer trainings internally and reduce the need to hire outside trainers
Reduce burdens
- Engage stakeholders to suggest approaches for bundling PRO collection with other data collection to lessen burden
- Continue to support existing users and train new providers
- Optimize PROM data display within electronic health records to allow for easy access and inclusion of other clinical data (e.g. vital signs, procedure dates) (see Chapter 10, Visualizations to Aid Interpretation)
Continuous quality improvement
- Audit PRO systems and provide feedback such as benchmarking to other clinics/organizations
- Incentivize providers to collect, view, and use PROs in care, by, for example, offering PRO use as a quality project or malpractice reduction credit
- Redesign the workflow iteratively as needed to optimize the system
- Engage providers and teams to evaluate the PRO program
Publicly celebrate successes
- Demonstrate how use of PROs improved quality of care and outcomes
- Convey the benefits and successes of the program at clinical/organizational meetings
F. How do you motivate patient end-users to use PRO systems?
Demonstrate to patients the value of PROs for their care
- Most importantly, review and discuss results with patients
- Develop informational materials like pamphlets or flyers that describe why PRO data are collected and how they are used to inform clinical decision-making
- Develop scripts about the value of PROs and tailor them for different members of the care team
Incentivize patient participation in PRO systems
- Ensure that patients’ results are discussed with them by a member of the care team
- Create patient friendly reports and data displays that can be effectively used in care, such as by displaying results over time
- Make sure patients can access their PRO results
- Demonstrate how PROs can be used for self-management
- Encourage patients’ personal motivation to complete PROs
Prompt patients to use PRO systems using these mechanisms
- Scheduler prompts when making the appointment
- Reception staff prompt upon arrival at the appointment
- Nurse champion/medical assistant/clinical support staff prompt during appointment
- Electronic systems prompt via reminders, patient portal messages
- Provider prompts when seeing the patient
- Research coordinator prompts, if patient is part of a study
- Patient navigator prompts
G. How might stakeholders be engaged in developing PRO systems?
Using methods for engagement
- Interviews or conversations with stakeholders about how to design and develop systems
- Delphi-methods to reach decisions across groups
- User-centered design strategies to inform the development of reports/visualizations that are informative and interesting
- Provider discussions with patients at the point of care regarding the value of PRO data to inform the patient’s individual care
Following a philosophy of engagement
- Create a “feedback loop” to increase transparency, improve trust, foster bi-directional conversation, and demonstrate how stakeholder input informs decision-making
- Engage stakeholders throughout the entire lifecycle, from conceptualization to implementation, evaluation, and sustainment
- More generally, advance health system capacity for patient engagement
Relevant Primary Resources
The information presented here is an overview of how to engage stakeholders. For more detailed information please see the following sources:
Background And Citing The Proteus-Practice Guide
Nothing in this Guide should be construed to represent or warrant that persons using this Guide have complied with all applicable laws and regulations. All individuals and organizations using this template have the responsibility for complying with the applicable laws and regulations or regulatory requirements for the relevant jurisdiction.
Each chapter of the Guide lists the key foundational resources that informed its content. To appropriately recognize the foundational resources, we encourage you to cite both the Guide and the relevant foundational resource(s). Recommended citations are provided here.
Suggested Citation
The PROTEUS Guide to Implementing Patient-reported Outcomes in Clinical Practice
A synthesis of resources. Prepared by Crossnohere N, Brundage M, Snyder C, and the Advisory Group, 2023. Available at: TheProteusConsortium.org.
Further Reading
The Guide draws primarily from the foundational resources cited in each chapter. Please click here to find a selection of other relevant references.