PROTEUS — PRACTICE
Selecting PROs and PROMs: Chapter 5
In this excerpt (Chapter 5) from the PROTEUS-Practice Guide, you’ll consider how to choose patient-reported outcomes (PROs) and measures (PROMs) relevant to the goals of your PRO system.
This webpage contains the entire contents of Chapter 5. You can also download the PROTEUS-Practice Guide by clicking here.
Key Points
- At the outset, there should be a clear rationale for patient-reported outcome (PRO) assessment and plan for use of the data. The outcomes selected should be relevant to patients and advancing the goals of the PRO system
- After identifying the outcomes of interest, stakeholders then select specific patient-reported outcome measures (PROMs). The ideal PROM has several characteristics, including that data from the PROM can be used to meet the goals of the PRO system, is psychometrically robust, and can be feasibly implemented in the context of interest
Overview
At the outset, stakeholders should consider what outcomes to collect in the PRO system. The information to collect depends on the relevance of the content to advancing the goals of the overarching PRO system, and to patients and other stakeholders. In addition to identifying what clinical area the outcome data is tied to, stakeholders should also consider whether they are interested in aspects such as symptoms, physical function, treatment, tolerability, or broader quality of life, as well as the type of outcome they are interested in (e.g. frequency, severity, bother).
After identifying the outcomes of interest, stakeholders can then select specific PROMs. The ideal PROM addresses the goals of the PRO system, is psychometrically robust, and can be feasibly implemented into the context of interest. Decision-makers must balance their interest in using a specific measure with measurement properties of that measure. In some cases, there may be many PROMs from which to choose. In other cases, no “perfect” PROM is available; this is especially the case when trying to use a single measure to meet multiple goals (e.g. it may be hard to find a measure that is relevant, psychometrically robust, and feasible to collect, and which also meets both the goals of informing patient care and value-based purchasing).
Outside of the content area, other attributes of PROMs that might influence their selection include whether the measure is generic or disease-specific, whether the measure is profile or preference-based, how burdensome it is for the patient to complete, how burdensome it is to collect and integrate into the PRO system, and the costs/licensing agreements required for use.
Harmonizing PROMs across the healthcare system can be useful to avoid including multiple PROMs within the same domain, minimize data collection and processing, and streamline PRO efforts. However, harmonizing approaches produce challenges, including less flexibility in measure selection by clinicians and researchers, and potentially less relevance to patients.
Questions and Considerations
A. How do you decide what PROs to collect?
Ensure that the outcomes are relevant to stakeholders
(see Chapter 3, Identifying, Engaging, and Training, Users and Stakeholders)
- Engage stakeholders in the process of identifying what outcomes to assess
- Include patients and caregivers in the PRO selection process for the co-production of healthcare
- Involve the clinical team to foster buy-in to collect and use the PRO data
- Involve research/analyst early in planning to support the selection of measures with evaluable data
- Challenges with engaging stakeholders include:
- Different perspectives across stakeholders on the relevance, purpose, and value of a given PRO/PROM and its ability to advance the goals of the PRO system
- Time and resource requirements
- Limitations in generalizability of input
Ensure that information on the outcomes will advance the goals of the PRO system
- If the goal of the system is to improve patient care, then the outcomes to be measured should be relevant to patients, and be useful to providers to inform clinical decision making (see Chapter 1, Defining Goals)
- If the goal of the system is tied to billing and payment, certain outcome measures might be required and/or recommended to enable economic analyses
Ensure content relevance
- Determine whether symptoms, functioning, quality of life, or a combination of these is of interest and relevant
- Evaluate what aspects are relevant (e.g. severity, frequency, interference, bother)
- Consider the preferences of patients and clinicians and the goals of the PRO system
Consider the use of core outcome sets
- Core outcome sets are standardized groups of outcomes/measures that are often designed through expert consensus and which are collected and reported together
- They are often inclusive of PROs, clinician reported outcomes, and clinical measures which together provide a more cohesive picture of patient health
- As they are standardized there is often greater opportunity to compare outcomes across settings and populations
- Numerous core outcome sets exist which span specific conditions and patient populations
Figure 5.1: Process of Selecting a PROM
B. What are the different types of measures that can be used to collect PROs?
Generic vs. disease-specific measures
- Generic measures may capture the more common health-related quality of life domains and allow comparison to population norms, but might not be sensitive to changes over time
- Disease-specific measures may be more sensitive to specific symptoms but might miss domains of health that are important to patients but unrelated to the specific condition
Profile vs. preference based
- Profile measures usually provide scores across multiple domains and are often longer to complete
- Preference measures use a single score aggregated across multiple PRO domains. Often the score is weighted based on the preferences of a set of stakeholders (e.g. general population, patients). Often, but not always, preference-based measures are generic rather than disease-specific
Single item vs. multi-item
- Multi-item scales typically provide more reliable and valid estimates but are also more burdensome
- Single item measures are quick to complete but less reliable, particularly at tracking changes over time
Static vs. computer adaptive test (CAT) measures
- CAT measures require the presence of and access to a validated item bank, and CAT software to facilitate the adaptive nature of the assessment
- CAT measures are more efficient and typically allow more domains to be assessed with a similar number of questions
- Static measures can be used on paper and computers but might lack the high degree of discernability of CAT measures and may take longer to complete
C. What response format should be used?
Response option formats
- Verbal descriptions of response options may be preferred if the outcome is difficult to describe numerically or if the population of interest has lower numeracy
- Numeric ratings may be preferred if the outcome is related to frequency, can otherwise be meaningfully explained numerically, or is commonly assessed using a rating scale (e.g. visual analogue scale for pain)
- If using multiple measures, consider the potential cognitive burden of differing response options
D. What are other desirable features of PROMs?
Robust psychometric properties in the context of interest
- Evidence of validity in the patient population(s) of interest
- Low measurement error such that the scores give a precise picture of the patient’s status
- Conceptually clear with published evidence on reliability, feasibility, and utility
- Appropriate reading level for the target population; generally lower is usable by a broader range of patients
- Normative values such as thresholds, severity values, and other reference values that can be used to contextualize patient health and functional status
- Responsive to changes over time and in response to clinically significant changes in health
- Recall periods that are appropriate for the outcome of interest (i.e. short enough to reflect current health status, but not so short that they miss the outcome of interest)
Feasible to implement
- Respondent burden: How difficult is a measure to complete? While shorter measures are often regarded as better to reduce burden, patients are willing to complete longer measures if there is a clear purpose and the concepts are meaningful to them
- Cost/licensing: Some measures may require permission to use or have fees associated with their use
- Proxy versions: Some measures may have validated proxy versions, which is important if there is concern about the patient population’s ability to report for themselves (see Chapter 4, Identifying Patients)
- Translations: Availability of linguistic and cultural translations of instruments
- Compatibility with PRO system: Feasibility to program, collect, store, and feedback the PROM data within PRO system
- Clinical champions: Stakeholders may have positive experiences with a given PROM
E. Should PROs/PROMs be harmonized across the system?
Benefits to PRO/PROM harmonization
- Reduces the total number of PROMs that need to be programmed into a system, and increases the feasibility of PROM adoption
- Decreases duplication of effort to identify relevant severity thresholds, normative values
- Reduces respondent burden if fewer PROs can be used to inform medical decisions in multiple clinical contexts
- Streamlines training of end-users as all are training on the same system and with a pre-defined set of measures
- Facilitates research and other non-patient care goals by contributing to common data elements, allowing for information sharing within and across systems
Burdens to PRO/PROM harmonization
- Requires buy-in and participation of multiple stakeholder groups to agree on a set of harmonized measures
- Requires top-down guidelines on the harmonization system
- Decreases clinician/researcher autonomy in measure selection
- Limits utility if harmonized measures do not address the specific clinical needs of a population or are not relevant to decision-making in each context
F. Is the PRO/PROM a good or poor fit for a given health system?
Indicators that a PRO/PROM is a good fit for the health system
- Information on the outcome will advance the goals of the PRO system
- The outcome is relevant and meaningful to patients and other stakeholders
- The types of measures and/or response options available to evaluate the outcome are feasible for the patient population and setting
- The PROM has evidence of psychometric validity in the context of interest
- The PROM is feasible to integrate into the PRO system
Indicators that a PRO/PROM is a poor fit for the health system
- Information on the outcome will not advance the goals of the PRO system
- The outcome is not necessarily relevant and/or meaningful to patients and other stakeholders
- The measures and/or response options are not feasible for the patient population and setting
- The PRO measure does not have evidence of psychometric validity in the context of interest
- The PRO measure may be difficult to integrate into the PRO system
Relevant Primary Resources
The information presented here is an overview of how to select PROs and PROMs. For more detailed information please see the following sources:
Background And Citing The Proteus-Practice Guide
Nothing in this Guide should be construed to represent or warrant that persons using this Guide have complied with all applicable laws and regulations. All individuals and organizations using this template have the responsibility for complying with the applicable laws and regulations or regulatory requirements for the relevant jurisdiction.
Each chapter of the Guide lists the key foundational resources that informed its content. To appropriately recognize the foundational resources, we encourage you to cite both the Guide and the relevant foundational resource(s). Recommended citations are provided here.
Suggested Citation
The PROTEUS Guide to Implementing Patient-reported Outcomes in Clinical Practice
A synthesis of resources. Prepared by Crossnohere N, Brundage M, Snyder C, and the Advisory Group, 2023. Available at: TheProteusConsortium.org.
Further Reading
The Guide draws primarily from the foundational resources cited in each chapter. Please click here to find a selection of other relevant references.