University of Wisconsin-Madison

CLOSE

ENHANCED BY  

World-class leader in pharmacy education, research, and service in the pharmaceutical, social, and clinical sciences.

Module 5: Using EHR Data to Measure Outcomes

Opioid Stewardship Toolkit

Opioid Stewardship Toolkit

icon of computer screen displaying a graphThe opioid overdose crisis now claims more than 40,000 lives in the United States each year. In order to address this problem, hospitals and health systems are responding by creating opioid stewardship programs with a goal to reduce opioid-related morbidity and mortality. Because hospitals and health systems vary, measurement is a critical part in assessing the impact of these programs, demonstrating improvement, and implementing a successful intervention. (Weiner et al., 2019)

Metrics from the electronic health records are ideal, discrete, and can be utilized to measure opioid treatment or intervention outcomes. Overall, metrics can be translated into reports that identify specific problem areas and offer guidance to improve opioid stewardship programs and patient-based outcomes. (American Hospital Association, 2020)

Increasingly, quality, accountability and payer organizations are recommending or requiring heath systems to provide data on opioid stewardship metrics. For example, in 2018 & 2019, the Joint Commission, a national an independent, not-for-profit standards-setting and accrediting body in health care, issued revised Pain Assessment and Management Standards for hospitals, accredited ambulatory care facilities, and critical access hospitals that require accredited hospitals to collect data on pain management and pain assessment to monitor whether opioids are being used safely and identify areas that need to be improved.

Clip duration:
00:01:39

Listen to Audio Clip

audio clip icon

Why EHR data is so important to measure opioid stewardship progress

Clip from Podcast with Nicole Green, Director of Ambulatory Pharmacy and Dan Cunningham, ThedaCare

Listen to Audio Clip with Transcript


Podcast duration:
00:24:56

Listen to Podcast

audio clip button - click to hear podcast

Full Interview with with Nicole Green, Director of Ambulatory Pharmacy and Dan Cunningham, ThedaCare

  • Highlights the role of a pharmacist in opioid stewardship efforts, including:
    • Provider education through CME presentation and academic detailing initiative focusing on 8 clinics in rural areas
    • Identifying opioid-related risks to inform opioid stewardship
  • How they have been able to support the pharmacist’s time spent on opioid stewardship
  • Reflections on lessons learned and advice for pharmacist colleagues in rural areas, such as how this role is not specific to ambulatory pharmacists, but rather, can be done by any pharmacist

Listen to Audio Clip with Transcript

Patient metrics from the electronic health record can be used to measure outcomes and assess opioid stewardship program efforts and goal completion. Tables 1-3 provide a starter set of metrics to consider. For each metric, the numerator and denominator columns show how calculations should be performed. Additionally, these tables show how different metrics could be utilized to evaluate specific goals of an opioid stewardship program or align with federal quality or accountability programs. Module 2 provides more information about opioid or pain-related quality and accountability metrics. For example, metrics can be used to gauge compliance with opioid-focused laws and regulations, assess outpatient or inpatient prescribing habits, assess individual patient opioid use, addiction potential, and abuse risk, or evaluate opioid-related services, including assessment and education. The metrics listed in the tables are considered lower effort by the American Hospital Association and may be easier for health systems to calculate. The lists are not fully inclusive, and other elements might be considered as well. In a 2020 publication, the American Hospital Association provided a comprehensive list of measures.

Thumbnail of the Quality Improvement and Care Coordination guide put out by CDCThe Centers for Disease Control and Prevention (CDC)’s guide for implementing the 2016 CDC Guideline for Prescribing Opioids for Chronic Pain includes additional measures in Appendix B, including ICD-10 codes that can be utilized when creating inclusion and exclusion criteria. Additionally, CDC provides an at-a-glance reference document listing the 16 recommended QI measures.

Table 1. Starter Set of Measures Focused on Acute Pain Management
Metric Numerator Denominator Goal Alignment with Federal Programs
Average total MME per prescription Average MME Number of opioid prescriptions Reduction in average total MME Medicaid ACS, MSSP
MME per opioid prescription Total MME Number of opioid prescriptions Decrease in MME Medicaid ACS, MSSP
Number of opioid prescriptions per prescriber at discharge Number of opioid prescriptions at discharge Patient volume per prescriber Decrease in total number of opioid prescriptions
Average MME dose administered per inpatient day Opioid MME dose administered per day Patient days with opioids administered Decrease in average MME dose administered TJC

Medicaid ACS = Medicaid Adult Core Set (ACS)
MSSP = Medicare Shared Savings Program
TJC = The Joint Commission

Table 2. Starter Set of Measures Focused on Harm Reduction
Metric Numerator Denominator Goal Alignment with Federal Programs
Percentage of patients with opioids and benzodiazepines co-prescribed Patients prescribed both opioids and benzodiazepines All patients Decrease in number of patients co-prescribed HEDIS, Medicaid ACS, HIQRP, MSSP
Naloxone prescribed for opioid overdose or high-risk patients Number of naloxone prescriptions Number of patients presenting with OD or opioid MME > 50 Increase in naloxone prescriptions
Opioid prescriptions > 90 MMEs daily Prescriptions > 90 MMEs daily All prescriptions Decrease in opioid prescriptions > 90 MME daily Medicaid ACS
Proportion of hospitalized patients administered naloxone Number of naloxone administrations per admission Discharges of adult patients over 12 months Decrease in naloxone administered in hospital HIQRP, TJC

HEDIS = Healthcare Effectiveness Data and Information Set
Medicaid ACS = Medicaid Adult Core Set (ACS)
HIQRP = Hospital Inpatient Quality Reporting Program
MSSP = Medicare Shared Savings Program
TJC = The Joint Commission

Table 3. Starter Set of Measures Focused on Identification and Treatment
of Opioid Use Disorders
Metric Numerator Denominator Goal Alignment with Federal Programs
Number of referrals of OUD/MOUD treatment Number of referrals ordered Number of patients identified with OUD Increase in referrals or use of addiction consult service Medicaid ACS TJC
New patient starts for MOUD MOUD initiated Number of patients identified with OUD Increase in number of new starts Medicaid ACS
Screening for OUD/SUD Number of risk assessments documented in EHR or percentage of patients screened Number of patients on opioids for longer than 6 weeks Increase in number of screens MIPS QM
Identification and planning for patients with OUD/in MOUD upon admission Number of plans documented Number of patients with OUD or MOUD treatment Increase in number of documented plans Medicaid ACS

Medicaid ACS = Medicaid Adult Core Set (ACS)
MIPS QM = Merit Based Incentive Payment System (MIPS) Quality measure
TJC = The Joint Commission

Overall, it is important to select a set of measures that aligns with the health system’s mission and values and also addresses the problems faced by the members of the community being served. Module 8 provides additional guidance on how health systems can establish partnerships with the community. Additionally, it may be helpful to consider which metrics will help meet the needs of the Quality Department, reimbursement rates, or third-party payers.

Metrics should be aligned with your health system’s opioid stewardship goals and priorities. Module 3 provides guidance and examples related to identifying the highest priority goals, which will in turn influence the identification of appropriate metrics.

The American Hospital Association Center for Health Innovation’s 2020 publication on opioid stewardship measurement offers the following criteria for opioid stewardship measurements:

Criteria for Opioid Stewardship Measurements

  1. Addresses a problem in the hospital/health system or community
  2. Supported with up-to-date and evidence-based internal guidelines, policies, or procedures
  3. Shows success or a need for improvement with established goals
  4. Longitudinal
  5. Supported with a level of evidence to positively influence patient outcomes
  6. Identifies variation between departments, units, or prescribers
  7. Balanced with counter measures to ensure desired changes do not cause new problems in the system

  (AHA, 2020)

Since pain management and opioid use is multifaceted, focusing on more than one measure may be needed to fully appreciate and improve the complexity of opioid stewardship efforts and patient outcome improvement. Using multiple metrics helps comprehensively evaluate programs on a system, provider, and patient level.

Additional health system-specific questions should be considered when selecting appropriate metrics.

Which metrics are easier or harder to obtain and/or calculate?

Certain metrics may be quicker for health systems to calculate. Additionally, it may be easier to populate certain numerators and denominators. Table 4 shows examples of metrics that are categorized by their degree of difficulty to complete.

Table 4. Examples of Opioid Stewardship Metrics by Degree of Difficulty to Complete
Simple Moderate Complex
  • Total number of opioid or
    benzodiazepine
    prescriptions dispensed by
    a provider
  • Total number of opioid or
    benzodiazepine
    prescriptions for an
    individual patient
  • Total number of patient
    encounters for pain
  • Proportion of patients with
    timely follow-up (1-4 weeks
    after dosage change)
  • Proportion of opioid
    discharge prescriptions
    exceeding 7 days
  • Naloxone prescriptions
    dispensed by provider
  • Naloxone prescriptions
    dispensed per
    month/quarter for facility
  • Number of naloxone
    reversals provided in
    hospital-based units
  • Total number of urine drug
    screens completed by a
    care area or provider
  • Functional Assessments,
    Pain Assessments, other
    assessments
  • Total number of opioid or
    benzodiazepine
    prescriptions dispensed in a
    specific practice or care area
  • Total number of concurrent
    opioid and benzodiazepine
    prescriptions by:
      • Provider
      • Unit
      • Practice or Care Area
  • Total morphine
    milliequivalents (MME) for
    each patient
  • Proportion of patients with
    MME ≥ 50 and ≥ 90
  • Average MME for a
    provider, practice area, or
    unit
  • Adverse drug events
    suspected to be caused by
    opioids or benzodiazepines
    for a specific patient
  • Positive urine drug screens
    for non-prescribed opioids
    for a specific patient or
    provider group
  • Number of patients in a
    treatment agreement
  • Number of patients that
    might require a treatment
    agreement due to
    prescription use and co-
    morbid conditions
  • Number of patients with
    two or more as needed
    opioid prescriptions for
    different indications
  • Proportion of patients with
    diagnosed opioid use
    disorder receiving
    treatment
  • Number of patients on
    long-term opioid therapy
    (i.e. > 90 days)
  • Number of overdose events
    per year
  • Number of overdose events
    that could have been
    prevented
  • Electronic Prescription
    Drug Monitoring Access
    before prescribing

What metrics are appropriate for a Pharmacy Resident or Pharmacy Student completing an Advanced Practice Pharmacy Experience project?

Student or resident projects may be utilized for calculating metrics to evaluate opioid stewardship programs. This can be exceptionally useful in systems where pharmacists are highly constrained by time or additional responsibilities. If utilizing a student or resident, it is important to consider the following questions:

  1. How long will the student or resident be at the site?
  2. What type of rotation is the student or resident on? Will this project align with the rotation goals, as well as the goals for opioid stewardship evaluation?
  3. Are the desired calculations fairly simple or more complex?
  4. What parts of the electronic health record (EHR) does the student or resident have access to?
  5. If needed, does the student or resident have access to the IT department or EHR company?

Consider the size of your institution and how quick is it to connect with the IT department

When selecting electronic health record (EHR) metrics, it is important to consider the characteristics of your health system or institution. Depending on the metrics being used, pharmacists and other members of the opioid stewardship team may need to collaborate with professionals in other areas of the system, including the IT Department. If collaboration is necessary, it is important to consider the other work that the IT team may be required to complete, as well as other ongoing IT work or projects. If working with IT, consider how many individuals work in the department or if certain members are assigned to assist the opioid stewardship efforts. IT resources that are allocated to quality, regulatory, or even provider optimization may be key resources that can be reallocated to opioid stewardship work, as this work affects patient-based outcomes in those areas. Additionally, prior to selecting metrics, it is important to understand the method of communicating with outside departments, resources, and other electronic systems. Efficiency in communication and collaborative and well-understood time frames for receiving responses or assistance are key to building strong relationships with IT partners.

Tech-savvy Pharmacists, Nurses, and Providers

IT Professionals

Quality / Compliance Department Professionals

Other health systems with the same EHR

EHR Vendor

Depending on the electronic health record (EHR), data needed to calculate certain metrics may be available directly on the front end. Pharmacists, clinicians, and other team members may have the ability to run reports based upon their role and make the appropriate calculations by themselves. However, not all EHR users may be aware of these options. Additionally, certain data points may not be directly available to all roles in the system. The following resources can be used to help with data collection:

Tech-savvy Pharmacists, Nurses, and Providers A helpful place to start is by utilizing colleagues. Depending on the size of the health system, certain clinicians may have formal or informal IT training. Colleagues with residencies, certifications, or years of experience in this area may have a better understanding of how certain data points can be obtained. If no formal training exists among the non-IT team, enlisting the help of those with a higher degree of EHR knowledge can be beneficial.
IT Professionals Leveraging the help of the IT professionals can be beneficial in determining what data or information is available through a system’s EHR. EHR’s can have a variety of versions, each with different functionality. IT professionals will have a better understanding of what information can be extracted from the health system’s EHR. Additionally, IT professionals can provide information on what data is available on the front end. They can help show pharmacists and others where to find these data and how to run reports. Finally, IT professionals can serve as an ongoing resource to accessing data from the back end of the EHR.
Quality/Compliance Department Professionals In addition to the IT Department, colleagues in the Quality Department can also assist with opioid stewardship efforts. Because these professionals work to measure health benefits and improve patient outcomes, they may be familiar with the data that is available or have ideas on different metrics that that can be measured. They may also be knowledgeable about where and how data enters the EHR. In addition, the Quality Department may have a better understanding of which metrics are tied to reimbursement or value-based care delivery and can help decide which measures would be most beneficial for the health system. For example, they may be knowledgeable about which opioid stewardship measures are recommended by third-party payers, or if there are Centers for Medicaid and Medicare Services quality measures that the health system must meet. As more and more health systems join Accountable Care Organizations (ACOs), the quality team may also be engaged in data collection for the ACO and can speak to opioid-based measures. Lastly, Quality Department professionals can help facilitate interactions between the opioid stewardship team and the IT Department, especially if the data is already being collected for a quality base measure.
Other health systems who use the same EHR To learn more about the data that is available through the EHR, connect with other health systems that use the same EHR system. Reaching out to practice sites within the same health system can be a beneficial first step. It is likely that these practice sites not only use the same EHR system, but a similar version with comparable functionality. Additionally, reach out to nearby hospitals or health systems that use the same EHR to leverage their expertise or experience. This provides an opportunity to discuss potential metrics, provided the other health system treats similar patient populations. Lastly, it may be beneficial to utilize interpersonal connections between pharmacists, contacts form previous projects and/or collaborations, pharmacy society connections, national society connections, or national list serv options.
EHR Vendor Resources provided by the EHR vendor can also be utilized to learn more about functionality, including which metrics can be obtained by pharmacists and/or the IT Department. Reach out to your IT Director to learn more. Many computer-based health system records have end user portals and frequently asked question forums. Additionally, attending an EHR health conference can always help build connections, allow for networking, and offer a hands-on option for talking with EHR professionals from other locations and states.

Use appropriate language and key words

graphic of word bubblesAs pharmacists, we learn to use patient-friendly language during patient encounters. Similarly, it is important to use appropriate language when communicating with Information Technology (IT) professionals. Medical language may not always be well-understood by the IT Department, as their background and expertise differ. Additionally, it is important to ask questions when certain terms are not understood. Similarly, IT professionals may use language that is not well-understood by health care clinicians. In these situations, it is important to ask for clarification, rather than making assumptions.

Be as specific as possible

Prior to asking the IT Department for assistance, it is important to have a clear idea of the exact data points needed and the outcomes being evaluated. This includes setting specific boundaries before obtaining and calculating the data. If looking at a specific patient population, make sure the population is well-described using inclusion and exclusion criteria, such as age, gender, race/ethnicity, disease state, type of visit, insurance type/status, etc. Additionally, if interested in specific medications or providers, include these parameters in data extraction. Lastly, data criteria should include a specific time frame as applicable, and IT should be made aware if patient identifiers are necessary.

Meet in person if necessary

Because calculations for opioid stewardship metrics can be complicated, communicating with the IT Department via email or electronic methods could lead to confusion. In order to improve understanding and avoid confusion, set up in-person meetings, schedule a virtual meeting, or plan a phone call to allow for questions and further discussion. These meetings may become routine/scheduled or may only need to occur initially as the team is determining which metrics to use. Moving forward, meetings may only need to happen as needed or when new metrics are considered. Additionally, it is important to remember that IT requests may be made through online systems, and the system may not provide enough space or characters to define your need or request. With that in mind, it is important to find your IT professional partner and summarize your request with two-way communication, ensuring that they have an opportunity to ask questions and understand the need for the data, as well as the impact of the data on patient care.

graphic of hands holding a tablet with EHR information displayedIt may also be helpful to think about what data is ‘discretely’ included in the EHR. What data is collected during a typical patient encounter and how the data is added to the system can be very helpful in determining what data elements could be pulled into a report. Ensure that all members of the team include their input on how to extract information from the EHR. Knowing this can give IT professionals a better idea of what information can be pulled into reports. Additionally, it may be helpful for clinical team members that will view the data and report it to leadership/ administration to provide the preferred format (list, table, etc.), as IT may have different options for presenting the data. Lastly, running reports using test patients may allow for a proof of concept report that can be further evaluated by the opioid stewardship data collection team and ensure that the report is pulling what is desired. Using a testing patient, when possible, can also ensure that no assumptions are causing data integrity concerns (i.e. overlooking key data or including data that is not accurate).

  • graphic of lightbulb popping out of a cardboard boxEstablish relationships with Information Technology (IT) colleagues as early as possible, even before assistance is needed
  • Work with IT colleagues to agree upon a timeframe that works for both parties and plan ahead to work around everyone’s schedules
  • Remain flexible with what IT professionals are able to provide – the opioid stewardship team may need to filter or manipulate the data further after it is pulled by the IT Department
  • If the data is not what was expected, think about how you can change parameters and communicate this to IT
  • In some cases, IT may have to reach out to the EHR company – be patient and plan accordingly
  • Utilizing ICD-10 codes can be useful for creating parameters and filtering data. However, make sure to explicitly state what each ICD-10 code represents.
  • Make sure to filter out duplicate records after the data is obtained
  • It is important to think critically about opioid stewardship measures. The individuals included in a metric calculation may not represent the entire population of patients at risk. On the other hand, every patient included in the calculation may not truly be high risk. Individual patient factors and clinical knowledge should be taken into consideration.
Clip duration:
(00:04:50)

Listen to Audio Clip

audio clip icon

Considerations about opioid stewardship data collection and input from colleagues in quality department

Clip from Podcast with Sarah Pagenkopf, Director of Pharmacy Services, Fort Health Care

Listen to Audio Clip with Transcript

Podcast duration: 00:44:37

Listen to Podcast

audio clip button - click to hear podcast

Sarah Pagenkopf, Director of Pharmacy Services and Tyler Prickette, Clinical Pharmacist at Fort Health Care

  • Getting buy-in from leadership to initiate opioid steward
  • Conducting a gap analysis
  • Identifying a champion leader
  • Setting goals and offering lessons learned and advice for pharmacist colleagues in rural settings

Listen to Podcast with Transcript

The following resources offer examples and step-by-step guidance specifically related to working with data to measure opioid stewardship outcomes.

In 2020 American Hospital Association (AHA) Center for Health Innovation published Stem the Tide: Opioid Stewardship Measurement Implementation Guide. The Guide addresses six critical elements that can support users through a process of implementing a data-driven approach to an opioid stewardship program and includes a curated Menu of Measures, which presents 30 ranked quality improvement measures categorized into the primary focus areas of acute pain management, harm reduction, and identification and treatment of opioid use disorder.

Thumbnail of the Quality Improvement and Care Coordination guide put out by CDCThe Centers for Disease Control and Prevention (CDC)’s guide for implementing the 2016 CDC Guideline for Prescribing Opioids for Chronic Pain includes additional measures in Appendix B, including ICD-10 codes that can be utilized when creating inclusion and exclusion criteria.
Additionally, CDC provides an at-a-glance reference document listing the 16 recommended QI measures.

Weiner SG, Price CN, Atalay AJ, Harry EM, Pabo EA, Patel R, Suzuki J, Anderson S, Ashley SW, Kachalia A. A Health System-wide Initiative to Decrease Opioid-related Morbidity and Mortality. Jt Comm J Qual Patient Saf. 2019;45(1):3-13.  This paper describes a framework for a new health system-wide opioid stewardship program, including the measures they used to track progress.

The College of HealthCare Information Management Executives (CHIME)  put together an Opioid Playbook  which has a chapter on ‘Creating your dashboard.’  The dashboard will play a critical role as a tool to view and monitor performance metrics and present other relevant data.

The Society of Hospital Medicine (SHM) developed the Reducing Adverse Drug Events Related to Opioids (RADEO) Implementation Guide which provides step-by-step guidance to assist hospital teams in implementing a quality improvement program to improve patient safety and reduce opioid-related adverse events for patients receiving opioids, including: Step 5: Choose metrics and develop a data collection plan.