The Cornerstone Series: The State of Data Use in EMS

Download Full PDF Here

It’s clear there is increasing interest among EMS leaders in using data to improve our ability to monitor our systems—often in real time—and to translate that data into information that can improve systems and ultimately patient care. Powered by Fitch & Associates’ experience in all 50 states over three decades, the series will be an up-to-date overview of how agencies can—and are—using their data. Future installments of the Cornerstone Series will cover:

  • Performance Measurement and Benchmarking
  • How to Measure Financial Performance
  • How Data Can Help Improve Safety
  • Beyond Response Time: Operational Measures in EMS
  • Using HR Data to Strengthen the Workforce
  • How Access to Hospital Outcome Measures Will Benefit EMS

To start the series, we did our own research to get a sense of how EMS agencies are using data currently, and the challenges they face.

Introduction to the Survey

When trying to make improvements within an EMS agency, measurement and evidence are essential tools in the decision-making process and in understanding the impact of any changes. Meaningful data can drive smart, evidence-based decisions and make EMS agencies more effective and efficient. Many EMS organizations are in the early stages of creating systems to use data for regular, meaningful analysis to inform operational, clinical and budgetary decision-making.

In an effort to learn more about the state of data collection, data analysis and data-driven decision-making in EMS, Fitch & Associates partnered with EMS1 to survey hundreds of EMS professionals and leaders. The results of the Cornerstone Survey, presented here, provide insight into how the EMS profession is using data right now—and how it isn’t.

The survey showed that most EMS agencies are collecting vast amounts of data, which is clearly an important first step. But the survey’s findings suggest that many agencies are struggling to devote the resources needed for meaningful data analysis— including personnel, time and technology. Information gleaned from agency data is, after all, essential to helping EMS agencies demonstrate their value to taxpayers, patients, city councils, state agencies, hospitals and other partners in a changing reimbursement landscape.

Many agencies could also do more to communicate to internal and external stakeholders the importance of data analysis and its role in helping the agency make smarter decisions about staffing, clinical care, resource needs and more.

Our goal is not only to assess the needs of the industry as EMS agencies ramp up their investments in data collection and analysis, but also to provide insight and assistance to  help you improve. In the coming months, we will be sharing articles, whitepapers, webinars and other resources as part of the Cornerstone Series to help EMS agencies—big and small— move toward evidence-based practices, meaningful data analysis and data-driven decision-making.

Key Findings

The Cornerstone Survey confirmed that many EMS agencies are collecting and using data to drive agency improvement, with an overwhelming majority (94.4 percent) of survey respondents saying data analysis is a part of their decision- making process.

Other interesting findings include:

  • EMS agencies track a wide variety of metrics related

to clinical information, operational efficiency, financial outcomes, the workforce and safety. The most commonly tracked metrics included total response time, scene time, billing collection rate, STEMI recognition, cardiac arrest return of spontaneous circulation (ROSC) and intubation success rate. Most agencies use a NEMSIS-compliant tool to capture patient information.

  • EMS agencies report they are regularly reviewing data reports for a number of purposes, frequently to examine response time and clinical skills performance. Agencies still struggle to gain access to hospital data in a way that’s reliable and able to be used
  • Many agencies face notable barriers to data collection and analysis, including time constraints, lack of expertise and software tools that don’t meet their data analysis
  • EMS agencies report being somewhat prepared for a shift toward pay-for-performance models, and many have begun measuring patient satisfaction and

About the Survey

Nearly 500 EMS professionals participated in the Cornerstone Survey, with a majority (61 percent) reporting that they were part of the senior leadership of an organization and most

of the remainder representing field supervisors and quality improvement coordinators. They represented a diverse range of service delivery models, with about 20 percent each from fire departments, private for-profit agencies and private nonprofits. The other 40 percent included government “third- service” agencies, hospital-based services and public-utility model organizations.

The survey results also represent agencies both large and small. More than a third reported that their agencies responded to between 1,000 and 10,000 EMS incidents per year. A significant proportion (21.8 percent) also came from large agencies that respond to more than 50,000 incidents per year. Survey respondents from organizations that respond to fewer than 1,000 incidents each year comprised 12.5 percent of the total survey sample.

How Data Is Being Collected and Used

Perhaps not surprisingly, the vast majority of EMS practitioners surveyed—more than 94 percent—stated that their agencies currently use data to drive improvement efforts. While more than two-thirds expressed confidence in the completeness and accuracy of the data their agencies are collecting, it is clear that they still frequently lack access to much of the information they need and only analyze a limited subset of the data. However, with more than three-fourths reporting use of a NEMSIS-compliant tool to capture patient information, the National EMS Information System has certainly aided local EMS organizations looking to have access to patient care data in order to measure performance.

Additionally, many respondents stated that their agency regularly incorporates data from other organizations into agency improvement processes. Most often, they stated that they incorporate data from dispatch centers (70.4 percent) or hospitals (56.9 percent). A significant percentage of respondents reported that their agency uses data from fire departments or other first responders (35.0 percent) or aeromedical services (20.1 percent). That more than half of the EMS professionals who responded said their agencies were using hospital data is significant, yet until that number reaches closer to 100 percent, we know that EMS still isn’t where it needs to be. As part of the healthcare continuum, and in order to improve patient care, EMS needs access to hospital information.

IMPLICATION

Although many EMS agencies report incorporating data from other organizations, more needs to be done to ensure EMS has the information it needs to serve patients and evaluate and improve care. Just over half of EMS agencies surveyed are receiving data from hospitals, which can provide information that is vital to the delivery of seamless care and to the quality improvement process. Every day, EMS providers across the country provide hospitals with patient data. Agencies need to turn those relationships into data-sharing partnerships, where hospitals consider it routine business to provide outcome information to their EMS colleagues.

What Are EMS Agencies Measuring?

While the survey revealed that EMS systems are tracking a wide range of measures, six stood out as the most commonly used, by more than 300 respondents. They include operational measures, a financial measure and several clinical measures:

  • Total response time
  • Scene time
  • Collection rate
  • STEMI recognition
  • Cardiac arrest return of spontaneous circulation (ROSC)
  • Intubation success rate

Other commonly examined statistics include overtime hours and costs, turn-out (or “chute”) time, stroke recognition and injury rates from lifting and moving patients.

In general, it is clear that what EMS measures is influenced more by what data is available than by what truly matters to patient outcomes. Operational process measures are still by far the most commonly viewed measures, with more than three-fourths of survey respondents reporting that response time and call volume are regularly examined. The majority of respondents also reported examining clinical process measures, such as skills performance and protocol compliance, while fewer than half said they regularly look at patient outcomes.

IMPLICATION

EMS too often relies on the data and measures that are easy to find and calculate, rather than the ones that can be most effective in assessing performance and making improvements. This often leads to changes that improve on those measures but may not make the system more effective.

Operational Data

More than half of the survey respondents stated their agency measured the following operational data points:

  • Total response time
  • Scene time
  • Chute/turn-out time
  • Call processing time
  • Total dispatch time

Given that the data most frequently collected and used by EMS are response times, it’s interesting that agencies were nearly split on how they define that measure, with about 42 percent using time from receiving the call to arriving on scene, and 38 percent starting the clock on dispatch instead.

If using response time to assess system health, it’s most important that an agency pick a definition and remain consistent in order to see any trends over time. However, with many EMS agencies benchmarking times against each other and various standards, it is critical to ensure that agencies are only comparing their times to those that are measured using the same definitions for when the clock should start and stop.

IMPLICATION

Operational data is most useful when it helps an agency understand how to make changes to become more efficient or better communicate its value and/or resource needs. Agencies should focus on data that helps:

1.    Understand how busy the system is, i.e., supply and demand of resources.

2.    Communicate to the community, partners and other stakeholders return on investment and justify the value of the EMS agency or growing resource needs.

Clinical Data

Measuring clinical metrics such as protocol compliance or clinical skills or procedure success rates is essential to knowing that the agency is properly treating patients and focusing training in the right areas. Clinical outcomes offer an important way for EMS agencies to justify their lifesaving value to the public and to policymakers. Most of the clinical metrics collected by EMS agencies are focused on clinical skills and procedure success rates and not patient outcomes, which could present challenges should reimbursement be more directly tied to outcomes in the future. More than 75 percent of respondents said their agencies measure return of spontaneous circulation, while only half of respondents said they look at survival-to- discharge, when assessing cardiac arrest outcomes. This is yet another example of how limited access to the right data can create barriers to good measurement, as ROSC is considered an inferior measure to survival-to-discharge. Looking only at ROSC can in fact sometimes cause EMS providers to think improvements are being made, even though increases in ROSC rates do not always lead to increases in survival.

Process measures, such as intubation and IV success rates, can help an agency assess its own training efforts, but it is unclear if there is any tie between skills success and clinical outcomes.

On an encouraging note, a large majority of survey takers reported that they assess how well their agencies identify stroke and STEMI patients, two clinical conditions where EMS can make a significant difference in people’s lives simply by recognizing the problem and getting the patient to the right care at the right time.

IMPLICATION

Given the increasing focus on patient outcomes, more outcome-based measures could be prioritized among the clinical data collected by EMS agencies. To do so, most EMS agencies will need to work with hospitals and other healthcare organizations to create a system for consistent tracking and reporting of patient clinical outcomes.

Even where clinical process measures continue to be a focus, it is important to align them with positive outcomes and to tie data on clinical process to training and improvement.

Other Data

Survey participants also reported collecting and analyzing information related to finance, safety and the workforce. The vast majority of EMS organizations said that they reviewed transport collection rates as well as cost and revenue per transport. More than 80 percent said they also regularly

look at overtime hours and associated costs. A much smaller percentage analyzes data on employee turnover, satisfaction and wellness.

A significant number of respondents are also looking at several health and safety issues, including injury rates from lifting and moving patients and vehicle collision rates. Only a handful are tracking data on crew fatigue and workload.

Barriers to Data Analysis

Data is only useful when analyzed in a way that places it within context and allows the information to drive or influence decisions. Survey respondents reported a number of barriers to meaningful data collection and analysis in EMS agencies.

Barriers to Data Collection and Analysis

A large majority of survey respondents (76.2 percent) cited

a lack of time as the biggest barrier to using data effectively. A majority also pointed to a lack of expertise and knowledge as an obstacle, and nearly half felt that they did not have the

technological capabilities necessary to perform adequate data collection.

Many survey respondents (19.6 percent) lacked confidence that data they collected would actually be used in the decision- making process. Other barriers cited include cost, lack of personnel, poor data quality, lack of appropriate software tools and a struggle to collaborate with other organizations such as hospitals.

Data Collection and Analysis Resources

As with many roles within EMS agencies, the task of collecting and analyzing data is often left to someone who has minimal training and experience, often a paramedic who teaches herself on the job before being rotated to another post. A majority of survey respondents (67.0 percent) stated that their agency does not have a dedicated data analyst. In cases where an agency does have a dedicated data analyst, about 40 percent of survey respondents stated this person was not formally trained in data analysis.

IMPLICATION

In order to effectively use data through meaningful data analysis and reporting, EMS agencies need to dedicate budget and staff time to hiring and training more data analysts. Return on investment needs to be assessed with budgeting staff time or new software tools for reporting and analysis to understand how more trained professionals or better tools could save time and money in the long run.

Time for data collection and analysis needs to be built into the agency’s workflow and staff job description to assure that this is a valued part of the agency’s operations. Much like an agency should set aside resources for fleet maintenance or medical equipment, there must be funding allocated for the collection and analysis of data. Where time and resources for data analysis are significantly limited, agencies can look to other entities, from local universities to health departments, for assistance. Partnering with other local EMS agencies or other healthcare providers to share the costs of the technology or personnel needed for data analysis may also facilitate greater data sharing and more comparative data analysis.

Technology Tools and Data Collection

As mentioned above, nearly half of survey respondents listed technology as one of the barriers to data collection and analysis. Some respondents mentioned the limitations of the software, while others mentioned its cost. The vast majority (72.6 percent) of respondents use software solutions to create automated data reports, yet clearly many don’t believe their current software is meeting their needs.

What Software Tools are Being Used?

The most frequently used software tools for EMS data reporting and analysis were those embedded within electronic PCR systems or CAD systems. Nearly half of survey respondents (49.7 percent) reported their agencies use Microsoft Excel or Access or a similar program. Some agencies reported using other solutions, such as FirstWatch and FirstPass or other third-party analysis programs.

Are Agencies Investing in New Software Tools?

Although most survey respondents use software in data reporting, and nearly half identified technology as one of the barriers to data collection and analysis, a majority (54.5 percent) stated their agencies were not discussing or had no immediate plan to invest in new technology to improve data collection and analysis.

Only 17.4 percent of respondents said their agency would soon make investments in technology to help with data collection and analysis. Another 12.4 percent of respondents said their agency had done so recently; 15.7 percent of respondents said they would like to invest in new software but did not have the funding to do so.

IMPLICATION

EMS agencies should work collaboratively with software companies and other healthcare agencies to articulate their need for software that is affordable and user-friendly and allows for sharing across agencies and organizations. New tools should be NEMSIS-compliant to allow for consistency and benchmarking between organizations. Where EMS agencies have invested in new software to help with data collection and analysis, they should be open to sharing their experience using the software with software developers as well as other agencies looking to invest in new tools.

Preparing EMS for Changes in Reimbursement

With the healthcare reimbursement system still in flux, it is unclear what payment for EMS will look like in the near and distant future. However, the shift away from fee-for-service to a pay-for-performance or value-based model is clearly underway and will have significant impact on EMS operations and finances. Part of that shift will likely involve the use of performance measurement to demonstrate effectiveness and efficiency, whether directly to payors or to other stakeholders, including patients and regulators.

When asked whether local city or county leadership or other stakeholders see value in performance measurement, the majority of survey respondents (63.2 percent) stated that they did. When it comes to preparedness for pay-for-performance in EMS, though, only a quarter of survey respondents said they felt “prepared” or “very prepared” for this change.

Measuring Patient Satisfaction

Within the pay-for-performance framework, the importance of tracking patient satisfaction and using it to rate providers or adjust reimbursement rates has gained traction. EMS agencies, too, are attempting to measure patient experience and satisfaction; 58.7 percent of survey respondents said that their agency was trying to track this metric. Of the agencies that tracked patient satisfaction and experience, 69 percent tracked it internally, about 18 percent measured it through a third-party, and 13 percent said it was not measured through either means. A majority of survey respondents (78.8 percent) responded that their agency measures patient satisfaction and experience using a mail-in survey, while about 20 percent said they used phone surveys.

IMPLICATION

As EMS prepares for changes in reimbursement models, it can look to other organizations in the healthcare system to gauge how they have responded to the move toward value-based reimbursement. Measuring patient satisfaction not only prepares EMS agencies for a possible use of this measure by payors, but it can provide important insight into how the agencies are perceived by the communities they serve.

Summary

The Cornerstone Survey revealed that EMS agencies’ leaders understand the importance of using information to drive decision-making but struggle to find the time, resources and knowledge to ensure they are accurately and consistently using data to measure the right things.

 

According to the survey, EMS agencies track a wide variety of metrics including measures of clinical outcomes, operational efficiency, financial outcomes, the workforce and safety. Across agencies, six metrics stood out as the most commonly tracked:

  • Total response time
  • Scene time
  • Collection rate
  • STEMI recognition
  • Cardiac arrest ROSC
  • Intubation success rate

Although many, if not most, EMS agencies are systematically tracking data, a number of organizations point to barriers to meaningful data analysis. Most often cited were lack of time, lack of knowledge and expertise, and lack of adequate technology. Just as EMS agencies should not be satisfied if paramedics are not trained properly to perform their jobs or do not have the proper tools to treat patients, they should not ignore the importance of having appropriate resources to use information and data to make decisions. Otherwise, it is nearly impossible for them to know how their agency is performing or whether changes are having the intended impact.

The results of the Cornerstone Survey conducted by Fitch & Associates suggest that EMS is in a time of transition regarding how agencies collect, analyze and use data. While this may be daunting, it also provides an opportunity for agencies to embrace data-driven decision-making and set up systems, budgets and personnel to align with this approach.