Print Page | Report Abuse | Sign In
News & Press: News and Announcements

Meeting CMS Measure Testing Requirements for QCDRs

Friday, March 20, 2020   (0 Comments)
Posted by: Kasia Januszewski
Share |

Published on March 18, 2020, on

It may not be as challenging as it sounds

CMS’ requirement that Qualified Clinical Data Registry (QCDR) quality measures used in the Quality Payment Program beginning in January 2021 must be fully tested is a major concern of measure stewards and developers. While CMS’ language may suggest a need for the long and hugely expensive process of empirically validating measures, the reality may be much less burdensome.

As with many specifics of the new program, CMS has provided little additional guidance on what “fully testing” measures means for QCDR measures. However, existing regulations define it by reference to National Quality Forum (NQF) endorsement criteria, which require different levels of testing for different measure types at different points in their life cycles. In general, testing requirements are less stringent for registry measures than for eCQMs, and for new measures than for existing measures, which NQF defines as those it has previously endorsed. (See figure A)

One caveat: while NQF defines an existing measure as one it has endorsed, CMS has not provided guidance on how they define new versus existing measures. We’ve reached out to CMS for clarification on this point and will announce any communication we receive in response. Until then, it may be prudent to treat any registry measure already in use as an existing measure. On the plus side, previous testing of existing measures is acceptable as long as it meets current standards. That could cut down on the need for new investments in measure testing.

(Begin figure A)

Figure A

NQF Minimum Measure Testing Requirements


© 2019 PCPI Foundation. All rights reserved.

* No reliability testing required if data element validity testing conducted and results are adequate; however, additional score level testing is encouraged

(end Figure A)

Registry Measures
For purposes of testing QCDR measure for inclusion in the Quality Payment Program, the NQF endorsement criteria may grant more flexibility to registry measures.

For both new and existing measures, a formal feasibility assessment is not required; however, the PCPI recommends informal assessments of feasibility be completed with resources readily available to QCDRs (e.g., clinical experts) to ensure data for a measure can be readily collected in the course of clinical care.

Demonstrating measure score reliability and face validity is required for new registry measures. These requirements may be met with resources that are relatively easily accessed, such as a handful of technical experts already associated with the QCDR or specialty society who manages the QCDR, and data from a small sample of providers at two or three locations which may already be available through the QCDR.

For existing registry measures, NQF requires demonstrating measure score reliability and empirical validity. However, face validity may be acceptable in a few circumstances with justification. For example, face validity may be acceptable for an existing measure used on a patient population too small to support statistical analysis. To determine whether your existing measure has sufficient justification for face validity, PCPI recommends QCDRs reach out to CMS or NQF for guidance.

Data element testing may also be done for new or existing measures, which satisfies both validity and reliability testing requirements; however, this method of testing can be costly and QCDRs should weigh their testing options prior to deciding on a methodology.

Testing requirements for eCQMs are more rigorous. Both new and existing measures must undergo a formal feasibility testing process using the NQF Feasibility Scorecard. Reliability and validity must be established through data element testing, and a Bonnie simulation test must be run.

These requirements generally require cooperation of data partners (i.e., test sites) to undertake. If a measure steward has limited funding available to pay partners for their participation in the electronic and manual data extraction that is required for this type of analysis, recruiting partners can be a challenge.

Going Digital
Current requirements for developing and testing measures are likely to get more stringent. At its Quality Conference this year, CMS announced that by 2030 all measures must be “digital.” The intent is to harness the rich clinical data available through EHRs and other sources to support creation of eCQMs and facilitate quality improvement and better patient outcomes. The challenges in integrating and standardizing data sources will be substantial.

In the near term, we will publish more on measure testing. For more information click here for our interactive Measure Testing Webinar series, launching today. It will help you keep up with this fast-changing and technically challenging topic.

Our Members

353 N. Clark St. Suite 1400-a, Chicago, IL 60654 | Phone: 312-757-PCPI | General Email: