Short Communication - Archives of Cancer Biology and Therapy (2020) Volume 1, Issue 2
VA-Radiation Oncology Quality Surveillance Program: Enhancing Quality Measure Data Capture, Measuring Quality Benchmarks and Ensuring Long Term Sustainability of Quality Improvements in Community Care
Evangelia Katsoulakis1, Rishabh Kapoor2, John Park3, Christina Chapman4, Abhi Solanki5, Lindsay Puckett1, Rebecca Hagan2, William Sleeman2, Jatinder Palta2, Michael Hagan2*
1Department of Radiation Oncology, Veterans Affairs James A. Haley, Tampa, FL, USA
2Department of Radiation Oncology Kansas City VA, USA
3South African Herbal Science and Medicine Institute, University of the Western Cape, Bellville 7535, South Africa
4Department of Radiation Oncology, University of Michigan and Michigan VA Ann Arbor Healthcare, USA
5Department of Radiation Oncology, Loyola University Chicago and Edward Hines Jr. VA hospital, Department of Radiation
Oncology, Medical College of Wisconsin and Zablocki VA Medical Center, Milwaukee, WI, USA
- *Corresponding Author:
- Michael Hagan
Received date: July 09, 2020; Accepted date: July 27, 2020
Citation: Katsoulakis E, Kapoor R, Park J, Chapman C, Solanki A, Puckett L, et al. VA-Radiation Oncology Quality Surveillance Program: Enhancing Quality Measure Data Capture, Measuring Quality Benchmarks and Ensuring Long Term Sustainability of Quality Improvements in Community Care. Arch Cancer Biol Ther. 2020; 1(2): 25-30.
Copyright: © 2020 Katsoulakis E, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Introduction: Delivery of high-quality cancer care improves oncologic outcomes, including survival and quality of life. The VA National Radiation Oncology (NROP) established the VA Radiation Oncology Quality Surveillance Program (VAROQS) which has developed clinical quality measures (QM) as a measure of quality indices in radiation oncology. We sought to measure quality in community care, assess barriers to data capture, and develop solutions to ensure long term sustainability of continuous quality improvement for veterans that receive dual care, both within the VA and in non-VA community care (NVCC).
Methods: From 2016-2018, the VA-ROQS project randomly selected three Veterans Integrated Service Networks (VISNs) for quality analysis using established QM for prostate cancer, specifically: VISNs 6, 16, and 22. NROP manually abstracted data for QM treated in NVCC, which was compared to the performance of the VA QM in the same VISN as well as globally for all VISNs in the VA.
Results: Out of the 723 NVCC cases that were examined, none were fully evaluable for all 25 Prostate quality metrics. QM were able to be assessed in only 28% of NVCC patients (n=208) reviewed. Only 12/25 (48%) of all Prostate QM were able to be compared between VA and NVCC. Out of the 12 available Prostate QM, 9 were performance, 2 were surveillance, while 1 was an aspirational measure. The overall >75% pass rate of all the expected performance QM measures for the VA was 13/14 (92%). For NVCC, of the available expected QM for comparison, 8 of which were high potential impact, only 1/9 (11%) QM received a >75% pass rate in all three NVCC VISNs. When examining the 8 high potential impact QM, the VA had a 100% pass rate.
Conclusions: There are challenges to obtaining data to perform QM assessment from community care. For cases where QM performance could be assessed, VA care outperformed non-VA care. VA-ROQS program is an ongoing quality improvement initiative and in order to ensure that quality is comprehensively collected for NVCC, we propose a web-based portal that will enable providers to directly upload anonymized treatment information and the DICOM treatment plan.
Quality measures, VA Radiation Oncology Quality Surveillance Program, Non-VA community care, Radiation
High quality cancer care improves patient survival and quality of life. Radiation plays an important role in cancer management, given that over 50% of all cancer patients receive radiation therapy as either a primary treatment or for palliation. Ensuring quality of radiotherapy specifically, is therefore important to achieving optimal patient outcomes. Radiation oncology utilizes advanced technologies including image-guided intensity-modulated radiotherapy, stereotactic radiosurgery, and stereotactic body radiation therapy. Both the technical complexities of these treatments as well as the clinical decision making require quality assurance both within the VA as well as in non-VA radiotherapy practices . The VA has made assessment of radiotherapy a major priority. The Veterans Health Administration-National Radiation Oncology Program (VHA-NROP) has been designated to oversee radiation oncology operations and ensure quality performance of all VHA radiation oncology services. As part of this effort, the VHA-NROP has already introduced quality metrics (QM) developed by disease-site experts into a peer review setting designed to operate in the background of daily practice . Every radiation oncology case treated in the VA has the possibility of extracting detailed comprehensive quality assessments which may be compared against national standards. In the pilot study which examined lung and prostate cancer cases, over 1567 assessed cases within the VHA Radiation oncology practices passed 82.4% of all QM and 88% of dose-volume measures.
Although assessing radiotherapy quality is important, it is complicated by the fact that veterans’ healthcare system is elaborate, and over 50% of Veterans are dual users of both VA and non-VA community care (NVCC) facilities, and this number is substantially higher in Radiation Oncology [3-5]. Notably, 15,000 veterans receive radiation oncology services at the VA annually and over 45,914 dually-enrolled veterans receive radiation therapy [1,6]. The numbers utilizing non-VA radiation oncology care may be further increased through the MISSION Act . This legislation broadened the criteria for community care eligibility, increasing the proportion who can opt to receive radiotherapy closer to home and has given veterans substantial access to healthcare in the community. Although this may be logistically easier for veterans, obtaining non-VA care records is a well-described challenge . Moreover, MISSION Act requires assessment of the quality of non-VA care. The communication and coordination for patient care between VA and non-VA community hospitals is often disjointed and has been reported to affect patient outcomes [9,10]. The frequent use of community care may therefore complicate the quality assurance process. Additionally, quality assessment of non-VA radiation oncology care is further complicated by the technical complexity of radiation oncology. Many of the key details that impact patient outcomes are documented in a separate electronic medical record that is unfamiliar to most medical records departments. The challenges with obtaining outside records, coupled with the technical aspects of radiotherapy, may therefore limit the VHANROP’s ability to ascertain quality metrics for non-VA care . Furthermore, continuity of care, which is a core
value in cancer survivorship requires comprehensive and
complete data on cancer episodic-care of patients for ongong
Regrettably, the current system for receiving Radiation Oncology treatment data from community providers is cumbersome, error prone, and inadequate. In this study, we sought to quantify ascertainment of prostate cancer quality measures from non-VA community care practices. Using the VA-ROQS, we assessed the availability of quality measure abstraction data from non-VA practices and performed a comparative analysis on quality measures in VA and NVCC practices. Finally, we propose solutions to streamline care in the VA for dually enrolled veterans in non-VA community care practices.
The VA-ROQS program has been described previously . Briefly, the pilot effort addressed intermediaterisk and high-risk prostate cancer (CaP) as well as stage IIIA/B non-small cell lung cancers (NSCLC) and limited stage small cell lung cancers (SCLC). QM are divided into 3 categories: expected performance metrics, those anticipated for the near future (aspirational QM), and QM for surveillance only. For the current study, NROP manually abstracted data to assess quality measures for prostate cancer patients treated in NVCC practices. From 2016-2018, the VA-ROQS project examined a total of 723 cases from NVCC practices. Three VISNs were randomly selected for analysis of community care: 6 (Mid-Atlantic), 16 (South Central), and 22 (Desert Pacific). All community care cases from each VISN (VISN CC) were included for analysis. During this time, the community care program changed from the Veterans Choice program (VCP) to the VA Mission Act. The VCP ended on June 6, 2019 and the VA Mission Act established a new community care program. VISN 6 community care data was obtained from when community care was under VCP. Community care data from both VISN 16 and 22 was obtained from when community care was under the VA Mission Act.
Data were manually abstracted from consult notes as well as patient treatment completion summary documents that are returned to the VA and scanned into the VA EHR system, CPRS. Data collected included diagnosis, staging, imaging, performance status, and use of androgen deprivation therapy (long vs short term). Unfortunately, simulation and dose-volume histogram (DVH)-based objectives, complete DICOM data set, toxicity information, survivorship care plan and follow-up assessments were not available from community care records as these are not typically sent to the VA for patients treated in the community. The data were used to score the 25 quality measures for prostate cancer. None of the 17 DVH measures were scored as DVH records were not available from community care. The evaluation of each quality measure required extensive decision-tree logic to construct a “pass” or a “fail” score. The quality measures are shown in Supplemental Table 1. We assessed the number of
patients with available community care data for each QM.
Each QM has its own passing criteria. The performance of
Quality measures was scored from NVCC and compared to
VA aggregate performance (all VA VISNs).
We used the same scale for VA and non-VA practice to assess percent pass rates for each QM: <25%, 25-50%, 51- 75%, and >75%. There are 25 QM in total, with 14 expected performance measures, 8 aspirational measures, and 3 surveillance measures. Of the 14 expected performances measures, 8 were identified with high potential impact including: GU QM2 Staging (PSA, T-stage, Gleason score, risk group), GU QM3 Imaging/Staging (pelvic CT/ MRI and bone scan with T99 or NaF PET), GU QM7 Use of 3D or IMRT, GU QM9 ADT for high risk, GU QM11 Daily Target localization (cone base for conventional fractionation scheme (dose levels greater than equal to 7400cGy at 180 to 200cGy per fraction), GU QM 15 Appropriate dose for post-prostatectomy (dose levels greater than or equal to 6000cGy but less than or equal to 7200cGy at 180-200cGy/fx and QM 17A Post-Implant Dosimetry Evaluation (complete post-op dosimetry including CT or MRI imaging, prostate V100 evaluation, Prostate D90 evaluation, Rectum V100 evaluation, and physician review).
Out of the 723 cases that were examined from NVCC, 208 cases (28%) were available for QM assessment. Only 12 of the 25 GU QM were available for comparison between VA and non-VA care. Out of these 12 GU QM, 9 were expected performance measures (GU QM1, 2, 3, 4, 7, 9, 11, 14, and 15), 2 were surveillance measures (GU QM10 and 12), and 1 was aspirational (GU QM5). None of the DVH measures were scored as the DVH records were not available from NVCC. The percent passing rates for all VA and NVCC QM measures by high potential impact, all expected performance measures, aspirational measures, and surveillance measures are shown in Figure 1.
All expected QM (14) were scored for VA care. The VA care had all expected performance QM available for scoring and for these the overall pass rate was 92% (13/14 expected performance QM) >75% pass rate. There was a total of 9 expected QM that were available for comparison of VA and NVCC. Of these 9 expected QM, 2 were expected workflow workup QM (QM1 and QM4) while the remaining 7 QM were high potential impact QM. The VA scored >75% for all 9 expected QM and in NVCC, only 1/9 (11%) QM scored >75% passing. There was one high potential impact QM (QM 17A Post implant dosimetric evaluation) which was not able to be scored from all three NVCC VISNs. There were two high potential impact QM (QM 15 Appropriate dose for post-prostatectomy and QM 3 Imaging/Staging for High Risk prostate) which were not able to be scored from NVCC in VISNs 16 &22. In addition, there were two QM (QM 7 Use of 3D or IMRT and QM 11 Daily Target Localization) which were not able to be scored from VISN 6. The performance of Quality measures which were scored from the community care providers were also compared to VA performance in the same geographical VISN to assess for any differences in quality based on geography and the results mirrored the comparison to VA aggregate data of all VISNs (data not shown).
The data which is needed to assess high potential impact QM are extracted from different portions of the clinical workflow namely the diagnostic workup section of the chart as well as from treatment planning sections of the chart. The percent passing rates for all VA and NVCC QM measures by clinical workflow, specifically Diagnosis & Intent Workup measures, Treatment Planning Measures, and Follow-up measures are shown in Figure 2. Half of the high potential impact measures are from the clinical workflow of treatment planning (QM 11, 14, 15, and 17A). Moreover, out of the 8 high potential impact measures, the VA scored 100% (8/8) high potential impact QM >75% pass rate. For NVCC, of the 8 of high potential impact QM, only 1/8 (12.5%) QM received a >75% pass rate in all three NVCC VISNs. VISN 6, which was from VCP, had better performance than VISNs 16 and 22 which were under Mission Act. When examining available expected QM measures VISN 6 NVCC had 50% (3/6) >75% pass rate, 33% (2/6) >50-75% pass rate, and 17% (1/6) >25-50% pass rate. VISN 16 NVCC had 33.3% (2/6) >75% pass rate, 0% (0/6) >50-75% pass rate, 33.3% (2/6) >25-50% pass rate and 33.3% (2/ 6) <25% pass rate. VISN 22 NVCC had 29% (2/7) >75% pass rate, 14% (1/7) >50-75% pass rate, 29% (2/7) >25-50% pass rate, and 29% (2/7) <25% pass rate.
There was only one Aspiration measure QM5 (Quality of
Life assessment at consultation) available for comparison VA and NVCC. The VA and NVCC VISN 6 had a >50-75%
pass rate for this measure while VISN 16 NVCC had >25-
50% pass rate, and VISN 22 NVCC had <25% pass rate.
There were two surveillance measures (QM 10 ADT for
intermediate risk prostate cancer and 12 Comprehensive
treatment pelvic lymph nodes for high risk prostate
cancer) that were available for comparison. The VA had
>75% pass rate for both measures. VISN 6 NVCC had
available information only for QM 10 with a >50-75% pass
rate. VISN 16 NVCC had <25% and >25-50% pass rate
for QM 10 and QM 12 respectively. VISN 2 had the lowest
score for surveillance measures QM10 and 12 with <25%
pass rate for both measures.
In this study, a significant number of NVCC cases, approximately 70%, did not have available information from quality could be assessed. These findings highlight the challenges of ensuring quality benchmarks are enforced in a large complex healthcare system such as the VA. Radiation Oncology is deeply rooted in a culture of QA, safety, and continuous quality improvement . This is evident in the experience of multicenter cooperative group trials where multi-faceted radiation oncology QA is necessary . The difficulty required, along with the time and resources needed to perform nationwide high quality QA is quite significant . Oncologic outcomes are directly tied to adherence to quality standards, including adherence to dosimetric parameters. Failure to adhere to dosimetric parameters in RTOG 9704, a phase 3 study looking at adjuvant chemotherapy vs chemoradiation, showed decreased survival . The prospective RTOG 0116 and 0128 brachytherapy trials for cervical cancer also noted an adverse impact on local recurrence and disease free survival for poorer quality placement of the brachytherapy sources . These results highlight an
opportunity for developing pipelines to streamline care
for dual users of both VA and non-VA community care
(NVCC) facilities and the importance of comparing DVH
parameters for both external beam radiotherapy, as well as
evaluation of brachytherapy metrics between the centers.
Future attempts will need to focus on extracting this data
in an efficient manner from the NVCC.
This work represents one of the first attempts of EHR medical data transfer and data migration from one EHR to another EHR in a large community cancer network. In all cases where QM performance could be assessed, the VA consistently outperformed non-VA care. There were 8 high potential impact QMs established by VA-ASTRO expert panel which may affect treatment outcomes and are important measures of quality care. Notably, out of the 8 high potential impact QM, the VA scored 100% (8/8) high potential impact QM >75% pass rate. For NVCC, of the 8 of high potential impact QM, only 1/8 (12.5%) QM received a >75% pass rate in all three NVCC VISNs. Fifty percent of the high potential impact QM are from the clinical workflow of treatment planning. The poor performance of the NVCC is largely because the clinical workflow involving treatment planning data is often not received by the VA and the process of data retrieval is neither standardized nor automated. Currently, treatment completion data from community providers are faxed back to the VA which then must be manually scanned. These scanned documents are uploaded to the VA’s EMR (CPRS) for continuity of care for the veteran. This process has several downsides as it requires physical work to send and retrieve the data, the received data has no standard format, completeness of the treatment data is variable and there is no simple way to verify the treatment data was ever received or scanned. In addition, the scanned data is not in a format conducive of automated data analysis. It has been our experience that most of the QM data required was not present after a careful manual inspection of the returned documents. VAROQS is an ongoing quality improvement initiative. To ensure that complete community-based data is collected and that QM are accurately scored from community care, we are developing a Web-based portal that will allow NVCC providers to directly upload anonymized treatment information and the corresponding DICOM treatment plan. The use of discrete structured forms allows accurate data validation. Additionally, by processing treatments electronically we can quickly detect if required data has not been sent back to the VA. Once valid and complete data is being returned, QMs can now be automatically generated for further analysis. This semi-automated, Webbased system capitalizes on the benefits of a subject matter expert and the power of a modern digital data management platform.
There are several limitations to our study. Data from only 3 of the 40 VISNs was manually abstracted retrospectively during the time that non-VA community care switched from VCP to MISSION Act. Future studies will allow us to implement the web-based system and re-assess the performance of the VISNs with the newly developed data abstraction tools. In addition, another goal of the QM data abstraction from NVCC providers is to reinforce the importance of quality care and feedback mechanisms to ensure continued adherence to quality in NVCC. We hope to integrate healthcare delivery nationally across geography and across complex EHRs in order to improve outcomes for the individual veteran as well as for the nonveteran patients in the community who will also benefit from community awareness of quality measures and general program improvements.
The current system for receiving Radiation Oncology treatment data from community providers is cumbersome, error prone, and fails to provide discrete data elements that can be used for measuring treatment quality. For all cases where QM performance could be assessed, VA care outperformed non-VA care. High potential impact quality measures were consistently unavailable in NVCC. VAROQS is an ongoing quality improvement initiative and in order to ensure that quality is accurately collected from community care, we propose a web based portal that will enable providers to directly upload anonymized treatment information and the DICOM treatment plan. We are optimistic that our pipeline will streamline care in the VA for dually enrolled veterans in non-VA community care practices and optimize quality care.
Conflicts of Interest
No potential conflicts of interest exist.
This work was conducted as a clinical operations analysis of the VA National Radiation Oncology Program Office. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
- Kapoor R, Moghanaki D, Rexrode S, Monzon B, Ray M,
Hulick PR, et al. Quality improvements of Veterans Health
Administration radiation oncology services through
partnership for accreditation with the ACR. Journal of the
American College of Radiology. 2018 Dec 1;15(12):1732-
- Hagan M, Kapoor R, Michalski J, Sandler H, Movsas B,
Chetty I, et al. VA-Radiation Oncology Quality Surveillance
Program. International Journal of Radiation Oncology
Biology Physics. 2020 Mar 1;106(3):639-647.
- Charlton ME, Mengeling MA, Schlichting JA, Jiang L,
Turvey C, Trivedi AN, et al. Veteran Use of Health Care
Systems in Rural States: Comparing VA and Non-VA
Health Care Use Among Privately Insured Veterans Under
Age 65. The Journal of Rural Health. 2016 Sep;32(4):407-
- Liu CF, Bolkan C, Chan D, Yano EM, Rubenstein LV,
Chaney EF. Dual use of VA and non-VA services among
primary care patients with depression. Journal of General
Internal Medicine. 2009 Mar 1;24(3):305-11.
- Hynes DM, Koelling K, Stroupe K, Arnold N, Mallin K,
Sohn MW, et al. Veterans’ access to and use of Medicare
and Veterans Affairs health care. Medical Care. 2007 Mar
- French DD, Bradham DD, Campbell RR, Haggstrom
DA, Myers LJ, Chumbler NR, et al. Factors associated
with program utilization of radiation therapy treatment
for VHA and medicare dually enrolled patients. Journal of
Community Health. 2012 Aug 1;37(4):882-7.
- America OHFCotUSo: The Mission Act 1-89, 2018.
- Miller LB, Sjoberg H, Mayberry A, McCreight MS, Ayele
RA, Battaglia C. The advanced care coordination program:
a protocol for improving transitions of care for dual-use
veterans from community emergency departments back
to the Veterans Health Administration (VA) primary care.
BMC Health Services Research. 2019 Dec;19(1):734.
- Tsilimingras D, Bates DW. Addressing postdischarge
adverse events: a neglected area. The Joint Commission
Journal on Quality and Patient Safety. 2008 Feb;34(2):85-
- Axon RN, Gebregziabher M, Everett CJ, Heidenreich
P, Hunt KJ. Dual health care system use is associated with
higher rates of hospitalization and hospital readmission
among veterans with heart failure. American Heart
Journal. 2016 Apr 1;174:157-63.
- Thaker NG, Sturdevant L, Jhingran A, Das P, Delclos
ME, Gunn GB, et al. Assessing the quality of a radiation
oncology case-based, peer-review program in an integrated
academic and community cancer center network. Journal
of Oncology Practice. 2016 Apr;12(4):e476-86.
- Fairchild A, Straube W, Laurie F, Followill D.
Does quality of radiation therapy predict outcomes of
multicenter cooperative group trials? A literature review.
International Journal of Radiation Oncology Biology
Physics. 2013 Oct 1;87(2):246-60.
- Abrams RA, Winter KA, Regine WF, Safran H,
Hoffman JP, Lustig R, et al. Failure to adhere to protocol
specified radiation therapy guidelines was associated
with decreased survival in RTOG 9704—a phase III trial
of adjuvant chemotherapy and chemoradiotherapy for
patients with resected adenocarcinoma of the pancreas.
International Journal of Radiation Oncology Biology
Physics. 2012 Feb 1;82(2):809-16.
- Viswanathan AN, Moughan J, Small W, Levenback
C, Iyer R, Hymes S, et al. The quality of cervical cancer
brachytherapy implantation and the impact on local
recurrence and disease-free survival in radiation therapy
oncology group prospective trials 0116 and 0128.
International Journal of Gynecologic Cancer. 2012 Jan