High quality cancer care improves patient survival and quality of life. Radiation plays an important role in cancer management, given that over 50% of all cancer patients receive radiation therapy as either a primary treatment or for palliation. Ensuring quality of radiotherapy specifically, is therefore important to achieving optimal patient outcomes. Radiation oncology utilizes advanced technologies including image-guided intensity-modulated radiotherapy, stereotactic radiosurgery, and stereotactic body radiation therapy. Both the technical complexities of these treatments as well as the clinical decision making require quality assurance both within the VA as well as in non-VA radiotherapy practices [1]. The VA has made assessment of radiotherapy a major priority. The Veterans Health Administration-National Radiation Oncology Program (VHA-NROP) has been designated to oversee radiation oncology operations and ensure quality performance of all VHA radiation oncology services. As part of this effort, the VHA-NROP has already introduced quality metrics (QM) developed by disease-site experts into a peer review setting designed to operate in the background of daily practice [2]. Every radiation oncology case treated in the VA has the possibility of extracting detailed comprehensive quality assessments which may be compared against national standards. In the pilot study which examined lung and prostate cancer cases, over 1567 assessed cases within the VHA Radiation oncology practices passed 82.4% of all QM and 88% of dose-volume measures.

Although assessing radiotherapy quality is important, it is complicated by the fact that veterans’ healthcare system is elaborate, and over 50% of Veterans are dual users of both VA and non-VA community care (NVCC) facilities, and this number is substantially higher in Radiation Oncology [3-5]. Notably, 15,000 veterans receive radiation oncology services at the VA annually and over 45,914 dually-enrolled veterans receive radiation therapy [1,6]. The numbers utilizing non-VA radiation oncology care may be further increased through the MISSION Act [7]. This legislation broadened the criteria for community care eligibility, increasing the proportion who can opt to receive radiotherapy closer to home and has given veterans substantial access to healthcare in the community. Although this may be logistically easier for veterans, obtaining non-VA care records is a well-described challenge [8]. Moreover, MISSION Act requires assessment of the quality of non-VA care. The communication and coordination for patient care between VA and non-VA community hospitals is often disjointed and has been reported to affect patient outcomes [9,10]. The frequent use of community care may therefore complicate the quality assurance process. Additionally, quality assessment of non-VA radiation oncology care is further complicated by the technical complexity of radiation oncology. Many of the key details that impact patient outcomes are documented in a separate electronic medical record that is unfamiliar to most medical records departments. The challenges with obtaining outside records, coupled with the technical aspects of radiotherapy, may therefore limit the VHANROP’s ability to ascertain quality metrics for non-VA care [1]. Furthermore, continuity of care, which is a core value in cancer survivorship requires comprehensive and complete data on cancer episodic-care of patients for ongong clinical management.

Regrettably, the current system for receiving Radiation Oncology treatment data from community providers is cumbersome, error prone, and inadequate. In this study, we sought to quantify ascertainment of prostate cancer quality measures from non-VA community care practices. Using the VA-ROQS, we assessed the availability of quality measure abstraction data from non-VA practices and performed a comparative analysis on quality measures in VA and NVCC practices. Finally, we propose solutions to streamline care in the VA for dually enrolled veterans in non-VA community care practices.

The VA-ROQS program has been described previously [2]. Briefly, the pilot effort addressed intermediaterisk and high-risk prostate cancer (CaP) as well as stage IIIA/B non-small cell lung cancers (NSCLC) and limited stage small cell lung cancers (SCLC). QM are divided into 3 categories: expected performance metrics, those anticipated for the near future (aspirational QM), and QM for surveillance only. For the current study, NROP manually abstracted data to assess quality measures for prostate cancer patients treated in NVCC practices. From 2016-2018, the VA-ROQS project examined a total of 723 cases from NVCC practices. Three VISNs were randomly selected for analysis of community care: 6 (Mid-Atlantic), 16 (South Central), and 22 (Desert Pacific). All community care cases from each VISN (VISN CC) were included for analysis. During this time, the community care program changed from the Veterans Choice program (VCP) to the VA Mission Act. The VCP ended on June 6, 2019 and the VA Mission Act established a new community care program. VISN 6 community care data was obtained from when community care was under VCP. Community care data from both VISN 16 and 22 was obtained from when community care was under the VA Mission Act.

Data were manually abstracted from consult notes as well as patient treatment completion summary documents that are returned to the VA and scanned into the VA EHR system, CPRS. Data collected included diagnosis, staging, imaging, performance status, and use of androgen deprivation therapy (long vs short term). Unfortunately, simulation and dose-volume histogram (DVH)-based objectives, complete DICOM data set, toxicity information, survivorship care plan and follow-up assessments were not available from community care records as these are not typically sent to the VA for patients treated in the community. The data were used to score the 25 quality measures for prostate cancer. None of the 17 DVH measures were scored as DVH records were not available from community care. The evaluation of each quality measure required extensive decision-tree logic to construct a “pass” or a “fail” score. The quality measures are shown in Supplemental Table 1. We assessed the number of patients with available community care data for each QM. Each QM has its own passing criteria. The performance of Quality measures was scored from NVCC and compared to VA aggregate performance (all VA VISNs).

We used the same scale for VA and non-VA practice to assess percent pass rates for each QM: <25%, 25-50%, 51- 75%, and >75%. There are 25 QM in total, with 14 expected performance measures, 8 aspirational measures, and 3 surveillance measures. Of the 14 expected performances measures, 8 were identified with high potential impact including: GU QM2 Staging (PSA, T-stage, Gleason score, risk group), GU QM3 Imaging/Staging (pelvic CT/ MRI and bone scan with T99 or NaF PET), GU QM7 Use of 3D or IMRT, GU QM9 ADT for high risk, GU QM11 Daily Target localization (cone base for conventional fractionation scheme (dose levels greater than equal to 7400cGy at 180 to 200cGy per fraction), GU QM 15 Appropriate dose for post-prostatectomy (dose levels greater than or equal to 6000cGy but less than or equal to 7200cGy at 180-200cGy/fx and QM 17A Post-Implant Dosimetry Evaluation (complete post-op dosimetry including CT or MRI imaging, prostate V100 evaluation, Prostate D90 evaluation, Rectum V100 evaluation, and physician review).

Out of the 723 cases that were examined from NVCC, 208 cases (28%) were available for QM assessment. Only 12 of the 25 GU QM were available for comparison between VA and non-VA care. Out of these 12 GU QM, 9 were expected performance measures (GU QM1, 2, 3, 4, 7, 9, 11, 14, and 15), 2 were surveillance measures (GU QM10 and 12), and 1 was aspirational (GU QM5). None of the DVH measures were scored as the DVH records were not available from NVCC. The percent passing rates for all VA and NVCC QM measures by high potential impact, all expected performance measures, aspirational measures, and surveillance measures are shown in Figure 1.

All expected QM (14) were scored for VA care. The VA care had all expected performance QM available for scoring and for these the overall pass rate was 92% (13/14 expected performance QM) >75% pass rate. There was a total of 9 expected QM that were available for comparison of VA and NVCC. Of these 9 expected QM, 2 were expected workflow workup QM (QM1 and QM4) while the remaining 7 QM were high potential impact QM. The VA scored >75% for all 9 expected QM and in NVCC, only 1/9 (11%) QM scored >75% passing. There was one high potential impact QM (QM 17A Post implant dosimetric evaluation) which was not able to be scored from all three NVCC VISNs. There were two high potential impact QM (QM 15 Appropriate dose for post-prostatectomy and QM 3 Imaging/Staging for High Risk prostate) which were not able to be scored from NVCC in VISNs 16 &22. In addition, there were two QM (QM 7 Use of 3D or IMRT and QM 11 Daily Target Localization) which were not able to be scored from VISN 6. The performance of Quality measures which were scored from the community care providers were also compared to VA performance in the same geographical VISN to assess for any differences in quality based on geography and the results mirrored the comparison to VA aggregate data of all VISNs (data not shown).

The data which is needed to assess high potential impact QM are extracted from different portions of the clinical workflow namely the diagnostic workup section of the chart as well as from treatment planning sections of the chart. The percent passing rates for all VA and NVCC QM measures by clinical workflow, specifically Diagnosis & Intent Workup measures, Treatment Planning Measures, and Follow-up measures are shown in Figure 2. Half of the high potential impact measures are from the clinical workflow of treatment planning (QM 11, 14, 15, and 17A). Moreover, out of the 8 high potential impact measures, the VA scored 100% (8/8) high potential impact QM >75% pass rate. For NVCC, of the 8 of high potential impact QM, only 1/8 (12.5%) QM received a >75% pass rate in all three NVCC VISNs. VISN 6, which was from VCP, had better performance than VISNs 16 and 22 which were under Mission Act. When examining available expected QM measures VISN 6 NVCC had 50% (3/6) >75% pass rate, 33% (2/6) >50-75% pass rate, and 17% (1/6) >25-50% pass rate. VISN 16 NVCC had 33.3% (2/6) >75% pass rate, 0% (0/6) >50-75% pass rate, 33.3% (2/6) >25-50% pass rate and 33.3% (2/ 6) <25% pass rate. VISN 22 NVCC had 29% (2/7) >75% pass rate, 14% (1/7) >50-75% pass rate, 29% (2/7) >25-50% pass rate, and 29% (2/7) <25% pass rate.

There was only one Aspiration measure QM5 (Quality of Life assessment at consultation) available for comparison VA and NVCC. The VA and NVCC VISN 6 had a >50-75% pass rate for this measure while VISN 16 NVCC had >25- 50% pass rate, and VISN 22 NVCC had <25% pass rate. There were two surveillance measures (QM 10 ADT for intermediate risk prostate cancer and 12 Comprehensive treatment pelvic lymph nodes for high risk prostate cancer) that were available for comparison. The VA had >75% pass rate for both measures. VISN 6 NVCC had available information only for QM 10 with a >50-75% pass rate. VISN 16 NVCC had <25% and >25-50% pass rate for QM 10 and QM 12 respectively. VISN 2 had the lowest score for surveillance measures QM10 and 12 with <25% pass rate for both measures.

In this study, a significant number of NVCC cases, approximately 70%, did not have available information from quality could be assessed. These findings highlight the challenges of ensuring quality benchmarks are enforced in a large complex healthcare system such as the VA. Radiation Oncology is deeply rooted in a culture of QA, safety, and continuous quality improvement [11]. This is evident in the experience of multicenter cooperative group trials where multi-faceted radiation oncology QA is necessary [12]. The difficulty required, along with the time and resources needed to perform nationwide high quality QA is quite significant [12]. Oncologic outcomes are directly tied to adherence to quality standards, including adherence to dosimetric parameters. Failure to adhere to dosimetric parameters in RTOG 9704, a phase 3 study looking at adjuvant chemotherapy vs chemoradiation, showed decreased survival [13]. The prospective RTOG 0116 and 0128 brachytherapy trials for cervical cancer also noted an adverse impact on local recurrence and disease free survival for poorer quality placement of the brachytherapy sources [14]. These results highlight an opportunity for developing pipelines to streamline care for dual users of both VA and non-VA community care (NVCC) facilities and the importance of comparing DVH parameters for both external beam radiotherapy, as well as evaluation of brachytherapy metrics between the centers. Future attempts will need to focus on extracting this data in an efficient manner from the NVCC.

This work represents one of the first attempts of EHR medical data transfer and data migration from one EHR to another EHR in a large community cancer network. In all cases where QM performance could be assessed, the VA consistently outperformed non-VA care. There were 8 high potential impact QMs established by VA-ASTRO expert panel which may affect treatment outcomes and are important measures of quality care. Notably, out of the 8 high potential impact QM, the VA scored 100% (8/8) high potential impact QM >75% pass rate. For NVCC, of the 8 of high potential impact QM, only 1/8 (12.5%) QM received a >75% pass rate in all three NVCC VISNs. Fifty percent of the high potential impact QM are from the clinical workflow of treatment planning. The poor performance of the NVCC is largely because the clinical workflow involving treatment planning data is often not received by the VA and the process of data retrieval is neither standardized nor automated. Currently, treatment completion data from community providers are faxed back to the VA which then must be manually scanned. These scanned documents are uploaded to the VA’s EMR (CPRS) for continuity of care for the veteran. This process has several downsides as it requires physical work to send and retrieve the data, the received data has no standard format, completeness of the treatment data is variable and there is no simple way to verify the treatment data was ever received or scanned. In addition, the scanned data is not in a format conducive of automated data analysis. It has been our experience that most of the QM data required was not present after a careful manual inspection of the returned documents. VAROQS is an ongoing quality improvement initiative. To ensure that complete community-based data is collected and that QM are accurately scored from community care, we are developing a Web-based portal that will allow NVCC providers to directly upload anonymized treatment information and the corresponding DICOM treatment plan. The use of discrete structured forms allows accurate data validation. Additionally, by processing treatments electronically we can quickly detect if required data has not been sent back to the VA. Once valid and complete data is being returned, QMs can now be automatically generated for further analysis. This semi-automated, Webbased system capitalizes on the benefits of a subject matter expert and the power of a modern digital data management platform.

There are several limitations to our study. Data from only 3 of the 40 VISNs was manually abstracted retrospectively during the time that non-VA community care switched from VCP to MISSION Act. Future studies will allow us to implement the web-based system and re-assess the performance of the VISNs with the newly developed data abstraction tools. In addition, another goal of the QM data abstraction from NVCC providers is to reinforce the importance of quality care and feedback mechanisms to ensure continued adherence to quality in NVCC. We hope to integrate healthcare delivery nationally across geography and across complex EHRs in order to improve outcomes for the individual veteran as well as for the nonveteran patients in the community who will also benefit from community awareness of quality measures and general program improvements.

The current system for receiving Radiation Oncology treatment data from community providers is cumbersome, error prone, and fails to provide discrete data elements that can be used for measuring treatment quality. For all cases where QM performance could be assessed, VA care outperformed non-VA care. High potential impact quality measures were consistently unavailable in NVCC. VAROQS is an ongoing quality improvement initiative and in order to ensure that quality is accurately collected from community care, we propose a web based portal that will enable providers to directly upload anonymized treatment information and the DICOM treatment plan. We are optimistic that our pipeline will streamline care in the VA for dually enrolled veterans in non-VA community care practices and optimize quality care.

No potential conflicts of interest exist.

This work was conducted as a clinical operations analysis of the VA National Radiation Oncology Program Office. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.