Abstract
Background: This paper analyses any possible association of various factors like gender, last COVID-19 PCR test results, BCG Vaccination, Seasonal Flu vaccination, occupation and confirmed case contact history with COVID-19 RDT results of the participants. COVID-19 will soon become endemic in Pakistan, the government should adopt COVID-19 RDT kits for trace, test and quarantine activities.
Methodology: Considering the availability of COVI-19 rapid diagnostic kits, 596 individuals all previously COVID-19 PCR tested were made part of this cross-sectional study. Simple random sampling was used for the selection of study participants. The whole study was conducted during September and October 2020.
Results: The major findings of this study are clearly showing that the Positive Likely hood ratio of the COVID-19 RDT Kits (LR+) is well above 1; similarly, the negative Likely hood ratio is approaching 0. On the other hand, the sensitivity and specificity 80% and 74% respectively. Similarly, study found statistically significant association was between RDT out comes and last PCR Test status, occupation and contact with COVID-19 positive individuals. While other variables like gender, BCG vaccination and history of seasonal flu vaccinations were found to have no significant associations with COVID-19 RDT Kit out comes.
Conclusion: Being the first study of its kind in Pakistan the major findings of this study are almost in line with the set hypothesis and objectives of this study and based on study findings it will be of high value to use COVID-19 RDT kits during mass screening especially during test, trace and quarantine activities.
Keywords
COVID-19, RDT (Rapid Diagnostic test), PCR, Positive and negative likely hood ratios
Summary Box
Q: What is already known on this subject?
COVID-19 is a new disease, not much is known about it. COVID-19 RDT Kits have been made and testing have shown promising results.
Q: What this study adds?
This study adds that in the days of COVID-19 pandemic the world will be facing multiple waves of this pandemic till the development of vaccine ,till that time the only option left with us Test trace and Quarantine(TTQ). With time shortage and resources wastage this study opens an opportunity of window and encourages to use COVID-19 Rapid diagnostic test kits on mass levels for activities like TTQ and bedside diagnosis.
Background
The COVID-19 infections are characterized by highly nonspecific manifestations including respiratory symptoms, fever, cough, difficulty in breathing. These symptoms are also seen as clinical presentations of other virus-related diseases including influenza [1,2]. This poses a challenge in identifying those patients with COVID-19 from individuals with other respiratory diseases. Therefore, there is a need for a diagnostic test that is rapid, accurate and cost efficient that may be used at point-of-care to screen and confirm suspected cases. Early case detection has been proven to have a dramatic effect in controlling infectious disease outbreaks [3]. The current, WHO-recommended gold standard for the diagnosis of COVID-19 is the qualitative detection of SARS-CoV-2 virus nucleic acid via reverse transcription polymerase chain reaction (RT-PCR). RT-PCR is reported to have a sensitivity of 95% and a specificity of 100%; for every 100 COVID-19 positive patients, RT-PCR would have a falsely negative result in 5 patients [4]. The test however, has limitations such as long turnaround times and complicated logistical operations that makes it infeasible as a rapid and simple field test option to screen and diagnose patients. Another proposed rapid, simple, and highly sensitive way to diagnose COVID-19 is through the qualitative detection of antibodies that are specific to SARS-CoV-2 instead of the direct detection and measurement of viral load through RT-PCR. Several studies have investigated the use of antibodies in the diagnosis of COVID-19 using ELISA [5-7] and lateral flow rapid test kits [8,9]. These studies show different sensitivity and specificity results, and different recommended timing of testing. Disparities between ELISA and lateral flow rapid test results may be due to the longer incubation time for ELISA compared to swift resolution for lateral flow tests, the slow kinetic dissociation rate of ELISA compared to a faster kinetic association rate in lateral flow tests, and the ELISA “capture” antibody and “detector” antibody designations may be reversed in lateral flow tests [10]. Thus, results from studies using ELISA and lateral flow tests should be analyzed separately. A rapid point-of-care lateral flow immunoassay test product was developed intended for qualitative detection of IgM/IgG in human blood within 15 minutes. It has been designed to be a complementary aid in the diagnosis of patients suspected to have the COVID-19 infection. Limitations of the test include the following: 1) It does not directly confirm virus presence, instead, it provides serological evidence of recent infection, 2) It is not known if the test will cross-react with antibodies to other coronaviruses and flu viruses [8]. Further, studies in patients with Severe Acute Respiratory Syndrome (SARS), a disease also caused by a coronavirus, show that IgM is detectable as early as 3 to 6 days from symptom onset, while IgG is detectable after 8 days, with peak titers at 15 to 20 days [9,11]. This potentially limits the clinical and public health utility of antibody tests for the early diagnosis of coronavirus infections. Clinical trials that investigate the accuracy and safety of IgM/IgG rapid test kits for diagnosis of COVID-19 patients are still limited.
Literature Review
This rapid review summarizes the available evidence on the accuracy and safety of lateral flow immunoassay (LFIA) IgM/IgG rapid test kits in diagnosing patients with COVID-19. Evidence on the accuracy and safety of the enzyme-linked immunosorbent assay (ELISA) test for COVID-19 are summarized in a separate rapid review.
Population: Symptomatic and asymptomatic COVID-19 patients and suspected COVID19 patients of any age, with any comorbidities, any severity
• Intervention: Antibody/Antigen test, IgM/IgG rapid test kit
• Comparator: Reverse Transcriptase Polymerase Chain Reaction (RT-PCR)
• Outcomes: Sensitivity, Specificity, Time to detection of antibodies
• Methods: randomized controlled trials (RCTs), nonrandomized studies, cohort studies, case-control studies, cross-sectional studies.
After comprehensive search and appraisal, two (2) completed studies (Appendix 1) and one (1) ongoing trial (Appendix 2) on the accuracy of IgM/IgG antibody test kits for diagnosing COVID-19 were identified. Li et al., examined 525 blood samples of clinically positive (including PCR test) (n = 397) and clinically negative(n=128)patients to determine these sensitivity and specificity of the IgM /IgG rapid test kit [8].On the other hand,Yingetal.,investigated179 patients who were PCR positive (n=90) And PCR negative (n=89) comparing the over-all sensitivity and specificity of the antibody test kit and when done between day 0-7,day8-15, or day16 and beyond [9].In both studies, a positive test result was defined as detection of either or both IgM and IgG antibodies to SARS-CoV-2. A negative test result was defined as nondetection of any or both IgM and IgG antibodies to SARSCoV- 2 [8,9]. The ongoing trial (NCT04316728) is designed to evaluate the clinical performance of IgM/IgG antibody test kits in the early diagnosis of COVID-19 in high-risk populations. The study will serially test uninfected healthcare workers and individuals with chronic conditions (n=200) and is expected to complete data collection by September 2020.
Critical appraisal
Two studies provided direct evidence on the accuracy of lateral flow immunoassay (LFIA)IgM/IgG rapid test kits in diagnosing COVID-19 compared with PCR and clinical picture as the reference standard. Both studies did not adequately describe the methods used to validate the accuracy of the test kits. As such, it is difficult to ascertain whether or not safeguards to ensure a good estimate of the test kit’s diagnostic accuracy were in place; the absence of these safeguards would tend to result in an overestimate of the test kit’s diagnostic accuracy. As such, the results of these two studies need to be interpreted with caution.
Accuracy outcomes
The overall accuracy of the rapid test from the two identified studies are summarized in Table 1.
| Author | Sample Size | Sensitivity | Specificity | Positive Likelihood Ratio (LR+) | Negative Likelihood Ratio (LR-) |
|---|---|---|---|---|---|
| Li et al. | 525 | 88.7% | 90.6% | 9.46 | 0.13 |
| Ying et al. | 179 | 85.6% | 91.0% | 9.52 | 0.16 |
Table 1: Accuracy of the rapid test from the two identified studies.
In majority of cases, Li et al. was not able to determine the number of days from symptom onset to the time the blood sample for the rapid test was collected. However, in a subset of patients from one institution (n=58), the blood samples were collected at day 8 to 33 after symptom onset [8]. Ying et al. reported the time from onset of illness to blood sample collection in 115 patients. The accuracy of the rapid test kit, stratified according to number of days of onset, is summarized in Table 2. The sensitivity (18.8%) of the rapid test was extremely low among those who had their blood samples collected within the first week of symptom onset [9]. Accuracy of IgM/IgG rapid test kits, stratified according to the number of days after onset of symptoms, Ying et al. [9].
| Day test done | Sample Size | Sensitivity | Specificity | Positive Likelihood Ratio (LR+) | Negative Likelihood Ratio (LR-) |
|---|---|---|---|---|---|
| 0 to 7 days | 25 | 18.8% | 77.8% | 0.84 | 1.05 |
| 8 to 15 days | 8 | 100.0% | 50.0% | 2.0 | 0.17* |
| 16 days | 82 | 100.0% | 64.3% | 2.8 | 0.01* |
Table 2: The accuracy of the rapid test kit, stratified according to number of days of onset, is summarized below.
Problem statement: COVID 19 is a new disease not much is known about it and it would soon become endemic in Pakistan.
Till 30th August 2020, 1215 confirmed cases have been reported and local transmission is on its peak with 7 infected districts in Balochistan. The current cases of COVID-19 are iceberg of diseases, there may be many more cases in community.
Rationale: The present standard for diagnosis of COVID-19 is through qualitative detection of COVID19 virus nucleic acid via reverse transcription polymerase chain reaction (RT-PCR). Due to long turnaround times and complicated logistical operations, a rapid and simple field test alternative is needed to diagnose and screen patients. There is delayed in results due to limited RT PCR Laboratory in Balochistan Province and its limited capacity to carry out daily PCR test. Even it was observed that results are declared after death of Positive case. As compared to RT-PCR, RDT is simple, cost effective and timely test to be conducted in field. As, Novel nature of disease, antibodies studies are conducted on limited level throughout Pakistan. Study is justifiable because so far, no published study has ascertained the association of various known factors with immunity status using COVID-19 RDT results among suspected, confirmed and exposed individuals so that in the long run Possible COVID-19 exposure and encounter event could be made from individual’s history while performing Trace, test and Quarantine (TTQ) activities in the field till the development of a vaccine.
Research Questions
Q: Is there any association between COVID-19 RDT outcomes and certain variable of interest like Gender, Last COVID-19 PCR results, BCG Vaccination, Seasonal Flu vaccination, Occupation and Close contact or living with confirmed case of COVID-19 in last 15-45 days?
Q: Is COVID-19 RDT Kits suitable for diagnostic Purposes?
Objectives
1. To ascertain the association of various known factors with immunity developed against COVID-19 using serological COVID-19 RDT for suspected, confirmed and exposed individuals.
2. To determine the Positive and Negative Likely hood Ratios for the COVID-19 RDT kits.
3. To ascertain clinical history based positive COVID-19 cases in the field while performing Test, Trace and Quarantine (TTQ) activities in the field till the development of a vaccine.
Hypothesis
H10: COVID-19 Serological Rapid Diagnostic test outcome has no association with Gender, Last COVID-19 PCR report, BCG vaccination, Seasonal flue vaccination, Occupation and Close contact or living history with confirmed case of COVID-19 in last 15-45 days.
H1a: COVID-19 Serological Rapid Diagnostic test outcome has association with Gender, Last COVID-19 PCR report, BCG vaccination, Seasonal flue vaccination, Occupation and Close contact or living history with confirmed case of COVID-19 in last 15-45 days.
H20: There exists no association between COVID-19 RDT Kits out comes and COVID-19 Diagnosis.
H2a: There exists an association between COVID-19 RDT Kits out comes and COVID-19 Diagnosis.
Research Methodology
A Cross-sectional study was performed, considering the availability of COVID-19 Rapid Diagnostic test kits a total of 596 individuals were included in this study. All of these participants were selected by Simple random sampling after acquiring the “Complete name list” of all the individuals whom had done their PCR lab test in Fatimah Jinnah Chest Hospital Quetta. The study setting was District Quetta Balochistan, the whole population of Balochistan was considered as sample population. Every individual of any age and gender belonging to any district of Balochistan who had done his or her COVID-19 PCR test at PCR Lab Fatimah Jinnah Chest hospital Quetta from 1st January 2020 till 30th August 2020 were included in the study regardless of their PCR test reports. Prior to data collection ethical review board at Director General Health Services Balochistan was approached for the approval. Informed consents were acquired from each participant after briefing them. Data collection was done through interviews using structured questioner. Later on, at the end of each interview a single fresh blood drop was acquired from every individual for COVID-19 Rapid Diagnostic test. The whole study was conducted during September and October 2020.
Descriptive statistics was used for age, sex, occupation, educations status, & blood groups of the study participants while inferential statistics like Chi-square was used to assess various associations. SPSS Software -19 versions was used to analyze the data. Similarly, positive and negative likely hood ratios calculated using 2x2 Table.
| S.NO | Socio-demographic Characteristics of the study participants |
Breakup (Where applied) | Percentages (%) | Number (n) |
|---|---|---|---|---|
| 01 | Gender | Male Participants | 77% | 459 |
| Female Participants | 23% | 137 | ||
| 02 | Qualification | Un Educated | 1% | 6 |
| Primary | 5% | 30 | ||
| Metric | 6% | 38 | ||
| Intermediate | 2% | 13 | ||
| Graduation | 5% | 28 | ||
| Masters | 2% | 9 | ||
| PhD | 0% | 0 | ||
| 03 | Co-Morbidities | DM | 2% | 9 |
| Asthma | 0% | 1 | ||
| HTN | 1% | 6 | ||
| Ovarian CA | 0% | 1 | ||
| Hepatitis | 0% | 1 | ||
| Non | 97% | 578 | ||
| 11 | Occupation | Health Care Provider | 49% | 292 |
| Others | 51% | 303 | ||
| 12 | BCG Vaccination status | BCG Vaccination NO |
38% | 226 |
| BCG Vaccination Yes | 62% | 370 | ||
| 13 | Seasonal Flu Vaccination status | Seasonal Flu vaccination Yes | 7% | 42 |
| Seasonal Flu vaccination NO |
93% | 554 | ||
| 14 | Contact History with a Positive case | History of any close contact or living with confirmed case of COVID-19 in last 01-45 days: NO | 41% | 244 |
| History of any close contact or living with confirmed case of COVID-19 in last 01-45 days: Yes | 59% | 352 | ||
| 15 | Blood Groups of the Participants |
A+ | 14% | 81 |
| A- | 1% | 6 | ||
| B+ | 14% | 82 | ||
| B- | 1% | 5 | ||
| AB+ | 4% | 22 | ||
| AB- | 0.16% | 1 | ||
| O+ | 15% | 88 | ||
| O- | 1.8% | 11 | ||
| unknown | 50.3% | 300 | ||
| 16 | Status of Last PCR Report of the Participants | Negative | 58% | 346 |
| Positive | 42% | 250 | ||
| 16 | Status of Last PCR Report of the Participants | Negative COVID-19 Rapid Diagnostic test |
51% | 304 |
| Positive COVID-19 Rapid Diagnostic test | 49% | 292 |
Table 3: Socio-democratic characteristics of the participants.
| COVID-19 PCR Report | ||||
|---|---|---|---|---|
| Positive | Negative | Total | ||
| COVID-19 RDT TEST Result |
POSITIVE | 200 | 90 | 290 |
| Negative | 50 | 256 | 306 | |
| TOTAL | 250 | 346 | 596 | |
Table 4: 2x2 table for Positive and Negative likely hood ratios calculations.
Results
Descriptive statistics
Inferential Statistics: The overall accuracy of the COVID-19 rapid diagnostic test among the study participants was found to be as below:
Positive Likely hood ratio = Sensitivity/1-Specificity
Putting the values
= 200/250 / 1-256/346
= 3.07
Negative Likely hood ratio = 1-Sensitivity/Specificity
Putting the values:
= 1-200/250 / 256/346
= 0.28
| When COVID-19 RDT Tests were Done |
Sample Size | Sensitivity | Specificity | Positive Likely Hood Ratio LR+ | Negative Likely Hood Ratio LR- |
|---|---|---|---|---|---|
| They were done after the 14 days of Quarantine & Last PCR Reports. |
596 | 39.20% | 86% | 2.8 | 0.7 |
Table 5: Overall Summary sheet.
It is clear from the analysis that the Positive Likely hood ratio LR+ is more than 1, while Negative Likely hood ratio is approaching 0. Hence COVID-19 RDT Kits could be used for diagnostic purposes in all Clinical and Screening setup and especially for Test trace and Quarantine (TTQ) activities.
Likewise, Chi-square test of association showed:
Statistically significant association was found between RDT out comes and Last PCR Test status, Occupation and Contact with COVID-19 positive individuals.
| RDT Results | X2 (df) | p-value | ||
|---|---|---|---|---|
| Positive | Negative | |||
| Last PCR Test Status | ||||
| Positive | 200(34%) | 50 (8.3%) | 194.52(1) | 0.00 |
| Negative | 90(15.10%) | 293(49.1%) | ||
| Gender | ||||
| Male | 186(31.2%) | 271(45.5%) | 3.480(1) | 0.062 |
| Female | 69(11.6%) | 70(11.7%) | ||
| BCG Mark of Right Arm | ||||
| Present | 147(24.7%) | 220(36.9%) | 2.910(1) | 0.085 |
| Absent | 108(18.1%) | 121(20.3%) | ||
| History of Seasonal Influenza vaccination | ||||
| Yes | 18(3%) | 20(3.4%) | 0.348(1) | 0.555 |
| No | 237(39.8%) | 321(53.9%) | ||
| Occupation-Health Care Provider | ||||
| Yes | 110(18.5%) | 183(30.7%) | 6.471(1) | 0.011 |
| No | 145(24.3%) | 158(26.5%) | ||
| Close contact or living with confirmed case of COVID-19 in last 01-45 days | ||||
| Yes | 109(18.3%) | 174(29.2%) | 4.012(1) | 0.045 |
| No | 146(24.5%) | 167(28%) | ||
Table 6: Chi-square test of association.
Discussion
The major findings of this study are almost in line with the set hypothesis and objectives of this study, this study is clearly showing that the Positive Likely hood ratio of the COVID-19 RDT Kits (LR+) is well above 1; similarly, the Negative Likely hood ratio is approaching 0. On the other hand, the Sensitivity and Specificity 80% and 74% respectively. Similarly, study found statistically significant association was between RDT out comes and Last PCR Test status, Occupation and Contact with COVID-19 positive individuals. While other variables like Gender, BCG Vaccination and history of seasonal flu vaccinations were found to have no significant associations with COVID-19 RDT Kit out comes.
The results from this study seems to be consistent with previous studies like in his study Li et al was able to report COVID-19 RDT kits sensitivity as 88.7%, specificity as 90.6%, LR+ = ratio as 9.46 while LR- = ratio as 0.13 [8]. In another study Ying et al has reported sensitivity to be 85.6%, Specificity as 91.0%, LR+ = 9.52 and LR- = 0.16 [9]. Which in comparison to this our study has reported _ sensitivity 80%, specificity of 74% respectively while LR+ = 3.07 and LR- = 0.28. All the findings of our study are almost identical. It can be clearly Seen that in all three studies COVID-19 RDT Kits (LR+) is well above 1; similarly, the negative likely hood ratio is approaching 0.
Conclusion
Being the first study of its kind in Pakistan the major findings of this study are almost in line with the set hypothesis and objectives of this study and based on study findings it will be of high value to use COVID-19 RDT kits during mass screening especially during test, trace and quarantine activities. These kits are reliable, cheap, easy to use and quick to produce results. With such promising likely hood ratios, it will be convenient to use these kits for bed site diagnosis of COVID-19.
References
1. Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet. 2020 Feb 15;395(10223):497-506.
2. Wang C, Yu H, Horby PW, Cao B, Wu P, et al. Comparison of patients hospitalized with influenza A subtypes H7N9, H5N1, and 2009 pandemic H1N1. Clinical Infectious Diseases. 2014 Apr 15;58(8):1095-103.
3. Nguyen T, Duong Bang D, Wolff A. 2019 novel coronavirus disease (COVID-19): paving the road for rapid detection and point-of-care diagnostics. Micromachines. 2020 Mar;11(3):306.
4. Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Eurosurveillance. 2020 Jan 23;25(3):2000045.
5. Zhao J, Yuan Q, Wang H, Liu W, Liao X, Su Y, et al. Antibody responses to SARS-CoV-2 in patients with novel coronavirus disease 2019. Clinical Infectious Diseases. 2020 Oct 15;71(16):2027-34.
6. To KK, Tsang OT, Leung WS, Tam AR, Wu TC, Lung DC, et al. Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. The Lancet Infectious Diseases. 2020 May 1;20(5):565-74.
7. Guo L, Ren L, Yang S, Xiao M, Chang D, Yang F, et al. Profiling early humoral response to diagnose novel coronavirus disease (COVID-19). Clinical Infectious Diseases. 2020 Jul 28;71(15):778-85.
8. Li Z, Yi Y, Luo X, Xiong N, Liu Y, Li S, et al. Development and clinical application of a rapid IgM-IgG combined antibody test for SARS-CoV-2 infection diagnosis. Journal of Medical Virology. 2020 Sep;92(9):1518-24.
9. Liu Y, Liu Y, Diao B, Ren F, Wang Y, Ding J, Huang Q. Diagnostic indexes of a rapid IgG/IgM combined antibody test for SARS-CoV-2. MedRxiv. 2020 Jan 1.
10. Hsieh HV, Dantzler JL, Weigl BH. Analytical tools to improve optimization procedures for lateral flow assays. Diagnostics. 2017 Jun;7(2):29.
11. Zhuoyue W, Xin Z, Xinge Y. IFA in testing specific antibody of SARS coronavirus. South China Journal of Preventive Medicine. 2003 Jan 1;29(3):36-7.