- Open Access
Evaluating patient reported outcomes in routine practice of patients with rheumatoid arthritis treated with biological disease modifying anti rheumatic drugs (b-DMARDs)
SpringerPlusvolume 4, Article number: 462 (2015)
In this study the concordance between the Routine Assessment of Patient Index Data 3 (RAPID3) and the Disease Activity Score 28-joint count (DAS28) was investigated in a clinical routine outpatient setting.
Patients and methods
A sample of 150 adult patients with stable RA treated with biological DMARDs (bDMARDs) was asked to complete the RAPID3 (digital or on paper) just before their outpatient routine visit during which DAS28 assessment took place. The RAPID3 correlation with and the agreement in four DAS28 categories was studied using Spearman’s rank order and Cohen’s observed kappa statistics respectively. The positive (PPV) and negative (NPV) predictive values were calculated to test whether RAPID3 could make distinction in active disease (DAS28 >3.2) or not.
A moderate correlation (ρ 0.576) and a poor kappa value of 0.13 were found in the whole study population. Patients reported a higher disease severity than was measured by DAS28. The PPV of RAPID3 for active disease by DAS28 was 0.59 (95 % CI 0.50–0.68) and the NPV was 0.91 (95 % CI 0.75–0.98) with a sensitivity and specificity of 96 and 40 % respectively.
While RAPID3 correlates to some extent with DAS28 at the group level, agreement between RAPID3 and DAS28 at the individual patient level is to poor to rely on RAPID3 results in monitoring patients with RA. RAPID3 tends to over-report disease activity as assessed by DAS28.
Rheumatoid arthritis (RA) is a chronic inflammatory disease that may impair daily functioning and quality of life due to pain, swelling and stiffness. The disease has an unpredictable course and the main treatment goal is to suppress disease activity in order to prevent joint damage and to improve daily living. Treatment of RA mainly includes disease modifying anti-rheumatic drugs (DMARDs) including biologicals. Both international and national treatment guidelines recommend assessing the disease activity using measures such as DAS28, and doing regular follow up assessments, in which the level of disease activity dictates the frequency of monitoring (Smolen et al. 2010; Deighton et al. 2010). Frequent collection of these patient data is challenging in a busy rheumatology practice. There are indications that the DAS28 or a quantitative joint count is not measured frequently enough in standard rheumatology care (Choy et al. 2012; Pincus and Segurado 2006).
It seems attractive to assess disease activity from the patient’s perspective using validated patient reported outcomes as an alternative or as an addition to laborious frequent joint assessments. Patient reported outcomes (PROs) are well established by groups of experts from the Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT)-working group (Felson et al. 1993). Patient global health (pGH) a PRO element is included as a recommendation in the 2010 ACR-EULAR guideline that proposed tighter definitions for clinical remission in clinical trials, and is now also established for clinical practice (Smolen et al. 2013). This implies that PROs have certain significance for routine clinical care.
The Routine Assessment of Patient Index Data (RAPID3) is an RA specific questionnaire on relevant patient domains including physical function, pain and global health and takes only 5 min to complete (Pincus et al. 2010). Here we have investigated by—comparing it to DAS28—whether RAPID3 is an appropriate tool to monitor disease activity and response to treatment in a real life setting.
This study was performed in the Atrium-Orbis medical centre hospital in Sittard, the Netherlands. The study was carried out according to the principles of the Declaration of Helsinki after ethical approval by the local ethics committee. All subjects gave their informed consent before participation in this study. Between May 2013 and April 2014 a sample was taken of consecutive adult patients with RA according to the ACR 1987 criteria that were on stable treatment with a biological DMARD (bDMARD). Clinical information such as age, sex, disease duration and medication was collected by a review of the medical records. Only patients with psychiatric illness or personality disorder were excluded.
RAPID3 is a fully patient driven outcome measure that can be expressed as a score and is composed of the 3 PRO measures of the ACR core data set; physical function, pain and patient global estimate(Pincus et al. 2008; Castrejon and Pincus 2012; Anderson et al. 2011, 2012). Both DAS28-ESR and RAPID3 disease activity values can be distinguished into four categories. Scores are classified for DAS28-ESR of >5.1, ≥3.2 to ≤5.1, ≥2.6 to <3.2, and <2.6 and represent high, moderate, low disease activity and remission, respectively. RAPID3 scores of >12, 6.1–12, 3.1–6, and ≤3 represent high, moderate, low severity and remission, respectively [(Anderson et al. 2011; Van der Heijde et al. 1993).
Four rheumatologists participated, and each assessed patients with rheumatoid arthritis during “real time” clinical consultations. Three qualified RA nurse practitioners (DAS28-ESR assessors) invited patients for a regular 3 monthly visit. Patients were e-mailed and asked to complete the RAPID3 questionnaire at home in the days prior the outpatient visit where the DAS28-ESR measurement was conducted. An online data portal named “Sermos E-communication in healthcare” allowed on-line availability of the patient-reported outcome questionnaire. The online data portal is protected and certified with ISO 9001/ISO 27001. Patients who were not able to complete an electronic RAPID3 completed a paper form before the outpatient visit.
The performed RAPID3 was compared with the DAS28-ESR on a 0-30 versus 1-10 scale, respectively. DAS28-ESR and RAPID3 scores were correlated using Spearman’s rank correlation and the agreement (also for each DAS28-ESR assessor and RAPID3 method) was investigated with Cohen’s kappa coefficient per category of outcome. The following values of agreement were attributed to Cohen’s kappa: ≤0.20, poor; 0.21–0.40, fair; 0.41–0.60, average; 0.61–0.80, good, and ≥0.81, very good (Landis and Koch 1977). The positive- and negative predictive values (PPV, NPV) as well as the corresponding sensitivity and specificity of RAPID3 were calculated to test if the patient reported outcome measure could make a distinction between active and non-active disease according to DAS28-ESR criteria. SPSS Statistics software version 17 (IBM Corp., Armonk, NY, USA) was used for statistical analysis.
We have screened 1195 RA patients for the study and 293 patients met entry criteria since they were on treatment with biological DMARDs. Seventy-two patients were not willing or could not participate, and another 71 patients did not perform a RAPID3 before their DAS28-ESR assessment. Sixty-nine (46 %) of the 150 included subjects completed a RAPID3 in a conventional manner on paper and, 81 (54 %) performed it digitally via their e-mail invitation in the week before their visit. The mean age of the included patients was 60 years, 67 % were female. All patients were treated with biological DMARDs and 34 % of them were treated in combination with methotrexate. The mean levels of DAS28-ESR (3.4 ± 1.4) and RAPID3 (11.7 ± 6.2) were above the cut off levels for moderate activity and -severity according to DAS28-ESR and RAPID3 respectively. In all four DAS28-ESR disease categories the RAPID3 sub scores for pain and global health had a relatively high share in the final RAPID3 score comparing to the function scores. Demographics and patient characteristics are depicted in Table 1.
Correlation between DAS28-ESR and RAPID3 scores
DAS28-ESR and RAPID3 score correlated moderately well (ρ = 0.576). The correlation coefficients between the DAS28-ESR and the individual RAPID3 components; patient physical function, pain and patient global estimate were respectively 0.569, 0.486 and 0.470, (all were statistically significant (p < 0.0001).
DAS28-ESR and RAPID3 in categories
The agreements between the RAPID3 and DAS28-ESR across categories are visualized in a scatterplot showing (mis) classifications (Fig. 1).
The calculated agreement on DAS28-ESR and RAPID3 in categories was poor; Cohen’s kappa: 0.13 (0.04–0.23), weighted kappa: 0.27. For assessor one, two and three separately agreement was similarly poor; Cohen’s kappa’s were 0.17, 0.10 and 0.11 respectively. The kappa’s for electronic and paper RAPID3 were 0.08 and 0.20 respectively and can also be stated as poor (Table 2).
While 96 % of the patients with a moderate to high DAS28-ESR had also a moderate to high RAPID3 score (sensitivity), only 40 % of the patients with a remission to low disease activity measured by DAS28-ESR showed remission to low disease severity according to RAPID3 (specificity).
The positive predictive value of a moderate to high RAPID3 score with regard to a moderate to high DAS28-ESR was 0.59 (95 % CI 0.50–0.68) while the negative predictive value of a low severity to remission RAPID3 with regard to finding a DAS28-ESR below 3.2 was 0.91 (95 % CI 0.75–0.98).
In our sample of RA patients treated with a bDMARD in common clinical practice the agreement between RAPID3 and DAS28-ESR was poor. We have found moderate associations at best, which were not clinically irrelevant. Our focus was not on finding a group-level association (which was indeed confirmed here) but rather on the level of individual agreement between two instruments developed to distinguish categories of disease activities: the DAS28 that integrates clinical and patient-reported outcome measures and the RAPID3 that is a combined score of 3 patient-reported outcomes. We have demonstrated here that there is tremendous over-call of disease activity when you rely on RAPID3 for measuring disease activity, and with DAS28-ESR as a reference. The overcall in RAPID3 disease severity could possibly be caused by comorbidities regarding the relatively high share of pain and global health scores in the final RAPID3 score across the different DAS28-ESR categories. It is obvious that this RAPID3 does not give a proper indication of DAS-measured disease activity across the four disease activity categories: Positive predictive values of RAPID3 to find active disease was moderate at best, and inappropriate to rely on in clinical practice. Negative predictive values (to find low disease activity or remission) were far better.
Introducing covariables like several DAS assessors and different methods to take RAPID3 questionnaires could have influence on the established concordance between the clinical and patient reported outcomes. However kappa values of individual assessors and RAPID3 method did not differ much and were all stated as poor. Since DAS28-ESR is not regarding tender and swollen joint counts in ankles and feet it could possibly underestimate disease severity according to RAPID3 in some patients. Moreover only relying on whatever disease parameter could be misleading in the clinical assessment of patients.
Other studies found Spearman rank correlation values in clinical routine between 0.43 and 0.91 ranging from a moderate to strong positive correlation (Pincus et al. 2010; Castrejon and Pincus 2012; Singh et al. 2012; Bossert et al. 2012; Kim et al. 2014). The Cohen`s unweighted kappa value indicative for concordance in our study is in line with kappa values found in other clinical care studies where kappa values varied between 0.16 and 0.26 and were calculated within the four disease categories (Pincus et al. 2008, 2010; Castrejon and Pincus 2012; Castrejón et al. 2013). Linear weighted kappa’s are also used in studies that addressed the agreement between RAPID3 and DAS28 and delivered relatively high kappa values 0.27 and 0.44 compared to non weighted kappa’s (Pincus et al. 2010; Castrejon and Pincus 2012; Kim et al. 2014). Only one clinical care study where 87 % of Indian patients had a DAS28-ESR >3.2 found a relatively high kappa value of 0.63 (Singh et al. 2012). Two trials found relatively high kappa values (0.25–0.36) in patient cohorts that consisted out of 73–96 % of patients with active disease (Pincus et al. 2011a, b). None of the references found observed kappa values (>0.80) providing good agreement.
Even when the RAPID3 is in poor agreement with DAS28-ESR, it is possible that RAPID3 is capable in tracking changes of DAS28-ESR score within the individual patient during longitudinal follow-up. The influence of comorbidities and disease duration on RAPID3 scoring and thereby the clinical reliability should also be topic for future research. Regarding this poor relation between the two measures in the studied population treated with bDMARDs the question is raised if the questionnaire is still up to date in patients treated with these modern agents. Differences in RAPID3 scoring in cDMARDs treated patients comparing to bDMARDs treated patients should also be addressed in future research.
Our study, like others, indicates discordance in opinion between patients and physicians with respect to disease burden. This study shows a poor match in category outcomes to conclude a meaningful clinical relation between DAS28 and RAPID3 in patients treated with bDMARDs. It can therefore not substitute a frequent joint assessment. Based on our results RAPID3 is possibly useful as a non-laborious pre-screening tool to identify patients with low disease activity on a population level.
Anderson JK, Zimmerman L, Caplan L, Michaud K (2011) Measures of rheumatoid arthritis disease activity: patient (PtGA) and provider (PrGA) Global Assessment of Disease Activity, Disease Activity Score (DAS) and Disease Activity Score with 28-Joint Counts (DAS28-ESR), Simplified Disease Activity Index (SDAI), Cl. Arthritis Care Res 63:S14–S36. doi:10.1002/acr.20621
Anderson J, Caplan L, Yazdani J et al (2012) Rheumatoid arthritis disease activity measures: American College of Rheumatology Recommendations for use in clinical practice. Arthritis Care Res 64:640–647. doi:10.1002/acr.21649
Bossert M, Prati C, Vidal C et al (2012) Evaluation of self-report questionnaires for assessing rheumatoid arthritis activity: a cross-sectional study of RAPID3 and RADAI5 and flare detection in 200 patients. Joint Bone Spine 79:57–62. doi:10.1016/j.jbspin.2011.03.014
Castrejon I, Pincus T (2012) Patient self-report outcomes to guide treat-to-target strategy in clinical trials and usual clinical routine care of rheumatoid arthritis. Clin Exp Rheumatol 30:S50–S55
Castrejón I, Pincus T, Soubrier M et al (2013) GUEPARD treat-to-target strategy is significantly more efficacious than ESPOIR routine care in early rheumatoid arthritis according to patient-reported outcomes and physician global estimate. Rheumatology 52:1890–1897. doi:10.1093/rheumatology/ket230
Choy E, Taylor P, McAuliffe S et al (2012) Variation in the use of biologics in the management of rheumatoid arthritis across the UK. Curr Med Res Opin 28:1733–1741. doi:10.1185/03007995.2012.731388
Deighton C, Hyrich K, Ding T et al (2010) BSR and BHPR rheumatoid arthritis guidelines on elegibility criteria for the first biological therapy. Rheumatology 49:1197–1199. doi:10.1093/rheumatology/keq006a
Felson DT, Anderson JJ, Boers M et al (1993) The American College of Rheumatology preliminary core set of disease activity measures for rheumatoid arthritis clinical trials. The Committee on Outcome Measures in Rheumatoid Arthritis Clinical Trials. Arthritis Rheum 36:729–740
Kim S, Park S, Bae J et al (2014) Performance of Routine Assessment of Patient Index Data 3 (RAPID3) for assessment of rheumatoid arthritis in clinical practice: differential agreement of RAPID3 according to disease activity categories. Rheumatol Int 34:1311–1318. doi:10.1007/s00296-014-3042-y
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174
Pincus T, Segurado OG (2006) Most visits of most patients with rheumatoid arthritis to most rheumatologists do not include a formal quantitative joint count. Ann Rheum Dis 65:820–822. doi:10.1136/ard.2005.044230
Pincus T, Swearingen CJ, Bergman MJ et al (2008) RAPID3 (Routine Assessment of Patient Index Data 3), a Rheumatoid Arthritis Index without Formal Joint Counts for Routine Care: Proposed Severity Categories Compared to Disease Activity Score and Clinical Disease Activity Index Categories. J Rheumatol 35:2136–2147. doi:10.3899/jrheum.080182
Pincus T, Swearingen CJ, Bergman MJ et al (2010) RAPID3 (Routine Assessment of Patient Index Data) on an MDHAQ (Multidimensional Health Assessment Questionnaire): agreement with DAS28 (Disease Activity Score) and CDAI (Clinical Disease Activity Index) activity categories, scored in five versus more than ninety seconds. Arthritis Care Res 62:181–189. doi:10.1002/acr.20066
Pincus T, Hines P, Bergman MJ et al (2011a) Proposed severity and response criteria for Routine Assessment of Patient Index Data (RAPID3): results for categories of disease activity and response criteria in abatacept clinical trials. J Rheumatol 38:2565–2571. doi:10.3899/jrheum.110262
Pincus T, Furer V, Keystone E et al (2011b) RAPID3 (Routine Assessment of Patient Index Data 3) severity categories and response criteria: Similar results to DAS28 (Disease Activity Score) and CDAI (Clinical Disease Activity Index) in the RAPID 1 (Rheumatoid Arthritis Prevention of Structural Damage) clinical trial of certolizumab pegol. Arthritis Care Res 63:1142–1149. doi:10.1002/acr.20481
Singh H, Gupta V, Ray S et al (2012) Evaluation of disease activity in rheumatoid arthritis by Routine Assessment of Patient Index Data 3 (RAPID3) and its correlation to Disease Activity Score 28 (DAS28) and Clinical Disease Activity Index (CDAI): an Indian experience. Clin Rheumatol 31:1663–1669. doi:10.1007/s10067-012-2070-9
Smolen JS, Aletaha D, Bijlsma JW et al (2010) Treating rheumatoid arthritis to target: recommendations of an international task force. Ann Rheum Dis 69:631–637. doi:10.1136/ard.2009.123919
Smolen JS, Landewe R, Breedveld FC et al (2013) EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2013 update. Ann Rheum Dis 73:492–509. doi:10.1136/annrheumdis-2013-204573
Van der Heijde DM, van’t Hof M, van Riel PL, van de Putte LB (1993) Development of a disease activity score based on judgment in clinical practice by rheumatologists. J Rheumatol 20:579–581
NB made substantial contribution to conception and design, acquisition of the data, and analysis plus interpretation of the data and has been involved in drafting the manuscript and revising it critically for important intellectual content. PT made substantial contribution to acquisition of data, analysis and interpretation of the data and has been involved in drafting the manuscript and revising it critically for important intellectual content. RJ have been involved in drafting the manuscript and revising it critically for important intellectual content. HK made substantial contribution to conception and design have been involved in drafting the manuscript and revising it critically for important intellectual content. RL made substantial contribution to conception and design, interpretation of the data and have been involved in drafting the manuscript and revising it critically for important intellectual content. All authors read and approved the final manuscript.
This study was supported by a grant from Pfizer.
Compliance with ethical guidelines
Competing interests The authors declare that they have no competing interests.
All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation.
About this article
- Rheumatoid arthritis (RA)
- Routine Assessment of Patient Index Data 3 (RAPID3)
- Disease Activity Score 28 (DAS28)
- Patient reported outcome (PRO)