Variations in risk assessment models may contribute to the existing gap between venous thromboembolism prophylaxis guidelines and adherence

Among hospitalized patients, venous thromboembolism (VTE) represents the second most common nosocomial condition, the second most common reason for an increased length of hospital stay, and the third most common cause of increased mortality (Zhan & Miller2003). Given the high morbidity and mortality associated with this disease, the American College of Chest Physicians (ACCP) calls for preventive strategies and risk assessment for all admitted patients. (Kahn et al.2012) However, despite the many published recommendations there appears to be a widening gap between guidelines and implementation of preventative strategies. The IMPROVE study showed that utilization of prophylaxis for admitted medical patients was around 60% among 15,000 hospitalized patients. (Tapson et al.2007) The ENDORSE study found an even lower number (less than 40%) among at risk medical patients (Cohen et al.2008). According to the 9^th ACCP guidelines, all medical patients on bed-rest with at least one additional risk factor should receive chemoprophylaxis while those at a high risk of bleeding should receive mechanical prophylaxis with graduated compression stockings (GCS) or intermittent pneumatic compression devices (IPCs). (Kahn et al.2012)

There are several strategies for improving the provision of VTE prophylaxis including: 1.) Risk assessment models (RAMs); 2.) Risk recognition strategies; and 3.) Universal prophylaxis (Nutescu2007). The 9^th ACCP guidelines favor risk recognition/assessment over universal prophylaxis. (Kahn et al.2012) RAMs may allow the clinician to assess the need for thromboprophylaxis and determine appropriate prophylaxis based on individual risk factors. RAMs for identifying patients at risk for VTE have been described in the literature (Arcelus et al.1991; Cohen & Alikhan2001; Thromboembolic Risk Factors (THRIFT) Consensus Group1992; Caprini et al.2001). RAMs vary greatly from ease of use, risk factor stratification, and decision support. Some RAMs may provide the clinician with a risk score but do not provide guidance regarding which prophylaxis fits the patient’s needs. Others provide decision support and guide the clinician toward either mechanical or chemoprophylaxis. In general however, these RAMs may be time consuming, lack internal validity, be difficult to use, and lack generalizability for all patient populations (Kahn et al.2012). However, despite this, RAMs continue to be used at healthcare institutions.

This paper presents an exploratory study looking at a convenience sample of RAMs to evaluate the reliability of these tools in assessing medical patients’ risk for DVT and clinical decision support with regard to appropriate thromboprophylaxis. This study focused on medical patients as this patient population has the greatest variability in terms of risk factors for DVT. The objectives were to determine: 1.) inter-RAM variability in DVT risk factors and contraindications; 2.) inter-rater variability and inter-RAM variability when applying a RAM to a standard case; and 3.) inter-rater and inter-RAM variability in outcome as far as type of prophylaxis.

A convenience sample of DVT RAMs was obtained from various institutions in the United States. Several e-mail requests for RAMs were sent to the American College of Clinical Pharmacy Adult medicine listserve that caters to about a thousand pharmacist subscribers from various institutions. The primary investigator reviewed the RAMs, seven of which were duplicate, incomplete or catered toward the surgical population and were subsequently discarded. Thirteen RAMs from various institutions were deemed appropriate and all institutional identifiers were removed from the RAMs to avoid bias. Four faculty members were recruited to provide an initial review of the various RAMs in determining patient eligibility for DVT prophylaxis. The 4 reviewers conducted an exploratory review of the thirteen RAMs to determine any variations in risk factors for DVT and contraindications to chemoprophylaxis within the RAMs. Based on this preliminary review, it was determined that significant variation existed between various RAMs which warranted a further review.

Preliminary review of the RAMs showed differences which precipitated further analysis of the RAMs by various clinicians. The type and number of risk factors varied greatly within each RAM. RAMs also differed in the number and type of contraindications to chemoprophylaxis. In total, there were 66 different risk factors (range 6–31) identified in the RAMs and there were 39 different contraindications to chemoprophylaxis (range 1–17). There were significant variations in the definitions of certain risk factors especially with regard to age and obesity. Some RAMs defined elderly as age >60 (61%) whereas other RAMs defined it as age >75 (15%). Some RAMs defined obesity as a body mass index (BMI) of > 30 (31%), while some defined it as a BMI>50 (8%).

The phase 2 review resulted in 390 separate assessments. Use of the RAMs on the 3 patient cases revealed significant inter-rater and inter-RAM variability. Kappa scores are displayed in Figure1 which showed significant disagreement between reviewers when applying each RAM to a standard patient case with a central tendency of 0.2-0.4. A multidimensional plot was created which displayed the significant spread of reviewer agreement (Figure1). There was statistically significant difference in the RAM effect when applied to each patient case (p=0.01). Patient 1 did not receive any chemoprophylaxis in 67% of the evaluations (range 0-80%), patient 2 in 27% of the evaluations (range 10-50%) and patient 3 in 2.3% of the evaluations (range 0-10%). There was statistically significant variation in the provision of chemoprophylaxis per RAM for patient 1 (p=0.001) and no significant variation for patients 2 and 3. When analyzing the rate of chemoprophylaxis per reviewer, there was statistically significant variation for patients 1 and 2 (p=0.026 and <0.0001 respectively) but not for patient 3 (p=0.123). On average, each reviewer spent 5–10 minutes to review the RAM and determine the most appropriate prophylaxis based on the patient characteristics.

This pilot study found significant inter-rater and inter-RAM variability with use of RAMs from various US healthcare institutions. Based on this initial data, it is clear there is a probability that there will be considerable variation in the ordering practices of DVT prophylaxis with the use of traditional RAMs. It is also clear that there is variation in the type and number of risk factors associated with DVTs. The new ACCP guidelines now elucidate that the risk factors for VTE in medical patients include increasing age (especially > 70 years), previous VTE, known thrombophilia, active cancer, heart failure, or respiratory failure, reduced mobility, and hormonal medications. The guidelines also provide a scoring system for these risk factors which should theoretically result in consistent risk assessment for all patients. In order to maintain consistency, using the risk factor and scoring elucidated by the guidelines may reduce the variability in the provision of DVT prophylaxis however there may still be possibility of inter-rater variability especially in terms of ambiguous risk factors such as ‘reduced mobility, recent trauma/surgery’ as well as ‘heart and respiratory failure’. In addition, the ACCP guidelines, state that those patients who are deemed high risk for bleeding should receive mechanical prophylaxis. High risk patient are those with multiple risk factors for bleeding or those with an active gastroduodenal ulcer, bleeding in the 3 months before admission, or a platelet count <50 × 10⁹/L. These contraindications for chemoprophylaxis leave room for provider interpretation especially with regard to ambiguous factors such as ‘bleeding in 3 months before admission’ or even interpretation of ‘multiple’ risk factors. As a case in point, would an 85 year old male who is immobile with a history of heart failure be considered high risk for bleeding? In this paper, we showed that there was considerable inter-rater variability even when the same RAM was utilized, this means that even though the guidelines have now clearly identified a risk scoring system, there still may be potential of missed prophylaxis for patients that would benefit from receiving prophylaxis. Individual institutions will have to identify the best way to determine risk for DVT. In this paper, we argue that RAMs may not be the best approach to identifying these patients.

This exploratory study has shown that RAMs may not be the ideal tool for determining the appropriate DVT prophylaxis for hospitalized patients and may result in significant inter-rater variability and suboptimal provision of DVT prophylaxis. Variation in RAMs may result in missed opportunities for providing appropriate prophylaxis to medical patients.

DV identified reviewers, sought out RAMs from various institutions, and drafted the manuscript. All authors read and approved the final manuscript.