Open Access

A concise drug alerting rule set for Chinese hospitals and its application in computerized physician order entry (CPOE)

SpringerPlus20165:2067

https://doi.org/10.1186/s40064-016-3701-4

Received: 19 May 2016

Accepted: 18 November 2016

Published: 1 December 2016

Abstract

Background

A minimized and concise drug alerting rule set can be effective in reducing alert fatigue.

Objectives

This study aims to develop and evaluate a concise drug alerting rule set for Chinese hospitals. The rule set covers not only western medicine, but also Chinese patent medicine that is widely used in Chinese hospitals.

Setting

A 2600-bed general hospital in China.

Methods

In order to implement the drug rule set in clinical information settings, an information model for drug rules was designed and a rule authoring tool was developed accordingly. With this authoring tool, clinical pharmacists built a computerized rule set that contains 150 most widely used and error-prone drugs. Based on this rule set, a medication-related clinical decision support application was built in CPOE. Drug alert data between 2013/12/25 and 2015/07/01 were used to evaluate the effect of the rule set.

Main outcome measure

Number of alerts, number of corrected/overridden alerts, accept/override rate.

Results

Totally 18,666 alerts were fired and 2803 alerts were overridden. Overall override rate is 15.0% (2803/18666) and accept rate is 85.0%.

Conclusions

The rule set has been well received by physicians and can be used as a preliminary medical order screening tool to reduce pharmacists’ workload. For Chinese hospitals, this rule set can serve as a starter kit for building their own pharmaceutical systems or as a reference to tier commercial rule set.

Keywords

Medication-related clinical decision support Chinese patent medicine Drug alerting rule Alert fatigue

Background

Computerized physician order entry (CPOE) with medication-related clinical decision support (CDS) is an effective solution to reduce drug-related problems and pharmacist workload (Hammar et al. 2015; Claus et al. 2015). Most medication-related decision support functions, such as dosage checking and drug–drug interaction (DDI) checking, are typically implemented by a set of computerized drug alerting rules. One major problem faced by drug alerting rules is the alert fatigue (Nanji et al. 2014), which is usually caused by highly exhaustive and sensitive rules. Recent related work shows override rates can be as high as 53.6% (Nanji et al. 2014), 87.6% (Topaz et al. 2015), and 93% (Bryant et al. 2013) respectively. To address this issue, lots of work has been focused on constructing minimized and concise drug rule sets. For example, Shah et al. (2006) built a tiered medication knowledge subset from a commercial knowledge base. The subset contains clinical significant drug contraindications, and only interrupts physicians for severe alerts. Phansalkar et al. (2012) developed a minimum set of 15 high-severity, clinically significant DDIs from several commercial knowledge bases. Classen et al. (2011) identified 7 most common DDIs by reviewing multiple sources. The public DDI knowledge base SFINX (Swedish, Finnish, INteraction X-referencing) tiers DDIs according to clinical significance (A-D), which enables threshold settings for automated warnings (Andersson et al. 2015).

Aim of the study

The aim of this study to build and evaluate a concise rule set suitable for Chinese hospitals. Compared to existing related work, this rule set not only covers the western medicine, but also includes various Chinese patent medicine (CPM) that is extensively used by Chinese hospitals. For example, a typical Chinese hospital (DaYi Hospital, ShanXi Province, China) uses 1981 drugs, and 462 (23.3%) are Chinese patent medicine.

Ethical approval

This study was approved by the medical ethics committee of DaYi Hospital. All collected data have been de-identified by the information department of the hospital.

Methods

Settings and materials

DaYi Hospital was established in 2011 and is the largest general hospital (2600-bed) in ShanXi Province, China. Until 2013, all the drug checking work in DaYi was performed manually by clinical pharmacists. At the drug dispensing time, the pharmacists would inspect medication orders submitted by the physicians. Unqualified orders would be returned to physicians and recorded by the pharmacists. The recorded medication errors between 2011 and 2013 were used to analyze the most frequent and error-prone drug rules. These records are the initial resource for building the concise rule set.

In 2013, we initiated the KTP (Knowledge Translation Platform) project (Zhang et al. 2015). One of KTP’s goals is to build a medication-related CDS for CPOE, in order to help pharmacists reduce work load and assist the drug checking process. At the beginning of KTP, a preliminary question is: whether to develop own medication-related CDS or use a commercial one. Although there are already mature commercial products on the Chinese market, e.g. Wolters Kluwer/Medicom PASS (Prescription Automatic Screening System), we have our own considerations for not choosing such off-the-shelf systems. (1) Although the rule base of commercial products may be much more comprehensive and detailed, it is still necessary to tier and routinely tailor the complete rule set to suit local hospital situations. For pharmacists, there is not much workload advantage over maintaining a local-developed rule set. (2) From the perspective of the KTP project, the pharmaceutical knowledge is an inseparable part of the entire knowledge base. Inside the KTP knowledge base, there are semantic relations between drug and other medical entities. For example, many clinical rules (e.g. if [Use of Aspirin] == true || [Use of Clopidogrel] == true, recommend [INR monitor]) and clinical treatment protocols (predefined order sets or clinical pathways) involves drug entities. If using third-party products, even if the vendors open their knowledge base or provide external access interfaces, the integration and interaction between different systems (e.g. mapping of drug entities across systems) can be complex and effort-taking. Therefore, we decided to develop an own system.

Information model

To implement a computerized rule set, an information model of drug alerting rules is designed (Fig. 1). It defines 11 rule types (Table 1), including dosage (single intake), daily dosage (accumulated intake), administration route, frequency, skin test, dissolvent, dissolvent dosage, DDI, contra-indication, and prescription restriction.
Fig. 1

The Information model for drug alerting rules

Table 1

Drug alerting rule types

Rule type

Description

Example

Dosage

Defines maximum dosage for one medical order

Maximum dosage of Ambroxol injection is 2 doses

[Dosage] ≤ 2 doses

Daily dosage

Defines maximum daily accumulated dosage

Maximum daily dosage of ShuXueNing injection (Ginkgo biloba extract) is 4 doses

[DailyDosage] ≤ 4 doses

Administration route

Defines allowed administration route

Cobamamide injection should be administrated by intramuscular injection

[AdministrationRoute] = {intramuscular}

Frequency

Defines allowed frequency

Ceftriaxone injection frequency should be qd. (1/day)

[Frequency] = {qd}

skin test

Defines whether skin test flag should be specified in the medication order, so as to remind the nurses

Lidocaine hydrochloride injection needs skin test

[SkinTest] = true

Dissolvent

Defines allowed dissolvent

Dissolvent for pHGF injection can only be 10% glucose injection

[Dissolvent] = {10% glucose}

Dissolvent dosage

Defines maximum dissolvent dosage

Dissolvent dosage for iron sucrose injection is 100 ml

100 ml ≤ [DissolventDosage] ≤ 100 ml

Pregnancy risk

Assigns each drug to FDA pregnancy category, which contains five categories: ABCDX. Category X should never be applied to pregnant patients

FDA pregnancy category of Ribavirin is X

[PregancyRiskLevel] = X

Drug-drug interaction (DDI)

Defines synergistic, antagonistic, etc. interactions between drugs

Warfarin and Vitamin K have antagonistic interaction

Interaction (Warfarin, Vitamin K)

Contra-indication

Defines drug-disease and drug-symptom conflicts

Clopidogrel cannot be used against patients with active peptic ulcer

[Contra-indication] = ”[active peptic ulcer] == false && [gastrointestinal hemorrhage] == false”, check passed if result is true

prescription restriction

Restricts the prescription of certain drugs for some departments or physicians

For third-line antibiotics such as Vancomycin, only chief physicians have prescription rights. Pediatrics departments cannot prescribe Vancomycin

[RestrictedDeptment] = {pediatrics}, [RestrictedPhyscian] = {ID1, ID2,…}

These rule types are designed according to pharmacists’ drug checking requirements. However, there are also other rule types, such as personalized dosing algorithms (e.g. children or elder patients with different body weights and body surface areas, or patients with renal insufficiency based on creatinine clearance). In the current development phase, we haven’t supported such rules because they require lots of patient context data, such as body weight, body surface area, Crcl rate, etc. These data mostly reside in heterogeneous formats in external systems, such as HIS (Hospital Information System), LIS (Laboratory Information System), EMR (Electronic Medical Record), etc. How to extract high-quality and well-structured data in expected formats from various sources is a non-trivial task. In the next development phase, we will try to solve this data acquisition problem and support more rule types.

Authoring tool

Based on the above information model, the database schema for drug alerting rules can be decided, and a corresponding rule authoring tool has been developed (Fig. 2). The tool was developed as a web-based application.
Fig. 2

Drug alerting rule authoring tool. a Main page for editing drug rules. The left panel is the drug list, where user can click one to edit. On the right side is the edit area, which contains three tab pages: basic info, interactions and contraindications. Basic info tab page defines basic rules such as skin test, dosage, etc. b Tab page for editing drug–drug interactions. Users can select drugs that have interactions with the current one. c Tab page for editing contraindication rules. Left panel is the context item (e.g. lab test, symptoms, vital signs, etc.) list used to define contraindicated conditions. The right side is a table of user-selected context items, and a graphical rule composer, as well as a textual rule expression editor

Results

Drug alerting rule set

Based on the recorded medication errors between 2011 and 2013, the pharmacists used the rule authoring tool to define a rule set that was able to cover the most widely used and error-prone drugs. The first version of the rule set was created in June 2013, and contained 150 drugs. The detailed rule set is provided in “Appendix”.

Medication-related CDS based on the rule set

With the rule set, a medication-related clinical decision support was developed and integrated into CPOE (Fig. 3). Reasoning of the rules is executed by a home-grown rule engine (refer to http://ktp.brahma.top/Display/TestRuleEngine, http://ktp.brahma.top/Pages/Evaluation/RuleEngine/Index.html). The CPOE was also developed by our research team, under the product name “MIAS (Medical Information Automation System)”. The interaction between CPOE and CDS was implemented by web services. Whenever the physician submits orders, CPOE will call the drug checking web service of CDS to trigger the rule engine. CDS-detected alerts are then returned to CPOE, and CPOE displays them to the physician as warnings (Fig. 3b). The physician can either cancel order submission or override the alert. All detected alerts are also sent to the notification area (Fig. 3a) for review. In exceptional cases due to patient status, physicians may state their reasons for overriding the alert. While reviewing the drug alerts, physicians can use infobutton (Fig. 3c) to retrieve related drug labels (Fig. 3d). For pharmacists, we provide a backend web portal for viewing the status (accepted or overridden) and override reason for each alert. The information flow of drug alert status is automatically directed and tracked by the system, which has greatly reduced the necessity of face-to-face communication and telephone calls between physicians and pharmacists.
Fig. 3

Medication-related clinical decision support in CPOE. a Notification area for drug alerts. User can review and process all triggered drug alerts in this area. b Drug alert message. c Infobutton for drug labels. d Retrieved drug label by Infobutton

In this system, only physicians have the right to change the status of an alert (accept or override). Pharmacist only have read-only rights for alert statuses, but they can edit (increase threshold or change rule content) or deactivate corresponding rules if they found many occurrences of an unreasonable alert.

Evaluation of the rule set in CPOE

The computerized rule set was first implemented in the inpatient CPOE on 2013/12/25 (The outpatient CPOE was provided by another vendor, and had not been integrated with our system). Until now, the system has been used in 49 inpatient departments for more than 2 years. In order to evaluate the actual effect of the rule set, system log data between 2013/12/25 and 2015/07/01 were collected. During this period, totally 68,182 inpatient visits were enrolled into the system and 2,747,140 medication orders were submitted.

For the submitted medication errors, totally 18,666 alerts were detected by the CDS, and 2803 alerts were overridden by physicians. Therefore, the overall override rate is 15.0% (2803/18,666), and accept rate is 85%. Among the 18,666 alerts, Chinese patent medicine (CPM) takes up 38.4% (7168 in 18,666).

According to Tables 2 and 3, several results caught our attention and we further analyzed these results.
Table 2

Drug alert analysis

Drug name

Drug name (Chinese)

Alert type

Alerts

Overridden alerts

Override rate (%)

Ambroxol injection

氨溴索注射液

Daily dosage

4938

22

0.4

Salvia TMP injection

丹参川芎嗪注射液

Daily dosage

4039

0

0.0

Injection esomeprazole

注射用埃索美拉唑

Dissolvent dosage

1261

1239

98.3

Thin Chi glycopeptide injection

薄芝糖肽注射液

Daily dosage

1050

2

0.2

Shuxuening injection

舒血宁注射液

Daily dosage

876

0

0.0

Fufangkushen injection

复方苦参注射液

Daily dosage

761

4

0.5

Lidocaine hydrochloride injection

盐酸利多卡因注射液

Skin test

691

287

41.5

Injection cefathiamidine

注射用头孢硫脒

Daily dosage

488

0

0.0

Injection thymopentin

注射用胸腺五肽

Administration route

413

277

67.1

Calcium gluconate injection

葡萄糖酸钙注射液

Dissolvent

307

0

0.0

Iron sucrose injection

蔗糖铁注射液

Dissolvent dosage

298

0

0.0

Injection ambroxol

注射用氨溴索

Administration route

248

0

0.0

Injection aminophylline

氨茶碱注射液

Dissolvent

229

161

70.3

Injection pantoprazole

注射用泮托拉唑

Dissolvent dosage

219

111

50.7

Yinxingdamo injection

银杏达莫注射液

Dissolvent dosage

203

102

50.2

Injection omeprazole

注射用奥美拉唑

Administration route

198

191

96.5

Injection pantoprazole

注射用泮托拉唑

Administration route

133

46

34.6

Injection of fat-soluble vitamins II

注射用脂溶性维生素II

Dissolvent

131

10

7.6

Ceftriaxone for injection

注射用头孢曲松

Frequency

116

56

48.3

Injection cefamandole ester

注射用头孢孟多酯

Prescription restriction

113

0

0.0

Injection pancreatic kallikrein

注射用胰激肽原酶

Administration route

113

0

0.0

Leucovorin injection

亚叶酸钙注射液

Administration route

112

0

0.0

Injection cefoxitin

注射用头孢西丁

Prescription restriction

110

0

0.0

Injection omeprazole

注射用奥美拉唑

Dissolvent dosage

103

61

59.2

Oxytocin injection

缩宫素注射液

Dissolvent

96

0

0.0

Heparin sodium injection

肝素钠注射液

Administration route

91

0

0.0

Sodium for injection cefodizime

注射用头孢地嗪钠

Prescription restriction

87

0

0.0

Alprostadil injection

前列地尔注射液

Administration route

80

28

35.0

Furosemide injection

呋塞米注射液

Dissolvent

70

51

72.9

Injection esomeprazole

注射用埃索美拉唑

Frequency

60

0

0.0

Salvia TMP injection

丹参川芎嗪注射液

Dissolvent dosage

57

0

0.0

Injectable piperacillin sodium and tazobactam sodium

注射用哌拉西林钠他唑巴坦钠

Prescription restriction

53

0

0.0

Cefoperazone sulbactam

注射用头孢哌酮舒巴坦

Prescription restriction

51

0

0.0

Kangai injection

康艾注射液

Dissolvent dosage

47

0

0.0

Leucovorin injection

亚叶酸钙注射液

Frequency

43

0

0.0

Levofloxacin injection

左氧氟沙星注射液

Dissolvent dosage

38

21

55.3

Injection torasemide

注射用托拉塞米

Frequency

38

0

0.0

Large plants Rhodiola injection

大株红景天注射液

Dissolvent dosage

37

0

0.0

Cefoperazone

注射用头孢哌酮

Prescription restriction

36

0

0.0

Xuebijing injection

血必净注射液

Dissolvent dosage

36

26

72.2

Injection of fat-soluble vitamins II

注射用脂溶性维生素II

Daily dosage

33

3

9.1

Ceftazidime for injection

注射用头孢他啶

Prescription restriction

30

0

0.0

Injection imipenem cilastatin sodium

注射用亚胺培南西司他丁钠

Prescription restriction

28

0

0.0

Sodium for injection aescinate

注射用七叶皂苷钠

Daily dosage

24

3

12.5

Torasemide injection

托拉塞米注射液

Frequency

23

0

0.0

Shuxuening injection

舒血宁注射液

Dissolvent

21

17

81.0

Injection of water-soluble vitamins

注射用水溶性维生素

Dosage

21

0

0.0

Amiodarone injection

胺碘酮注射液

Dissolvent

20

15

75.0

Injection ulinastatin

注射用乌司他丁

Frequency

20

0

0.0

Meropenem for injection

注射用美罗培南

Prescription restriction

19

0

0.0

Polyene phosphatidylcholine injection

多烯磷脂酰胆碱注射液

Dissolvent

19

11

57.9

Injection pantoprazole

注射用泮托拉唑

Dissolvent

18

8

44.4

Insulin injection

胰岛素注射液

DDI

17

3

17.6

Fluconazole injection

氟康唑注射液

Prescription restriction

16

0

0.0

Injection esomeprazole

注射用埃索美拉唑

Dosage

15

0

0.0

Sodium for injection aescinate

注射用七叶皂苷钠

Dosage

15

0

0.0

Vancomycin injection

注射用万古霉素

Prescription restriction

14

0

0.0

Vitamin C injection

维生素C注射液

DDI

13

3

23.1

Injection omeprazole

注射用奥美拉唑

Dissolvent

13

12

92.3

Methylprednisolone sodium succinate injection

注射用甲泼尼龙琥珀酸钠

DDI

11

1

9.1

Injection carbazochrome sodium sulfonate

注射用卡络磺钠

Dissolvent

11

7

63.6

Itraconazole oral solution

伊曲康唑口服液

Prescription restriction

10

0

0.0

Fufangkushen injection

复方苦参注射液

Dosage

10

0

0.0

Flurbiprofen injection

氟比洛芬酯注射液

Dosage

10

0

0.0

Injection lentinan

注射用香菇多糖

Dosage

10

0

0.0

Other low occurrence drug alerts (i.e. fired alert count <10)

155

25

16.1

Total

18,666

2803

15.0

Table 3

drug alert analysis grouped by rule types

Alert type

Alerts

Overridden alerts

Override rate (%)

Daily dosage

12,212

34

0.3

Dissolvent dosage

2299

1560

67.9

Administration route

1391

542

39.0

Dissolvent

964

312

32.4

Skin test

691

287

41.5

Prescription restriction

595

0

0.0

Frequency

300

56

18.7

Dosage

151

5

3.3

DDI

63

7

11.1

Total

18,666

2803

15.0

  1. 1.

    Among the detected alerts, “daily dosage” rule type has the highest alert occurrence rate (12,212 alerts in total 18,666). We dived into the “daily dosage” alerts, and found four of the top five drugs are CPM, i.e. “Salvia TMP injection (4039 alerts)”, “Thin Chi glycopeptide injection (1050 alerts)”, “Shuxuening injection (876 alerts)” and “Fufangkushen Injection (761 alerts)”, which are responsible for the majority of “daily dosage” alerts. CPM is mostly extracted or manufactured from Chinese traditional herbs. Compared to western synthesized chemical medicine, though herbs take much longer time to take effect, they also have fewer side effects and adverse reactions. In fact, CPM usually plays an auxiliary or supportive role in treatment regimens. For this reason, some physicians relaxed their vigilance and didn’t pay enough attention when using CPM. This also explains why CPM has a noticeable percentage in all the detected alerts (38.4%).

     
  2. 2.

    The “dissolvent dosage” rule type has the highest override rate (67.9%). The 67.9% override rate is remarkably high compared to other rule types, which means about 2/3 “dissolvent dosage” alerts have been overridden. We consulted with the clinical pharmacists, and found many alerts were related to patients with certain conditions, e.g. renal deficiency or heart failure. For such patients, it is reasonable to use smaller dosage than required by the drug fact sheet. Such false-positive cases have added up to the overridden alerts. To address this issue, we are currently considering using more patient context data to exclude such false-positive alerts.

     
  3. 3.

    The “skin test” rule type has the second highest override rate (41.5%). Investigation reveals that this high override rate is caused by the discrepancy in physicians’ understanding of the “skin test” rule. In this system, the skin test rule is not designed as a mandatory requirement for the current specific patient, but a general risk reminder for nurses. That means, if there is potential allergic risk (either from medical literature or drug fact sheet) for a certain drug, physicians should set the skin test flag for corresponding medication orders. If not, the skin test rule will give an alert. When it comes to the drug administrating phase, the nurses will investigate this flag as well as patient’s specific conditions (e.g. known allergy history towards certain drugs) to judge whether skin test is needed. However, many physicians treated the “skin test” rule as patient-specific flags, i.e. if a certain drug has potential allergic risk, but the physician already knows the current patient is not allergic to this drug, he/she will not set the flag and override the skin test alert.

     

Besides the above analysis for certain rule types, there are also high alert occurrence and override rates for several individual drugs, which are caused by different reasons and need case-by-case investigation. Base on these periodical retrospective analyses, pharmacists can continually improve the rule set (e.g. change threshold, revise rule content, deactivate rules) to better suit clinical use.

Discussion

The primary contribution of this study is a concise drug alerting rule set oriented to Chinese hospitals. As the rule set was built based on the historical data from a large-scale (2600-bed) general hospital with high patient throughput (e.g. 68,182 inpatient visits from 2013/12/25 to 2015/07/01), the rule set should be able to reflect the medication use profile of large populations and may serve as a reference for other Chinese hospitals.

In this study, the computerized rule set can be used as a “preliminary screening tool” against physicians’ medication orders. In DaYi Hospital, pharmacists need to check 4968 medication orders per day on average, and unqualified orders have to be returned to physicians. This is a time-consuming and laborious work. With the drug alerting CDS, many potential mistakes can be ruled out before they reach the final checkpoint of pharmacists. According to the evaluation result, physicians have revised 85% of detected medication orders. In the long run, the system will not only alleviate the workload of pharmacists (many drug use errors can be revised by the physicians without pharmacists’ intervention) but also enhance the workflow efficiency (avoid the “reject-revise-resubmit” process).

This study has several limitations or arguments:
  1. 1.

    The proposed rule set is not suitable for procedural drug rules. For example, the preparation of azithromycin solution is a multi-step procedure. First, azithromycin is dissolved with sterilized water to formulate into 0.1 g/ml. Then, add it to 250–500 ml 0.9% NaCl or 5% glucose solution to get a 1.0–2.0 mg/ml concentration. This procedural logic cannot be easily represented as a single succinct dissolvent rule.

     
  2. 2.

    The current rule set doesn’t support complex personalized dosing algorithms. In certain contexts, such as children or elder patients with different body weights and body surface areas, or patients with renal insufficiency based on creatinine clearance, more complicated personalized dosing algorithms are needed. To support them, the information model needs further extension to represent such individualized knowledge.

     
  3. 3.

    DDI rule subtyping. In current system implementation, all DDI rules are treated as one rule type. However, it’s better to design more DDI sub-types in order to achieve more fine-grained alerts. For example, the SFINX project (Andersson et al. 2015) tiers DDIs according to clinical significance (A–D), which enables fine-grained threshold settings for automated warnings.

     
  4. 4.

    Lack of complete evaluation. In this study, the accept and override rates can be easily calculated from the log data. However, it is not so easy to calculate accuracy and specificity, which requires reviewing every overridden alert in order to identify true positives and false positives. In the future, we will build a “closed-looped” alert tracking workflow, in which the state changes (either by physicians or pharmacists) and change reasons (e.g. why physician override an alert, and why pharmacists reject overriding an alert) of each alert are tracked and logged by the system.

     
  5. 5.

    Use of clinically identified ADEs. ADEs (adverse drug events) are valuable data for analyzing drug use and medication-related CDS. In China, we have a multi-level ADE reporting mechanism. Level I: Physicians submit detected ADE and related clinical data (patient demographics, symptoms, drug use info, etc.) to the hospital’s pharmacy department. Level 2: Pharmacists submit confirmed ADEs to drug regulatory authorities, i.e. China SFDA (a counterpart of US FDA). Level 3: China SFDA evaluates drug risks based on nation-wide collected ADEs. Although this ADE-reporting mechanism is well designed, it’s a sad reality that it hasn’t lived up to its maximum benefit, largely due to the wide-spread under-reporting problems. Most ADE events were concealed or neglected in daily practices, and the few reported ADEs cannot be used as a solid and complete data source for analyzing physicians’ drug use and evaluating our rule set. To address this issue, we are currently cooperating with clinical pharmacists to detect unreported ADEs from clinical documents (e.g. patient daily progress notes) by natural language processing (NLP) technologies.

     
  6. 6.

    Coverage of the rule set. One basic assumption of this study is that drug alerts conform to Pareto-alike distribution, where small portion of drug rules accounts for the majority of alerts. As a supporting case, one US study in 2005 (Reichley et al. 2005) used a commercial drug alerting rule set. It contains 48,262 rules for 1537 drugs, but 90% of alerts are focused on 58 drugs. From their daily work experience, the pharmacists in DaYi hospital also hold the same opinion that small set of drugs generate majority of errors. However, to further verify this assumption, a further evaluation is needed to get the coverage rate of the rule set. This requires a full set for all drugs on the Chinese market, and a parallel comparison of the full set and concise set on a large-scale and long-term patient drug use data set. A coverage rate greater than 80% should be ideal. Otherwise, more rules may have to be added to the rule set.

     
  7. 7.

    Another problem of the rule set is how to keep up with the latest clinical evidence. Occasionally published guidelines or case reports will necessitate adding or revising rules. For example, the China SFDA (State Food and Drug Administration) periodically publish ADE (adverse drug events) reports collected all round the country. A well-maintained rule set should keep up with these public sources. Currently, our research team is developing a semi-automatic program based on NLP, which will help pharmacists extract structured contents from the public ADE reports.

     

Generally speaking, the overall 85.0% accept rate indicates the rule set has been well received by physicians [compared to the override rates reported in other recent studies, e.g. 53.6% (Nanji et al. 2014), 87.6% (Topaz et al. 2015)] and is effective in reducing pharmacists’ workload. Moreover, the pharmacists are continually analyzing (i.e. analyze those drug alerts with high override rates), improving (e.g. raise alert threshold to reduce false positive alerts) and expanding (i.e. add more drugs and rules) the drug rule set, which will further improve its accuracy and coverage. However, due to the various complex and individualized patient statuses, such a computerized rule set is never meant to substitute the routine work of pharmacists, but can be used an effective supportive tool.

Conclusions

In this study, a concise drug alerting rule set for Chinese hospitals was constructed by pharmacists. The case study in a Chinese hospital indicates the medication-related CDS based on the rule set has been well received by physicians. For other hospitals, they may use this rule set as a starter kit for building their own medication-related CDS systems or use it to tier commercial rule bases.

Declarations

Authors’ contributions

YZ and XL made the data analysis and wrote the manuscript. WC provided clinical advisory opinions to the study result. HL and HD supervised the entire study. QS further processed the results, and made the graphs and charts. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

Please contact corresponding author for data request.

Funding

This study is supported by Chinese National High-tech R&D Program (2012AA02A601), Humanities and Social Sciences Foundation of Ministry of Education of China (15YJC630106), and Natural Science Foundation of Zhejiang Province of China (LQ16G020006). This study is also supported by the Research Center of Information Technology & Economic and Social Development, Zhejiang Province, China.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
School of Computer Science and Information Engineering, Zhejiang Gongshang University
(2)
College of Biomedical Engineering and Instrument Science, Zhejiang University
(3)
Department of Pharmacy, DaYi Hospital
(4)
Children’s Hospital, Institute of Translational Medicine, School of Medicine, Zhejiang University
(5)
Management School, Hangzhou Dianzi University

References

  1. Andersson ML, Böttiger Y, Bastholm-Rahmner P et al (2015) Evaluation of usage patterns and user perception of the drug–drug interaction database SFINX. Int J Med Inform 84:327–333. doi:https://doi.org/10.1016/j.ijmedinf.2015.01.013 View ArticleGoogle Scholar
  2. Bryant A, Fletcher G, Payne T (2013) Drug interaction alert override rates in the meaningful use era: no evidence of progress. Appl Clin Inform 5:802–813. doi:https://doi.org/10.4338/ACI-2013-12-RA-0103 View ArticleGoogle Scholar
  3. Classen DC, Phansalkar S, Bates DW (2011) Critical drug-drug interactions for use in electronic health records systems with computerized physician order entry: review of leading approaches. J Patient Saf 7:61–65. doi:https://doi.org/10.1097/PTS.0b013e31821d6f6e View ArticleGoogle Scholar
  4. Claus BO, Colpaert K, Steurbaut K et al (2015) Role of an electronic antimicrobial alert system in intensive care in dosing errors and pharmacist workload. Int J Clin Pharm 37:387–394. doi:https://doi.org/10.1007/s11096-015-0075-6 View ArticleGoogle Scholar
  5. Hammar T, Lidström B, Petersson G et al (2015) Potential drug-related problems detected by electronic expert support system: physicians’ views on clinical relevance. Int J Clin Pharm 37:941–948. doi:https://doi.org/10.1007/s11096-015-0146-8 View ArticleGoogle Scholar
  6. Nanji KC, Slight SP, Seger DL et al (2014) Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 21:487–491. doi:https://doi.org/10.1136/amiajnl-2013-001813 View ArticleGoogle Scholar
  7. Phansalkar S, Desai AA, Bell D et al (2012) High-priority drug–drug interactions for use in electronic health records. J Am Med Inform Assoc 19:735–743. doi:https://doi.org/10.1136/amiajnl-2011-000612 View ArticleGoogle Scholar
  8. Reichley RM, Seaton TL, Resetar E et al (2005) Implementing a commercial rule base as a medication order safety net. J Am Med Inform Assoc 12:383–389. doi:https://doi.org/10.1197/jamia.M1783 View ArticleGoogle Scholar
  9. Shah NR, Seger AC, Seger DL et al (2006) Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 13:5–11. doi:https://doi.org/10.1197/jamia.M1868 View ArticleGoogle Scholar
  10. Topaz M, Seger DL, Slight SP et al (2015) Rising drug allergy alert overrides in electronic health records: an observational retrospective study of a decade of experience. J Am Med Inform Assoc. doi:https://doi.org/10.1093/jamia/ocv143 Google Scholar
  11. Zhang Y, Li H, Duan H et al (2015) Mobilizing clinical decision support to facilitate knowledge translation: a case study in China. Comput Biol Med 60:40–50. doi:https://doi.org/10.1016/j.compbiomed.2015.02.013 View ArticleGoogle Scholar

Copyright

© The Author(s) 2016