Open Access

Effects of the performance management information system in improving performance: an empirical study in Shanghai Ninth People’s Hospital

  • Yinghui Cui1,
  • Zhengyi Wu1Email author,
  • Yao Lu1,
  • Wenzhong Jin1,
  • Xing Dai1 and
  • Jinxi Bai1
SpringerPlus20165:1785

https://doi.org/10.1186/s40064-016-3436-2

Received: 23 March 2016

Accepted: 29 September 2016

Published: 13 October 2016

Abstract

Improving the performance of clinical departments is not only the significant content of the healthcare system reform in China, but also the essential approach to better satisfying the Chinese growing demand for medical services. Performance management is vital and meaningful to public hospitals in China. Several studies are conducted in hospital internal performance management, but almost none of them consider the effects of informational tools. Therefore, we carried out an empirical study on effects of using performance management information system in Shanghai Ninth People’s Hospital. The main feature of the system is that it provides a real-time query platform for users to analyze and dynamically monitor the key performance indexes, timely detect problems and make adjustments. We collected pivotal medical data on 35 clinical departments of this hospital from January 2013 until December 2014, 1 year before and after applying the performance management information system. Comparative analysis was conducted by statistical methods. The results show that the system is beneficial to improve performance scores of clinical departments and lower the proportion of drug expenses, meanwhile, shorten the average hospitalized days and increase the bed turnover rate. That is to say, with the increasing medical services, the quality and efficiency is greatly improved. In a word, application of the performance management information system has a positive effect on improving performance of clinical departments.

Keywords

HospitalClinical departmentsPerformance managementInformation systemEmpirical study

Background

Over the past decades, the healthcare reform in China has already achieved some profound success. The new comprehensive and deepening reform plan, launched in 2013, marked the explicit goals to accelerate the public hospital reform, fulfill government responsibility and establish scientific and effective medical performance management system (Wong et al. 2016; Yip et al. 2012; Blumenthal and Hsiao 2005; Xin 2015; Cheng 2015). Therefore, performance management is vital and meaningful to public hospitals in China, and the critical challenge for them is how to improve the performance of clinical departments that is essential to ensure they are providing high-quality and efficiency medical services.

Several studies are done in hospital internal performance management, but nearly no studies take into consideration the effects of informational tools. With the rapid development of healthcare information technologies and the comprehensive construction of hospital information systems, plenty of hospitals in China built the medical big data information integration platform with clinical data repository (CDR) as the core component, thus achieving data integration, information sharing and cooperative processing of multiple information systems in hospital (Mezghani et al. 2015; Lin et al. 2012; Acampora et al. 2013; Blumenthal 2009; Jiang et al. 2014). Meanwhile, establishing performance management information system (PMIS) based on this medical big data information integrated platform to help supervise and appraisal performance of clinical departments.

So, does the application of PMIS facilitate improved performance of clinical departments? This paper takes Shanghai Ninth People’s Hospital as the research object, and describes the construction and functions of PMIS. In particular, we analyzed the medical data and performance scores on 35 clinical departments that before and after applying the intelligent performance management information system, then conclusions are drawn accordingly. Although similar studies might be done in other hospitals in Shanghai, to our best knowledge, Shanghai Ninth People’s Hospital is the first to complete the big data analysis and forecast in advance while the analysis shows abnormity.

The rest of papers are organized as follows: “Performance management information system” section introduces the construction of this system and describes its characteristics and functions in details. The results of medical data analyses are given in “Case study and results” section. Finally, last section gives summary and conclusion of this paper.

Performance management information system (PMIS)

Shanghai Ninth People’s Hospital is a large general hospital integrated with medical treatment, teaching and scientific research, which is characterized by oral medicine and plastic surgery. The hospital contains more than 1000 inpatient beds and 300 dental chairs, thereby allowing for 58,000 hospitalizations, nearly 2.97 million outpatients and emergency patients each year. In July 2013, the hospital cooperated with Microsoft (China) to implement the construction of medical big data information integration platform which takes clinical data repository (CDR) as the core component, and the overall system architecture as shown in Fig. 1.
Fig. 1

Overall system architecture of the information integration platform

Because of the history of the development of healthcare information technologies, clinical data are typically stored in relational databases (Yao et al. 2015a). Therefore, the hierarchical structure was designed to integrate various hospital information systems (such as RIS, HIS and PACS, etc.), thus realizing clinical data exchanging, resource sharing, interoperability and unified management of different departments and heterogeneous information systems. The clinical data repository (CDR) is a patient-centric, integrated, real-time, life-long data repository that can aggregate clinical data from multiple sources, and it has stored historical and real-time data from January 2008 until the present and the new data automatically import to CDR every 3 s. On that basis, we built some specific clinical data warehouses (CDW) for our specific purposes to serve on the medical service, clinical study and management of the hospital. Finally, we developed some external applications and tools, such as Business Operation Platform, Quality Control Platform, Patient-Centric View, Medical Research Information System (MedRis) and performance management information system (PMIS).

Figure 2 describes the logical architecture of the performance management information system (PMIS) which has been used formally in January 2014, and it has six modules: Target Setting and Planning, Performance program, Performance monitoring and improvement, Assessment and Evaluation, Performance Review, System Configuration. User could choose “graph” or “table” to check statistics. The performance management information system (PMIS) could achieve functions as follows:
Fig. 2

Logical architecture of the performance management information system

  • Dock with the real business systems, get clinical data from the CDR and automatically calculate the amount of work, the KPI score and bonus distribution of each clinical department. Subjective evaluation data such as patient satisfaction are scored at mobile termination and read automatically to the PMIS through data interface.

  • Calculate workload target, KPI base value and ratio of each clinical department automatically and realize the function of accounting and distribution according to the configuration of foundation information.

  • Analyze and process basic data, show and export the results in various forms.

  • Based on user’s right and limit to classify management. Hospital leader, clinical director and head of Performance Management Office have different system operational permissions.

Results were satisfactory when users were asked about their perceived benefit of applying this performance management information system. The system provides a real-time query platform for users to dynamically monitor the key performance indexes and analyze medical data, thereby, timely find problems and make adjustments. Users are aware of the need to apply the system and they consider it is contributed to provide high-quality and efficiency medical service, and it will clearly facilitate improved performance.

Case study and results

This section presents the results obtained in the data analysis. To illustrate the effect of performance management information system (PMIS), we collected data on 35 clinical departments in Shanghai Ninth People’s Hospital from January 2013 until December 2014, and divided the data into two groups. The first group data is from January to December in 2013, and the other set of data is from January to December in 2014 after applying the performance management information system. Do comparative analysis between the two sets of data by general statistical description and repetitive measurement deviation analysis with SPSS 20.0 statistical software. All the data in this case study come from the medical big data information integration platform.

Analysis of the performance scores

Managers of the Performance Management Office calculate monthly performance score of each clinical department according to key performance appraisal indexes which take patient satisfaction, quantity and quality of medical service, grade of disease, control of healthcare expenses and medical professionalism as core contents.

Analyze performance scores by means of repetitive measurement deviation analysis. This case has two repeated measurement factors, time and year. So the result of Mauchly’s test of sphericity gives the Epsilon correction coefficient, and the Greenhouse–Geisser corrected results should be taken (Johnson and Wichern 2007; Huang 2002), as shown in Table 1. Different time levels have significant differences (P < 0.05), and the score differences between 2013 and 2014 also have statistical significance (P < 0.05).
Table 1

Test of within-subjects effects

Measure: MEASURE_1

Source

Type III sum of squares

df

Mean square

F

Sig.

Time

 Sphericity assumed

111.315

11

10.120

5.672

0.000

 Greenhouse–Geisser

111.315

7.113

15.649

5.672

0.000

 Huynh–Feldt

111.315

9.185

12.119

5.672

0.000

 Lower-bound

111.315

1.000

111.315

5.672

0.023

Error (time)

 Sphericity assumed

667.281

374

1.784

  

 Greenhouse–Geisser

667.281

241.850

2.759

  

 Huynh–Feldt

667.281

312.301

2.137

  

 Lower-bound

667.281

34.000

19.626

  

Year

 Sphericity assumed

139.626

1

139.626

40.953

0.000

 Greenhouse–Geisser

139.626

1.000

139.626

40.953

0.000

 Huynh–Feldt

139.626

1.000

139.626

40.953

0.000

 Lower-bound

139.626

1.000

139.626

40.953

0.000

Error (years)

 Sphericity assumed

115.919

34

3.409

  

 Greenhouse–Geisser

115.919

34.000

3.409

  

 Huynh–Feldt

115.919

34.000

3.409

  

 Lower-bound

115.919

34.000

3.409

  

Time × year

 Sphericity assumed

115.701

11

10.518

5.307

0.000

 Greenhouse–Geisser

115.701

6.179

18.726

5.307

0.000

 Huynh–Feldt

115.701

7.701

15.025

5.307

0.000

 Lower-bound

115.701

1.000

115.701

5.307

0.027

Error (time × year)

 Sphericity assumed

741.290

374

1.982

  

 Greenhouse–Geisser

741.290

210.069

3.529

  

 Huynh–Feldt

741.290

261.827

2.831

  

 Lower-bound

741.290

34.000

21.803

  
Combining with Fig. 3, we clearly find the performance scores in 2014 are generally higher than 2013, and the increase of performance scores is statistically significant. It demonstrates that the performance scores are effectively improved when the performance management information system is applied, and the informational tool has a positive effect on improving performance of clinical departments.
Fig. 3

Profile diagram of interaction effect

Analysis of the medical service quality and efficiency

To further clarify the influence of this informational tool, a study about the quality and efficiency of medical service was carried out. We analyzed the pivotal medical data on 35 clinical departments from January 2013 until December 2014, 1 year before and after applying the performance management information system.

Descriptive statistics of the medical service are given in Table 2. Compared with 2013, medical service quantity of 2014 maintained continuous rapid growth. The number of outpatients and emergency patients has rapidly increased to over 2.67 million, and the average annual growth rate is 9.68 %. Meanwhile, the number of inpatients was 4.96 % higher than in 2013. Especially the number of surgery has a more than 30 % jump over the same period in 2013.
Table 2

Descriptive statistics of the medical service

Variable

2013

2014

Annual growth rate (%)

Mean

SD

Total

Mean

SD

Total

Number of outpatients and emergency patients

69,747.89

47,940.729

2,441,176

76,498.8

54,980.618

2,677,458

9.68

Number of inpatients

2359.36

1585.723

51,906

2476.27

1656.776

54,478

4.96

Number of surgery

4232.87

7836.144

135,452

5505.72

10,312.137

176,183

30.04

With the deepening reform of healthcare system in China, controlling the proportion of drug cost accounting for total medical expenditure has become a significant objective and great challenge for public hospitals (Liang et al. 2014; Yao et al. 2015b; Song et al. 2014; Yang et al. 2013). Curtailing the drug cost proportion is beneficial to guide the rational medication and control the unreasonable increase of drug expenditure (World Health Organization 2002; Quick et al. 1991; Guan et al. 2011; Le Grand et al. 1999). Figure 4 shows the comparative analysis result of drug cost proportion between 2013 and 2014, and the drug cost proportion decreased after the implementation of the performance management information system in 2014 with statistical significance (P < 0.05). It revealed that the application of such technology in the hospital performance appraisal had an important significance to keep the high efficiency, which promoted sustainable development of public hospital and better satisfy the people’s growing demand of medical services.
Fig. 4

Profile diagram of drug cost proportion

Average hospitalized days and bed turnover rate are comprehensive indexes to reflect the medical service efficiency. Shortening the average hospitalized days and increasing the bed turnover rate could effectively improve the performance of clinical departments (Lee and Wang 2006; Tseng et al. 2015; Escuriet et al. 2015; Mullen 2004; Bonfill et al. 2013). For more homogeneity, we analyzed these two indexes of surgical departments and non-surgical departments individually, as shown in Figs. 5 and 6. The x-axis represents month, the main y-axis represents average hospitalized days and the secondary y-axis represents bed turnover rate. As we have seen in Fig. 5, the red line shows the average hospitalized days from January to December of 2014 is completely below the blue line which reflects the average hospitalized days in same period of 2013, and the purple bar represents bed turnover rate of 2014 is nearly all higher than the green bar depicts bed turnover rate of 2013. Almost the same situation is observed in Fig. 6. This led us to draw the conclusion that the average hospitalized days decreased both in surgical departments and non-surgical departments after using the performance management information system, meanwhile, the bed turnover rate had a greater improvement. The result indicates the application of this performance management information system has a practical significance in improving the efficiency of medical service.
Fig. 5

Average hospitalized days and bed turnover rate of surgical departments in 2013–2014

Fig. 6

Average hospitalized days and bed turnover rate of non-surgical departments in 2013–2014

Conclusions

This study represented the constructions and functions of the performance management information system in Shanghai Ninth People’s Hospital that can provide a real-time query platform for users to monitor performance indexes and analyze medical data, thereby, timely detect problems and make adjustments. Additionally, we compared performance scores and crucial medical data on 35 clinical departments of this hospital before and after applying this system. The results show that the performance scores are greatly improved when the performance management information system is used, and the drug cost proportion which could guide the rational medication and control the unreasonable increase of drug expenditure is curtailed. Furthermore, it effectively shortened the average hospitalized days and increased the bed turnover rate. In other words, with the growing demand of medical services, the quality and efficiency of medical service has a significant improvement. Therefore, we could draw a conclusion accordingly that the implementation of performance management information system has a positive effect on improving performance of clinical departments. Long-term assessment for such system is in progress to make the databased more reliable and we anticipate that PMIS system could be implement in other hospitals in the future.

Declarations

Authors’ contributions

YHC participated in the design of the study, performed the statistical analysis and drafted the manuscript. ZYW conceived of the study, and participated in its design and coordination and helped to draft the manuscript. YL and WZJ participated in the construction and description of the performance management information system. XD participated in the design of the study. JXB participated in the data collection and analysis. All authors read and approved the final manuscript.

Acknowledgements

This work was supported by the Shanghai Municipal Commission of Health and Family Planning Scientific Research Program (grant no. 20124439).

Competing interests

The authors declare that they have no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Department of Shanghai Ninth People’s Hospital, Shanghai JiaoTong University School of Medicine

References

  1. Acampora G, Cook DJ, Rashidi P, Vasilakos AV (2013) A survey on ambient intelligence in healthcare. Proc IEEE 101(12):2470–2494View ArticleGoogle Scholar
  2. Blumenthal D (2009) Stimulating the adoption of health information technology. N Engl J Med 360(15):1477–1479View ArticlePubMedGoogle Scholar
  3. Blumenthal D, Hsiao W (2005) Privatization and its discontents—the evolving Chinese health care system. N Engl J Med 353:1165–1170View ArticlePubMedGoogle Scholar
  4. Bonfill X, Roqué M, Aller MB, Osoria D, Foradada C, Vives A, Rigau D (2013) Development of quality indicators from systematic reviews: the case of hospital delivery. Implement Sci 8:42View ArticlePubMedPubMed CentralGoogle Scholar
  5. Cheng T-M (2015) Shanghai’s track record in population health status: what can explain it? Comment on “Shanghai rising: health improvements as measured by avoidable mortality since 2000. Int J Health Policy Manag 4(9):631–632View ArticlePubMedPubMed CentralGoogle Scholar
  6. Escuriet R, White J, Beeckman K, Frith L et al (2015) Assessing the performance of maternity care in Europe: a critical exploration of tools and indicators. BMC Health Serv Res 15:491View ArticlePubMedPubMed CentralGoogle Scholar
  7. Guan X, Liang H, Xue Y, Shi L (2011) An analysis of China’s national essential medicines policy. J Public Health Policy 32:305–319View ArticlePubMedGoogle Scholar
  8. Huang JY (2002) Multivariate statistical analysis. Hwa Tai Publishing, TaipeiGoogle Scholar
  9. Jiang P, Winkley J, Zhao C, Munnoch R, Min G, Yang LT (2014) An intelligent information forwarder for healthcare big data systems with distributed wearable sensors. IEEE Syst J 99:1–9Google Scholar
  10. Johnson RA, Wichern DW (2007) Applied multivariate statistical analysis. Pearson, Upper Saddle RiverMATHGoogle Scholar
  11. Le Grand A, Hogerzeil HV, Haaijer-Ruskamp FM (1999) Intervention research in rational use of drugs: a review. Health Policy Plan 14:89–102View ArticlePubMedGoogle Scholar
  12. Lee WF, Wang YH (2006) A review of empirical studies on productivity and efficiency in Taiwan’s healthcare industry. Fu-Jen J Med 4:230–251Google Scholar
  13. Liang X, Xia T, Zhang X, Jin C (2014) Governance structure reform and antibiotics prescription in community health centres in Shenzhen, China. Fam Pract 31(3):311–318View ArticlePubMedGoogle Scholar
  14. Lin C, Lin I-C, Roan J (2012) Barriers to physicians’ adoption of healthcare information technology: an empirical study on multiple hospitals. J Med Syst 36(3):1965–1977View ArticlePubMedGoogle Scholar
  15. Mezghani E, Exposito E, Drira K, Da Silveira M, Pruski C (2015) A semantic big data platform for integrating heterogeneous wearable data in healthcare. J Med Syst 39:185View ArticlePubMedGoogle Scholar
  16. Mullen PM (2004) Using performance indicators to improve performance. Health Serv Manag Res 17:217–228View ArticleGoogle Scholar
  17. Quick JD, Laing RO, Ross-Degnan DG (1991) Intervention research to promote clinically effective and economically efficient use of pharmaceuticals: the International Network for Rational Use of Drugs. J Clin Epidemiol 44(Suppl 2):57S–65SView ArticlePubMedGoogle Scholar
  18. Song Y, Bian Y, Petzold M, Li L, Yin A (2014) Effects of the National Essential Medicine System in reducing drug prices: an empirical study in four Chinese provinces. J Pharm Policy Pract 7:12View ArticlePubMedPubMed CentralGoogle Scholar
  19. Tseng S-F, Lee T-S, Deng C-Y (2015) Cluster analysis of medical service resources at district hospitals in Taiwan, 2007–2011. J Chin Med Assoc 78:732–745View ArticlePubMedGoogle Scholar
  20. Wong HT, Guo YQ, Chiu MYL, Chen S, Zhao Y (2016) Spatial illustration of health-care workforce accessibility index in China: how far has our 2009 health-care reform brought us? Aust J Rural Health 24:54–60View ArticlePubMedGoogle Scholar
  21. World Health Organization (2002) Promoting rational use of medicines: core components. WHO Policy Perspect Med 5:1–6Google Scholar
  22. Xin H (2016) Experiences and lessons from urban health insurance reform in China. Popul Health Manag 19(4):291–297View ArticlePubMedGoogle Scholar
  23. Yang L, Liu C, Ferrier JA, Zhou W, Zhang X (2013) The impact of the National Essential Medicines Policy on prescribing behaviours in primary care facilities in Hubei province of China. Health Policy Plan 28:750–760View ArticlePubMedGoogle Scholar
  24. Yao Q, Tian Y, Li P-F, Tian L-L, Qian Y-M, Li J-S (2015a) Design and development of a medical big data processing system based on hadoop. J Med Syst 39(3):23View ArticlePubMedGoogle Scholar
  25. Yao Q, Liu C, Ferrier JA, Liu Z, Sun J (2015b) Urban–rural inequality regarding drug prescriptions in primary care facilities—a pre-post comparison of the National Essential Medicines Scheme of China. Int J Equity Health 14:58View ArticlePubMedPubMed CentralGoogle Scholar
  26. Yip WC, Hsiao WC, Chen W, Hu S, Ma J, Maynard A (2012) Early management of China’s huge and complex health-care reforms. Lancet 379:833–842View ArticlePubMedGoogle Scholar

Copyright

© The Author(s) 2016