Usability assessment of an electronic health record in a comprehensive dental clinic
© Suebnukarn et al.; licensee Springer. 2013
Received: 13 February 2013
Accepted: 30 April 2013
Published: 12 May 2013
In this paper we present the development and usability of an electronic health record (EHR) system in a comprehensive dental clinic.The graphic user interface of the system was designed to consider the concept of cognitive ergonomics.The cognitive task analysis was used to evaluate the user interface of the EHR by identifying all sub-tasks and classifying them into mental or physical operators, and to predict task execution time required to perform the given task. We randomly selected 30 cases that had oral examinations for routine clinical care in a comprehensive dental clinic. The results were based on the analysis of 4 prototypical tasks performed by ten EHR users. The results showed that on average a user needed to go through 27 steps to complete all tasks for one case. To perform all 4 tasks of 30 cases, they spent about 91 min (independent of system response time) for data entry, of which 51.8 min were spent on more effortful mental operators. In conclusion, the user interface can be improved by reducing the percentage of mental effort required for the tasks.
Electronic health record (EHR) is a generic term for integrated, computer-based, health information systems, accessible at the point of care. EHRs were classified on the basis of the International Organization for Standardization (ISO) definition (ISO/DTR 20514 2004). According to this definition, the EHR means a repository of patient data in digital form, stored and exchanged securely, and accessible by multiple authorized users. It contains retrospective, concurrent, and prospective information and its primary purpose is to support continuing, efficient and quality integrated health care. Some of the major advances in EHR use in health care during the past four and a half decades have dealt with relatively mundane matters such as approaches to capturing and storing information, communicating it, retrieving it, and producing and distributing reports (Mantas 2002). These capabilities have greatly reduced transcription errors, improved legibility of reports, eliminated redundancy, facilitated billing and financial functions, and provided a wide variety of other benefits, which indirectly do affect patient safety, health care quality, and efficacy (Poissant et al. 2005;Weiskopf & Weng 2013). Despite their promise, reports of EHR-related safety hazards are now emerging (Singh et al. 2011). Many currently unanswered, legal, ethical, and financial questions threaten the widespread adoption and use of EHRs. Key legal dilemmas that must be addressed in the near-term pertain to the extent of clinicians’ responsibilities for reviewing the entire computer-accessible clinical synopsis from multiple clinicians and institutions, the liabilities posed by overriding clinical decisions support warnings and alerts, and mechanisms for clinicians to publically report potential EHR safety issues (Sittig & Singh 2011).
The main thrust of information technology efforts recently has been to encourage broad adoption of EHRs, and clinical systems and applications based on the EHR that provide various kinds of functionality (Takian et al. 2012). Dentists are increasingly adopting EHRs in their practices (Zvárová et al. 2008;Cederberg & Valenza 2012;Tokede et al. 2013). Adoption rate is high among academic centers, but less so at community hospitals and in office practices. Estimates of adoption vary, depending on what one considers the benefits of an EHR are, ranging from the ability to review laboratory and radiology results or other limited functionality to that of a system that includes computerized physician order entry. Dentists are increasingly adopting EHRs, and are using structured data entry interfaces to enter data such that the data can be easily retrieved and exchanged (Walji et al. 2013). Although some studies have discussed the benefits of EHR in dentistry (Zvárová et al. 2008;Tokede et al. 2013;Hippmann et al. 2010), Poor usability is one of the major barriers against optimum usage of electronic health records (EHRs). It is common for user resistance to challenge implementation efforts. As more dental clinics undertake EHR implementations, a solid understanding of how to foster acceptance of EHR is necessary to reap the benefits of clinical error reduction, improved quality of care, and decreased healthcare costs (Jenkings & Wilson 2007). The principles and practices of human-computer interactions can be used to bolster user satisfaction and increase usability, thereby increasing the chances of being successful in the implementation of EHR (Garrard 2000).
In this paper we present the development and usability of an electronic health record (EHR) system in a comprehensive dental clinic. The graphic user interface of the system has been designed to consider the concept of cognitive ergonomics. The user interface of the EHR was analyzed using the Cognitive Task Analysis (CTA) method called “Goals, Operators, Methods, and Selection rules” (GOMS) (Saitwal et al. 2010) and an associated technique called “Keystroke Level Model” (KLM) (Chi & Chung 1996).The GOMS method was used to evaluate the user interface of the EHR by identifying all sub-tasks of a given task and classifying them into mental (internal) or physical (external) operators. KLM was used to predict task execution time required to perform the given task through the application of its standard set of operators. Our goal is to suggest areas of improvement to promote flexibility in the user interface.
Materials and methods
Electronic oral health record system
In this study, we focus on the uses of EHR in a comprehensive dental clinic of final year undergraduates. The dental students take care of the patients in the comprehensive care clinic under supervisions of the faculty members. The dental clinic EHR connects with the hospital EHR. All related medical information, e.g. past medical history, laboratory tests, can be retrieved from the hospital EHR. It integrates specifications of all aspects of oral health care, including history, signs and complaints, prescriptions and procedures for holistic oral health care. The dental section of the system is an open source that can plug in to the hospital EHR. The hospital EHR is the open source program and operates in client–server architecture. The hospital information system is able to connect to the infrastructures and networks of Community hospitals, Province hospitals and health care centers. The data is recorded in a Database Server and used in client/server applications.
Usually, existing electronic oral health records allow the practitioner to electronically document patient care, allow claims transactions to be more quickly and reliably processed as well as being able to communicate with records from other healthcare disciplines. The EHR developed in this study has additional functions. The graphic user interface of each oral health status item has been designed to consider the concept of cognitive ergonomics. Cognitive ergonomics (Hoc 2008), as defined by the International Ergonomics Association, is concerned with mental processes, such as perception, memory, reasoning, and motor response, as these processes affect the interactions among humans and other elements in a system. The graphic user interface of each oral health status item has been designed to follow steps in status recording, from chief complaint to hygiene and periodontal condition, defect and restoration. Graphics for each status item have no ambiguity and are easy to remember.
To evaluate the user interface of the EHR, we randomly selected 30 paper based cases that had oral examinations for routine clinical care by final year undergraduates. Ten final year undergraduates were recruited and instructed on the use of the EHR and the requirements of the oral health status recording. All participants gave their written informed consent which was approved by the Ethical Review Board of the University. The students had no experience of using EHR and familiarized themselves with the systems interface for 1 week, but they did not familiarize themselves with the task. After the familiarization process, the students recorded the oral health status of the 30 case. Inter-rater reliability was then calculated to determine agreement among students who independently conducted GOMS analysis for each task.
“Locating the patient” task using GOMS along with KLM technique
Cognitive distribution Mental/Physical operator
Average Time (s) to task completion by each participant
Retrieve the name of the patient
Extend the mouse to the patient name
Click on the located patient name
Extend the mouse to “Oral examination” button
Click on “Oral examination” button
A GOMS analysis for 4 tasks and their subtask
Locating the patient
step 1 Retrieve the name of the patient [M]
step 2 Extend the mouse to the patient name [P]
step 3 Click on the located patient name [P]
step 4 Extend the mouse to “Oral examination” button [P]
step 5 Click on“Oral examination” button [P]
Enter chief complaint
step 1 Think of location in main menu [M]
step 2 Extend hand towards mouse and go to chief complaint menu [P]
step 3 Click on the chief complaint menu [P]
step 4 Retrieve the patient's chief complaint [M]
step 5 Type the patient's chief complaint [P]
step 6 Extend hand towards mouse and go to save button [P]
step 7 Click on save button [P]
step 8 Extend hand towards mouse and go to back button [P]
step 9 Click on back button and go to main page [P]
Document coding of tooth status
step 1 Retrieve the data of tooth [M]
step 2 Extend hand towards mouse and go to tooth number button/select tooth number [P]
step 3 Click on tooth number button [P]
step 4 Search for coding of tooth status/ select tooth options [M]
step 5 Extend hand towards mouse and go to coding of tooth status button/select tooth status [P]
step 6 Click on tooth status [P]
step 7 Review coding of tooth status [M]
step 8 Click on coding of tooth status [P]
step 9 Extend hand towards mouse and go to back button [P]
step 10 Click on back button and go to main page [P]
Save the data
step 1 Review all of tooth status [M]
step 2 Extend hand towards mouse and go to save button [P]
step 3 Click on save button [P]
As far as forth operator – mental operator (M) is concerned, it is estimated that those operators take 1.2 s on average.
Summary results of 4 prototypical tasks showing total number of steps, distribution of mental and physical operators, execution times, and inter-rater reliability
Execution time (s)
Total time (s)
Locating the patient
Enter chief complaint
Document coding of tooth status
Save the data
Average mental operator (%)
Total execution time for all tasks (s)
Total execution time for all tasks (%)
Steps for each of the 4 tasks were further classified as either mental [M] or physical [P] operators depending upon their cognitive distributions based on the GOMS classification. For 30 cases, the results show that of the total operators, 35.30 % of the steps were mental.
The second half of Table 3 estimates the total amount of time the user would take to execute each task in all 30 cases. These values reflect the time a user has spend interacting with the EHR and do not include the time in examining a patient. Further, these estimates are based on the assumption that the users are expert computer users. Execution time for the 4 tasks ranged from a low of 69 s to high of 1911.6 s. The mental operators accounted for about 56.89% of the total time.
Inter-rater reliability was also assessed between the mental/ physical operator classification results by two evaluators (see Table 3). These values range between 0.997 (substantial agreement) and 1 (almost perfect agreement) with the average of 0.9, indicating good reliability of the evaluation method.
EHR systems have the potential to address many limitations of paper-based records and have been described by the Institute of Medicine as an essential technology for medical care (Kassirer 1995).Dental schools have recently begun an ambitious program of converting undergraduate, graduate, and faculty clinics from paper to EHRs (Cederberg & Valenza 2012;Tokede et al. 2013). The functional requirements of EHRs in dental schools are different from those in medicine and nursing. Surveys of students, faculty, and staff before and after implementation indicated that users had mixed feelings about the system in terms of efficiency and time required compared with paper charts. As reported by Walji et al. (2009), the users found that the electronic patient record improved patient care and that they would recommend such a system to dentists starting a new practice. It is essential that the dentists understand the potential benefits of using EHRs in their practices not only for patient care but also for outcome measurements (when linked with other health and social care datasets), quality improvement, public health surveillance, and research.
Researchers in medicine, ergonomics, health economics, cognitive science, and biomedical informatics have conducted several studies on the design of EHRs (Liu et al. 2009;Collins et al. 2011;Cho et al. 2010;Blobel et al. 2003). The general approach that has been used in analyzing the basis for human performance is known as cognitive task analysis (CTA) (Militello & Hutton 1998). Its purpose is to capture the way the mind works―to capture cognition. CTA could describe the basis for skilled performance that is being studied. In using CTA, cognitive scientist try to capture what people are thinking about, what they are paying attention to, the strategies they are using in making decisions, what they are trying to accomplish, what information they discard, and what they know about the way a process works. In studying system usability, there are several possible task analysis techniques that are used to detail the interactions between users and systems. This is done to facilitate and enhanced system design, procedures, training, and support. Task analysis is ideally used when designing a system (Shachak et al. 2009). Using task analysis early in the design process is one way to integrate the study of human processes, including user capabilities and limitations, into the final product. After changes have been made based on task analysis findings, it is important to re-analyze the resulting system to ensure no unforeseen consequences developed as a result of the process. Task analysis could have assisted several authors mentioned in the literature review. The studies by Murff and Kannry (Murff & Kannry 2001) and Payne et al. (2003) investigated various ways that design affects how physicians carry out their work and what physician preferences are. The knowledge gained from these studies is in hindsight. In contrast, a thorough task analysis, by improving the design of a system, can save time and money upfront by reducing the need for revisions. Payne et al. (2003) mentioned that more research in the design stages could have prevented the creation of numerous unnecessary order sets. In this study, we used the cognitive task analysis method called “Goals, Operators, Methods, and Selection rules” and an associated technique called “Keystroke Level Model”. The results suggest that there are many opportunities to improve the efficiency of information delivery and task performance to reduce system complexity. For example, designers could try to simplify the interface and improve the organization to integrate relevant information and data using less windows and screens, or redesign some tasks and functions. Mental workload could be reduced by providing information reminders and recognition-based assistance. For example, some functions like “fill in the blank” could be replaced with “select from a droplist”. Furthermore, we can use colour to classify each main menu on the user interface to reduce mental recall, and also improve the user-friendly interface. For future work, we plan to evaluate the usability of the EHR system on behalf of the user, as in function and representation.
This study investigated the current user interface of the EHR in the dental student clinic using the cognitive task analysis. The study reveals that the total number of steps required for doing all given tasks. On the other hand, the study also shows that the mental operators are the main part of the total step operators. Further analysis of the execution time shows that more than half of the time is spent in performing mental operators, which can lead to mental fatigue for the users due to extended mental workload for long periods of time. The user interface can be improved by reducing the percentage of mental effort required for the tasks.
- Blobel B, Stassinopoulos G, Pharow P: Model-based design and implementation of secure, interoperable EHR systems. AMIA Annu Symp Proc 2003, 96-100.Google Scholar
- Cederberg RA, Valenza JA: Ethics and the electronic health record in dental school clinics. J Dent Educ 2012, 76: 584-589.Google Scholar
- Chi CF, Chung KL: Task analysis for computer-aided design (CAD) at a keystroke level. Appl Ergon 1996, 27: 255-265. 10.1016/0003-6870(96)00012-9View ArticleGoogle Scholar
- Cho I, Kim J, Kim JH, Kim HY, Kim Y: Design and implementation of a standards-based interoperable clinical decision support architecture in the context of the Korean EHR. Int J Med Inform 2010, 79: 611-622. 10.1016/j.ijmedinf.2010.06.002View ArticleGoogle Scholar
- Collins SA, Bakken S, Vawdrey DK, Coiera E, Currie L: Model development for EHR interdisciplinary information exchange of ICU common goals. Int J Med Inform 2011, 80: e141-149. 10.1016/j.ijmedinf.2010.09.009View ArticleGoogle Scholar
- Garrard CS: Human-computer interactions: can computers improve the way doctors work? Schweiz Med Wochenschr 2000, 130: 1557-1563.Google Scholar
- Hippmann R, Dostálová T, Zvárová J: Voice-supported electronic health record for temporomandibular joint disorders. Methods Inf Med 2010, 49: 168-172.View ArticleGoogle Scholar
- Hoc JM: Cognitive ergonomics: a multidisciplinary venture. Ergonomics 2008, 51: 71-5. 10.1080/00140130701801124View ArticleGoogle Scholar
- ISO/DTR 20514: Health Informatics – Electronic Health Record – Definition, Scope, and Context. 2004. http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=39525 Google Scholar
- Jenkings KN, Wilson RG: The challenge of electronic health records (EHRs) design and implementation: responses of health workers to drawing a ‘big and rich picture’ of a future EHR programme using animated tools. Inform Prim Care 2007, 15: 93-101.Google Scholar
- Kassirer JP: The next transformation in the delivery of health care. N Engl J Med 1995, 332: 52-54. 10.1056/NEJM199501053320110View ArticleGoogle Scholar
- Liu H, Hou XQ, Hu G, Li J, Ding YQ: Development of an EHR system for sharing - a semantic perspective. Stud Health Technol Inform 2009, 150: 113-117.Google Scholar
- Mantas J: Electronic health record. Stud Health Technol Inform 2002, 65: 250-257.Google Scholar
- Militello LG, Hutton RJ: Applied cognitive task analysis (ACTA): a practitioner’s toolkit for understanding cognitive task demands. Ergonomics 1998, 41: 1618-1641. 10.1080/001401398186108View ArticleGoogle Scholar
- Murff HJ, Kannry J: Physician satisfaction with two order entry systems. J Am Med Inform Assoc 2001, 8: 499-509. 10.1136/jamia.2001.0080499View ArticleGoogle Scholar
- Payne TH, Hoey PJ, Nichol P, Lovis C: Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system. J Am Med Inform Assoc 2003, 10: 322-329. 10.1197/jamia.M1090View ArticleGoogle Scholar
- Poissant L, Pereira J, Tamblyn R, Kawasumi Y: The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc 2005, 12: 505-516. 10.1197/jamia.M1700View ArticleGoogle Scholar
- Saitwal H, Feng X, Walji M, Patel V, Zhang J: Assessing performance of an Electronic Health Record (EHR) using Cognitive Task Analysis. Int J Med Inform 2010, 79: 501-506. 10.1016/j.ijmedinf.2010.04.001View ArticleGoogle Scholar
- Shachak A, Hadas-Dayagi M, Ziv A, Reis S: Primary care physicians’ use of an electronic medical record system: a cognitive task analysis. J Gen Intern Med 2009, 24: 341-348. 10.1007/s11606-008-0892-6View ArticleGoogle Scholar
- Singh H, Classen DC, Sittig DF: Creating an oversight infrastructure for electronic health record-related patient safety hazards. J Patient Saf 2011, 7: 169-174. 10.1097/PTS.0b013e31823d8df0View ArticleGoogle Scholar
- Sittig DF, Singh H: Legal, ethical, and financial dilemmas in electronic health record adoption and use. Pediatrics 2011, 127: e1042-1047. 10.1542/peds.2010-2184View ArticleGoogle Scholar
- Takian A, Petrakaki D, Cornford T: Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems. BMC Health Serv Res 2012, 30: 105.View ArticleGoogle Scholar
- Tokede O, Walji M, Ramoni R: Treatment planning in dentistry using an electronic health record: implications for undergraduate education. Eur J Dent Educ 2013, 17: e34-e43. 10.1111/j.1600-0579.2012.00759.xView ArticleGoogle Scholar
- Walji MF, Taylor D, Langabeer JR 2nd, Valenza JA: Factors influencing implementation and outcomes of a dental electronic patient record system. J Dent Educ 2009, 73: 589-600.Google Scholar
- Walji MF, Kalenderian E, Tran D, Kookal KK, Nguyen V, Tokede O, White JM, Vaderhobli R, Ramoni R, Stark PC, Kimmes NS, Schoonheim-Klein ME, Patel VL: Detection and characterization of usability problems in structured data entry interfaces in dentistry. Int J Med Inform 2013, 82: 128-138. 10.1016/j.ijmedinf.2012.05.018View ArticleGoogle Scholar
- Weiskopf NG, Weng C: Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc 2013, 20: 144-151. 10.1136/amiajnl-2011-000681View ArticleGoogle Scholar
- Zvárová J, Dostálová T, Hanzlícek P: Electronic health record for forensic dentistry. Methods Inf Med 2008, 47: 8-13.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.