Healthc Inform Res Search


Healthc Inform Res > Volume 18(1); 2012 > Article
Heo, Hwang, Kim, Cho, Lee, Kim, Kim, Baek, and Yoo: Comparing the Certification Criteria for CCHIT-Certified Ambulatory EHR with the SNUBH's EHR Functionalities



This study aims to investigate the suitability of electronic health record (EHR) systems in Korea for global certification and to propose functions for future global systems by comparing and analyzing the certification criteria for Certification Commission for Health Information Technology (CCHIT) Certified Ambulatory EHR with BESTCare, which is the EHR system at Seoul National University Bundang hospital.


Domain expert groups were formed to analyze the inclusion of BESTCare functions and the types of differences for each of the CCHIT Certified 2011 Ambulatory EHR Certification Criteria. The types of differences were divided into differences in functions (F), differences in business processes (B), and differences in government policies (P).


Generally, the criteria that showed differences in functions pertained to the connection between the diagnosis/problem list and order, the alert and warning functions for medication-diagnosis interactions, and the reminder/instruction/notification messages related to the patient's immunization status; these absent functions were enhanced clinical decision support system (CDSS) functions related to patient safety and healthcare quality. Differences in government policies were found in the pharmacy's electronic prescription functions, while differences in business processes were found in the functions constrained by the local workflow or internal policy, which require some customization.


Functions that differed between the CCHIT certification criteria and the BESTCare system in this study should be considered when developing a global EHR system. Such a system will need to be easily customizable to adapt to various government policies and local business processes. These functions should be considered when developing a global EHR system certified by CCHIT in the future.

I. Introduction

The term "Healthcare Information System Certification" refers to a system for follow-up management products, services, and compliance with standard specifications for healthcare information systems (e.g., order communication system, electronic medical record system, picture archiving and communication system) [1,2]. A certification system is designed to build the foundation to maintain a health information system that is efficient, ensures interoperability, and secures the reliability of products [3].
With the recent passage of the economic stimulus bill in the United States, interest in electronic health record (EHR) systems for the healthcare industry has increased worldwide [3]. This bill presents a program to provide incentives to clinics and hospitals that adopt EHR systems, ensuring interoperability across various healthcare standards. Numerous EHR systems have been developed as a result of such incentive programs, and many clinics and hospitals are adopting EHR systems certified by the Certification Commission for Health Information Technology (CCHIT). In response to the changes in the US healthcare industry, some companies in South Korea are trying to enter the U.S. healthcare market on the basis of their experience in establishing EHR systems for hospitals in South Korea [4,5].
There are five lists of functional criteria for the certification of healthcare information systems, including COPIC (222 elements), Bureau of Primary Health Care (BPHC, 266 elements), Health Level Seven (HL7, 138 elements), CCHIT (315 elements), and Practice Fusion (PF, 252 elements) [6-10]. The HL7, CCHIT, and PF EHR functionality lists each have three major categories. HL7's three major categories are direct care and supportive and information infrastructure; CCHIT's categories are functionality, interoperability, and security and reliability; and PF's categories are features and functions, interoperability, and security and reliability [11]. This study examined and analyzed CCHIT, which contains the most elements. Given that the U.S. government urged the extensive use of health information technology (HIT) and the routine use of EHR, the U.S. HIT associations have established CCHIT as a voluntary private organization to certify HIT products. Currently, CCHIT operates four working groups (WGs). In addition, CCHIT develops EHR certification criteria and procedures, as well as certification criteria for the network connecting healthcare information systems. By dividing the target areas for the development of certification criteria into the ambulatory, inpatient, emergency department, and enterprise domains, CCHIT also develops and utilizes the certification criteria for functionality, interoperability, and security [4,5,12]. The certification criteria for functionality are developed to assess functionality related to the creation and management of EHR as well as to digitalize administrative tasks. The certification criteria for interoperability are developed to assess interoperability related to information exchange and connection between different EHR systems, and the certification criteria for security are developed to assess the security measures that protect patient information in the EHR system [2]. The EHR (BESTCare) at Seoul National University Bundang Hospital is the first system outside of North America to receive a stage 7 certification from Healthcare Information and Management Systems Society (HIMSS) analytics EMR adoption model [13]; it is a full EHR system implemented with standard terminology, standard practice guidelines, a real-time medication system using the radio frequency identification (RFID) system for drugs, the clinical decision support system (CDSS) for antibiotics, blood transfusions and diagnostic tests, and the ability to exchange electronic medical information between clinics and hospitals using the international standard.
By comparing and analyzing the certification criteria for CCHIT Certified Ambulatory EHR in the U.S. and the functionality of BESTCare, this study aims to investigate the suitability of EHR systems in Korea to meet the global certification requirements as well as to examine the necessary functions in detail. Such study is needed to propose functions for future global systems.

II. Methods

For this study, domain expert groups were formed to review all criteria presented in the CCHIT Certified 2011 Ambulatory EHR Certification Criteria April 7, 2010 [4,5].
The expert groups were composed of medical staff (2 doctors, 6 nurses), system developers (5 people), and security specialists (1 system security specialist, 1 security manager). Based on expert group's review, all experts get together at the same spot, shared each of all functions, checked with the test script and classified inclusion or exclusion of BESTCare functionality according whether the suggested test script can be executed or not. For functions that were not included, the deficiencies were classified as functions (F), business processes (B), and government policies (P). Differences in functions (F) indicate the absence of a function itself, while differences in business processes (B) refer to functions that are limited or different because of policies that are internal to the hospital system or workflow. Differences in government policies (P) refer to functions that are restricted by Korean domestic medical law, i.e., these policies cannot be implemented because of the differences in policy between the U.S. and South Korea.

III. Results

In order to check how well BESTCare Ambulatory HER functionality suits certification standards which the international standard suggests, we investigated whether each CCHIT Ambulatory EHR function was included or not in BESTCare Ambulatory EHR system, 69.9% (200) of 286 CCHIT criteria were mapped. Table 1 shows the results of mapping by category (functionality, interoperability, and security). For each category (functionality, 229 criteria; interoperability, 8 criteria; and security, 49 criteria), the mapping rates were 67.7% (155), 12.5% (1), and 89.8% (44), respectively.

1. Functionality Category

1) Differences in functions

Table 2 shows the functionality criteria in which differences were found between the U.S. CCHIT Ambulatory EHR certification criteria and BESTCare.
A "compiling list" group is a set of criteria related to problems, medication, allergies, and adverse reactions. This term refers to management functions under the categories "Allergies and adverse reactions," "Medications," "Problems," "Diagnostic tests to order," and Medications to order." A detailed examination of this group has shown that the "Allergies and adverse reactions" category is missing functions that can explicitly record whether there was an allergy review (the ID used and the date of review) and functions that display patient allergy lists and the date of information entry for allergies. The "Medications" list lacked functions to explicitly display that the patient has no prescriptions and functions to enter and check new prescriptions. Additionally, a function is needed to record why certain drugs may be excluded from the current medication list of medications. The "Problem management" list must be able to connect orders and prescriptions for medication for one or more problems/diagnoses when the patient information is protected. Additionally, the category "Orders for diagnostic tests" lacked a functionality that could capture the details of how the tests were related to the diagnosis. These deficiencies emerged because connections between the current principal diagnosis and the medications prescribed were not managed on the database.
The following alerts and practical medication-identification functions were required to be added to the category "Orders for medications": 1) display a dose calculator for patient-specific dosing based on weight; 2) alert the user if the drug-interaction information is outdated; 3) add reminders for necessary follow-up tests based on the medication prescribed; 4) alert the user when a new medication is prescribed/ordered that no drug-interaction, allergy, and formulary checking will be performed against an uncoded or free-text medication; and 5) identify medication samples dispensed, including the lot number and expiration date.
The category "Orders and referral management in the "Creating orders" group also emphasized the connection between the problem/diagnosis and the order/medication by requiring the addition of functions related to connections between the problem/diagnosis and the prescription ordered.
The "Managing workflow" list has some communication functions within the EHR system, but the following necessary functions were absent: 1) clinical task assignment and routing: create and assign tasks by user or user role; designate a task as completed; present a list of tasks by user or user role; re-assign and route tasks from one user to another user; remove a task without completing the task; 2) inter-provider communication: document verbal/telephone communication in the patient record; support messaging between users; and 3) manage immunizations: capture, in a discrete field, an allergy/adverse reaction to a specific immunization.
For the criteria "Organizing patient data," a detailed implementation of the following functions was required for each category: 1) manage clinical documents and notes: filter, search or order notes by associated diagnosis within a patient record; display patient notes in a manner that distinguishes them from other content in the system; graph height and weight over time; display modified notes in full, including both the original content and any changes, corrections, clarifications, addenda, etc.; 2) manage patient advance directives: indicate that a patient has completed advance directive(s); indicate the type of advance directives; indicate when advance directives were last reviewed; and 3) managing patient demographics: maintain and make available historic information for demographic data including prior names, addresses, phone numbers, and email addresses; search information by patient's first and last name.
The criteria for receiving and displaying information are related to test results, patient consent, authorizations, and clinical documents from outside the practice. These criteria include the categories "Capture external clinical documents," "Health-record output," "Identification and maintenance of patient records," "Result management," "Report generation," and "Health-record summaries." The missing functions are as follows: 1) retrieve indexed, scanned documents by document type and date; 2) define one or more reports as the formal health record for disclosure purposes; 3) merge information from two patient records into a single record; 4) indicate normal and abnormal results based on data from the original data source; notify and forward a result; and 5) produce reports based on the absence of specific clinical data; save report parameters to generate subsequent reports; modify one or more parameters of a saved report specification when generating a report using that specification.
The category "Decision support" is a criteria group related to alerts and reminders for disease management, preventive services, and wellness. Its missing functions are summarized below: 1) providing notifications and reminders related to immunizations and identifying functions for disease management as well as for preventive and wellness services based on patient demographic and clinical data; 2) providing individualized alerts and functions for updating/overriding guidelines for disease management or prevention, modifying rules for guideline-related alerts, and establishing criteria for disease management as well as for preventive and wellness services based on patient demographic and clinical data; and 3) support for drug-interaction alerts: set the severity level at which drug interaction warnings should be displayed; display, on demand, potential drug-diagnosis interactions; check for a potential interaction between newly documented allergies and the patient's current medications.
For example, the following test script is required during a certification test for immunization.
[Test Script 4.81] Generate reminder letters for patients who are due or overdue for a DTaP immunization booster: either automatically generate a letter to a patient (either Emily Jones or Will Haynes) that automatically includes content specifying what services are due or automatically generate a letter to all patients who are due for a specified service (DTaP).

2) Differences in business processes

Criteria that showed differences in the business process were found in the "Problem management" and "Result management" categories (Table 3). Missing elements were not implemented in the BESTCare system because functions to capture, maintain, and display the free-text comments associated with a given problem/diagnosis are currently restricted by hospital policy. As an authorized user in the current system can verify a result at any time without forwarding information, such a function was found to be unnecessary.

3) Differences in government policies

The criteria in which differences arise from government policies are shown in Table 4. For example, in South Korea, the national healthcare policy requires verification of the patient's medical eligibility; in the U.S., a different evaluation system is used. Other differences lie in the transmission of electronic prescription to pharmacies, the ability to verify a patient's eligibility for electronic prescription coverage, and the differences in the ability to refill a prescription without re-entering data.

2. Interoperability Category

Table 5 shows the results of mapping BESTCare over the CCHIT interoperability criteria.
CCHIT laboratories must be able to receive lab results using the Health Level 7 (HL7) v2.5.1 message standard. This function has also become available in the BESTCare system following the increased awareness of international standards in Korea.
Most functions related to the "Medications/ePrescribing" category were connected with pharmacies, such as sending electronic prescriptions or requesting refills, as seen in Table 5. Although these functions are limited by government policies in Korea, they can be considered essential to fulfilling the international requirements.
The "Clinical documentation" category requires functions to generate, file, and display patient summary documents in the Healthcare Information Technology Standards Panel (HITSP) C32/Continuity of Care Document (CCD) standard format, but because BESTCare can only generate, send, and display referral notes or consultation notes in the HL7 clinical document architecture (CDA), the functions were considered to be absent. In addition, when CCD documentation is generated, at least the patient demographics, the medication list coded with RxNORM or NCC, the allergy list coded with RxNORM, and the UNII information must be included.

3. Security Category

Criteria related to security are composed of the following categories: Access control, Amendments, Audit, Authentication, Backup/Recovery, Data integrity Auditability, Documentation, Manage clinical documentation, and Technical services. Of 43 criteria, 5 functions absent from BESTCare were confirmed as functional differences. Table 6 contains a detailed CCHIT criteria list showing the differences in function.
Currently, when BESTCare is used to view the full content of a finalized note (Amendments category, Table 6), all modifications are logged; however, as BESTCare does not display the modifications on a screen in the finalized note format, this function was classified as absent and requiring supplementation.
The ability to manage logs and auditable events was also implemented in BESTCare, but it is not possible to include or exclude auditable events by setting the program parameters or to export time stamps in a coordinated universal time (UTC) standard time (i.e., 1994-11-05T13:15:30-05:00) format.
As the procedures to scan the system and installation media for well-known malware (Technical Services category) were not documented, the related functions were mistakenly classified as absent. Furthermore, standards-based encryption was required (but not currently available) when storing protected health information (PHI) on a portable device.

IV. Discussion

EHRs must satisfy the qualification criteria for certification because they support clinical decision-making, input physicians' orders, and provide electronic healthcare information from other sources, capturing and querying data related, for example, to healthcare quality. Providers must also prove "meaningful use," including the electronic exchange of information to improve the quality and coordination of treatment via electronic prescriptions and quality-measure reports [14]. Furthermore, products with an emphasis on approaches such as usefulness, interoperability, and affordability should be used and reviewed in the market to allow decision-making.
A certification system is a way to improve the quality of EHR products and secure interoperability, but is not a legal sanction. The participation of stakeholder groups and the opportunity for hearings must be conducted through a standardized, transparent, and objective process that involves input from stakeholders. Additionally, a testing process that reflects the reality of use should be developed. It is necessary to select certification criteria based on the CCHIT certification process (such as EHR functionality, interoperability, and security) that are appropriate for Korean society and to develop a test process that reflects the needs of Korean health-care providers. For the healthcare information system in Korea to secure global competitiveness and dominate the international healthcare market, it must be equipped with a healthcare information system platform and solutions that meet international standards. Standardized healthcare information technology is necessary to developing healthcare information services that ensure convenience, efficiency, and interoperability. In addition, to provide safe medical services and minimize users' reluctance or resistance, it is necessary to maintain the security and confidentiality of the healthcare data as well as to devote technical, legal, and ethical resources to the protection of privacy.
Through this study, we recognized the development of a CDSS that emphasizes patient-centered service in healthcare and identified the major EHR function that supports patients in prevention-related decision-making. CDSS ensures the interoperability of EHRs in addition to a common infrastructure of healthcare information. Through the development of standards, services, and tools required for operation, CDSS will protect patients' interests and rights via improvements to patient safety and the prevention of medical malpractice, as well as by preventing the waste of resources [15-18].
Furthermore, the consideration of a wider variety of CDSS in the next global EHR system may yield a medical service that more faithfully tends towards care based on practice guidelines, with patient-centered care that contributes to enhanced patient surveillance and monitoring and reduced medication errors. Such a service is expected to significantly benefit the nation and the public interest, enhancing patient safety and management of service quality, as well as saving unnecessary medical expenses.


This work was supported by the Industrial Strategic Technology Development Program (grant 10038690, Global Healthcare Software Framework Development) funded by the Ministry of Knowledge Economy (MKE).


No potential conflict of interest relevant to this article was reported.


1. Shin HM. The future of health care services and digital. Microsoftware 2010;319:156-163.

2. Ha KS, Park HY, Lee SI, Kim JW, Hwang H, Cho EY, Heo EY, Lee MH, Yun GS. EHR system functionality and exchange of medical information. 2010. Seoul, Korea: Center for Interoperable EHR.

3. Kim HS, Cho H, Lee IG. Design and development of an ehr platform based on medical informatics standards. J Intell Inf Syst 2011;21:456-462.

4. CCHIT certified 2011 ambulatory HER certification criteria [Internet]. Certification Commission for Health Information Technology. c2012. cited at 2012 Mar 20. Chicago (IL): CCHIT; Available from:

5. CCHIT certified 2011 certification handbook [Internet]. Certification Commission for Health Information Technology. c2012. cited at 2012 Mar 20. Chicago (IL): CCHIT; Available from:

6. BPHC electronic medical record resources [Internet]. Bureau of Primary Health Care cited at 2012 Mar 20. Available at:

7. Benchmarks for electronic medical record systems [Internet]. COPIC. c2010. cited at 2012 Mar 20. Denver (CO): COPIC; Available from:

8. COPIC. Benchmarks for electronic medical record systems: a decision tool and resource. 2001. Denver (CO): COPIC Insurance.

9. Health level seven international (HL7) [Internet]. 2012. cited at 2012 Mar 20. Ann Arbor (MI): Health Level Seven International; Available from:

10. Rowley R. Practicing without paper charts: one clinic's experience. c2005. cited at 2012 Mar 20. Leawood (KS): American Academy of Family Physicians; Available from:

11. Drury BM. Ambulatory EHR functionality: a comparison of functionality lists. J Healthc Inf Manag 2006;20:61-70. PMID: 16429960.

12. Shin HM. Digital hospital architecture and our attitude. Microsoftware 2010;322:152-159.

13. HIMSS analytics Asia: stage 7 hospitals. Healthcare Information and Management Systems Society. 2012. cited at 2012 Mar 20. Chicago (IL): HIMSS; Available from:

14. Electronic health records and meaningful use. Office of the National Coordinator for Health Information Technology. 2012. cited at 2012 Mar 20. Washington: US Department of Health & Human Services; Available from:

15. Kim SH. Health information technology standardization trend and interpolation. HN Focus 2007;14:44-49.

16. Kim IG. Center for Interoperable EHR. Interoperability standards for EHR harmonization and tools. 2010. Seoul, Korea: Elsevier Korea.

17. Park WS, Seo SW, Lee MJ, Kim ON. Health information privacy and security. 2010. Seoul, Korea: Center for Interoperable HER.

18. Cho IS, Kim JA, Go YT, Song SH, Park RW. Clinical decision support system. 2010. Seoul, Korea: Center for Interoperable HER.

Table 1
Inclusion of criteria in the BESTCare system for each of the CCHIT category

Values are presented as number (%). F, B, and P mean differences in functions, local business processes, and government policies, respectively.

CCHIT: Certification Commission for Health Information Technology.

Table 2
Differences in functionality

CCHIT: Certification Commission for Health Information Technology.

Table 3
Difference in business processes among functionality criteria
Table 4
Differences in government policies among functionality criteria

CPT: current procedural terminology.

Table 5
Interoperability criteria and the inclusion of EHR (BESTCare) functions at Seoul National University Bundang Hospital

F and P mean differences in functions and government policies.

ORU: observation result, HITSP: Healthcare Information Technology Standards Panel, CCD: Continuity of Care Document, EHR: electronic health record.

Table 6
Criteria among the security category that is not included in the BESTCare
Share :
Facebook Twitter Linked In Google+ Line it
METRICS Graph View
  • 1 Crossref
  • 1   Scopus
  • 1,261 View
  • 6 Download
Related articles in Healthc Inform Res


Browse all articles >

Editorial Office
1618 Kyungheegung Achim Bldg 3, 34, Sajik-ro 8-gil, Jongno-gu, Seoul 03174, Korea
Tel: +82-2-733-7637, +82-2-734-7637    E-mail:                

Copyright © 2021 by Korean Society of Medical Informatics.

Developed in M2community

Close layer
prev next