Samsung Medical Center (SMC) is a Korean tertiary hospital with a capacity of 2000 beds. The Korean government has guided all hospitals to implement the concurrent DUR (cDUR) system by the end of 2011. To follow this government guideline, the SMC organized a task force (TF) in July 2011. The TF included doctors, pharmacists, nurses, administrators, and IT specialists. Its mission was to design, develop, and implement the cDUR system before 2012. The cDUR system has been tightly integrated with the SMC's computerized physician order entry (CPOE) system in the outpatient clinical setting. Through gathering opinions from several TF meetings, the cDUR system was developed in December 2011 and adopted for outpatients since January 1, 2012 without any trouble. According to the government guidelines, the checking of DUR conflict events is mandatory in hospitals in order for its benefits to be realized [12].

In Korea, the concept of the DUR was introduced in the 1980s, but it had not been seriously discussed until early 2000. At the end of 2011, the Korean government issued a guideline for the cDUR system to be adopted in all hospitals. If the guideline is not followed, the National Health Insurance Review Agency (NHIRA) may retrospectively limit reimbursement of the hospital's claim. This system automatically checks whether a prescription meets the criteria developed by NHIRA at the time of prescribing. DUR conflict events are based on drug-drug interactions, such as duplicate medication, contraindicated drug interaction, etc., for outpatients. Implementation of the DUR system by Korea hospitals is the first case in the world in which all healthcare providers have used the system nationwide [13].

The first step of the cDUR system is that the doctor enters drug information into the CPOE system (①), as shown in Figure 1. Next, the entered data are exchanged with the national DUR server to check for conflict events (②). The checked results are returned to the doctor's computer display (③). When there are no conflict events, the prescription is stored into the national DUR database, and the prescription process is finished (④). In the case of DUR conflict events, the doctor has to enter the reason why the outpatient needs the drugs (⑤). When the doctor does not have any reason to proceed with the prescription, the prescription has to be canceled or changed (⑥).

Real prescription data during one week from January 2 to 8, 2012 was analyzed immediately after adoption of the cDUR system. During the period, the number of outpatients was 40,908. Among them, 21,701 (53.0%) were prescribed drugs by doctors. The number of DUR conflict events was 1,752, which comprised 4.3% of outpatients. Duplicate medication was found in 99.3% of DUR conflict events, and contraindicated drug interaction accounted for 0.7% [11]. The time required for entering a reason for the DUR conflict was about 31.6 seconds in the outpatient clinical setting.

According to a literature review, there have been several studies on the D&M IS Success Model of the HIS [14,15,16,17,18]. Studies regarding the success of the DUR system were rare, and there was also a lack of analysis of the impact of doctors' resistance. As seen in the system architecture (Figure 1), the DUR system is very complex, and this leads to an increase in the time required for the prescription process in the clinical setting. This complicated work should be observed to be helpful to improve patient safety. However, doctors may consider it as a laborious task in their already time-pressing outpatient clinical setting, although they recognize its usefulness. Thus, the process may invoke doctors' resistance even if they use it under compulsion. Thus, this study focused on the success of the DUR system based on the augmented D&M IS Success Model, which used doctors' resistance as a socio-technological measure. This framework is the same as that of the D&M IS Success Model in that it is based on qualities, such as system, information, and services [19]. The major difference is that this study excluded the variable 'use' because it was not statistically significant for mandatory systems [20]. Another very specialized construct is newly added doctors' resistance in this study to examine whether this has an impact on user satisfaction or user usefulness in a statistically significant fashion or not. The net benefit was specified as 'user usefulness' to reflect doctors' viewpoint, since doctors are the main users of the DUR system. Pertaining to net benefit, benefits for patients or organizations may also be considered. Any improvement of patient service, satisfaction of patients, cost issues of organizations, etc., can be examples of net benefit. However, this study selected 'user usefulness' in that it is meaningful for the system to be evaluated in stakeholders' interests, as recommended by Seddon et al. [21]. Hence, this study tried to measure whether the DUR system could be useful in improving patient safety or quality for doctors to care patients or not. Review of previous studies led to the conclusion that DUR systems could be considered in relation to the following possible benefits: 'useful for making prescription', 'improving decision-making', and 'giving doctors positive impact'. Considering these aspects, the research model of this study is presented in Figure 2.

Hypothesis 1 (H1): The overall quality of the DUR system positively impacts user satisfaction but user satisfaction is negatively impacted by doctors' resistance.

Hypothesis 2 (H2): User satisfaction positively impacts user usefulness but user usefulness is negatively impacted by doctors' resistance.

Hypothesis 3 (H3): Both user satisfaction and user usefulness negatively impact doctors' resistance.

The questionnaire used in this study was composed of six variables based on the hypotheses (Appendix 1). In the system quality variable, the measured items were ease of use, response time, and system reliability. In the information quality variable, the measured items were understandability, accuracy, and timeliness. In the service quality variable, the measured items were sincerity, right time, and high service quality. The measured items in the user satisfaction variable were overall satisfaction, user friendliness, and positive attitude. In the case of doctors' resistance, the measured items included workload increase, time consumption and no tangible benefits. Usefulness, decision-making, and positive benefits were included in the variable measuring user usefulness.

The measures for system quality, information quality, user satisfaction, and user usefulness were mainly based on studies by van der Meijden et al. [22] in 2003, Jen and Chao [14] in 2008, and Petter and Fruhling [16] in 2011. The measures for doctors' resistance were based on challenges or barriers described by Lee et al. [8] in 1996, Sittig et al. [5] in 1999, Murff and Kannry [23] in 2001, Carroll et al. [9] in 2002, Bates and Gawande [7] in 2003, Berger and Kichak [6] in 2004, Aarts et al. [10] in 2007, and Ventura et al. [24] in 2011. A 5-point Likert-type scale was used for all items with 1 being 'strongly disagree', 3 being 'neutral', and 5 being 'strongly agree'.

A survey of doctors who have used computers to enter prescriptions was conducted over a period of 3 weeks from February 22 to March 13, 2012. The questionnaire was based on the self-reported usage pattern. An informal informed consent was included at the head of the questionnaire. IBM SPSS Statistics ver. 20 (IBM, Armonk, NY, USA) was used for data analysis. Although the variables and measures were already validated in previous researches, validity was checked through factor analysis as a confirmatory method because some of them were newly combined in this study. To check the validity between variables and measures, factor analysis with IBM SPSS Statistics ver. 20 was performed. The analysis of results was based on convergent validity, discriminant validity, and nomological validity [25]. The extraction method of factor analysis was principle component analysis, and the rotation method was Equamax with Kaiser normalization. After analysis, measures with low construct validity, low convergent validity, or low discriminant validity were excluded. With Cronbach's α, the reliability of the measures was assessed. Multiple linear regression analysis was used to examine the hypotheses. In addition, in-depth interviews with doctors, who have rich experience in leading IT projects at the hospital, were conducted to discuss the results of data analysis.

The Institutional Review Board (IRB) review has been obligatory for all research related to human subjects since February 1, 2013 in Korea. According to the IRB office, this study is eligible for a waiver of IRB review because this study did not store any personal information which could be used to identify respondents.

Of 650 doctors who entered prescriptions in January 2012, 54 answered the questionnaires. Of the returned 54 questionnaires, 20 were invalidated because 7 were returned incomplete, and 13 were insincere. This yielded a total of 34 usable questionnaires for data analysis with a validation rate of 63%. The response rate was 5.3%, that is, 34 valid surveys out of 650 doctors. Respondents of valid questionnaires were 68% male; 71% were between the ages of 30 and 50; 50% were professors, 15% were fellows, and 35% were residents; 47% had working experience of more than 10 years, 12% had 5 to 10 years, and 41% had less than 5 years. The distribution of specialties was 32% in internal medicine, 35% in surgery, 9% in pediatrics, 6% in neurology, 6% in family medicine, and 12% in other areas. Even though the response rate was very low, this distribution was very similar to the distribution of doctors in the hospital. Therefore, this survey was able to cover a representative sample of the hospital.

According to the results of checking the validity between variables and measures, as shown in Table 1, response time and reliability are relevant to system quality; understandability, accuracy, and timeliness are relevant to information quality; sincerity, right time, and high service quality are relevant to service quality; overall satisfaction and user friendliness are relevant to user satisfaction; increase in workload and time consumption are relevant to doctors' resistance; usefulness, decision-making, and positive benefits are relevant to user usefulness.

However, ease of use was relevant to user satisfaction rather than system quality in this study, although it was the most frequently addressed measure of system quality. Thus, it was excluded from further analysis. Positive attitude was not relevant to user satisfaction, even though it was recommended as a measure of user satisfaction [22]. It was also excluded from this analysis. From the perspective of nomological validity, no tangible benefits were also excluded since it is strongly related to doctors' resistance rather than to user satisfaction according to the literature review [6].

According to a result of the reliability assessment with 6 variables and 15 measures except ease of use, positive attitude and no tangible benefits, Cronbach's α for the 5 variables indicated good reliability with reliability values higher than 0.7 representing the minimal standard [25]. System quality had a slightly weak reliability of 0.640. The mean, standard deviation, and Cronbach's α for the variables are listed in Table 2. The means of all variables were above 3.0. Pearson correlation coefficients indicated low and high existent correlations among the variables examined for the DUR system. Among the variables, doctors' resistance showed a strong correlation with user satisfaction, whereas it was less correlated with user usefulness.

Typically, studies on IS success use structural equation modeling (SEM) for large sample sizes. However, due to the small sample size of the present study, multiple linear regression, rather than SEM, was adopted to examine the relationships among the hypotheses in this research. The results of the regression analysis are presented in Table 3.

System quality, information quality, service quality, and doctors' resistance (H1) did not behave as hypothesized. For user satisfaction, the regression model was significant (R² = 0.529, p = 0.000). Of the 4 variables, only service quality was significant with a positive beta (β = 0.611, p = 0.001), whereas doctors' resistance was negatively significant (β = -0.279, p = 0.035). Thus, hypothesis H1 was partially supported, as shown in Table 3.

Similar to hypothesis H1, hypothesis H2 was partially supported, as shown in Table 3. With regard to user usefulness, the regression model was significant (R² = 0.377, p = 0.001). User satisfaction had a significant positive association (β = 0.429, p = 0.002), while doctors' resistance had no significant association.

Hypothesis H3 was not supported, as shown in Table 3. For doctors' resistance, the regression model was not strongly significant (R² = 0.222, p = 0.020) and neither user satisfaction nor user usefulness was associated with doctors' resistance, while user satisfaction was suggestively significant (β = -0.370, p = 0.097).