The meetings of experts, such as task forces or standing committees for special issues often make important decisions, but decision-making systems can be costly and timeconsuming [123], and such systems cannot respond quickly. The rapid advancement of information technology (IT) services can help to overcome this difficulty.
In this paper, we propose a new system, called the Rapid Opinion Collecting System (ROCS), that utilizes public IT services from short message services (SMS) and e-mails to cloud services such as Google Forms [45]. The objective of this study was to demonstrate the efficiency of the ROCS by analyzing response rates and times.
II. Methods
This study was approved by the Institutional Review Board (IRB) of Asan Medical Center (No. 2014-0690). Informed consent requirement was waived by the IRB. The study was conducted from November 19, 2012 to December 30, 2012 (6 weeks). The study participants were 10 board-certified emergency physicians who worked in four different regional emergency centers.
Another board-certified emergency physician who was a professor of a general hospital was assigned as the moderator. All 20 cases were designed by analyzing emergency room medical records. Each case was anonymized by the removal of personal identifiers, such as patient identification number and name. Each case was given a random number. The moderator activated each case by random number. Sixteen cases were activated on weekdays and four cases on weekend days. Among the 16 weekday cases, four were activated in the evening (from 6 PM to 11 PM). When activating a case, the moderator sent an SMS via mobile phone and an email using Google Forms [45]. The overall procedure of the ROCS is explained in Figure 1. The receivers could answer the survey by simply replying to the e-mail or logging in to the Google Forms website. Google Forms summarized the survey results and response times automatically. Therefore, the time required for the analysis was shortened. The activation time was set by the time the e-mail was sent. A timely response was defined as one that was received within 2 hours from activation. During the study, the 10 participating physicians had no information on the study period, case activation time, the total number of activations, or the criteria for a timely response. IBM SPSS ver. 20 (IBM Corporation, Armonk, NY, USA) was used for all statistical analysis. A chi-square test, Fisher exact test, and a Mann-Whitney U-test were used. We considered p<0.05 as statistically significant.
III. Results and Discussion
The ROCS can successfully collect professional opinions on a public health issue. Table 1 shows the details of the ROCS responses received in our study. The overall response rate was 97.5%, and the timely response rate was 87.0%. The response rates were high for all cases whether it was a weekend day or evening (Table 2). Only the timely responses in the evening were relatively low (75.0%, p=0.02), as shown in Table 2. Although a method for promoting timely responses in the evening is necessary, the feasibility of the ROCS for the collection of expert opinions in a relatively short time was demonstrated.
There have been few studies using online survey tools, such as Google Forms [678910]. As far as we know, this study is the first attempt to apply an online tool to the rapid collection of expert opinions on a public health issue. By using well-known IT services, such as SMS, e-mail, and Google Forms, the ROCS can collect experts' opinions in a relatively short time compared with traditional off-line meetings. Also, this system operates with almost zero cost and can give a prompt response on a public health issue. The proposed ROCS may be applied to various situations, such as bioterrorism attacks, disasters, and epidemic outbreak, which require an emergency response [1112131415].
This study had several limitations. We did not conduct a test from 11 PM to 8 AM. The responses during those times might have been lower than the results in this study. SMS and e-mails are one-way push services. Because there is no confirmation of receipt, transmission failure could have occurred. Also, there is a probability of quality dropping. Timely response was defined by authors on an arbitrary basis. All study participants were emergency physicians. Therefore, the results of the study might not be generalizable to professionals with standard working hours or days.
In conclusions, the potential of the ROCS was demonstrated; it showed a high response rate and a relatively short response time. The proposed system may help to overcome the limitations of the traditional off-line expert opinion aggregation model.
Acknowledgments
This work was supported by grants from the Ministry of Health and Welfare–Korea Centers for Disease Control & Prevention (Grant No. 2831-304-320-01).
Conflict of Interest: No potential conflict of interest relevant to this article was reported.
World Health OrganizationGeneva, SwitzerlandWorld Health Organization2002SethiDCopenhagen, DenmarkWorld Health Organization2008PedenMMGeneva, SwitzerlandWorld Health Organization2008RehaniMMBerrisTInternational Atomic Energy Agency study with referring physicians on patient radiation exposure and its tracking: a prospective survey using a web-based questionnaire201225e001425Google[place unknown]Google2015cited at 2017 Apr 1Available from: https://www.google.com/forms/about/de la Fuente ValentinLPardoADelgado KloosCUsing third party services to adapt learning material: a case study with Google FormsCressUDimitrovaVSpechtMHeidelberg, GermanySpringer2009744750RehaniMMBerrisTRadiation exposure tracking: survey of unique patient identification number in 40 countries2013200477677923521447GaoFA case study of using a social annotation tool to support collaboratively learning2013177683ReynaJProceedings of World Conference on Educational Media and Technology2010 Jun 29Toronto, Canada15661572PeacockJGGrandeJPAn online app platform enhances collaborative medical student group learning and classroom management201638217418025782601IrvinCBNouhanPPRiceKSyndromic analysis of computerized emergency department patients' chief complaints: an opportunity for bioterrorism and influenza surveillance200341444745212658241BushLMAbramsBHBeallAJohnsonCCIndex case of fatal inhalational anthrax due to bioterrorism in the United States2001345221607161011704685ChretienJPTomichNEGaydosJCKelleyPWReal-time public health surveillance for emergency preparedness20099981360136319542047KulldorffMHeffernanRHartmanJAssuncaoRMostashariFA space-time permutation scan statistic for disease outbreak detection200523e5915719066HopeKDurrheimDNd'EspaignetETDaltonCSyndromic Surveillance: is it a useful tool for local outbreak detection?200660537437516680907
Schematic diagram of Rapid Opinion Collecting System.
Analysis of the responses and timely responses for the Rapid Opinion Collecting System
A timely response was defined as a response within 2 hours of activation.
IQR: interquartile range.
Comparison of activation days and times
A timely response was defined as a response within 2 hours of activation.