Estimating Cognitive Load in a Mobile Personal Health Record Application: A Cognitive Task Analysis Approach

Article information

Healthc Inform Res. 2023;29(4):367-376
Publication date (electronic) : 2023 October 31
doi : https://doi.org/10.4258/hir.2023.29.4.367
1Department of Biostatistics and Medical Informatics, Faculty of Medicine, Akdeniz University, Antalya, Türkiye
2Department of Computer Engineering, Alanya Alaaddin Keykubat University, Antalya, Türkiye
Corresponding Author: Hasibe Yıldız Department of Biostatistics and Medical Informatics, Faculty of Medicine, Akdeniz University, Antalya 07058, Türkiye. Tel: +905511544160, E-mail: yildizhasibe@hotmail.com (https://orcid.org/0000-0003-1971-5164)
Received 2023 June 5; Revised 2023 September 21; Accepted 2023 October 18.

Abstract

Objectives

Mobile health applications that are designed without considering usability criteria can lead to cognitive overload, resulting in the rejection of these apps. To avoid this problem, the user interface of mobile health applications should be evaluated for cognitive load. This evaluation can contribute to the improvement of the user interface and help prevent cognitive overload for the user.

Methods

In this study, we evaluated a mobile personal health records application using the cognitive task analysis method, specifically the goals, operators, methods, and selection rules (GOMS) approach, along with the related updated GOMS model and gesture-level model techniques. The GOMS method allowed us to determine the steps of the tasks and categorize them as physical or cognitive tasks. We then estimated the completion times of these tasks using the updated GOMS model and gesture-level model.

Results

All 10 identified tasks were split into 398 steps consisting of mental and physical operators. The time to complete all the tasks was 5.70 minutes and 5.45 minutes according to the updated GOMS model and gesture-level model, respectively. Mental operators covered 73% of the total fulfillment time of the tasks according to the updated GOMS model and 76% according to the gesture-level model. The inter-rater reliability analysis yielded an average of 0.80, indicating good reliability for the evaluation method.

Conclusions

The majority of the task execution times comprised mental operators, suggesting that the cognitive load on users is high. To enhance the application’s implementation, the number of mental operators should be reduced.

I. Introduction

Personal health records (PHRs) have many benefits, such as helping people manage their health records by enabling access to health information and facilitating communication by building bridges between patients and healthcare providers [1,2]. Initially, PHRs were web-based applications, but in recent years, the use of mobile PHR (mPHR) applications has increased. In 2021, there were 5.29 billion mobile phone users [3] and more than 350,000 mHealth applications were available in the app stores [4]. Despite the numerous potential advantages of PHRs, poor usability remains a significant barrier to patient acceptance and adoption of PHRs [5,6].

Usability is defined by the International Organization for Standardization as “the effectiveness, efficiency, and satisfaction with which specific users achieve specified goals in particular environments” [7]. The amount of resources required to complete a task is related to efficiency. In general, efficiency can have both an explicit, physical component, relating to the speed required to finish a task, and a mental or cognitive component, related to the mental resources needed for a task. Poor system usability caused by inefficient, ineffective, and complex designs increases the mental effort due to the human cognitive architecture, which involves a limited working memory [8] and a theoretically unlimited long-term memory that shapes cognitive schemas. Usability increases when a user’s working memory is suitable for concentrating on the details of the information to be used [9]. Designs that force the user to retain too much information in working memory at once are likely to fail. A well-designed system should not exceed the capacity of the user’s short-term memory [10,11]. A poor design can quickly deplete individuals’ very limited pool of cognitive resources. In such cases, as cognitive load increases, user performance decreases, and the likelihood of making mistakes rises rapidly [12].

Cognitive task analysis (CTA) is a method used to reveal cognitive overload that examines and analyzes how people’s mental processes work while performing a task competently [13]. CTA is considered one of the best methods for effectively analyzing the physical and mental procedures involved in a task [14]. This method focuses on comprehending and quantifying how users solve problems by assigning specific tasks that involve reasoning and then observing how they make decisions [15]. Human factors experts employ CTA to enhance the design of decision support systems, human-computer interfaces, and training programs [16].

The goals, operators, methods, selection rules (GOMS) model [17] is a CTA method that characterizes human-computer interaction in terms of user goals, the actions taken (operators), and methods employed to achieve these goals. The GOMS model aids in assessing the physical and mental effort required to attain a goal by identifying the steps necessary to reach the desired objective when using an interface. Research [18,19] has shown that scrutinizing the fundamental task steps can help eliminate unnecessary processes and streamline task execution for optimal efficiency. This model enables designers to quantitatively estimate the time required or the efficiency of a given user interface for a specific task by breaking down the user’s steps and summing the execution time of each step, all without conducting detailed user tests [20]. Goals frequently comprise sub-goals, representing users’ intentions for using the system. Operators encompass basic actions necessary to interact with a system (e.g., pressing a button, swiping the screen), while methods denote sequences of operators users employ to achieve each goal. When multiple methods exist for achieving a goal, context-based selection rules specify which method to use.

Card et al. [21] developed an estimation tool, known as the keystroke-level model (KLM), based on the GOMS model, to estimate the duration of physical and mental activities on desktop computers using the mouse and keyboard. The fundamental concept behind KLM is to estimate the execution time of a task by listing the sequence of operators that a user employs to complete a task, summing the predetermined durations of these operators, and subsequently calculating the time required for task completion. However, since the KLM was developed for desktop computer interactions, it cannot be applied to evaluate the use of smart devices, which have become increasingly prevalent in our lives due to technological advancements. Recognizing this limitation, researchers have proposed new models [19,2125] based on the KLM method specifically tailored for mobile devices, aimed at assessing interactions with emerging technology.

In this study, we used CTA to assess the cognitive complexity and execution time of the Republic of Türkiye’s national mPHR application e-Nabız.

II. Methods

1. CTA

This study used the GOMS model to analyze the cognitive complexity of the e-Nabız mPHR application. Two proposed GOMS analysis techniques—the updated GOMS model [20] and the gesture-level model [22,23]—were applied to estimate the execution times of the determined prototypical tasks using the device Samsung Note10+ (Android OS, screen size of 6.8 inches/17.27 cm). A mental operator and three physical operators were used, as suggested in these models (Table 1), since these are sufficient to evaluate the e-Nabız application.

Operators and execution times of the models used in this study

Physical operators (e.g., prepare finger/pointing [P], tap [T], and drag [D]) are the movements associated with hand gestures needed to accomplish tasks. The definition of basic operators may differ based on various descriptors such as “prepare finger” in the updated GOMS model and “pointing” in the gesture-level model, although they describe the same movement.

Mental operators include “mentally initialing a task (MI),” “mentally deciding, or choosing (MD),” “mentally retrieving (MR),” “mentally finding (MF),” and “mentally verifying (MV),” as proposed by Li et al. [24], and these operators give detailed information about the mental state of a user. MI refers to being mentally prepared for a task and takes place when a task is initialized. MD occurs when the user has to make a decision in case of two or more options. MR refers to a mental state that a user needs to recall information. The MF operator takes place in case there is an information or an object that needs to be searched on the screen, and MV is the mental operator used when a user confirms that the targeted page or result has been reached on the screen. Unlike physical operators, mental operators are not observable user behavior; their placements are guided by heuristic rules based on psychological assumptions about users, as proposed by Card et al. [21].

2. E-Nabız

The national PHR application of the Republic of Türkiye, which served 10 million users in 2019, experienced a remarkable surge in its user base, reaching 25 million users during the coronavirus disease 2019 pandemic [25]. The system seamlessly integrates real-time data from all healthcare institutions (public, private, or university hospitals), providing a platform where citizens can access their PHRs. Users can access a wealth of information, including laboratory results, radiology images, prescription and medication details. Additionally, with patient consent, healthcare providers or family members can also access this information. The menu structure diagram of the mPHR application is shown in Figure 1.

Figure 1

Menu structure diagram of e-Nabız.

3. Procedures

First, the most common 10 tasks a user can carry out with e-Nabız were determined: T1 “making an appointment,” T2 “canceling an appointment,” T3 “finding a previously taken radiology image/report,” T4 “reviewing physician visit information on a specific date,” T5 “adding reminder information to the prescribed medication from the last doctor visit,” T6 “entering allergy information,” T7 “reading the package insert of a previously prescribed drug,” T8 “recording medication side effects,” T9 “reviewing information about a diagnosed condition,” and T10 “view details from previous reports.” If necessary, each task was divided into subtasks.

Second, each task was broken down into its elementary steps according to the GOMS method and each step was described as a physical or mental operator (Table 1). The locations of mental operators were determined as proposed by Card et al. [21] and detailed based on their usage as suggested by Li et al. [24].

Lastly, two proposed GOMS analysis techniques—the updated GOMS model [20] and the gesture-level model [22]— were applied to estimate the execution times of the 10 prototypical tasks. As seen in Table 2, the times of these operators were recorded separately for both models and added. The execution time of mental operators was fixed, as supposed by Card et al. [21].

Time calculation of subtask 1 (“view hospital visits list)” of task 4 (“view past physician visits”) with the updated GOMS model (UG) and gesture-level model (GL)

This process was conducted independently by two evaluators (HY and NZ) who are experienced in usability evaluation. Cohen’s kappa test (SPSS version 23; IBM Corp., Armonk, NY, USA) was applied to calculate the interrater agreement.

Ethical approval was granted by the Akdeniz University Faculty of Medicine Clinical Research Ethics Committee (02.10.2019/899).

III. Results

Table 3 provides an overview of the estimated completion times for each task in seconds, based on both the updated GOMS model and the gesture-level model. These times exclusively accounted for task completion by expert users and did not factor in errors. Processes unrelated to task fulfillment were disregarded. The identified 10 tasks exhibited varying step counts, ranging from 17 to 121, with an average of 39.8 steps. Each step was categorized as either mental or physical, with an average of 47.71% representing mental operator steps.

Summary results of GOMS analysis and execution times of the 10 prototypical tasks according to the updated GOMS model and the gesture-level model

The task completion times were computed using both the updated GOMS model and the gesture-level model. In the updated GOMS model, these times ranged from 16.40 seconds, the shortest completion time for task 9, to 101.85 seconds, the completion time for task 1. In the gesture-level model, completion times ranged from 15.14 to 97.06 secconds. The mental step completion times constituted 72.61% and 75.91% of the total completion time in the updated GOMS model and gesture-level model, respectively.

The inter-rater reliability values ranged from 0.68 (task 7) to 0.88 (task 1), with an average of 0.80 for the 10 tasks indicating good reliability of the evaluation method.

Table 4, using task 8 as an example, demonstrates how the 10 specific tasks were deconstructed into individual steps and categorized as either physical or mental operators. A further examination refined the mental operators, specifying the required mental actions. The analysis reveals that the MF, MV, and MR operators accounted for 52.72%, 21.2%, and 16.3% of the total mental operators, respectively. Table 5 displays the distribution and quantity of mental operators across tasks. The calculated average execution times in both models are presented in Figure 2.

GOMS analysis for task 8 (“recording medication side effects”)

Number and total of mental tasks per task

Figure 2

Average execution time for mental and physical operators in tasks according to the updated GOMS model and gesture-level model. GOMS: goals, operators, methods, selection rules.

IV. Discussion

This study aimed to estimate the efficiency of e-Nabız, the Republic of Türkiye’s national mPHR application, in terms of cognitive load and task completion times. The calculation of task completion times revealed that, according to the updated GOMS model, all tasks took approximately 5.70 minutes (342.10 seconds), while the gesture-level model estimated it as 5.45 minutes (327.25 seconds). In the updated GOMS model, mental operators constituted around 73% of the total time required to complete tasks, whereas in the gesture-level model, this ratio was approximately 76%. This notably high proportion of mental operators may lead to excessive cognitive overload and, consequently, an increased likelihood of user errors.

The average time for mental operators was calculated at 1.35 seconds according to the updated GOMS model, while that for physical operators was 0.44 seconds. In contrast, the gesture-level model utilized a time of 0.37 seconds for physical operators. It is important to note that these times may be longer for novice users, as the models are based on expert users, and the potential for errors among less experienced users is not factored into these calculations.

Among the 10 prototypical tasks, the tasks of “making an appointment” and “adding reminder information to the prescribed medication from the last doctor visit” had the highest total number of steps and mental steps a user needs to go through, respectively. The need for 121 steps (56.2% of which were mental) to make an appointment can potentially lead to mental load and fatigue for the user. Similarly, the task of adding a medication reminder involved a significant total step count (80) and a substantial percentage of mental operators (55%), increasing the likelihood of user errors that could adversely affect their health.

Saitwal et al. assessed the performance of an Electronic Health Record (EHR) system using CTA, evaluating 14 tasks, and they noted that 37% of the time required for task completion was attributed to mental operators, [13]. In another study evaluating the usability of a dental EHR system, ten users were asked to complete four typical tasks in 30 cases. The study’s findings revealed that mental operators accounted for 35.30% of the total number of operators and 56.89% of the total time [28]. In our study, we found that the average percentage of mental operators was 47.71%, comprising a significant portion of the total execution time (73%). Interestingly, the results from these prior studies, even though they centered on EHR systems, align with our PHR evaluation study, highlighting the substantial presence of mental operators.

GOMS analysis, when used alone or in combination with other usability tests, provides valuable insights for system design and evaluation. It allows the prediction of the sequence of physical and mental operators a user will employ to complete a task, aiding in design decisions regarding whether to add a new sequence or remove inefficient ones [26]. Rasmussen and Kushniruk [18] identified inefficient sequences of user interactions through video analysis and observations. Using GOMS-KLM analysis, they demonstrated an example of a redesigned system that eliminated inefficient interaction sequences, resulting in a remarkable 44.6% reduction in theoretical task completion time. A more recent usability assessed the physical and cognitive efficiency of both the original and redesigned electronic medical record (EMR) interfaces. Their evaluation included various metrics such as the NASA Task Load Index, KLM-GOMS, and eye-tracking analysis. While their findings revealed a significant decrease in response time only for one specific task, they observed substantial reductions in the mental workload across multiple tasks in the redesigned EMR [27].

We have not identified any similar studies that evaluate mobile health applications on smartphones using the GOMS model. Existing assessments in the literature primarily focus on applications used on desktop computers and employ the KLM model. Consequently, it is not possible to directly compare our findings with prior data.

In the context of this study, we calculated task execution times using two distinct models specifically developed for mobile devices, and the results from both models closely aligned with each other. Any minor time variations could likely be attributed to the varying dimensions of the devices employed in the models. As such, mHealth evaluations can be effectively carried out using either model, although the actual calculated times will naturally vary based on the dimensions of the specific smart device used.

In this research, we chose to evaluate the 10 most frequently performed tasks to assess mobile PHRs. However, for increased reliability, evaluating all tasks would be ideal. Given the extensive range of smartphone models available in the market, the completion times for tasks are likely to differ due to variations in smartphone sizes. Furthermore, it is important to note that our evaluation was conducted exclusively on the Android operating system, but similar calculations can be applied to other smart device systems available in the market.

The need to validate the GOMS model for mobile devices is apparent; however, the ever-evolving landscape of phone models poses a challenge. Adapting the GOMS model to accommodate these constantly changing devices is necessary. The rapid turnover of mobile phone models in the market restricts the feasibility of validation efforts. Additionally, it is essential to note that task time calculations are primarily based on expert users, and these times may vary for novice users. Furthermore, potential errors, interruptions, and system response times while using the application are not factored into these calculations.

Moreover, there exists a web version of the mPHR, providing users with an alternative means to access and manage health information. Differences may exist between the mobile application and the web version, and the latter can be assessed using KLM models designed for desktop computer usage.

The findings from this research underscore a significant usability challenge related to cognitive overload in a mobile app with millions of users. These results can inform the development of more user-friendly mobile health applications, thereby enhancing their usability. Reducing the cognitive load during the use of mobile health applications can contribute to the increased adoption and effective utilization of these apps, ultimately benefiting public health.

Acknowledgments

This study was supported by the Akdeniz University Scientific Research Project Administration Division (Project ID. 5416).

Notes

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

References

1. Tang PC, Lansky D.. The missing link: bridging the patient-provider health information gap. Health Aff (Millwood) 2005;24(5):1290–5. https://doi.org/10.1377/hlthaff.24.5.1290.
2. Showell C.. Barriers to the use of personal health records by patients: a structured review. PeerJ 2017;5:e3268. https://doi.org/10.7717/peerj.3268.
3. Kemp S. Digital 2021 October Global Statshot Report [Internet] Singapore: DataReportal; 2022. [cited at 2023 Oct 31]. Available from: https://datareportal.com/reports/digital-2021-october-global-statshot?rq=health.
4. Franklin R. 11 surprising mobile health statistics [Internet] San Antonio (TX): Mobius MD; [cited at 2023 Oct 31]. Available from: https://mobius.md/2021/10/25/11-mobile-health-statistics.
5. Pachunka E, Windle J, Schuetzler R, Fruhling A. Natural-setting PHR usability evaluation using the NASA TLX to measure cognitive load of patients. In : Proceedings of the 52nd Hawaii International Conference on System Sciences; 2016 Jan 8–11; Maui, HI. p. 3954–63.
6. Ehrler F, Weinhold T, Joe J, Lovis C, Blondon K.. A Mobile app (BEDSide Mobility) to support nurses’ tasks at the patient’s bedside: usability study. JMIR Mhealth Uhealth 2018;6(3):e57. https://doi.org/10.2196/mhealth.9079.
7. International Organization for Standardization. Ergonomics of human-system interaction - Part 11: Usability: Definitions and concepts Geneva, Switzerland: International Organization for Standardization; 2018. (ISO 9241-11:2018).
8. Baddeley A.. Working memory: theories, models, and controversies. Annu Rev Psychol 2012;63:1–29. https://doi.org/10.1146/annurev-psych-120710-100422.
9. Cooper G. Research into cognitive load theory and instructional design at UNSW Sydney, Australia: School of Education Studies, University of New South Wales; 1998.
10. Balogh J, Cohen M, Giangola JP. Voice user interface design: minimizing cognitive load [Internet] Hoboken (NJ): InformIT; 2004. [cited at 2023 Oct 31]. Available from: https://www.informit.com/articles/article.aspx?p=170792.
11. Zu T, Hutson J, Loschky LC, Rebello NS. Use of eye-tracking technology to investigate cognitive load theory [Internet] Ithaca (NY): arXiv.org; 2018. [cited at 2023 Oct 31]. Available from: https://arxiv.org/abs/1803.02499.
12. Tracy JP, Albers MJ. Measuring cognitive load to test the usability of web sites. In : Proceedings of the 53rd Annual Conference of the Society for Technical Communication; 2006 May 7–10; Las Vegas, NV. p. 256–60.
13. Militello LG, Hutton RJ.. Applied cognitive task analysis (ACTA): a practitioner’s toolkit for understanding cognitive task demands. Ergonomics 1998;41(11):1618–41. https://doi.org/10.1080/001401398186108.
14. Saitwal H, Feng X, Walji M, Patel V, Zhang J.. Assessing performance of an Electronic Health Record (EHR) using Cognitive Task Analysis. Int J Med Inform 2010;79(7):501–6. https://doi.org/10.1016/j.ijmedinf.2010.04.001.
15. Kushniruk AW, Patel VL. Cognitive approaches to the evaluation of healthcare information systems. In : Anderson JG, Aydin CE, eds. Evaluating the organizational impact of healthcare information systems New York (NY): Springer; 2005. p. 144–73. https://doi.org/10.1007/0-387-30329-4_6.
16. Klein G, Militello L. Some guidelines for conducting a cognitive task analysis. Advances in human performance and cognitive engineering research Bingley, UK: Emerald Group Publishing Limited; 2001. p. 163–99. https://doi.org/10.1016/S1479-3601(01)01006-2.
17. Card SK, Moran TP, Newell A. The psychology of human-computer interaction Boca Raton (FL): CRC Press; 2018.
18. Rasmussen R, Kushniruk A.. The long and twisting path: an efficiency evaluation of an electronic whiteboard system. Stud Health Technol Inform 2013;183:174–8.
19. Gross M, Roberts C, Stinson K, Wiltman S.. Improving medical device usability by reducing complexity using a novel predictive models-based user interface assessment tool. Hum Factors Healthc 2023;3:100041. https://doi.org/10.1016/j.hfh.2023.100041.
20. Setthawong P, Setthawong R.. 2019;Updated Goals Operators Methods and Selection Rules (GOMS) with touch screen operations for quantitative analysis of user interfaces. Int J Adv Sci Eng Inf Technol 2019;9(1):258–65. https://doi.org/10.18517/IJASEIT.9.1.7865.
21. Card SK, Moran TP, Newell A.. The keystroke-level model for user performance time with interactive systems. Commun ACM 1980;23(7):396–410. https://doi.org/10.1145/358886.358895.
22. Nystrom A. Gesture-level model: a modified Keystroke-level model for tasks on mobile touchscreen devices [master’s thesis] Uppsala, Sweden: Uppsala University; 2018.
23. Kim D, Myung R.. A unit touch gesture model of performance time prediction for mobile devices. J Ergon Soc Korea 2016;35(4):277–91. http://doi.org/10.5143/JESK.2016.35.4.277.
24. Li H, Liu Y, Liu J, Wang X, Li Y, Rau PL. Extended KLM for mobile phone interaction: a user study result. In : Proceedings of the Extended Abstracts on Human Factors in Computing Systems; 2010 Apr 10–15; Atlanta, GA. p. 3517–22. https://doi.org/10.1145/1753846.1754011.
25. Republic of Turkiye, Ministry of Health. General directorate of health information systems [Internet] Ankara: Republic of Turkiye; 2021. [cited at 2023 Oct 31]. Available from: https://sbsgm.saglik.gov.tr/TR,73567/her-4-kisiden-biri-e-nabiz-kullaniyor.html.
26. John BE, Kieras DE. Using GOMS for user interface design and evaluation: which technique? ACM Trans Comput Hum Interact 1996;3(4):287–319. https://doi.org/10.1145/235833.236050.
27. Al Ghalayini M, Antoun J, Moacdieh NM.. Too much or too little? Investigating the usability of high and low data displays of the same electronic medical record. Health Informatics J 2020;26(1):88–103. https://doi.org/10.1177/1460458218813725.
28. Suebnukarn S, Rittipakorn P, Thongyoi B, Boonpitak K, Wongsapai M, Pakdeesan P.. Usability assessment of an electronic health record in a comprehensive dental clinic. SpringerPlus 2013;2:220. https://doi.org/10.1186/2193-1801-2-220.

Article information Continued

Figure 1

Menu structure diagram of e-Nabız.

Figure 2

Average execution time for mental and physical operators in tasks according to the updated GOMS model and gesture-level model. GOMS: goals, operators, methods, selection rules.

Table 1

Operators and execution times of the models used in this study

Updated GOMS
5″ (12.7 cm) device, experienced user
Gesture-level model
iPhone 7: 4.7″ (11.9 cm), expert users


Operator Time (s) Operator Time (s)
Prepare finger (E): Move finger to touch a specific part of the touchscreen 0.5 Pointing (P): Move finger from one part of the screen to another 0.34

Tap (T): Touch screen with finger 0.2 Tap (T): Tap an element on the screen 0.25

Drag (D): Move finger over surface 0.5 Drag (D): Move finger across the screen without losing contact with the screen 0.44

Mental (M) 1.35 Mental (M) 1.35

GOMS: goals, operators, methods, selection rules.

Table 2

Time calculation of subtask 1 (“view hospital visits list)” of task 4 (“view past physician visits”) with the updated GOMS model (UG) and gesture-level model (GL)

Step Step description Cognitive distribution Operator Time (s)
UG GL
1 Preparing for the task Mental MI 1.35 1.35
2 Thinking about where to find doctor visits Mental MF 1.35 1.35
3 Pointing finger to the screen Physical P 0.50 0.34
4 Scroll down the screen two times Physical 2*D 1.00 0.88
5 Recognizing the “All My Hospital Visits” button Mental MF 1.35 1.35
6 Pointing the finger on the “All My Hospital Visits” button Physical P 0.50 0.34
7 Tap the “All My Hospital Visits” button Physical T 0.20 0.25
8 Confirming found physician visits Mental MV 1.35 1.35
Goal accomplished (sum) 7.60 7.21

GOMS: goals, operators, methods, selection rules, MI: mentally initialing a task, MF: mentally finding, MV: mentally verifying, P: pointing, D: drag, T: tap.

Table 3

Summary results of GOMS analysis and execution times of the 10 prototypical tasks according to the updated GOMS model and the gesture-level model

Task Total steps Operator % Mental Execution time (s)

Updated GOMS Gesture-level model



Mental Physical Mental Physical Total Mental Physical Total
1 Making an appointment 121 53 68 43.80 71.55 30.30 101.85 71.55 25.51 97.06

2 Canceling an appointment 23 13 10 56.52 17.55 3.30 20.85 17.55 2.95 20.5

3 Finding a previously taken radiology image/report 18 9 9 50.00 12.15 4.60 16.75 12.15 3.87 16.02

4 Reviewing physician visit information on a specific date 32 16 16 50.00 21.6 7.70 29.3 21.6 6.42 28.02

5 Adding reminder information to the prescribed medication from the last doctor visit 80 36 44 45.00 48.6 16.90 65.5 48.6 14.28 62.88

6 Entering allergy information 30 13 17 43.33 17.55 6.20 23.75 17.55 4.66 22.21

7 Reading the package insert of a previously prescribed drug 25 11 14 44.00 14.85 5.30 20.15 14.85 4.26 19.11

8 Recording medication side effects 34 16 18 47.06 21.6 10.20 31.8 21.6 9.26 30.86

9 Reviewing information about a diagnosed condition 18 8 10 44.44 10.8 5.60 16.4 10.8 4.65 15.45

10 View details from previous reports 17 9 8 52.94 12.15 3.60 15.75 12.15 2.99 15.14

Sum of steps 398 184 214
Mean of mental operator steps (%) 47.71
Total fullfilment duration (s) 248.40 93.70 342.10 248.40 78.85 327.25
Total fullfilment duration (%) 72.61 27.39 100 75.91 24.09 100

GOMS: goals, operators, methods, selection rules.

Table 4

GOMS analysis for task 8 (“recording medication side effects”)

Subtask Step Step description Operator
Subtask-1: View prescription list 1 Preparing for the task MI
2 Searching “My Recipes” button MF
3 Pointing finger to the screen Physical
4 Scroll the screen two times Physical
5 Finding the “My Recipes” button MF
6 Putting the finger on the “My Prescriptions” button Physical
7 Tap the “My Recipes” button Physical

Subtask-2: View the desired recipe 8 Remembering which physician prescribed the medicine on which date MR
9 Search in the list MF
10 Deciding if the physician visit is on the List MD
11 Pointing finger to the screen Physical
12 Scrolling the screen until finding the physician visit Physical
13 Finding the physician and the date the medicine was written MF
14 Putting the finger on the date the medicine was written Physical
15 Touching the date, the medicine was written Physical

Subtask-3: View the desired drug and its information 16 Remembering the name of the drug MR
17 Finding the name of the drug from the list MF
18 Pointing to the name of the drug Physical
19 Tapping the name of the drug Physical
20 Confirming that the correct drug has been selected MV

Subtask-4: Entering side effects 21 Noticing the “Add” button in the “My Side Effects” field on the incoming page MF
22 Move finger to “Add” button Physical
23 Tap the “Add” button Physical
24 Noticing the “Symptom” field on the incoming screen MF
25 Pointing finger to “Symptom” field Physical
26 Tap the “Symptom” field Physical
27 Pointing finger to virtual keyboard Physical
28 Remembering symptoms MR
29 Entering the symptoms Physical
30 Confirming correct entry MV
31 Detecting “OK” button MF
32 Moving finger to “OK” button Physical
33 Tap the “OK” button Physical
34 Confirming entered information MV

GOMS: goals, operators, methods, selection rules, MI: mentally initialing a task, MF: mentally finding, MR: mentally retrieving, MD: mentally deciding, or choosing, MV: mentally verifying.

Table 5

Number and total of mental tasks per task

MI MR MF MD MV Sum
Task1 1 8 30 4 10 53
Task2 1 1 7 1 3 13
Task3 1 1 5 0 2 9
Task4 1 4 6 0 5 16
Task5 1 7 18 1 9 36
Task6 1 2 7 0 3 13
Task7 1 2 6 1 1 11
Task8 1 3 8 1 3 16
Task9 1 1 5 0 1 8
Task10 1 1 5 0 2 9
Sum 10 30 97 8 39 184
% 5.43 16.30 52.72 4.35 21.20 100

MI: mentally initialing a task, MF: mentally finding, MR: mentally retrieving, MD: mentally deciding or choosing, MV: mentally verifying.