Modification of Case-Based Reasoning Similarity Formula to Enhance the Performance of Smart System in Handling the Complaints of in vitro Fertilization Program Patients

Article information

Healthc Inform Res. 2022;28(3):267-275

Publication date (electronic) : 2022 July 31

doi : https://doi.org/10.4258/hir.2022.28.3.267

Paminto Agung Christianto ¹^,²

, Eko Sediyono ¹

, Irwan Sembiring ¹

¹Department of Computer Science, Satya Wacana Christian University, Salatiga, Indonesia

²College of Informatics and Computer Management (STMIK), Widya Pratama, Pekalongan, Indonesia

Corresponding Author: Paminto Agung Christianto, Department of Computer Science, Widya Pratama College of Informatics and Computer Management (STMIK), Patriot Street No. 25 Pekalongan City, Central Java Province, Indonesia. Tel: +628995929394, E-mail: p_a_chr@stmik-wp.ac.id (https://orcid.org/0000-0002-4518-7433)

Received 2021 August 3; Revised 2021 October 7; Revised 2022 March 10; Accepted 2022 April 24.

Abstract

Objectives

Eighty percent of in vitro fertilization (IVF) patients have high anxiety levels, which influence the success of IVF and drive IVF patients to quickly report any abnormal symptoms. Rapid responses from fertility subspecialist doctors may reduce patients’ anxiety levels, but fertility subspecialist doctors’ high workload and their patients’ worsening health conditions make them unable to handle IVF patients’ complaints quickly. Research suggests that smart systems using case-based reasoning (CBR) can help doctors handle patients quickly. However, a prior study reported enhanced accuracy by modifying the CBR similarity formula based on Lin’s similarity theory to generate the Chris case-based reasoning (CCBR) similarity formula.

Methods

The data were validated through interviews with two fertility subspecialist doctors, interviews with two IVF patients, a questionnaire administered to 17 community members, the relevant literature, and 256 records with data on IVF patients’ complaints and how they were handled. An experiment compared the performance of the CBR similarity formula algorithm with the CCBR similarity formula algorithm.

Results

A confusion matrix showed that the CCBR similarity formula had an accuracy value of 52.58% and a precision value of 100%. Fertility subspecialist doctors stated that 89.69% of the CCBR similarity formula recommendations were accurate.

Conclusions

We recommend applying a combination of the CCBR similarity formula and a minimum reference value of 80% with a CBR smart system for handling IVF patients’ complaints. This recommendation for an accurate system produced by the CBR similarity formula may help fertility subspecialist doctors handle IVF patients’ complaints.

Keywords: Anxiety; Complaint; Doctors; Fertility; Patients

I. Introduction

Infertility is a problem throughout the world that is experienced by 48.5 million couples. In vitro fertilization (IVF) is a technology used to solve infertility, but the success rate of IVF programs is still low in many countries, such as Thailand (about 30.7%–38.6%), the United States (28%–29%), European countries (17%–25%), the United Kingdom (24%–25%), Australia (19%–20%), and Japan (9%–15%) [1].

The success rate of IVF in Indonesia is also still low (about 29%), and approximately 17% of IVF patients have miscarriages [1]. Despite the low success rate of IVF programs, the number of couples interested in undergoing IVF keeps increasing every year [2,3]. Sixty-eight percent of IVF patients must travel relatively long distances to access IVF services [4].

Eighty percent of patients undergoing IVF have high anxiety levels [2,5,6]. These high anxiety levels prompt 82% of IVF patients to complain about abnormal symptoms in their body to fertility subspecialist doctors through short message services or WhatsApp [2,3]. IVF patients often make these requests regardless of fertility subspecialist doctors’ difficult conditions [7] and demand the best services [2,3]. However, fertility subspecialist doctors: face a heavy workload [8], and their patients may experience worsening health conditions [2,3]. These constraints make fertility subspecialist doctors unable to quickly handle IVF patients’ complaints [2,3], although waiting a long time for responses from a fertility subspecialist doctor increases IVF patients’ anxiety [2,3]. In contrast, a quick response to a complaint may reduce IVF patients’ anxiety levels [9,10]. Since high anxiety levels influence the success of IVF, measures are needed to lower IVF patients’ anxiety [11].

The coronavirus disease 2019 pandemic has required health services to take steps to prioritize the health of IVF patients and fertility subspecialist doctors, making online health systems a necessity [12]. Online systems can be used by anyone, from any location [13]. Previous research has shown that online health smart systems can help doctors with their tasks by quickly handling patients’ complaints [14], facilitating monitoring pregnant mothers’ health [15], and predicting IVF cycles’ success rates [16]. Case-based reasoning (CBR) is widely applied to smart health systems [16].

The accuracy of CBR can be improved through modifications, such as chi-square CBR [17] and combination of CBR with a genetic algorithm [18].

No minimum value for CBR [19] systems has been specified to determine the level of information accuracy. Nonetheless, a general recommendation in the literature is that a similarity value ≥80% is considered relatively accurate, while a similarity value <80% is considered to indicate relative inaccuracy.

The aim of this study was to examine IVF patients’ and fertility subspecialist doctors’ responses to a smart system handling the complaints of IVF patients and to examine the performance of a modified CBR similarity formula and a minimum reference value of 80% in the smart system for handling IVF patients’ complaints.

II. Research Stages

The research was conducted in the following stages:

The data were validated through triangulation [20].
A questionnaire was distributed to the community “Pejuang Tangguh Permata Hati (Community of IVF patients who are struggling to get pregnant through the IVF program at Permata Hati Clinic, Yogyakarta)” to which IVF patients belonged.
A survey was directly conducted in some cities in Central Java Province and Yogyakarta Province in 2018–2019, which found that only four hospitals provided IVF services [4].
Based on the “Regulation of the Minister of Health of the Republic of Indonesia No. 69 Year 2008,” the research data included information on IVF patients’ complaints and how they were handled, without data on IVF patients’ identity. The total research data comprised 256 records.
Based on the “Regulation of the Minister of Health of the Republic of Indonesia No. 30 Year 2019” and the “Regulation of the Indonesian Medical Council No. 87 Year 2020,” we certified that the validator of the results of this research is a member of the Indonesian IVF Association with at least 5 years of experience in handling IVF.
A direct survey in some cities in Central Java Province and Yogyakarta Province in 2018–2019 found that there were only five fertility subspecialist doctors.
The CBR similarity formula was modified by adopting Lin’s similarity theory [21,22], referred to as the Chris case-based reasoning (CCBR) similarity formula.
An experiment used a computer program to compare the performance of the CBR and CCBR similarity formula algorithms.
A confusion matrix was used to measure the smart system’s performance in handling 97 complaints of IVF patients. (10) This research’s scope was the handling of IVF patients’ complaints in Central Java Province and Yogyakarta Province.

The researchers received two ethical clearances from the Faculty of Health, Pekalongan University (No.33/B.02.01/KEPK/I/2022) and the Faculty of Medicine, Duta Wacana Christian University (No.1384/C.l6/FK/2022).

Figure 1 presents the research stages.

Figure 1

Research stages. IVF: in vitro fertilization, CBR: case-based reasoning, CCBR: Chris case-based reasoning.

III. Results

1. Timing of Research Data Collection

The interviews with two fertility subspecialist doctors were conducted at different times (September 2019 to February 2021). The interviews with two IVF patients were conducted in August 2019. The questionnaires were distributed in October 2019 to the members of “Komunitas Pejuang Tangguh Permata Hati,” and 17 members responded. The community coordinator explained that only a few members responded to the questionnaire because not all of the community members were willing to share information.

2. Research Data Processing

Based on the result of crosschecking the fertility subspecialist doctors’ responses with those of the IVF patients, it was found that both groups supported the smart system for handling the complaints of IVF patients. Fertility subspecialist doctors agreed on the requirement that the system’s recommendations must be accurate. IVF patients supported this system since they understood that fertility subspecialist doctors were busy, but still wanted their complaints to be quickly handled. More specifically, 53% of the community members strongly agreed and 35% agreed with the use of the smart system for handling the complaints of IVF patients.

Table 1 presents the results of cross-checking the data sources, with results from the interviews with fertility subspecialist doctors, complaint data, data on the handling of IVF patients’ complaints, and the relevant literature [18,21,23].Objective weights for scoring were obtained from the assessment by experienced fertility subspecialist doctors [18]. Table 2 presents the results of consensus between the two fertility subspecialist doctors for the weight scores used in the experiment. The interval of the weight scores was 0–1 [23].

Table 1

Identification of types of IVF patients’ complaints

Table 2

Complaint weighting

3. Modification of the CBR Similarity Formula

CBR is based on records of similar past situations, which are saved on a case-by-case basis and used to solve current problems [19]. If an old case was identical to a new case, its value was 1; otherwise, its value was 0 [24]. The CBR similarity level was measured using the following similarity formula [2]:

CBR=∑i=1nABi*Ki∑i=1nKi,

where A is a new case, B is an old case, and K is B’s weight value.

Lin’s similarity theory [21,22] states “Examining the similarity and difference between A and B must take different perspective and calculate the level of similarity in each perspective. The optimal similarity value is the average value of similarity value of each perspective.”

There are two perspectives for calculating the level of similarity between A and B. A’s perspective on B is calculated using the following formula:

A on B=∑i=1nABi*Ki∑i=1nKi.

B’s perspective on A is calculated using the following formula:

B on A=∑i=1nBAi*K′i∑i=1nK′i.

An optimal similarity value means the average value of similarity value of A’s perspective on B and B’s perspective on A. Thus, a modification of the CBR similarity formula or CCBR similarity formula is obtained:

CCBR=(∑i=1nABi.Ki∑i=1nKi)+(∑i=1nBAi.K′i∑i=1nK′i)2,

where A is a new case, B is an old case, K is B’s weight value, and K' is A’s weight value.

4. CBR Algorithm and CCBR Algorithm

Figure 2 shows the CBR and CCBR similarity formula algorithms, which are applied to the retrieval process, as shown in Figure 3. The experiment tested two algorithms with a minimum reference value of 80% on the smart system for handling the complaints of IVF patients. The system saved 159 data records of IVF patients’ complaints and complaint handling.

Figure 2

Algorithms of the CBR similarity formula and the CCBR similarity formula. CBR: case-based reasoning, CCBR: Chris case-based reasoning.

Figure 3

CBR system flow. IVF: in vitro fertilization, CBR: casebased reasoning, CCBR: Chris case-based reasoning.

IV. Discussion

1. IVF Patients’ Complaints and Fertility Doctors’ Responses

Fertility subspecialist doctors and IVF patients supported the smart system for handling the complaints of IVF patients, provided that the system’s recommendations must be accurate.

2. Accuracy of the Similarity Value of the CBR Similarity Formula and the CCBR Similarity Formula

Figure 4 shows 97 similarity values produced by the CBR and CCBR similarity formulas. The horizontal axis denotes the order of test data, and the vertical axis presents the similarity values.

Figure 4

Similarity values. CBR: casebased reasoning, CCBR: Chris case-based reasoning.

The CBR similarity formula showed better performance than CCBR similarity formula, since 91.8% of system recommendations had a similarity value ≥80%, while this was the case for only 42.3% of the recommendations using the CCBR similarity formula.

The first performance assessment was conducted by comparing the 97 system recommendations of the CBR and CCBR similarity formulas with how the 97 IVF patients’ complaints were handled. The results are presented in Table 3.

Table 3

Performance of the CBR and CCBR similarity formulas

The highest similarity value of the system recommendation was inaccurate, despite having a CBR similarity formula value of 100%; thus, a minimum value of 80% cannot be used as a reference threshold for the CBR model with the CBR similarity formula.

All system recommendations in the CCBR similarity formula with a similarity value ≥80% were 100% accurate. The highest inaccurate similarity value was 79.7%; thus, it can be concluded that a minimum value of 80% can be used as a reference to determine the accuracy of the system recommendations made using the CCBR similarity formula.

Based on the minimum value reference of 80%, measurement performance was evaluated using a confusion matrix. In Figure 5, the horizontal axis denotes the classification of accuracy values and precision values for each similarity formula, while the vertical axis presents the values. Accuracy refers to the similarity of all system recommendations to the data on how IVF patients’ complaints were handled, while precision denotes the similarity of system recommendations with a similarity value ≥80% to the data on IVF patients’ complaint handling.

Figure 5

Comparison between accuracy and precision values. CBR: case-based reasoning, CCBR: Chris case-based reasoning.

The confusion matrix showed that the accuracy of the CCBR similarity formula was better than that of the CBR similarity formula. The low accuracy value of CBR (19.59%) is similar to that of 17.32% reported in previous research [25]. The difference in the accuracy value between the CCBR similarity formula and the CBR similarity formula of 32.99% also corresponds to reported in previous research of 35%, with CBR showing lower accuracy [26].

The precision value of the CCBR similarity formula was 100% because all system recommendations with a similarity value ≥80% were declared to be 100% accurate. Thus, this precision value satisfied the requirement of the fertility subspecialist doctors and IVF patients that the system’s recommendations must be accurate.

The validation results of two fertility subspecialist doctors stated that 89.69% of the recommendations for the CCBR similarity formula were accurate.

There was a significant difference between the accuracy value of the CCBR similarity formula produced from the confusion matrix calculation (52.58%) and that produced from fertility subspecialist doctors’ validation. The reason for this difference is that 47.42% of the accurate system recommendations evidently had a similarity value <80%; thus, based on the CBR system flow, the system recommendations could not be directly accessed by IVF patients, but must first be checked by a fertility subspecialist doctor.

The low number of CBR system recommendations declared accurate by the fertility subspecialist doctors can be explained by two factors. First, there were different complaints with the same weight values (Table 2). Second, the CBR similarity formula did not adopt Lin’s similarity theory; thus, no optimal similarity value was obtained [22].

3. Conclusion and Research Opportunities

The CCBR similarity formula and the minimum reference value of 80% applied in the CBR system are capable of providing quality health services [27] through the provision of accurate system recommendations; thus, this approach is capable of helping fertility subspecialist doctors handle IVF patients’ complaints.

The fact that the research data were only collected from hospitals in Central Java Province and Yogyakarta Province means that the research results cannot be taken as the standard method, since further research should use data from many provinces in Indonesia to test the performance of the CCBR similarity formula. The combination of CBR with rule-based reasoning (RBR) can improve the accuracy of smart system output [24]; thus, research needs to be carried out on the combination of the CCBR similarity formula with an RBR model.

Notes

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

References

1. Hanoum IF, Boediono A, Pangestu M, Haryadi D, Widad S, Dasuki D. Microvolume of 0.1 μL gama sleeved cryoloops for blastocyst vitrification of assisted reproductive technology patients. Jurnal Kesehatan Reproduksi 2015;2(1):47–52. https://doi.org/10.22146/jkr.7127.

2. Christianto PA, Sediyono E, Sembiring I. Case-based reasoning modifications for intelligent systems in handling in vitro fertilization (IVF) patients post embryo transfer. In : Proceedings of 2020 International Seminar on Application for Technology of Information and Communication (iSemantic); 2020 Sep 19–20; Semarang, Indonesia. p. 109–14. https://doi.org/10.1109/iSemantic50169.2020.9234270.

3. Christianto PA, Sediyono E, Sembiring I, Wijono S. Intelligent system of handling in vitro fertilization (IVF) patients post embryo transfer to reduce the level of patient anxiety and help fertility doctors quickly answer patient questions In : Triwiyanto , Nugroho HA, Rizal A, Caesarendra W, eds. Proceedings of the 1st International Conference on Electronics, Biomedical Engineering, and Health Informatics. Singapore: Springer; 2021. p. 183–96. https://doi.org/10.1007/978-981-33-6926-9_17.

4. Bennett L, Pangestu M. Regional reproductive quests: cross-border reproductive travel among infertile Indonesian couples. Asia Pac Viewp 2017;58(2):162–74. https://doi.org/10.1111/apv.12160.

5. Guven PG, Cayir Y, Borekci B. Effectiveness of acupuncture on pregnancy success rates for women undergoing in vitro fertilization: a randomized controlled trial. Taiwan J Obstet Gynecol 2020;59(2):282–6. https://doi.org/10.1016/j.tjog.2020.01.018.

6. Capuzzi E, Caldiroli A, Ciscato V, Zanvit FG, Bollati V, Barkin JL, et al. Is in vitro fertilization (IVF) associated with perinatal affective disorders? J Affect Disord 2020;277:271–8. https://doi.org/10.1016/j.jad.2020.08.006.

7. Klitzman R. Infertility providers’ and patients’ views and experiences concerning doctor shopping in the USA. Hum Fertil (Camb) 2019;22(4):238–45. https://doi.org/10.1080/14647273.2017.1406155.

8. Facchin F, Leone D, Tamanza G, Costa M, Sulpizio P, Canzi E, et al. Working with infertile couples seeking assisted reproduction: an interpretative phenomenological study with infertility care providers. Front Psychol 2020;11:586873. https://doi.org/10.3389/fpsyg.2020.586873.

9. Chen S, Guo X, Wu T, Ju X. Exploring the online doctor-patient interaction on patient satisfaction based on text mining and empirical analysis. Inf Process Manag 2020;57(5):102253. https://doi.org/10.1016/j.ipm.2020.102253.

10. Borghi L, Leone D, Poli S, Becattini C, Chelo E, Costa M, et al. Patient-centered communication, patient satisfaction, and retention in care in assisted reproductive technology visits. J Assist Reprod Genet 2019;36(6):1135–42. https://doi.org/10.1007/s10815-019-01466-1.

11. Gozuyesil E, Karacay Yikar S, Nazik E. An analysis of the anxiety and hopelessness levels of women during IVFET treatment. Perspect Psychiatr Care 2020;56(2):338–46. https://doi.org/10.1111/ppc.12436.

12. Hallberg D, Salimi N. Qualitative and quantitative analysis of definitions of e-health and m-health. Healthc Inform Res 2020;26(2):119–28. https://doi.org/10.4258/hir.2020.26.2.119.

13. Vaishya R, Javaid M, Khan IH, Haleem A. Artificial intelligence (AI) applications for COVID-19 pandemic. Diabetes Metab Syndr 2020;14(4):337–9. https://doi.org/10.1016/j.dsx.2020.04.012.

14. Iftikhar P, Kuijpers MV, Khayyat A, Iftikhar A, DeGouvia De Sa M. Artificial intelligence: a new paradigm in obstetrics and gynecology research and clinical practice. Cureus 2020;12(2):e7124. https://doi.org/10.7759/cureus.7124.

15. Bras de Guimaraes B, Martins L, Metello JL, Ferreira FL, Ferreira P, Fonseca JM. Application of artificial intelligence algorithms to estimate the success rate in medically assisted procreation. Reprod Med 2020;1(3):181–94. https://doi.org/10.3390/reprodmed1030014.

16. Song K, De Jonckheere J, Zeng X, Koehl L, Yuan X, Zhao X. Development of a data-based interactive medical expert system for supporting pregnancy consultations: general architecture and methodology. IFAC-PapersOnLine 2019;52(19):67–72. https://doi.org/10.1016/j.ifacol.2019.12.109.

17. Morgan DL. Commentary—after triangulation, what next? J Mix Methods Res 2019;13(1):6–11. https://doi.org/10.1177/1558689818780596.

18. Gou X, Xu Z, Wang X, Liao H. Managing consensus reaching process with self-confident double hierarchy linguistic preference relations in group decision making. Fuzzy Optim Decis Mak 2021;20(1):51–79. https://doi.org/10.1007/s10700-020-09331-y.

19. Malathi D, Logesh R, Subramaniyaswamy V, Vijayakumar V, Sangaiah AK. Hybrid reasoning-based privacy-aware disease prediction support system. Comput Electr Eng 2019;73:114–27. https://doi.org/10.1016/j.compeleceng.2018.11.009.

20. Low N, Hesser J, Blessing M. Multiple retrieval case-based reasoning for incomplete datasets. J Biomed Inform 2019;92:103127. https://doi.org/10.1016/j.jbi.2019.103127.

21. Zhang H, Dai G. Research on traffic decision making method based on image analysis case based reasoning. Optik 2018;158:908–14. https://doi.org/10.1016/j.ijleo.2018.01.007.

22. Lin D. An information-theoretic definition of similarity. In : Proceedings of the 15h International Conference on Machine Learning (ICML); 1998 Jul 24–27; Madison, WI. p. 296–304.

23. Bentaiba-Lagrid MB, Bouzar-Benlabiod L, Rubin SH, Bouabana-Tebibel T, Hanini MR. A case-based reasoning system for supervised classification problems in the medical field. Expert Syst Appl 2020;150:113335. https://doi.org/10.1016/j.eswa.2020.113335.

24. Thike PH, Xu Z, Cheng Y, Jin Y, Shi P. Materials failure analysis utilizing rule-case based hybrid reasoning method. Eng Fail Anal 2019;95:300–11. https://doi.org/10.1016/j.engfailanal.2018.09.033.

25. Takkar S, Singh A. Impact of genetic optimization on the prediction performance of case-based reasoning algorithm in liver disease. Int J Perform Eng 2017;13(4):348–61. https://doi.org/10.23940/ijpe.17.04.p2.348361.

26. Adeniyi DA, Wei Z, Yang Y. Risk factors analysis and death prediction in some life-threatening ailments using chi-square case-based reasoning (χ² CBR) Model. Interdiscip Sci 2018;10(4):854–74. https://doi.org/10.1007/s12539-018-0283-6.

27. Agarwal N, Biswas B. Doctor consultation through mobile applications in India: an overview, challenges and the way forward. Healthc Inform Res 2020;26(2):153–8. https://doi.org/10.4258/hir.2020.26.2.153.

Article information Continued

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Type of complaint	Interview result with fertility subspecialists		Data of IVF program patients’ complaints and complaint handling
Type of complaint	Doctor 1	Doctor 2
Per vaginal bleeding [23]	Existed	Existed	Recorded
Supra-pubic pain [21]	Existed	Existed	Recorded
Fever [23]	Existed	Existed	Recorded
Nausea and vomiting [23]	Existed	Existed	Recorded
Insomnia [27]	Existed	Existed	Recorded

Complaint type	Complaint level	Criteria	Weight score
Per vaginal bleeding	Normal	Normal	0
	Little	Appearing blood spots	0.1
	Medium	Coming out blood volume of about 1 tablespoon	0.5
	Severe	Menstruation-like coming out blood	1

Supra-pubic pain	Normal	Normal	0
	Mild	Able to do normal activities	0.1
	Medium	Inhibiting activities	0.5
	Severe	Unable to do any activity	1

Fever	Normal	Body temperature of <37.2°C	0
	Mild	Body temperature of 37.2°C–37.5°C	0.1
	Medium	Body temperature of 37.5°C–40°C	0.5
	Severe	Body temperature of >40°C	1

Nausea and vomiting	Normal	Normal	0
	Mild	Able to do normal activities	0.11
	Medium	Inhibiting activities	0.5
	Severe	Unable to do any activity	1

Insomnia	Normal	>7 hours	0
	Mild	6–7 hours	0.25
	Medium	5–6 hours	0.5
	Severe	<5 hours	1

Value ranges	CBR similarity formula		CCBR similarity formula

	Number of similarity values	Number of accurate results	Number of similarity values	Number of accurate results
100%	84.5%	9.8%	9.3%	100%

80%–99.9%	7.2%	16.9%	33.0%	100%

<80%	8.2%	16.9%	57.7%	82.1%