Clinical Evaluation of Digital Therapeutics: Present and Future

Article information

Healthc Inform Res. 2022;28(3):188-197
Publication date (electronic) : 2022 July 31
doi : https://doi.org/10.4258/hir.2022.28.3.188
Department of Clinical Pharmacology and Therapeutics, Seoul National University College of Medicine and Hospital, Seoul, Korea
Corresponding Author: Kyung-Sang Yu, Department of Clinical Pharmacology and Therapeutics, Seoul National University College of Medicine and Hospital, 101 Daehakro, Jongno-gu, Seoul 03080, Korea. Tel: +82-2-3668-7833, E-mail: ksyu@snu.ac.kr (https://orcid.org/0000-0003-0921-7225)
Received 2022 March 24; Revised 2022 July 3; Accepted 2022 July 14.

Abstract

Objectives

Digital therapeutics (DTx) are software-based therapeutic interventions based on clinical evidence. Randomized clinical trials (RCTs) are often the source of clinical evidence, similar to conventional drugs or medical devices. However, novel approaches such as the use of real-world data or digital biomarkers are also utilized. This article aimed to review how DTx products have been clinically evaluated.

Methods

DTx products approved by the US Food and Drug Administration as of 2020 were reviewed and products with sufficient published information were selected. Pivotal clinical trials were analyzed according to the elements of the Consolidated Standards of Reporting Trials (CONSORT) guideline. Case reviews were presented for other clinical evaluation strategies, considering the small number of publications.

Results

Most approved DTx products used RCTs for clinical evaluations. Similar to conventional RCTs, parallel-group designs with statistical hypothesis testing were adopted. However, DTx trials were often not blinded due to practical issues and involved various comparator groups. In addition, DTx products could be readily evaluated in home-based settings and delivered through the internet. Other evaluation approaches included retrospective analyses using insurance claims data or usage data, which enabled long-term evaluations of effectiveness. Digital biomarkers obtained from real-time and continuous log data were also used to improve the objectiveness of endpoints.

Conclusions

RCTs accounted for the majority of DTx evaluations. The designs of DTx trials were comparable to those of drug or device trials, but blinding and comparator elements were often different. Furthermore, the use of real-world data and digital biomarkers are also being tried.

I. Introduction

The Digital Therapeutic Alliance defines digital therapeutics (DTx) as “evidence-based therapeutic interventions driven by high-quality software programs to prevent, manage, or treat a medical disorder or diseases” [1,2]. We summarized the three major characteristics of DTx based on its definition [2]. First, DTx are based on “software.” They can be considered as examples of software as a medical device (SaMD), which is a classification used by the US Food and Drug Administration (FDA) [3]. This means that the “software itself,” rather than the hardware upon which it is deployed, is classified as a medical device. Thus, DTx can be freely implemented on a regular smartphone or a tablet instead of being specifically installed on approved medical devices.

Second, DTx are “therapeutic” interventions. DTx are similar to drugs and traditional medical devices in that they should have “therapeutic” effects. This is an important point that distinguishes DTx from general health care applications.

Third, DTx should be “evidence-based.” The characteristic of being “evidence-based” means that appropriate medical evidence is required based on the risk level of DTx. Therefore, it is often required that clinical trial results should be published in peer-reviewed journals and/or reviewed by regulatory agencies, and that real-world evidence and device performance data should be obtained and analyzed [4].

The Digital Therapeutic Alliance classifies DTx into the following three categories according to the purpose: treat a disease, manage a disease, improve a health function [5]. It is recommended to conduct appropriate validation processes for each of these categories. Early DTx were focused on preventing/monitoring medical diseases or disorders or optimizing medication [4]. However, in recent years, increasingly many DTx applications have also been approved as independent therapeutics, mainly in the field of psychiatry [4].

The clinical evaluation of DTx could be compared to the evaluation of efficacy and safety for drug products, as DTx and drugs share similar properties. In general, the effectiveness of drugs has been confirmed through randomized, double-blind, placebo- or active-controlled clinical trials [6]. The clinical phase should be preceded by preclinical toxicological evaluations. DTx are similarly evaluated through clinical trials. However, due to the nature of DTx as software, DTx are exempt from preclinical evaluations that are mandatory for drugs [6]. In addition, blinding and assigning comparators, which are key elements in clinical trials of drugs, are often difficult to conduct due to the inherent properties of DTx as medical devices [6]. Hence, the clinical evaluation of DTx products requires an integrated approach that reflects the characteristics of DTx as both therapeutics and medical devices [6].

In this sense, the evaluation process for DTx could refer to the clinical evaluation processes of drugs; however, key features of medical devices should also be considered. Faris and Shuren [7] listed the following characteristics that distinguish clinical trials of medical devices from those of drugs, which can be equally applied to DTx: (1) device trials tend to enroll fewer participants than drug trials, (2) many device trials assess iterative improvements of previous-generation devices, (3) the device design or procedure may be modified during the trial, (4) device trials are less likely to be blinded or randomized than drug trials, (5) adaptive designs are increasingly common, and (6) existing data can partially or fully substitute for prospective trial data.

However, the use of existing or retrospective data to substitute for prospective trial data suggests that real-world data could be utilized in the clinical evaluation process. Although the definition of real-world data varies among authors, this term usually refers to data collected from real clinical settings rather than randomized clinical trials (RCTs) [8]. The scope of real-world data incorporates claims data, patient registry, post-marketing surveillance data, and pragmatic clinical trials [8].

From this standpoint, the evaluation of DTx through real-world data can be considered as an optional approach, supplementing RCTs [7]. In particular, DTx can be validated with a historical control group based on existing registry data, which is a strategy that has been used in clinical trials of medical devices [7,9]. In addition, since DTx are based on software, real-time log data are collected. Therefore, clinical evaluation methods using machine learning or artificial intelligence are being actively discussed [10].

In this review, we discuss the evaluation strategies of DTx from two different perspectives: clinical trials and real-world data. New approaches such as digital biomarkers are also discussed. Based on these perspectives, requirements for the global approval of DTx are suggested.

II. Clinical Evaluation of DTx through Clinical Trials

1. Similarities to Clinical Trials of Drug Products

Confirmatory RCTs are the gold standard for the clinical evaluation of drugs. RCTs usually involve statistical tests to prove superiority, equivalence, or non-inferiority by comparing a treatment group and a placebo or active control group under a controlled environment. The essentials of RCTs are described in the Consolidated Standards of Reporting Trials (CONSORT) guideline [11], by which the quality of the results is evaluated. Table 1 shows the key elements of RCTs included in the CONSORT guideline [12].

Items related to the methodology of clinical evaluation used in the CONSORT statement

Currently FDA-approved DTx products have been clinically evaluated through RCTs, similarly to the process for drug products. The clinical trial designs for the selected FDA-approved DTx products on the market that have sufficient clinical trial information as of 2020 [1,2] are summarized in Table 2 [1322]. All the clinical trials analyzed herein adopted a randomized, parallel group design. In addition, clinical endpoints validated in conventional drug clinical trials were evaluated using predefined statistical tests. These aspects suggest that the methodology of conventional RCTs can be consistently applied for the evaluation of DTx. The methodology was implemented irrespective of whether DTx were developed as independent therapeutics or adjunctive therapies for other approved drugs [6].

Summary of the clinical evaluation of digital therapeutics through clinical trials

2. Distinctive Characteristics of DTx Clinical Trials

DTx clinical trials had the following distinctive characteristics from conventional RCTs. The characteristics of DTx trials are summarized in Table 3 [13,15,16,1821,23].

Characteristics of clinical trials of digital therapeutics

Blinding for DTx is often difficult. In conventional drug clinical trials, a placebo drug with an identical shape to that of the active treatment drug is often implemented. However, as DTx products are software with various forms, a “placebo” is often not feasible or possible. Instead, a “sham control,” which is occasionally used in evaluations of medical devices, is implemented. However, since it is often difficult to establish blinding of the sham control, clinical trials are instead conducted with an open-label design regarding the treatments. In particular, when the two treatment groups are conventional therapy versus conventional therapy plus DTx, blinding cannot be applied unless a separate sham control is prepared. This feature is shown in Table 2, where most trials had an open rather than a blinded design.

In addition, DTx have various forms of comparators. As mentioned above, DTx trials often involve a sham control. The sham control for DTx trials can be quite variable, unlike the placebos used in drug trials. Sham controls can be constructed in a way that excludes or transforms the main features of the intervention and leaves only the auxiliary function. For example, Pear Therapeutics’ Somryst used a separate software that does not include a therapeutic effect for insomnia, named HealthWatch, as a sham control. The software was designed to include only minor elements of the software being tested (e.g., the interactive interface) [16]. Similar cases can also be found for Akili’s Endeavor [19] or the wearable NightWare [20] with the vibration function removed.

When DTx are not independent therapeutics, each step of using DTx could be analyzed as a comparator. A representative case is WellDoc’s Bluestar, which is used for glycemic control in patients with diabetes [18]. In this study, three-subdivided comparators were implemented: feedback through a smartphone, having the attending physician check only the data log, and having the attending physician check the final analyzed data [18].

Finally, DTx are available in decentralized environments. DTx as independent therapeutics have primarily focused on psychiatric conditions [16]. These DTx products had various indications ranging from drug/opioid use disorders that require facility-level treatment (such as reSET and reSETO [12,14]) to insomnia, which can be treated at home [16]. The diversity of the clinical settings where DTx products are provided indicates that DTx products could be utilized in decentralized environments.

Another important aspect of DTx trials is digital delivery. Since DTx are software, they could be delivered through the internet. In addition, patient recruitment from internet communities or social media could be more readily performed. In a trial using Pear Therapeutics’ Somryst, participants were recruited through Facebook. Patient screening was also conducted through a web portal [16]. This concept is closely related to that of the decentralized clinical trial [24] and has the advantage of enabling recruitment of a large number of participants with little cost.

III. Evaluation Using Real-World Data

Clinical evaluations using claims data can be used to evaluate long-term clinical effects that are difficult to assess in clinical trials. Several cases that utilized retrospective data for the evaluation of DTx have been found. In the case of Pear Therapeutics’ reSET and reSET-O, cost-effectiveness was analyzed in 351 opioid use disorder patients based on post-market insurance claims data [25,26]. Similarly, retrospective cohort data were used to analyze the therapeutic effects [27] or conduct an economic analysis [28] of DTx for patients with type 2 diabetes and hypertension.

Usage data from DTx for managing chronic diseases or optimizing medication compliance are also utilized to evaluate long-term clinical effects. NaturalCycle has obtained market approval for the purpose of contraception (menstrual cycle management); its menstrual cycle predictions were validated through an analysis of 18,548 person-years of menstrual cycle data obtained from 22,785 women [29]. Perx Health’s mobile application to improve medication adherence also confirmed improvements in medication compliance based on data obtained from the application users [30]. As these types of DTx could readily collect large amounts of usage data, clinical evaluations using usage data are expected to be increasingly common.

IV. Concepts of Digital Biomarkers and Exposure-Response of DTx

The therapeutic effects of DTx have been evaluated through validated endpoints in conventional clinical trials. However, recent attempts to use digital endpoints are also gaining attention [6]. Clinical evaluations of therapeutics for central nervous system–related disorders often face difficulties due to subjective and highly variable self-reported endpoints [6]. These characteristics necessitate a larger number of participants, higher study costs, and longer study periods [6].

Digital endpoints can be a favorable alternative as they could be collected in real time with little cost. The development of digital endpoints is aligned with DTx, where considerable amounts of log data are obtained in real time. Accordingly, if clinical feasibility is confirmed, digital endpoints can make a major contribution to improving evaluation efficiency.

As the number and scope of DTx continue to expand, attempts to develop biomarkers specifically for DTx as clinical indicators are also gaining attention. An example is an attempt to distinguish between amyloid-positive and amyloid-negative patients by analyzing gait changes in the early phase of Alzheimer disease [6,31]. Since gait change data were difficult to obtain in real time in the past, analyses were limited. However, due to the spread of digital devices, it has become possible to collect larger amounts of data, accelerating the development and validation of gait changes as a digital biomarker.

Under these circumstances, the importance of artificial intelligence is being emphasized [10,32,33]. As the data obtained from DTx are continuous data obtained in real time through sensors (e.g., gait, inhaler usage, blood sugar changes, and sleep patterns), artificial intelligence can be used to recognize patterns from these data [32]. This should be coupled with real-time monitoring functions, which are automated, efficient, expandable, and easy to operate [32].

As the scope of DTx has recently expanded to virtual reality, more biomarkers have become available for DTx [34]. A variety of information, such as behavior and facial expressions, can be collected from patients wearing virtual reality equipment, and the data can be used to develop biomarkers for treatment or prognoses through artificial intelligence [34]. A previous study evaluated executive dysfunction by analyzing patterns of brain waves and eye movement data obtained from 360° virtual reality equipment through a machine learning algorithm [35]. Considering that virtual reality-based DTx are being actively applied for pain management [36], digital biomarkers could be potentially developed as objective evaluation tools for pain assessment.

DTx could also be interpreted from the perspective of an exposure-response relationship [4]. Exposure to a drug corresponds to the dose, administration interval, and concentration, and the corresponding concepts for DTx may be time, frequency, and duration of DTx use [4]. The response to DTx can be assessed using conventional biomarkers, but also can be evaluated using novel digital endpoints. Further studies on the exposure-response relationship of DTx are necessary since a standard analytical method has not been established [4].

V. Discussion

We found that most examples of the clinical evaluation of DTx used clinical trials, which are the standard for the evaluation of drugs and medical devices. However, novel approaches that use real-world data and digital biomarkers are also on the way. An analogy between the elements of DTx and components of chemical drugs suggests the future direction of DTx [37]. The “active ingredient” of DTx corresponds to the component that shows a therapeutic effect, and it is necessary to validate its efficacy using the aforementioned clinical evaluation methodologies. The “excipient” of DTx is the user interface that maximizes the efficacy of the active ingredient [37]. As DTx products require more active engagement from patients than drugs, the importance of the “excipient,” or the user interface, cannot be overlooked. In other words, the socio-cultural background of the patients who will use DTx should be taken into account to achieve the desired therapeutic effects. In order for DTx to receive global approval, global standards for the clinical evaluation of the effectiveness of the “active ingredients” should be established, while local considerations for “excipients” should simultaneously be taken into account.

Acknowledgments

The authors thank Sungyeun Bae, Sae Im Jeong, WooKyung Chung, Minji Kim, Ji Yun Park, and JungJin Oh for helping with the literature review. This research is partly based on the KOSMI Issue Report (2021), which was supported by the Ministry of Health and Welfare, Republic of Korea.

Notes

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

References

1. Patel NA, Butte AJ. Characteristics and challenges of the clinical pipeline of digital therapeutics. NPJ Digit Med 2020;3(1):159. https://doi.org/10.1038/s41746-020-00370-8.
2. Hong JS, Wasden C, Han DH. Introduction of digital therapeutics. Comput Methods Programs Biomed 2021;209:106319. https://doi.org/10.1016/j.cmpb.2021.106319.
3. Shuren J, Patel B, Gottlieb S. FDA regulation of mobile medical apps. JAMA 2018;320(4):337–8. https://doi.org/10.1001/jama.2018.8832.
4. Chung JY. Digital therapeutics and clinical pharmacology. Transl Clin Pharmacol 2019;27(1):6–11. https://doi.org/10.12793/tcp.2019.27.1.6.
5. Digital Therapeutics Alliance. DTx product categories [Internet] Arlington (VA): Digital Therapeutics Alliance; 2021. [cited at 2022 Jul 24]. Available from: https://dtxalliance.org/wp-content/uploads/2021/01/DTA_FS_DTx-Product-Categories_010521.pdf.
6. Sverdlov O, van Dam J, Hannesdottir K, Thornton-Wells T. Digital therapeutics: an integral component of digital innovation in drug development. Clin Pharmacol Ther 2018;104(1):72–80. https://doi.org/10.1002/cpt.1036.
7. Faris O, Shuren J. An FDA viewpoint on unique considerations for medical-device clinical trials. N Engl J Med 2017;376(14):1350–7. https://doi.org/10.1056/NEJMra1512592.
8. Makady A, de Boer A, Hillege H, Klungel O, Goettsch W, ; (on behalf of GetReal Work Package 1). What is real-world data? a review of definitions based on literature and stakeholder interviews. Value Health 2017;20(7):858–65. https://doi.org/10.1016/j.jval.2017.03.008.
9. Carroll JD, Shuren J, Jensen TS, Hernandez J, Holmes D, Marinac-Dabic D, et al. Transcatheter valve therapy registry is a model for medical device innovation and surveillance. Health Aff (Millwood) 2015;34(2):328–34. https://doi.org/10.1377/hlthaff.2014.1010.
10. Fogel AL, Kvedar JC. Artificial intelligence powers digital medicine. NPJ Digit Med 2018;1:5. https://doi.org/10.1038/s41746-017-0012-2.
11. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg 2012;10(1):28–55. https://doi.org/10.1016/j.ijsu.2011.10.001.
12. Lee JS, Ahn S, Lee KH, Kim JH. Korean translation of the CONSORT 2010 Statement: updated guidelines for reporting parallel group randomized trials. Epidemiol Health 2014;36:e2014029. https://doi.org/10.4178/epih/e2014029.
13. Campbell AN, Nunes EV, Matthews AG, Stitzer M, Miele GM, Polsky D, et al. Internet-delivered treatment for substance abuse: a multisite randomized controlled trial. Am J Psychiatry 2014;171(6):683–90. https://doi.org/10.1176/appi.ajp.2014.13081055.
14. Chaple M, Sacks S, McKendrick K, Marsch LA, Belenko S, Leukefeld C, et al. A comparative study of the therapeutic education system for incarcerated substance-abusing offenders. Prison J 2016;96(3):485–508. https://doi.org/10.1177%2F0032885516636858.
15. Christensen DR, Landes RD, Jackson L, Marsch LA, Mancino MJ, Chopra MP, et al. Adding an internet-delivered treatment to an efficacious treatment package for opioid dependence. J Consult Clin Psychol 2014;82(6):964–72. https://doi.org/10.1037/a0037496.
16. Christensen H, Batterham PJ, Gosling JA, Ritterband LM, Griffiths KM, Thorndike FP, et al. Effectiveness of an online insomnia program (SHUTi) for prevention of depressive episodes (the GoodNight Study): a randomised controlled trial. Lancet Psychiatry 2016;3(4):333–41. https://doi.org/10.1016/s2215-0366(15)00536-2.
17. US National Library of Medicine. Study of efficacy of PEAR-004 in schizophrenia [Internet] Bethesda (MD): ClinicalTrials.gov; 2018. [cited at 2022 Jul 25]. Available from: https://ClinicalTrials.gov/show/NCT03751280.
18. Quinn CC, Shardell MD, Terrin ML, Barr EA, Ballew SH, Gruber-Baldini AL. Cluster-randomized trial of a mobile phone personalized behavioral intervention for blood glucose control. Diabetes Care 2011;34(9):1934–42. https://doi.org/10.2337/dc11-0366.
19. Kollins SH, DeLoss DJ, Canadas E, Lutz J, Findling RL, Keefe RS, et al. A novel digital intervention for actively reducing severity of paediatric ADHD (STARS-ADHD): a randomised controlled trial. Lancet Digit Health 2020;2(4):e168–78. https://doi.org/10.1016/s2589-7500(20)30017-0.
20. US National Library of Medicine. Traumatic nightmares treated by NightWare (To Arouse Not Awaken) [Internet] Bethesda (MD): ClinicalTrials.gov; 2019. [cited at 2022 Jul 25]. Available from: https://ClinicalTrials.gov/show/NCT04040387.
21. Merchant RK, Inamdar R, Quade RC. Effectiveness of population health management using the propeller health asthma platform: a randomized clinical trial. J Allergy Clin Immunol Pract 2016;4(3):455–63. https://doi.org/10.1016/j.jaip.2015.11.022.
22. Chrystyn H, Safioti G, Buck D, Granovsky L, Calderon E, Li T, et al. Real-life inhaler technique in asthma patients using the electronic ProAir Digihaler. Eur Respir J 2019. 54(Suppl 63)PA4258. https://doi.org/10.1183/13993003.congress-2019.PA4258.
23. Park DE, Shin YJ, Park E, Choi IA, Song WY, Kim J. Designing a voice-bot to promote better mental health: UX design for digital therapeutics on ADHD patients. In : Proceedings of CHI: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems; 2020 Apr 25–30; Honolulu, HI. p. 1–8. https://doi.org/10.1145/3334480.3382948.
24. Khozin S, Coravos A. Decentralized trials in the age of real-world evidence and inclusivity in clinical investigations. Clin Pharmacol Ther 2019;106(1):25–7. https://doi.org/10.1002/cpt.1441.
25. Velez FF, Ruetsch C, Maricich Y. Evidence of long-term real-world reduction in healthcare resource utilization following treatment of opioid use disorder with reSETO, a novel prescription digital therapeutic. Expert Rev Pharmacoecon Outcomes Res 2021;21(4):519–20. https://doi.org/10.1080/14737167.2021.1939687.
26. Velez FF, Colman S, Kauffman L, Ruetsch C, Anastassopoulos K. Real-world reduction in healthcare resource utilization following treatment of opioid use disorder with reSET-O, a novel prescription digital therapeutic. Expert Rev Pharmacoecon Outcomes Res 2021;21(1):69–76. https://doi.org/10.1080/14737167.2021.1840357.
27. Guthrie NL, Berman MA, Edwards KL, Appelbaum KJ, Dey S, Carpenter J, et al. Achieving rapid blood pressure control with digital therapeutics: retrospective cohort and machine learning study. JMIR Cardio 2019;3(1):e13030. https://doi.org/10.2196/13030.
28. Nordyke RJ, Appelbaum K, Berman MA. Estimating the impact of novel digital therapeutics in type 2 diabetes and hypertension: health economic analysis. J Med Internet Res 2019;21(10):e15814. https://doi.org/10.2196/15814.
29. Berglund Scherwitzl E, Linden Hirschberg A, Scherwitzl R. Identification and prediction of the fertile window using NaturalCycles. Eur J Contracept Reprod Health Care 2015;20(5):403–8. https://doi.org/10.3109/13625187.2014.988210.
30. Wiecek E, Torres-Robles A, Cutler RL, Benrimoj SI, Garcia-Cardenas V. Impact of a multicomponent digital therapeutic mobile app on medication adherence in patients with chronic conditions: retrospective analysis. J Med Internet Res 2020;22(8):e17834. https://doi.org/10.2196/17834.
31. Muir-Hunter SW, Wittwer JE. Dual-task testing to predict falls in community-dwelling older adults: a systematic review. Physiotherapy 2016;102(1):29–40. https://doi.org/10.1016/j.physio.2015.04.011.
32. Palanica A, Docktor MJ, Lieberman M, Fossat Y. The need for artificial intelligence in digital therapeutics. Digit Biomark 2020;4(1):21–5. https://doi.org/10.1159/000506861.
33. Kim M, Choi HJ. Digital therapeutics for obesity and eating-related problems. Endocrinol Metab (Seoul) 2021;36(2):220–8. https://doi.org/10.3803/enm.2021.107.
34. Riva G, Wiederhold BK, Di Lernia D, Chirico A, Riva EFM, Mantovani F, et al. Virtual reality meets artificial intelligence: the emergence of advanced digital therapeutics and digital biomarkers. Annu Rev Cyberther Telemed 2019;17:3–7.
35. Borgnis F, Baglio F, Pedroli E, Rossetto F, Riva G, Cipresso P. A simple and effective way to study executive functions by using 360° videos. Front Neurosci 2021;15:622095. https://doi.org/10.3389/fnins.2021.622095.
36. Abbadessa G, Brigo F, Clerico M, De Mercanti S, Trojsi F, Tedeschi G, et al. Digital therapeutics in neurology. J Neurol 2022;269(3):1209–24. https://doi.org/10.1007/s00415-021-10608-4.
37. Recchia G, Capuano DM, Mistri N, Verna R. Digital therapeutics: what they are, what they will be. Acta Sci Med Sci 2020;4(3):1–9. https://doi.org/10.31080/ASMS.2020.04.0575.

Article information Continued

Table 1

Items related to the methodology of clinical evaluation used in the CONSORT statement

Section Item# Checklist item
Trial design 3a Description of trial design (such as parallel, cross-over, split-mouth) including allocation ratio.
3b Important changes to methods after trial commencement (such as eligibility criteria), with reasons.

Participants 4a Eligibility criteria for participants.
4b Settings and locations where the data were collected.

Interventions 5 The interventions for each group with sufficient details to allow replication, including how and when they were actually administered.

Outcomes 6a Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed.
6b Any changes to trial outcomes after the trial commenced, with reasons.

Sample size 7a How the sample size was determined.
7b When applicable, explanation of any interim analyses and stopping guidelines.

Randomization

 Sequence generation 8a Method used to generate the random allocation sequence.
8b Type of randomization; details of any restriction (such as blocking and block size).

 Allocation concealment mechanism 9 Mechanism used to implement the random allocation sequence (such as sequentially numbered opaque envelopes), describing any steps taken to conceal the sequence until interventions were assigned.

 Implementation 10 Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions.

 Blinding 11a If done, who was blinded after assignment to interventions (such as participants, treatment providers, those assessing outcomes) and how.
11b If relevant, description of the similarity of interventions.

 Statistical methods 12a Statistical methods used to compare groups for primary and secondary outcomes.
12b Methods for additional analyses, such as subgroup analyses and adjusted analyses.

Adapted from Moher et al. [11].

Table 2

Summary of the clinical evaluation of digital therapeutics through clinical trials

Digital therapeutics Manufacturer Patients Trial design Randomization Blinding Interventions Total number of participants Primary endpoint
reSET [13,14] Pear Therapeutics Patients with drug use disorder Parallel design Stratified randomization (1:1) Open Twice per week for 12 weeks
  1. Conventional treatment + reSET

  2. Conventional treatment group

507 Reduction rates in drug use (via both self-reporting and urine drug test) and treatment retention rates
reSET-O [15] Pear Therapeutics Patients with opium use disorder Parallel design Stratified randomization Open 12 weeks
  1. Local community drug abuse treatment + reSET-O

  2. Local community drug abuse treatment + therapist consultation

206 Maximum abstinence period of opium and cocaine intake
Somryst [16] Pear Therapeutics Patients with chronic insomnia (excluding major depression symptoms) Parallel design Stratified randomization (1:1) Sham control group 6 weeks
  1. SHUTi: Insomnia treatment program

  2. HealthWatch: sham control group

1,149 Depression symptoms lasting for 6 months (PHQ-9)
PEAR-004 [17] Pear Therapeutics Schizophrenia patients Parallel design Random allocation (1:1) Single blinding of evaluator, sham control group 16 weeks
  1. Medication + PEAR-004

  2. Medication + sham control group

112 1. Change in Positive and Negative Syndrome Scale (PANSS) with reference to the base scale2. Dropout rate
Bluestar [18] WellDoc Patients with diabetes Parallel design Cluster random allocation Single (researcher) 1 year
  1. Typical therapy (control group)

  2. Feedback via smartphone

  3. Feedback via smartphone + primary care physician checking raw data

  4. Feedback via smartphone + primary care physician checking the analyzed data

213 HbA1c level after 1 year
Endeavor [19] Akili Pediatric attention deficit hyperactivity disorder (ADHD) patients Parallel design Stratified randomization (1:1) Quadruple blinding (patients, physicians, researchers, evaluators) sham control group 5 days per week for 4 weeks
  1. AKL-T01: DTx

  2. AKL-T09: Similar to AKL-T01 but not ADHD related program

348 Change in Test of Variables of Attention-Attention Performance Index (TOVA API) with reference to the base index
NightWare [20] NightWare Patients with post-traumatic stress disorder (PTSD) Parallel design Random allocation (1:1) Quadruple blinding (patients, physicians, researchers, evaluators), sham control group 60 days
  1. NightWare Therapeutic System: wearable devices and vibration stimulation provided

  2. Sham NightWare: wearable devices without vibration stimulation provided

270 Change in Pittsburgh Sleep Quality Index (PSQI) with reference to the base index
Propeller [21] Propeller Health Patients with asthma or chronic obstructive pulmonary disease Parallel design Stratified randomization (1:1) Open 1 year
  1. Propeller Health System: inhaler DTx linked to smartphone

  2. Control: Sensor only with no feedback

495 Change in short-acting beta 2 agonist use
ProAir Digihaler [22] Teva Patients with asthma or chronic obstructive pulmonary disease Parallel design Random allocation (1:1) Open 12 weeks
  1. ProAir Digihaler: inhaler linked to a digital system

  2. Control: general inhaler

333 Change of Asthma Control Test (ACT) scores with reference to the base score

Table 3

Characteristics of clinical trials of digital therapeutics

Components Characteristics Cases
Trial design Most trials are conducted with parallel designs similar to conventional RCTs, but a crossover design may also be attempted depending on the indication.
  • - RCT divided into general therapy + DTx and general therapy [15]

  • - A 2×2 crossover design of voice-bot DTx and a control group in patients with ADHD [23]

Randomization Stratified randomization is most often performed, but in the case of facility-based processing, cluster randomization is also performed by facility.
  • - Randomization of treatment groups stratified by major variables such as treatment facility, main abuse component, and drug ban period [13]

  • - A clinical study conducted with random allocation of cluster groups by primary care facilities in the state of Maryland [18]

Blinding In case of evaluating in addition to the existing therapy, the evaluation is conducted in an open trial design, but a sham control group may be created, and blinding may be applied.
  • - A study comparing typical therapy + DTx and typical therapy groups in an open way [15]

  • - A study using quadruple blinding by creating a sham control group with a placebo app excluding the function for ADHD treatment [19]

Participants If the purpose is to prevent disease, manage disease, or optimize medication, many participants are patients with chronic diseases. If the DTx are used for therapeutic purposes, participants are often patients with psychiatric diseases. For relatively mild diseases, there have also been attempts to recruit participants from web communities.
  • - A study providing feedback on the use of an asthma medication inhaler linked to a smartphone [21]

  • - A study recruiting and screening patients with insomnia, excluding depression, through Facebook [16]

Control Group Control groups vary depending on the device associated with the DTx, and in many cases, a sham control group is created by removing several functions. When the functions are complex, a control group may be created by subdividing the functions.
  • - A program without insomnia treatment function [16]

  • - Wearable devices without vibration function [20]

  • - A digital inhaler application without feedback [21]

  • - A study comparing diabetes management effects by subdividing control groups according to the use of simple feedback, raw log data, and analyzed data [19]

Number of participants The number of participants vary from small pilot studies to large-scale research with over 1,000 participants.
  • - A study in which 1,149 patients with insomnia were registered through patient recruitment, promotion, and screening on a web portal [16]

Adverse events Few adverse events related to DTx were reported.
  • - No adverse events related to DTx were reported among 1,149 patients who participated in a clinical trial for insomnia [16]

DTx: digital therapeutics, RCT: randomized controlled trials, ADHD: attention deficit hyperactivity disorder.