Clinical Evaluation of Digital Therapeutics: Present and Future

Article information

Healthc Inform Res. 2022;28(3):188-197

Publication date (electronic) : 2022 July 31

doi : https://doi.org/10.4258/hir.2022.28.3.188

Ki Young Huh, Jaeseong Oh, SeungHwan Lee, Kyung-Sang Yu

Department of Clinical Pharmacology and Therapeutics, Seoul National University College of Medicine and Hospital, Seoul, Korea

Corresponding Author: Kyung-Sang Yu, Department of Clinical Pharmacology and Therapeutics, Seoul National University College of Medicine and Hospital, 101 Daehakro, Jongno-gu, Seoul 03080, Korea. Tel: +82-2-3668-7833, E-mail: ksyu@snu.ac.kr (https://orcid.org/0000-0003-0921-7225)

Received 2022 March 24; Revised 2022 July 3; Accepted 2022 July 14.

Abstract

I. Introduction

The Digital Therapeutic Alliance defines digital therapeutics (DTx) as “evidence-based therapeutic interventions driven by high-quality software programs to prevent, manage, or treat a medical disorder or diseases” [1,2]. We summarized the three major characteristics of DTx based on its definition [2]. First, DTx are based on “software.” They can be considered as examples of software as a medical device (SaMD), which is a classification used by the US Food and Drug Administration (FDA) [3]. This means that the “software itself,” rather than the hardware upon which it is deployed, is classified as a medical device. Thus, DTx can be freely implemented on a regular smartphone or a tablet instead of being specifically installed on approved medical devices.

Second, DTx are “therapeutic” interventions. DTx are similar to drugs and traditional medical devices in that they should have “therapeutic” effects. This is an important point that distinguishes DTx from general health care applications.

Third, DTx should be “evidence-based.” The characteristic of being “evidence-based” means that appropriate medical evidence is required based on the risk level of DTx. Therefore, it is often required that clinical trial results should be published in peer-reviewed journals and/or reviewed by regulatory agencies, and that real-world evidence and device performance data should be obtained and analyzed [4].

The Digital Therapeutic Alliance classifies DTx into the following three categories according to the purpose: treat a disease, manage a disease, improve a health function [5]. It is recommended to conduct appropriate validation processes for each of these categories. Early DTx were focused on preventing/monitoring medical diseases or disorders or optimizing medication [4]. However, in recent years, increasingly many DTx applications have also been approved as independent therapeutics, mainly in the field of psychiatry [4].

The clinical evaluation of DTx could be compared to the evaluation of efficacy and safety for drug products, as DTx and drugs share similar properties. In general, the effectiveness of drugs has been confirmed through randomized, double-blind, placebo- or active-controlled clinical trials [6]. The clinical phase should be preceded by preclinical toxicological evaluations. DTx are similarly evaluated through clinical trials. However, due to the nature of DTx as software, DTx are exempt from preclinical evaluations that are mandatory for drugs [6]. In addition, blinding and assigning comparators, which are key elements in clinical trials of drugs, are often difficult to conduct due to the inherent properties of DTx as medical devices [6]. Hence, the clinical evaluation of DTx products requires an integrated approach that reflects the characteristics of DTx as both therapeutics and medical devices [6].

In this sense, the evaluation process for DTx could refer to the clinical evaluation processes of drugs; however, key features of medical devices should also be considered. Faris and Shuren [7] listed the following characteristics that distinguish clinical trials of medical devices from those of drugs, which can be equally applied to DTx: (1) device trials tend to enroll fewer participants than drug trials, (2) many device trials assess iterative improvements of previous-generation devices, (3) the device design or procedure may be modified during the trial, (4) device trials are less likely to be blinded or randomized than drug trials, (5) adaptive designs are increasingly common, and (6) existing data can partially or fully substitute for prospective trial data.

However, the use of existing or retrospective data to substitute for prospective trial data suggests that real-world data could be utilized in the clinical evaluation process. Although the definition of real-world data varies among authors, this term usually refers to data collected from real clinical settings rather than randomized clinical trials (RCTs) [8]. The scope of real-world data incorporates claims data, patient registry, post-marketing surveillance data, and pragmatic clinical trials [8].

From this standpoint, the evaluation of DTx through real-world data can be considered as an optional approach, supplementing RCTs [7]. In particular, DTx can be validated with a historical control group based on existing registry data, which is a strategy that has been used in clinical trials of medical devices [7,9]. In addition, since DTx are based on software, real-time log data are collected. Therefore, clinical evaluation methods using machine learning or artificial intelligence are being actively discussed [10].

In this review, we discuss the evaluation strategies of DTx from two different perspectives: clinical trials and real-world data. New approaches such as digital biomarkers are also discussed. Based on these perspectives, requirements for the global approval of DTx are suggested.

II. Clinical Evaluation of DTx through Clinical Trials

1. Similarities to Clinical Trials of Drug Products

Confirmatory RCTs are the gold standard for the clinical evaluation of drugs. RCTs usually involve statistical tests to prove superiority, equivalence, or non-inferiority by comparing a treatment group and a placebo or active control group under a controlled environment. The essentials of RCTs are described in the Consolidated Standards of Reporting Trials (CONSORT) guideline [11], by which the quality of the results is evaluated. Table 1 shows the key elements of RCTs included in the CONSORT guideline [12].

Table 1

Items related to the methodology of clinical evaluation used in the CONSORT statement

Currently FDA-approved DTx products have been clinically evaluated through RCTs, similarly to the process for drug products. The clinical trial designs for the selected FDA-approved DTx products on the market that have sufficient clinical trial information as of 2020 [1,2] are summarized in Table 2 [13–22]. All the clinical trials analyzed herein adopted a randomized, parallel group design. In addition, clinical endpoints validated in conventional drug clinical trials were evaluated using predefined statistical tests. These aspects suggest that the methodology of conventional RCTs can be consistently applied for the evaluation of DTx. The methodology was implemented irrespective of whether DTx were developed as independent therapeutics or adjunctive therapies for other approved drugs [6].

Table 2

Summary of the clinical evaluation of digital therapeutics through clinical trials

2. Distinctive Characteristics of DTx Clinical Trials

DTx clinical trials had the following distinctive characteristics from conventional RCTs. The characteristics of DTx trials are summarized in Table 3 [13,15,16,18–21,23].

Table 3

Characteristics of clinical trials of digital therapeutics

Blinding for DTx is often difficult. In conventional drug clinical trials, a placebo drug with an identical shape to that of the active treatment drug is often implemented. However, as DTx products are software with various forms, a “placebo” is often not feasible or possible. Instead, a “sham control,” which is occasionally used in evaluations of medical devices, is implemented. However, since it is often difficult to establish blinding of the sham control, clinical trials are instead conducted with an open-label design regarding the treatments. In particular, when the two treatment groups are conventional therapy versus conventional therapy plus DTx, blinding cannot be applied unless a separate sham control is prepared. This feature is shown in Table 2, where most trials had an open rather than a blinded design.

In addition, DTx have various forms of comparators. As mentioned above, DTx trials often involve a sham control. The sham control for DTx trials can be quite variable, unlike the placebos used in drug trials. Sham controls can be constructed in a way that excludes or transforms the main features of the intervention and leaves only the auxiliary function. For example, Pear Therapeutics’ Somryst used a separate software that does not include a therapeutic effect for insomnia, named HealthWatch, as a sham control. The software was designed to include only minor elements of the software being tested (e.g., the interactive interface) [16]. Similar cases can also be found for Akili’s Endeavor [19] or the wearable NightWare [20] with the vibration function removed.

When DTx are not independent therapeutics, each step of using DTx could be analyzed as a comparator. A representative case is WellDoc’s Bluestar, which is used for glycemic control in patients with diabetes [18]. In this study, three-subdivided comparators were implemented: feedback through a smartphone, having the attending physician check only the data log, and having the attending physician check the final analyzed data [18].

Finally, DTx are available in decentralized environments. DTx as independent therapeutics have primarily focused on psychiatric conditions [16]. These DTx products had various indications ranging from drug/opioid use disorders that require facility-level treatment (such as reSET and reSETO [12,14]) to insomnia, which can be treated at home [16]. The diversity of the clinical settings where DTx products are provided indicates that DTx products could be utilized in decentralized environments.

Another important aspect of DTx trials is digital delivery. Since DTx are software, they could be delivered through the internet. In addition, patient recruitment from internet communities or social media could be more readily performed. In a trial using Pear Therapeutics’ Somryst, participants were recruited through Facebook. Patient screening was also conducted through a web portal [16]. This concept is closely related to that of the decentralized clinical trial [24] and has the advantage of enabling recruitment of a large number of participants with little cost.

III. Evaluation Using Real-World Data

Clinical evaluations using claims data can be used to evaluate long-term clinical effects that are difficult to assess in clinical trials. Several cases that utilized retrospective data for the evaluation of DTx have been found. In the case of Pear Therapeutics’ reSET and reSET-O, cost-effectiveness was analyzed in 351 opioid use disorder patients based on post-market insurance claims data [25,26]. Similarly, retrospective cohort data were used to analyze the therapeutic effects [27] or conduct an economic analysis [28] of DTx for patients with type 2 diabetes and hypertension.

Usage data from DTx for managing chronic diseases or optimizing medication compliance are also utilized to evaluate long-term clinical effects. NaturalCycle has obtained market approval for the purpose of contraception (menstrual cycle management); its menstrual cycle predictions were validated through an analysis of 18,548 person-years of menstrual cycle data obtained from 22,785 women [29]. Perx Health’s mobile application to improve medication adherence also confirmed improvements in medication compliance based on data obtained from the application users [30]. As these types of DTx could readily collect large amounts of usage data, clinical evaluations using usage data are expected to be increasingly common.

IV. Concepts of Digital Biomarkers and Exposure-Response of DTx

The therapeutic effects of DTx have been evaluated through validated endpoints in conventional clinical trials. However, recent attempts to use digital endpoints are also gaining attention [6]. Clinical evaluations of therapeutics for central nervous system–related disorders often face difficulties due to subjective and highly variable self-reported endpoints [6]. These characteristics necessitate a larger number of participants, higher study costs, and longer study periods [6].

Digital endpoints can be a favorable alternative as they could be collected in real time with little cost. The development of digital endpoints is aligned with DTx, where considerable amounts of log data are obtained in real time. Accordingly, if clinical feasibility is confirmed, digital endpoints can make a major contribution to improving evaluation efficiency.

As the number and scope of DTx continue to expand, attempts to develop biomarkers specifically for DTx as clinical indicators are also gaining attention. An example is an attempt to distinguish between amyloid-positive and amyloid-negative patients by analyzing gait changes in the early phase of Alzheimer disease [6,31]. Since gait change data were difficult to obtain in real time in the past, analyses were limited. However, due to the spread of digital devices, it has become possible to collect larger amounts of data, accelerating the development and validation of gait changes as a digital biomarker.

Under these circumstances, the importance of artificial intelligence is being emphasized [10,32,33]. As the data obtained from DTx are continuous data obtained in real time through sensors (e.g., gait, inhaler usage, blood sugar changes, and sleep patterns), artificial intelligence can be used to recognize patterns from these data [32]. This should be coupled with real-time monitoring functions, which are automated, efficient, expandable, and easy to operate [32].

As the scope of DTx has recently expanded to virtual reality, more biomarkers have become available for DTx [34]. A variety of information, such as behavior and facial expressions, can be collected from patients wearing virtual reality equipment, and the data can be used to develop biomarkers for treatment or prognoses through artificial intelligence [34]. A previous study evaluated executive dysfunction by analyzing patterns of brain waves and eye movement data obtained from 360° virtual reality equipment through a machine learning algorithm [35]. Considering that virtual reality-based DTx are being actively applied for pain management [36], digital biomarkers could be potentially developed as objective evaluation tools for pain assessment.

DTx could also be interpreted from the perspective of an exposure-response relationship [4]. Exposure to a drug corresponds to the dose, administration interval, and concentration, and the corresponding concepts for DTx may be time, frequency, and duration of DTx use [4]. The response to DTx can be assessed using conventional biomarkers, but also can be evaluated using novel digital endpoints. Further studies on the exposure-response relationship of DTx are necessary since a standard analytical method has not been established [4].

V. Discussion

We found that most examples of the clinical evaluation of DTx used clinical trials, which are the standard for the evaluation of drugs and medical devices. However, novel approaches that use real-world data and digital biomarkers are also on the way. An analogy between the elements of DTx and components of chemical drugs suggests the future direction of DTx [37]. The “active ingredient” of DTx corresponds to the component that shows a therapeutic effect, and it is necessary to validate its efficacy using the aforementioned clinical evaluation methodologies. The “excipient” of DTx is the user interface that maximizes the efficacy of the active ingredient [37]. As DTx products require more active engagement from patients than drugs, the importance of the “excipient,” or the user interface, cannot be overlooked. In other words, the socio-cultural background of the patients who will use DTx should be taken into account to achieve the desired therapeutic effects. In order for DTx to receive global approval, global standards for the clinical evaluation of the effectiveness of the “active ingredients” should be established, while local considerations for “excipients” should simultaneously be taken into account.

Article information Continued