Digital Twins: A Futuristic Artificial Intelligence Methodology for Design and Analysis of Clinical Trials

The U.S. Food and Drug Administration (FDA) issued a discussion paper on the use of artificial intelligence and machine learning (AI/ML) in drug and biologic development [1]. The paper indicates that FDA is expecting AI/ML to be used in molecule design, protocol writing, patient and site selection, adherence, clinical endpoint assessment, etc.

One interesting item is the notion of digital twin design and analysis. Cosmologists claim that each of us has an exact twin somewhere in the multiverse, but FDA is expecting that digital twins of clinical trial patients to be created in silico and used in analysis. FDA envisions a one-arm trial with all patients receiving the experimental treatment. Using real world data (RWD) from multiple sources such as electronic health records, medical claims, registries, etc. and physiological and molecular process models AI/ML can predict what the patient outcome would be had the patient not received treatment. Data analysis would consider the differences between observed and digital twin measures for efficacy and safety endpoints.

Several researchers have already begun to work in this area. Laubenbacher, Sluka and Glazier indicate that medical digital twins are already being used in in conjunction with patient characteristics in real time to monitor insulin requirements and drive an implanted pump and to optimize heart surgeries [2]. They suggest that digital twins might someday be essential in developing treatments for highly transmissible viral infections such as SARS-Cov-2.

In the area of clinical trials, the European Medicines Agency has recently issued a document indicating that they are considering a prognostic covariate adjustment model (PROCOVA ™). that has some resemblance to digital twins [3]. For each patient in a clinical trial this method of covariate adjustment adds a prognostic score as a covariate in a trial where patients receive experimental treatment on one arm while the control arm consists of patients receiving control treatment on previous trials. The prognostic score is derived from patient characteristics and RWD by AI/ML. It indicates the relative outcome of a patient on the control treatment. The prognostic score is added to patient baseline covariates to adjust the primary efficacy endpoint in the experimental treatment group. This method aims to reduce sample size and increase precision in estimation.

Schuler, Walsh and Hall et al describe a procedure similar to PROCOVA which they apply to a clinical trial for Alzheimer’s Disease [4]. They demonstrate statistical properties such as Type I error control, asymptotic variance, and potential sample size reduction.

The digital twin trial envisioned by FDA is a matched pair design. There is only one arm and all patients on that arm will receive experimental treatment. This would not be a randomized trial, but enrollment might be easier than in a two-arm trial where a potential patient would have only a 50% chance of receiving experimental treatment. Each patient will be compared to his/her digital twin. AI/ML would be used to follow a digital patient with same physiology, molecular structure, health history and baseline variables. This digital twin patient will be constructed from scanning thousands of electronic health records, medical claims, registries etc. The only difference between the patient and his/her twin would be the experimental treatment taken by the enrolled patient.

If the primary clinical endpoint is dichotomous, the matched pair analysis with the 2x2 table showing the agreement and disagreement between pairs would be applied [5]. If the endpoint is numerical, including change scores, the common test of the hypothesis that the mean difference is zero would be utilized. For a time-to-event analysis like survival time the digital twin is formed by following the twin longitudinally along with the patent. There could be a difference in event occurrence (censoring). Several time-to-event methods for paired times including possibility of censoring exist [6,7]. There is also the possibility that the emerging methodology of generalized pairwise comparisons could be used to score differences on paired data [8].

Similar analyses could be applied to safety data.

It is not surprising that digital twin methodology would be proposed at this time. There is a push among all regulatory agencies to use novel clinical trial design and analysis methods. At the same time an interest in employing AI/ML and real-world data (RWD) has appeared. The digital twin trial is not a randomized trial and, thus, its immediate use might be for Phase II trials and postmarket trials, For the latter, this will be the first time for patients not eligible for the pivotal trial such as diabetics, pregnant women, patients taking steroids, etc. could receive the approved drug. Such a patient is not often interested in enrolling in a two-arm trial. For drugs receiving accelerated approval, this methodology could be used in the post-approval trial commitment. It is often difficult to get patients to enroll in these required post-approval trials because patients can now receive the approved drug from their community physician and would want to avoid the possibly of being randomized to placebo or standard of care.

Many researchers will be dismissive of the digital twin design claiming that it is merely a glorified historical control trial. It is not likely that regulators would approve a treatment where the pivotal trial is digital twin. Much has been written of the many inferential problems with RWD [9,10]. Researchers have already reported problems in trying to replicate clinical trial results with RWD [11]. Also, there may be challenges to the way the AI/ML constructed the digital twin. Sponsors could have chosen digital twin methodologies and data sources most likely to show a favorable outcome for the experimental drug. The huge pool of data to construct the digital twin changes every minute making it difficult to evaluate or standardize.

The proportional hazards assumption in time to event regression analysis has long been problematic [12]. Several promising alternatives have been proposed, namely restricted mean survival time [13] and “win statistics” [8, 14]. In hypothesis testing, Bayes factor has been proposed as an alternative to the much-faulted p-value [15]. If these methods are presented at all it is as supportive evidence of efficacy and/or safety of experimental treatments. Of course, such results are usually only presented if they yield positive results for the experimental treatment. It is likely that digital twin methodology will join the ranks of these worthy methodologies.

Digital twin methodology is a creative and interesting entry in novel design and analysis methods for clinical trials. It is likely to find its way into early phase and post market trials but we are not likely to see the digital twins design as a pivotal trial any time soon.

None.

None.

1. U.S. Food and Drug Administration (2023) Using Artificial Intelligence & Machine Learning in the Development of Drug and Biological Products: Discussion Paper and Request for Feedback.
2. Laubenbacher R, Sluka JP, Glazier JA, et al. (2021) Using digital twins in viral infection. Science. 371(6534): 1105-1106.
3. European Medicines Agency (2022) DRAFT Qualification opinion for Prognostic Covariate Adjustment (PROCOVA ™).
4. Schuler A, Walsh D, Hall D, et al. (2022) Increasing the efficiency of randomized trial estimates via liner adjustment for a prognostic score. International Journal of Biostatistics 18(2):329-356.
5. Miettinen, OS (1968) The matched pairs design in the case of all-or-none responses. Biometrics 24(2): 339-352.
6. Holt JD, Prentice RL (1974) Survival analyses in twin studies and matched pair experiments Biometrika 61(1): 17-30.
7. O Brien PC, Fleming TR (1987) A paired Prentice-Wilcoxon test for censored paired data. Biometrics 43(1): 169-180.
8. Buyse M (2010) Generalized pairwise comparisons of prioritized outcomes in the two-sample problem. Statistics in Medicine 29: 3245-3257.
9. Collins R, Bowman L, Landray M, et al. (2020) The magic of randomization versus the myth of real-world evidence. New England Journal of Medicine 382(7): 674-678.
10. Ewer MS, Herson J (2022) Cardiovascular adverse events in oncology trials: understanding and appreciating the differences between clinical trial data and real-world reports. Cardio-Oncology 8(1): 13.
11. Wang SV, Schneeweiss S (2023) Emulation of randomized clinical trials with nonrandomized database analyses: Results of 32 clinical trials. JAMA 329(16): 1376-1385.
12. Cox DR (1972) Regression models and life tables (with discussion). Journal of the Royal Statistical Society B 34: 187-220.
13. Pak K, Uno H, Kim DH, et al. (2017) Interpretability of cancer clinical trial results using restricted mean survival time as an alternative to the hazard ratio. JAMA Oncology 3(12): 1692-1696.
14. Pocock SJ, Ariti CA, Collier TJ, et al. (2012) The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities. European Heart Journal 33(2): 176-182.
15. Kass RE, Rafferty AE (1995) Bayes factors. Journal of the American Statistical Association 90: 773-795.