Presented July 30, 2018 at the Joint Statistical Meeting (JSM) of the American Statistical Association (ASA) in Vancouver, British Columbia, Canada.
Objective: We hypothesize that using predicted outcomes from our ML models as covariates in the analysis of a clinical trial’s primary endpoint will give a substantial boost to study power over traditional covariates.
Methods: We used the PRO-ACT database to develop Gradient Boosting Machine (GBM) models that predict total ALSFRS-R score and % expected VC. We made predictions for all patients in PRO-ACT using 10-fold cross-validation (CV). We than ran clinical trial simulations where we randomly sampled from the PRO-ACT population of patients, assigned patients equally to two groups, then analyzed (ANOVA) the primary endpoint of interest three different ways; 1.) unadjusted, 2.) adjusted for a single baseline covariate, & 3.) adjusted for the predicted outcome from our ML models as a covariate. For each simulation, we calculated the % mean squared error (MSE) reduction from each method of covariate adjustment compared to the unadjusted analysis. We used the MSE from each covariate adjusted analysis to perform power calculations (assuming a fixed treatment effect and sample size) and calculated the % power boost from each method of covariate adjustment compared to the unadjusted analysis. We summarized data across all simulations and provide visual summaries of the results from analysis of the simulated clinical trial populations. We repeated a similar process for Alzheimer’s disease using the Coalition Against Major Diseases (CAMD) database and predicting the Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAScog).
Conclusions: Using predictions of ALS outcomes from our machine-learning models as covariates in the analysis of clinical trial primary endpoints has been shown, through simulations, to give on average more than a 10% boost to study power across disease endpoints studied. In addition to extensive work done in predictive modeling for ALS outcomes, preliminary work in Alzheimer’s shows similarly promising results.
Authors: Danielle Beaulieu, Albert A. Taylor, Andrew Conklin, Jonavelle Cuerdo, Mike Keymer, David L. Ennist