Meta-prediction of coronary artery disease risk

Overall Summary

Study Background and Main Findings

This study addresses the critical public health challenge of Coronary Artery Disease (CAD), a leading cause of global morbidity and mortality, by aiming to improve upon current risk prediction methods. Existing tools often show limited accuracy, particularly in diverse populations or specific subgroups, and struggle to effectively integrate the growing wealth of genetic information. The primary objective was to develop and validate a novel machine learning framework – termed 'meta-prediction' – that integrates a wide range of unmodifiable risk factors (like age and numerous genetic predispositions summarized by Polygenic Risk Scores, or PRSs) with modifiable factors (clinical measurements, lifestyle information) to generate more accurate, personalized, and actionable 10-year CAD risk estimates.

Methodologically, the researchers utilized large-scale data from the UK Biobank (UKBB), strategically dividing participants into two groups: one with existing CAD ('prevalent cohort') and one initially free of CAD who were followed over time ('incident cohort'). The core innovation involved a two-stage process: first, training numerous baseline predictive models on the prevalent cohort to estimate various risk factors and diagnoses; second, using the outputs (predictions) from these baseline models as new input features – 'meta-features' – along with directly measured data, to train a final ensemble machine learning model (specifically XGBoost) predicting the 10-year risk of developing CAD in the incident cohort. This hierarchical approach allowed the model to learn complex patterns and interactions from over 1,700 initial features, ultimately selecting the 50 most informative ones, including 15 meta-features and 22 PRSs.

The resulting meta-prediction model demonstrated significantly improved performance. Within the UKBB test set, it achieved high discrimination (Area Under the Curve, AUC, a measure of model accuracy where 1 is perfect and 0.5 is random chance, was 0.84). Crucially, this high performance was largely maintained (AUC 0.81) upon external validation in the independent and diverse All of Us (AoU) research program cohort, indicating good generalizability. The model substantially outperformed standard clinical risk scores like the Pooled Cohort Equations (PCE) and QRISK3 (average AUC improvement >10%, average Area Under the Precision-Recall Curve improvement 67%), and also showed better risk reclassification (Net Reclassification Index 0.14-0.21). Performance gains were particularly notable in subgroups traditionally considered low-risk. Furthermore, model interpretation using SHAP values identified key risk drivers, and simulations suggested the framework could potentially guide personalized interventions by predicting differential risk reduction based on an individual's genetic profile and risk subgroup.

In conclusion, the study presents a powerful, integrative meta-prediction framework that significantly advances CAD risk prediction accuracy and generalizability compared to current standards. By effectively leveraging comprehensive genetic and non-genetic data through a sophisticated machine learning approach, the framework offers potential for more precise risk stratification and personalized prevention strategies. The findings underscore the importance of incorporating broad genetic information and complex interactions to capture individual CAD susceptibility more effectively, paving the way towards precision cardiology.

Research Impact and Future Directions

This study successfully developed and validated a sophisticated machine learning framework for predicting 10-year Coronary Artery Disease (CAD) risk, demonstrating superior performance compared to established clinical scores and previous research models. The 'meta-prediction' approach, integrating a vast array of genetic (numerous Polygenic Risk Scores - PRSs) and non-genetic factors (clinical, lifestyle, biomarker data) through intermediate predictive steps ('meta-features'), represents a significant methodological advance.

The model's high accuracy (AUC 0.84 in UK Biobank, 0.81 in the diverse All of Us cohort) and improved ability to reclassify individuals into more appropriate risk categories (Net Reclassification Index 0.14-0.21 vs standard scores) underscore its potential clinical utility. Particularly noteworthy is its enhanced performance in subgroups often considered lower risk by traditional methods (e.g., younger individuals, females), suggesting it captures risk pathways missed by simpler models. The framework's ability to simulate differential intervention benefits based on genetic risk profiles offers a promising avenue towards personalized prevention strategies, although these simulations are currently model-based hypotheses requiring prospective clinical validation.

While the findings are compelling, it is crucial to recognize the study's observational nature; the model identifies complex associations, but direct causation for all contributing factors is not established by this work alone. Although validated in the diverse All of Us cohort, further assessment in specific underrepresented populations and age groups is warranted. The practical implementation of such a complex model faces hurdles beyond predictive accuracy, including seamless integration into clinical workflows, clinician training and acceptance, regulatory approval, and demonstration of cost-effectiveness. The infrastructure for such deployment is emerging but not yet widespread.

In essence, this research provides a powerful demonstration of how advanced machine learning, applied to large-scale, multi-modal data including comprehensive genetics, can significantly refine cardiovascular risk prediction. It marks a substantial step towards more personalized and precise prevention of CAD. However, translating this potential into routine clinical impact requires overcoming implementation challenges and further validating the personalized intervention aspect through dedicated clinical trials.

Critical Analysis and Recommendations

Highlights Key Performance Metrics and Superiority (written-content)
Observation: The abstract highlights key quantitative performance metrics (AUC 0.84 in UK Biobank, 0.81 in All of Us) and the model's superiority over existing methods. Methodological Context: These metrics result from testing the final meta-prediction model on hold-out internal and external validation cohorts. Clinical/Practical Significance: Demonstrates the model's high discriminative accuracy and potential impact compared to current standards. Implementation Implications: Strong performance metrics are crucial for justifying further development and potential clinical adoption.
Section: Abstract

Emphasizes Actionability and Personalization (written-content)
Observation: The abstract mentions the framework's ability to generate individualized risk reduction profiles and notes that genetic risk influences intervention benefits. Methodological Context: This stems from model simulations where risk factors are perturbed. Clinical/Practical Significance: Suggests potential clinical utility beyond risk stratification, enabling tailored prevention strategies. Implementation Implications: Points towards a future application where the model could guide personalized treatment decisions, pending further validation.
Section: Abstract

Explicitly State Study Design (written-content)
Issue: The abstract does not explicitly state the study design (observational cohort study). Impact: While inferable, explicitly stating the design provides immediate context about the nature of the evidence (associations derived from observation) for readers scanning the abstract, enhancing clarity without requiring further reading.
Section: Abstract

Critiques Prior Integration Strategies (written-content)
Observation: The introduction effectively reviews prior attempts to integrate genetic and clinical data, noting their limitations (e.g., marginal improvements from linear combinations, failure to capture interactions). Methodological Context: This review establishes the scientific gap the current study aims to fill. Clinical/Practical Significance: Justifies the need for the novel, more complex meta-prediction framework proposed in the paper. Implementation Implications: Underscores why simpler approaches may be insufficient for achieving personalized risk prediction.
Section: Introduction

Clearly Introduces the Novel Framework (written-content)
Observation: The introduction clearly articulates the proposed 'omnigenic, integrative, meta-prediction framework' and its differentiating features (numerous PRSs, ML for interactions, meta-feature integration). Methodological Context: This defines the study's unique methodological contribution. Clinical/Practical Significance: Sets expectations for how the framework aims to improve upon previous work. Implementation Implications: Provides a conceptual blueprint of the novel approach.
Section: Introduction

Explicitly State Central Hypothesis (written-content)
Issue: The introduction outlines the framework and goals but lacks an explicit, concise statement of the central hypothesis. Impact: Adding a clear hypothesis (e.g., that the framework will achieve superior accuracy and stratification by capturing complex interactions) would provide a sharper focus and directly link the proposed methods to the expected advantages.
Section: Introduction

Rigorous Comparative Evaluation Shows Superior Performance (written-content)
Observation: The meta-prediction model significantly outperformed established clinical scores (PCE, QRISK3, PREVENT) and research models across multiple metrics (e.g., AUROC >10% higher, AUPRC 67% higher on average). Methodological Context: Comparisons were made on a hold-out test set using standard evaluation metrics. Clinical/Practical Significance: Provides strong quantitative evidence for the model's superior predictive ability compared to current standards. Implementation Implications: Supports the potential for the model to replace or augment existing clinical risk assessment tools.
Section: Results

Strong External Validation and Generalizability (written-content)
Observation: The model maintained high performance (AUC 0.81) when validated externally in the diverse All of Us (AoU) cohort, with similar performance across European, African, and Hispanic ancestry groups. Methodological Context: A streamlined model was tested on the independent AoU dataset. Clinical/Practical Significance: Demonstrates the model's robustness and generalizability beyond the UKBB training data, suggesting broader applicability. Implementation Implications: Increases confidence in the model's potential utility across different populations, although further validation in specific groups is still beneficial.
Section: Results

Demonstrates Potential for Personalized Intervention Guidance (written-content)
Observation: Simulations showed that the predicted absolute risk reduction from interventions (lowering LDL, HbA1c, SBP) varied significantly based on individuals' underlying genetic risk and their assigned risk subgroup. Methodological Context: Intervention effects were simulated by altering feature values in the trained model and observing changes in predicted risk. Clinical/Practical Significance: Highlights the framework's potential to guide personalized prevention by identifying individuals likely to derive the most benefit from specific interventions. Implementation Implications: Suggests a future clinical application for tailoring treatment intensity, contingent on prospective validation.
Section: Results

Enhance Clinical Interpretation of Risk Subgroups (written-content)
Issue: While distinct risk subgroups were identified via SHAP value clustering, the Results section primarily describes their differentiating features statistically without offering much clinical interpretation. Impact: Briefly connecting the differentiating features (e.g., high CAD PRS dominance vs. other factors) to potential underlying clinical or pathophysiological profiles within the Results would make the significance of these subgroups more immediately apparent and tangible.
Section: Results

Clear Summary of Core Findings and Superiority (written-content)
Observation: The Discussion effectively synthesizes the main finding – the meta-prediction framework's superior and more generalizable performance compared to existing standards. Methodological Context: This summarizes the key results from comparative analyses. Clinical/Practical Significance: Clearly states the paper's primary contribution and its potential impact on CAD risk prediction. Implementation Implications: Provides a concise take-home message regarding the framework's advantages.
Section: Discussion

Emphasizes Importance of Genetic Risk and Meta-Features (written-content)
Observation: The Discussion emphasizes the crucial role of genetic risk, particularly via meta-features derived from unmodifiable factors (genetics, age, sex), in achieving superior prediction. Methodological Context: This interpretation is based on SHAP value analysis showing the high importance of these features. Clinical/Practical Significance: Reinforces the study's theme about the power of integrating genetics and highlights potential mechanisms driving the model's success. Implementation Implications: Underscores the value of incorporating comprehensive genetic data in risk models.
Section: Discussion

Elaborate on Mechanisms of Differential Intervention Benefit (written-content)
Issue: The Discussion notes that genetic risk mediates differential intervention benefits but doesn't explore potential mechanisms. Impact: Speculating on how genetics might influence intervention response (e.g., pharmacogenetics, baseline risk levels, pathway interactions) would add depth, moving beyond observation towards biological/clinical implications and stimulating further research into personalized prevention mechanisms.
Section: Discussion

Explicit Description of Meta-Prediction Framework (written-content)
Observation: The Methods explicitly detail the novel meta-prediction strategy, explaining the two-stage process involving baseline models (trained on prevalent cohort) generating meta-features for the final incident model. Methodological Context: This describes the core innovation of the study. Clinical/Practical Significance: Allows understanding of how diverse information streams are integrated. Implementation Implications: Provides the blueprint necessary for others to understand and potentially replicate or adapt the approach.
Section: Methods

Rigorous External Validation Strategy (written-content)
Observation: The Methods detail the external validation strategy using the All of Us (AoU) cohort, including feature mapping, handling data differences, and testing specific streamlined/generalizable models. Methodological Context: This describes the steps taken to assess model robustness in an independent, diverse population. Clinical/Practical Significance: Significantly strengthens the study's claims of generalizability and potential broad applicability. Implementation Implications: Provides evidence supporting the model's potential use beyond the initial development cohort.
Section: Methods

Specify Clustering Cut Height or Cluster Number Determination Method (written-content)
Issue: The Methods state that subgroup identification used hierarchical clustering with fixed-height tree cutting to define five clusters, but the specific cut height or the method used to determine 'five' as the optimal number is not provided. Impact: This missing parameter hinders the exact reproducibility of the subgroup analysis, a key component supporting the personalized intervention findings.
Section: Methods

Section Analysis

Abstract

Key Aspects

Problem Context and Study Aim: The study addresses the critical public health challenge posed by Coronary Artery Disease (CAD), a primary global cause of death and illness. It highlights the limitations of current risk prediction methods and proposes a novel meta-prediction framework. This framework aims to provide more accurate, actionable, and personalized CAD risk estimates by integrating both fixed (age, genetics) and modifiable (clinical measurements, lifestyle) risk factors, moving beyond traditional risk assessment strategies.
Meta-Prediction Methodology and Cohort Strategy: A sophisticated meta-prediction methodology was developed using data from the extensive UK Biobank cohort, encompassing approximately 2,000 predictive features. The approach involved a two-stage process: initially training baseline predictive models on a prevalent CAD cohort, and subsequently using the outputs of these models as 'meta-features' to train a final incident CAD risk prediction model on a separate incident CAD cohort. This hierarchical structure allows for the integration of complex information derived from baseline characteristics and risk factor trajectories.
Model Composition and Predictive Performance: The resulting 10-year incident CAD risk model, incorporating 15 derived meta-features and multiple polygenic risk scores (PRSs), demonstrated high predictive accuracy. It achieved an Area Under the Curve (AUC) of 0.84 within the UK Biobank test set. Crucially, its performance was validated in an independent, diverse cohort from the All of Us research program, yielding an AUC of 0.81, thereby indicating good generalizability and outperforming both standard clinical risk scores and prior integrative models.
Personalized Risk Reduction and Genetic Influence: Beyond risk stratification, the developed framework offers significant potential for clinical translation by enabling personalized prevention strategies. It can generate individualized risk reduction profiles, quantifying how standard clinical interventions might lower a specific person's CAD risk. A key finding is that an individual's underlying genetic risk significantly influences the potential benefit derived from these interventions, underscoring the importance of incorporating genetic data for tailoring preventive care effectively.

Strengths

Clear Problem Statement and Objective
The abstract clearly articulates the significant public health problem of CAD and the critical need for improved, personalized risk prediction, effectively setting the stage for the study's objectives.

"Coronary artery disease (CAD) is a leading cause of morbidity and mortality worldwide, and accurately predicting individual risk is critical for prevention. Here we aimed to integrate unmodifiable risk factors, such as age and genetics, with modifiable risk factors, such as clinical and biometric measurements, into a meta-prediction framework that produces actionable and personalized risk estimates." (Page 1)
Concise Methodological Summary
The abstract concisely summarizes the complex meta-prediction methodology, including the use of distinct cohorts for training baseline and incident models and the concept of using baseline model outputs as meta-features.

"To power our meta-prediction approach, we stratified the UK Biobank into two primary cohorts: first, a prevalent CAD cohort used to train predictive models...and, second, an incident CAD cohort using, in part, these baseline models as meta-features to train a final CAD incident risk prediction model." (Page 1)
Highlights Key Performance Metrics
Key quantitative results, including the model's high performance (AUC 0.84 in UK Biobank, 0.81 in All of Us) and its superiority over existing methods, are effectively highlighted, demonstrating the model's potential impact.

"The resultant 10-year incident CAD risk model...achieves an area under the curve of 0.84. In an independent test cohort from the All of Us research program, this model achieved an area under the curve of 0.81...outperforming standard clinical scores and previously developed integrative models." (Page 1)
Emphasizes Actionability and Personalization
The abstract successfully conveys the translational potential of the framework by mentioning its ability to generate individualized risk reduction profiles and the finding that genetic risk influences intervention benefits, suggesting clinical utility.

"Moreover, this framework enables the generation of individualized risk reduction profiles by quantifying the potential impact of standard clinical interventions. Notably, genetic risk influences the extent to which these interventions reduce overall CAD risk, allowing for tailored prevention strategies." (Page 1)

Suggestions for Improvement

Explicitly State Study Design
This low-impact improvement would enhance immediate clarity for readers scanning the abstract. The Abstract section serves as a standalone summary, and explicitly stating the study design provides fundamental context upfront. Adding this information would slightly strengthen the abstract by immediately orienting the reader to the nature of the evidence presented, reinforcing the observational basis of the findings without requiring them to infer it or read further.

"To power our meta-prediction approach, we stratified the UK Biobank into two primary cohorts: first, a prevalent CAD cohort..." (Page 1)

Implementation: In the sentence describing the data source, incorporate the study design type. For example, modify the sentence starting 'To power our meta-prediction approach...' to something like: 'Using data from the large-scale observational UK Biobank cohort, we stratified participants into two primary groups to power our meta-prediction approach...'

Introduction

Key Aspects

Limitations of Current CAD Risk Prediction and PRS Utility: The introduction contextualizes the study by highlighting the limitations of current Coronary Artery Disease (CAD) risk prediction, particularly concerning the integration and clinical utility of Polygenic Risk Scores (PRSs). It notes the debate surrounding PRS effectiveness, especially for risk stratification in diverse populations, and argues that simple linear models are insufficient due to CAD's heterogeneity and genetic complexity. This establishes the need for more sophisticated approaches capable of handling complex genetic and non-genetic interactions for personalized interventions.
Critique of Prior PRS and Clinical Data Integration Methods: Previous attempts to combine PRSs and clinical risk factors into unified frameworks are reviewed and critiqued. Strategies like linear combinations of single CAD PRSs with traditional factors, or incorporating risk-factor-associated variants into single PRSs, yielded only marginal improvements and obscured individual factor contributions and interactions. Even combining multiple PRSs and clinical features linearly, or initial attempts with machine learning (ML), failed to significantly boost accuracy or identify the complex interactions needed for personalized recommendations, highlighting a gap in developing truly actionable, genetically informed ML models.
Proposed Omnigenic, Integrative, Meta-Prediction Framework: The paper introduces its novel approach: an 'omnigenic, integrative, meta-prediction framework'. This framework is distinct because it adopts an omnigenic hypothesis (incorporating numerous PRSs reflecting diverse biological pathways), utilizes an ML structure designed to detect interactions between risk factors (genetic and non-genetic) and potential interventions, and employs meta-prediction. Meta-prediction here involves integrating predictions from multiple baseline models (predicting various risk factors and diagnoses) as inputs ('meta-features') for the final CAD event risk model.
Framework Differentiation, Goals, and Potential Impact: The framework aims to advance CAD risk prediction by providing generalizable prospective estimates and enabling personalized prevention strategies. Key goals include disentangling inherited risk (captured by age, sex, genetics) from acquired risk (influenced by measured biomarkers and lifestyle factors) for a more comprehensive assessment. By detecting interactions (e.g., how CAD PRS modifies the benefit of lipid lowering), the framework is designed to identify personalized interventions, ultimately moving towards precision prevention of CAD.

Strengths

Establishes Clear Need and Context
The Introduction effectively establishes the context by highlighting the limitations of current CAD risk prediction, particularly the debated utility of PRSs and the shortcomings of simple linear models in capturing the complexity of CAD.

"Furthermore, due to the heterogeneity of CAD as a disease and the genetic complexity of CAD predisposition, simple linear models are destined to fail to produce generalizable risk estimates and connections to multiple personalized interventions." (Page 2)
Critiques Prior Integration Strategies
The section provides a concise yet critical review of previous attempts to integrate genetic and clinical data, outlining why these approaches (linear combinations, brute-force feature inclusion) yielded only marginal improvements and failed to capture necessary interactions for personalization.

"Similarly, combining individual CAD and risk factor PRSs with a wide range of candidate features in a linear manner allows for the identification of key genetic and clinical risk contributors; however, this brute-force strategy again leads to marginal improvements in risk prediction accuracy and fails to identify interactions required for personalized risk reduction recommendations..." (Page 2)
Clearly Introduces the Novel Framework
The Introduction clearly articulates the proposed 'omnigenic, integrative, meta-prediction framework,' explicitly stating its key differentiating features, such as incorporating numerous PRSs, using ML to detect interactions, and integrating predictions for multiple factors.

"Thus, our omnigenic, integrative, meta-prediction framework is differentiated from this previous work by showing generalizable prospective prediction, adopting an omnigenic hypothesis by incorporating numerous PRSs in a unified risk prediction framework, incorporating those PRSs in an ML framework that enables the detection of interactions between risk factors and interventions..." (Page 2)
Defines Framework Goals and Aims
The text successfully outlines the specific goals of the framework, such as disentangling inherited from acquired risk and facilitating individualized risk assessment, thereby setting clear expectations for the study's aims and potential contributions.

"By incorporating both unmodifiable risk predictions (based on age, sex and genetic factors) and modifiable risk predictions (which incorporate measured biomarkers), this framework aims to disentangle inherited from acquired risk, facilitating a more comprehensive and individualized risk assessment." (Page 2)

Suggestions for Improvement

Explicitly State Central Hypothesis
This low-impact improvement would enhance the clarity of the study's central argument. The Introduction section effectively outlines the framework and its goals, but explicitly stating the core hypothesis would provide a sharper focus. Adding a concise hypothesis statement would strengthen the Introduction by clearly articulating the expected outcome of applying the novel framework, directly linking the proposed methodology to its anticipated advantages over existing approaches.

"Thus, our omnigenic46–48, integrative, meta-prediction framework is differentiated from this previous work by showing generalizable prospective prediction, adopting an omnigenic hypothesis by incorporating numerous PRSs in a unified risk prediction framework, incorporating those PRSs in an ML framework that enables the detection of interactions between risk factors and interventions allowing for the detection of personalized interventions... and using a meta-prediction framework that integrates predictions for numerous contributing risk factors and diagnoses at baseline and in the future to make an ultimate prediction about CAD event risk." (Page 2)

Implementation: At the end of the paragraph introducing the omnigenic framework, add a sentence stating the main hypothesis. For example: 'We hypothesize that this omnigenic, integrative, meta-prediction framework will achieve substantially higher accuracy and better risk stratification for incident CAD compared to existing clinical scores and simpler PRS integration methods, particularly by capturing complex interactions.'

Results

Key Aspects

Cohort Definition and Characteristics: The study utilized data from the UK Biobank (UKBB), defining distinct prevalent (N=179,508, 9.02% CAD cases at baseline) and incident (N=160,159, 9.02% CAD cases during 10-year follow-up) cohorts from participants aged 40-69. These large, well-characterized cohorts provided the statistical power for model training and evaluation. The prevalent cohort was used to train baseline prediction models, while the incident cohort was used for training and testing the final 10-year CAD risk prediction model.
Meta-Feature Generation and Feature Selection: A key methodological innovation involved generating 296 'meta-features' by training predictive models on the prevalent cohort for various baseline and future diagnoses/biomarkers, then using these predictions as inputs for the final incident model. Feature selection using SHAP reduced 1,727 initial features (1,431 measured + 296 meta) to a final set of 50, comprising 13 directly measured features (lipids, biomarkers, etc.), 22 Polygenic Risk Scores (PRSs) for CAD and related traits, and 15 derived meta-features (predicting diagnoses like angina, ASCVD, MI, mental illness, sleep duration). This approach integrates diverse information streams.
Meta-Prediction Model Performance (UKBB): The final meta-prediction model demonstrated high performance in predicting 10-year incident CAD risk within the UKBB hold-out test set (N=32,032). It achieved excellent discrimination with an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.84 and an Area Under the Precision-Recall Curve (AUPRC) of 0.38. The model effectively stratified risk, with cumulative 10-year CAD incidence ranging from 0.3% in the lowest predicted risk percentile to 62.9% in the highest.
Superior Performance Compared to Benchmarks: The meta-prediction model significantly outperformed established clinical risk scores (PCE, QRISK3, PREVENT) and previous PRS-based research models. On average, the AUROC was >10% higher (0.84 vs. 0.72-0.79) and the AUPRC was 67% higher (0.38 vs. 0.19-0.29) than benchmarks. Survival analysis yielded a concordance index (C-index) of 0.82, also exceeding benchmarks (0.72-0.78). This superior performance highlights the benefit of the integrative meta-prediction approach.
Improved Risk Reclassification: The model demonstrated substantial improvements in risk reclassification compared to standard clinical scores. Using standard clinical risk thresholds, it achieved Net Reclassification Indices (NRIs) of 0.20 over PCE, 0.14 over QRISK3, and 0.21 over PREVENT. This indicates the model more accurately assigns individuals to appropriate risk categories, correctly identifying higher risk in cases missed by standard scores and lower risk in controls misclassified as high-risk.
Robust Performance Across Subpopulations: Model performance remained superior across various subpopulations stratified by standard risk factors (e.g., age, sex, baseline risk scores, specific biomarkers, PRSs). Notably, the greatest gains in predictive accuracy (measured by relative AUPRC improvement) were observed in groups traditionally considered low-risk (e.g., low PCE/QRISK3/PREVENT scores, females, younger individuals), suggesting the model effectively captures risk driven by factors beyond traditional markers, likely through complex genetic interactions.
Model Explanation via SHAP Values: Model interpretability was explored using SHAP values. The analysis revealed the importance of meta-features, particularly those predicting baseline cardiometabolic diagnoses using unmodifiable factors (genetics, age, sex). Thirteen of the 15 meta-features predicted cardiometabolic conditions or related factors like mental illness and sleep duration. Age, sex, and several CAD PRSs were the most frequent contributors within these meta-features, underscoring the complex interplay of factors driving risk prediction.
Identification and Characterization of CAD Risk Subgroups: Unbiased clustering of individuals based on their SHAP value profiles identified five distinct CAD risk subgroups within the incident cohort cases. These subgroups were significantly differentiated by 44 predictive features, with large effect sizes observed for features like specific CAD PRSs and meta-features predicting baseline ASCVD based on unmodifiable factors. This demonstrates the model's ability to identify patient clusters with potentially different underlying risk drivers beyond just the overall risk score.
Differential Response to Interventions by Genetics and Subgroups: Simulations showed that the predicted benefit from clinical interventions (lowering LDL, HbA1c, SBP) varied based on underlying genetic risk and risk subgroup. Individuals with higher genetic risk for CAD derived greater absolute risk reduction from intensive LDL lowering. Similarly, response to HbA1c and SBP lowering varied by genetic risk (T2D PRS, SBP PRS) and risk subgroup, with some subgroups showing markedly greater benefit. This highlights the framework's potential to guide personalized prevention strategies.
External Validation and Generalizability (All of Us Cohort): The meta-prediction framework's generalizability was confirmed through external validation using a streamlined model trained on UKBB and tested on the diverse All of Us (AoU) cohort. The model maintained high accuracy (AUROC 0.81 in full AoU cohort, 0.78 in sub-cohort with complete data), outperforming conventional scores. Performance was near-equivalent across European, African, and Hispanic ancestry groups in AoU, demonstrating broad applicability.
Performance of Fully Generalizable Genetic Model: To further address potential overfitting and demonstrate robustness, a fully generalizable genetic model was developed, excluding any PRSs derived using UKBB data. This model still achieved strong performance (AUROC 0.80, AUPRC 0.28) and good calibration, outperforming clinical tools and non-UKBB PRS models. This reinforces the validity of integrating comprehensive genetic information, even when restricted to PRSs developed independently of the primary dataset.

Strengths

Clear Cohort Definition
The Results section clearly defines the study cohorts (prevalent and incident) derived from the UK Biobank, providing essential demographic and clinical characteristics, which establishes a solid foundation for understanding the subsequent analyses.

"The final study population after quality filtration and case–control assignment included 339,667 participants aged between 40 and 69 who enrolled in the UK Biobank (UKBB) from 2006 to 2010. Among the participants, 44.95% were male and 55.05% were female, with 9.02% having a diagnosis of CAD." (Page 2)
Systematic Feature Generation and Selection
The process of generating meta-features and selecting the final 50 features for the model is described systematically, detailing the types of features included (measured, PRSs, meta-features) and their origins, enhancing methodological transparency.

"After meta-feature generation, a total of 1,727 features were considered for inclusion in the final 10-year incident CAD risk ML model via feature selection. Initial approximate feature selection resulted in 200 top-ranked predictive features (Supplementary Table 4) of which the 50 most important features were selected for final model inclusion via final model training and Shapley Additive Explanation (SHAP)-based feature selection." (Page 2)
Comprehensive Model Performance Reporting
The performance of the meta-prediction model is thoroughly reported using multiple standard metrics (AUROC, AUPRC, sensitivity, specificity, F1 score, Brier's score) and visualized effectively (Fig 1c-f), allowing for a comprehensive assessment of its predictive power.

"The model had an area under the receiver operating characteristic curve (AUROC) of 0.84 (95% confidence interval (CI) 0.83–0.85) (Fig. 1e) and area under the precision-recall curve (AUPRC) of 0.38 (95% CI 0.36–40) (Fig. 1f)." (Page 2)
Rigorous Comparative Evaluation
The study rigorously compares the meta-prediction model against numerous established clinical risk scores (PCE, QRISK3, PREVENT) and PRS-based models using multiple metrics (AUROC, AUPRC, C-index, NRI, IDI), clearly demonstrating its superior performance.

"The AUROC of the meta-prediction model was on average >10% higher than existing approaches (0.84 versus 0.72–0.79), with AUPRCs 67% higher on average (0.38 versus 0.19–0.29) (Fig. 1e,f)." (Page 4)
Demonstrates Robustness Across Subpopulations
The analysis extends beyond overall performance to demonstrate the model's robustness across various clinically relevant subpopulations (stratified by age, sex, baseline risk scores, biomarkers, PRSs), highlighting its particular strength in identifying risk within traditionally low-risk groups.

"The meta-prediction approach resulted in superior performance across all strata explored, with an average 2.2-fold improvement in CAD event enrichment in the top percentile of CAD risk, an average 1.2-fold improvement in AUROC and an average 2-fold improvement in AUPRC per stratum, as observed for the overall cohort (Fig. 2 and Supplementary Table 9)." (Page 5)
Effective Model Interpretation and Subgroup Analysis
The use of SHAP values for model explanation (Fig 3) and subgroup identification via clustering (Fig 4) provides valuable insights into feature contributions and identifies distinct risk profiles, enhancing model interpretability and potential clinical utility.

"As a first step, we identified risk subgroups in an unbiased manner by clustering individuals by the SHAP values contributing to their overall CAD risk prediction. This approach effectively groups individuals who arrived at their final risk prediction owing to similar risk factor profiles." (Page 6)
Demonstrates Potential for Personalized Intervention
The simulation of clinical interventions (LDL, HbA1c, SBP lowering) and the analysis of differential response based on genetic risk and identified subgroups (Fig 5) effectively demonstrate the framework's potential for personalized risk reduction strategies.

"Across the population overall, for individuals with a PCE ≥ 7.5%, an accepted threshold for the initiation of cholesterol-lowering therapy, the degree of absolute CAD risk reduction increases with increasing CAD genetic risk (Fig. 5a)." (Page 6)
Strong External Validation and Generalizability
The external validation using a diverse cohort (All of Us) and the development of a generalizable genetic model demonstrate the framework's robustness and potential applicability beyond the initial UKBB training data, including across different ethnicities.

"The streamlined model trained on UKBB achieved an AUROC of 0.81, with no loss of accuracy when tested in the full AoU cohort (AUROC 0.81) (Extended Data Fig. 5a)." (Page 6)

Suggestions for Improvement

Explicitly State Rationale for Intervention Thresholds
This low-impact improvement would enhance clarity regarding the intervention simulations. The Results section presents compelling simulations of risk reduction (Fig 5), but the rationale for selecting the specific clinical thresholds (e.g., PCE ≥ 7.5%, HbA1c ≥ 6%, SBP ≥ 140 mmHg) to define the 'at-risk' groups for these simulations isn't explicitly stated here. Briefly mentioning the clinical relevance or guideline basis for these thresholds within this section would provide immediate context for the reader evaluating the intervention analysis. Adding this detail directly supports the interpretation of Figure 5 and the subsequent discussion on differential responses. This enhancement would strengthen the transparency of the simulation methodology presented in the Results.

"The clinical interventions, thresholds used to identify at-risk individuals, perturbed biomarker and their clinical targets are described in Supplementary Table 12." (Page 6)

Implementation: When introducing the analysis shown in Figure 5 (paragraph starting 'Genetics and differential response...'), briefly incorporate the basis for the chosen thresholds. For example: 'Using our trained models, we simulated the influence of risk-reducing interventions... The clinical interventions, clinically relevant thresholds used to identify at-risk individuals (e.g., PCE ≥ 7.5% for statin consideration, HbA1c ≥ 6% indicating prediabetes/diabetes, SBP ≥ 140 mmHg indicating hypertension), perturbed biomarker and their clinical targets are described in Supplementary Table 12.'
Enhance Clinical Interpretation of Risk Subgroups
This medium-impact improvement would enhance the interpretation of the identified risk subgroups. The Results section successfully identifies five distinct CAD risk subgroups using SHAP value clustering (Fig 4a) and details the features differentiating them (Fig 4b, 4c). However, the clinical interpretation or potential phenotypic meaning of these statistically derived subgroups could be briefly elaborated upon here. Connecting the key differentiating features (e.g., high genetic risk via specific PRSs, high ASCVD meta-feature prediction) to potential underlying clinical profiles would help translate the statistical findings into more tangible concepts for the reader within the Results section itself. This addition would strengthen the narrative by providing initial insights into why these subgroups are different, complementing the detailed characteristics in the supplement. This enhancement directly aids in understanding the significance of the subgroup analysis presented in Figure 4.

"Notably, the three CAD PRSs (from PGS000337, PGS003446 and PGS003356) and the meta-feature representing baseline prediction of atherosclerotic cardiovascular disease (ASCVD) onset using only unmodifiable risk factors showed the greatest degree of stratification across the risk subgroups. These features differentiated the subgroups to a greater extent than age or sex as measured by η2 (Fig. 4c)." (Page 6)

Implementation: After describing the features with the largest effect sizes differentiating the subgroups (e.g., end of the paragraph discussing Fig 4c), add a sentence or two providing a brief clinical interpretation. For example: 'These features differentiated the subgroups to a greater extent than age or sex as measured by η2 (Fig. 4c), suggesting these subgroups may represent distinct underlying pathophysiological pathways, such as those driven primarily by inherited atherosclerotic predisposition (high CAD PRS and ASCVD meta-feature subgroups) versus those potentially influenced more by other factors despite similar overall predicted risk.'

Non-Text Elements

Fig. 1| Overview of cohort construction, model development and performance...

Full Caption

Fig. 1| Overview of cohort construction, model development and performance assessment for 10-year incident CAD risk meta-prediction in the UKBB.

Figure/Table Image (Page 3)

First Reference in Text

For incident CAD prediction, we assembled a cohort of 160,159 participants, with 9.02% being diagnosed with CAD after baseline over the 10-year follow-up period (Fig. 1a).

Description

Study Population Origin: This part of the figure (Panel a) visually outlines how the researchers selected the groups of people for their study from a larger dataset called the UK Biobank (UKBB). It starts with an initial pool of 502,414 individuals.
Exclusion Process: The diagram shows that certain individuals were removed (excluded) based on specific criteria. For example, 15,207 people were excluded because they lacked necessary data (like age, sex, or genetic information) or withdrew consent. Another large group of 147,540 was excluded due to issues with their electronic health records (EHR), such as insufficient follow-up time or number of entries, or developing the disease after the study's 10-year timeframe.
Incident CAD Cohort Derivation: After exclusions, the remaining 487,207 individuals with genotype data were divided. One key group formed is the 'incident CAD cohort', consisting of 160,159 participants. This group includes individuals who did not have Coronary Artery Disease (CAD - a condition affecting the heart's blood supply) at the start but were monitored for 10 years. The text specifically mentions this cohort size.
Incident CAD Cohort Composition: Within this incident CAD cohort of 160,159 people, the figure and text state that 14,445 individuals (which corresponds to 9.02% mentioned in the text, though the exact percentage isn't in the figure panel itself) developed CAD during the 10-year follow-up period. The remaining 145,714 served as controls (people who did not develop CAD during that time).
Prevalent CAD Cohort Derivation: Panel 'a' also depicts the derivation of a 'prevalent CAD cohort' (179,508 participants), which includes individuals who already had a CAD diagnosis at the beginning of the study (16,190 cases vs 163,318 controls). This cohort is used for different analyses, like training initial models.

Scientific Validity

Systematic Cohort Definition: The systematic approach to cohort definition and participant exclusion based on predefined criteria (genotype availability, EHR data sufficiency) is appropriate for establishing well-defined prevalent and incident cohorts for prediction modeling.
EHR Data Filtering Criteria: The criteria for EHR data sufficiency (minimum entries, follow-up duration) are explicitly stated (though briefly in the figure, likely detailed in Methods), which adds rigor to the control group definition and minimizes bias from incomplete follow-up.
Methodological Separation of Cohorts: The clear separation of prevalent (existing disease) and incident (new disease) cohorts is methodologically sound, as they serve distinct purposes in model training (baseline prediction vs. prospective prediction).
Statistical Power: The large sample sizes derived (N=160,159 for incident cohort) provide substantial statistical power for the subsequent prediction model development and validation.
Plausibility of Event Rate: The number of incident cases (14,445) relative to the total incident cohort size (160,159) yields an event rate (9.02%) that appears plausible for a 10-year follow-up in this age group, although comparison to UKBB overall rates would be informative.

Communication

Flowchart clarity: The flowchart format effectively illustrates the sequential filtering process applied to the UK Biobank population to derive the study cohorts. The visual flow from the initial large number to the final prevalent and incident cohorts is intuitive.
Numerical transparency: The specific numbers for exclusions and final cohort sizes are clearly presented at each stage, enhancing transparency and allowing readers to understand the scale of data filtration.
Cohort distinction: The distinction between the prevalent and incident cohorts is clearly demarcated, which is crucial for understanding the subsequent modeling approaches.
Exclusion criteria detail: While the main flow is clear, the reasons for exclusion could be slightly more detailed directly within the flowchart, although it's acknowledged that space is limited and details are likely in the Methods section.

Table 1 | Reclassification analysis in the UKBB population

Figure/Table Image (Page 4)

First Reference in Text

Additional demographic and clinical characteristics of the prevalent and incident risk cohorts are available in Table 1.

Description

Purpose of the Table: This table compares a new prediction model ('Meta-prediction') against three existing clinical risk scores (PCE, QRISK3, PREVENT) used to estimate the risk of developing Coronary Artery Disease (CAD).
Method and Population: It uses a method called 'reclassification analysis' performed on a specific group of people from the UK Biobank study (the test set of the incident CAD cohort, involving 2,889 people who developed CAD ('Events') and 29,143 who did not ('No event') over the study period).
Reclassification Concept: Reclassification analysis checks how many individuals are assigned to different risk categories (e.g., low, borderline, intermediate, high risk) by the new model compared to the old scores. The table shows the percentage of individuals moving between these categories.
Example Reclassification Values: For each comparison (Meta-prediction vs. PCE, vs. QRISK3, vs. PREVENT), the table shows percentages of individuals reclassified. For instance, when comparing against PCE, among those who did not have an event, the meta-prediction model moved 23.88% of individuals from the PCE intermediate-risk category (7.5-20%) to its own low-risk category (<5%). Conversely, among those who did have an event, it moved 17.03% from the PCE intermediate-risk category to its own intermediate-risk category.
Net Reclassification Index (NRI): The table calculates a summary statistic called the Net Reclassification Index (NRI). This index quantifies the overall improvement in classification, considering both correct upward movement for cases (events) and correct downward movement for controls (no events), while penalizing incorrect movements. A positive NRI suggests the new model is better at classifying people.
Key NRI Results: The calculated NRI values indicate the meta-prediction model achieved an improvement over the existing scores: NRI = 0.20 compared to PCE, NRI = 0.14 compared to QRISK3, and NRI = 0.21 compared to PREVENT.

Scientific Validity

Appropriateness of Reclassification Analysis: Reclassification analysis is a standard and appropriate method for evaluating the incremental value of a new prediction model or biomarker beyond existing ones, particularly concerning changes in clinical risk categories.
Clinical Relevance of Thresholds: The use of clinically relevant risk thresholds (e.g., 7.5% for PCE/PREVENT, 10% for QRISK3) for defining risk categories makes the analysis directly applicable to potential clinical decision-making scenarios.
Validity of NRI Calculation: Calculating the Net Reclassification Index (NRI) provides a quantitative summary of the improvement, complementing visual inspection of the reclassification tables. The calculation appears correctly applied based on the standard formula (sum of net improvements for events and non-events).
Use of Hold-Out Test Set: The analysis is performed on a hold-out test set (n=32,032 total, split into events and no-events), ensuring the evaluation is independent of the model training data, which is crucial for assessing generalizability.
Consistency of Findings: The results consistently show positive NRI values across comparisons with three different standard scores (PCE, QRISK3, PREVENT), strengthening the conclusion that the meta-prediction model offers improved risk stratification.
Consideration of Metric Limitations: While NRI is informative, it has known limitations (e.g., dependence on the number and choice of categories). The authors supplement this with other metrics (AUROC, AUPRC in Fig 1e,f and continuous NRI/IDI in Supplementary Table 8), providing a more comprehensive evaluation.

Communication

Table Structure: The table structure effectively presents the cross-tabulation required for reclassification analysis, clearly showing movement between risk categories for both events (cases) and no-events (controls).
Use of Standard Thresholds: Using standard clinical risk thresholds (e.g., <5%, 5-7.5%, 7.5-20%, ≥20% for PCE and PREVENT; <10%, 10-20%, ≥20% for QRISK3) facilitates comparison with established clinical practice.
Symbol Clarity: The use of plus (+) and minus (-) signs to indicate favorable and unfavorable reclassification, respectively, is intuitive, although a brief note defining this in the legend would be beneficial.
Data Presentation (Percentages): Presenting percentages within each cell allows for easy interpretation of the magnitude of reclassification within each subgroup.
NRI Presentation: The final Net Reclassification Index (NRI) is clearly presented and highlighted (bolded) for each comparison, summarizing the overall improvement.
Labeling Clarity: The column headers clearly distinguish between 'No event' and 'Event' populations, and the row/column labels specify the risk categories for both the meta-prediction model and the comparator score.
Sample Size Information: The sample sizes for events (n=2,889) and no-events (n=29,143) are provided, adding context to the percentages shown.

Fig. 2 | Comparative performance of meta-prediction stratified by standard risk...

Full Caption

Fig. 2 | Comparative performance of meta-prediction stratified by standard risk factors in the UKBB population.

Figure/Table Image (Page 5)

First Reference in Text

The meta-prediction approach resulted in superior performance across all strata explored, with an average 2.2-fold improvement in CAD event enrichment in the top percentile of CAD risk, an average 1.2-fold improvement in AUROC and an average 2-fold improvement in AUPRC per stratum, as observed for the overall cohort (Fig. 2 and Supplementary Table 9).

Description

Purpose of Comparison: This figure compares the performance of the study's new 'Meta-prediction' model against three existing clinical risk scores (PCE, QRISK3, PREVENT) for predicting Coronary Artery Disease (CAD).
Stratification by Risk Factors: The comparison is not just overall, but is broken down ('stratified') across different subgroups of the UK Biobank population. These subgroups are defined by various standard risk factors, such as age categories (<55 vs ≥55 years), sex (Male vs Female), levels of specific biological measurements (like LDL cholesterol, HbA1c - a measure of blood sugar control, SBP - systolic blood pressure), body measurements (like WHR - waist-hip ratio, BMI - body mass index), and genetic predisposition scores (Polygenic Risk Scores or PRS for CAD, Type 2 Diabetes, LDL, etc.).
Performance Metrics: Performance is evaluated using three different measures shown in three stacked panels: 1) 'Incident CAD in top percentile (%)' shows how concentrated the actual CAD cases are within the group predicted to be at highest risk (top 1%); 2) 'AUROC' (Area Under the Receiver Operating Characteristic curve) measures the model's ability to distinguish between people who will get CAD and those who won't (higher is better, 1.0 is perfect, 0.5 is random chance); 3) 'AUPRC' (Area Under the Precision-Recall curve) is another measure of predictive accuracy, particularly useful when the event (like CAD) is relatively rare (higher is better).
Visual Representation: Visually, for each risk factor stratum (e.g., 'Age <55 years'), there are bars representing the performance of each model (Meta-prediction in blue, others in different colors) according to the three metrics. The figure generally shows that the blue bar (Meta-prediction) is higher than the others across most strata and metrics.
Overall Performance Trend: The text mentions, and the figure visually supports, that the meta-prediction model shows superior performance on average across these different groups, with about a 1.2-fold improvement in AUROC and a 2-fold improvement in AUPRC compared to the other scores.
Highlighted Improvements: Annotations at the bottom highlight specific factors (Age, Sex, CAD-PRS) where the meta-prediction model showed particularly strong relative improvement in AUPRC, indicated by numerical fold changes (e.g., 1.2x for Age, 2.0x for Sex).

Scientific Validity

Importance of Stratified Analysis: Stratifying the performance analysis by known risk factors is a crucial step to assess model robustness and identify potential subgroup-specific performance variations. This goes beyond overall performance metrics and evaluates fairness and applicability across different demographic and clinical profiles.
Relevance of Stratification Factors: The selection of stratification factors covers a relevant range of demographic (age, sex), clinical (PCE, QRISK3, PREVENT scores), biomarker (LDL, HbA1c, SBP), anthropometric (WHR, BMI), and genetic (multiple PRSs) variables pertinent to CAD risk.
Use of Multiple Performance Metrics: Evaluating performance using multiple metrics (enrichment, AUROC, AUPRC) provides a more comprehensive picture than relying on a single metric. AUPRC is particularly important given the likely class imbalance in incident CAD prediction.
Consistency of Results Across Strata: The consistent outperformance of the meta-prediction model across nearly all strata and metrics strongly supports the claim of its superiority and robustness compared to standard clinical scores and PRS-only approaches implicitly compared via stratification.
Identification of High-Impact Subgroups: The analysis highlights subgroups where the meta-prediction model offers particularly large gains (e.g., low-risk strata according to traditional scores, females, low WHR/HbA1c/SBP), suggesting it effectively captures risk missed by conventional methods, which aligns with the authors' interpretation.
Statistical Significance within Strata: While the figure shows clear trends, the statistical significance of the performance differences within each stratum is not directly presented in the figure itself (though likely available in supplementary materials like the referenced Table 9), which would further strengthen the conclusions.

Communication

Multi-metric presentation: The three-tiered structure, presenting enrichment, AUROC, and AUPRC, allows for a multi-faceted comparison of model performance within a single figure.
Comprehensive stratification: Stratifying by a wide range of clinical and genetic risk factors (PCE, age, sex, multiple PRSs, biomarkers like LDL, HbA1c, SBP, anthropometrics like WHR, BMI) effectively demonstrates the model's robustness across diverse subgroups.
Consistent color-coding: The consistent color-coding for the different prediction models (Meta-prediction, PCE, QRISK3, PREVENT) across all strata facilitates easy visual comparison.
High/Low Strata Comparison: Including both high and low strata for each risk factor allows direct comparison of performance differences within subgroups (e.g., performance in low vs. high LDL groups).
Clear labeling of strata: The x-axis labels clearly identify the stratification factor and the specific PRS used (e.g., 'CAD-PRS (PGS003356)', 'LDL PRS (PGS000892)').
Annotations and summary plot: The annotations showing the average fold change in AUPRC for specific factors (Age, Sex, CAD-PRS) and the bubble plot summarizing relative AUPRC differences provide a useful summary, highlighting key findings directly on the figure, although the bubble plot's interpretation requires careful reading of the caption/text.
Information density vs. clarity: While visually dense due to the amount of information, the layout is logical and allows for systematic comparison across factors and metrics.

Fig. 3 | SHAP summary plot of features in the meta-prediction framework in the...

Full Caption

Fig. 3 | SHAP summary plot of features in the meta-prediction framework in the UKBB population.

Figure/Table Image (Page 7)

First Reference in Text

Example SHAP plots for non-CAD or non-CAD component meta-features are provided on the right side of Fig. 3.

Description

Purpose and Examples: This part of Figure 3 provides detailed examples of how specific 'meta-features' (predictions generated by intermediate models) that are not directly related to Coronary Artery Disease (CAD) contribute to the overall CAD risk prediction. The examples shown are predictions for '20-year mental illness', 'Any-onset nonischemic cardiomyopathy (CMP - a disease of the heart muscle)', and 'Present sleep duration'.
SHAP Plot Explanation: Each example uses a SHAP (SHapley Additive exPlanations) plot. SHAP values quantify the impact of each input feature (e.g., age, a specific genetic score) on the prediction output for that specific meta-feature. A positive SHAP value means the feature pushed the prediction higher (e.g., towards predicting mental illness), while a negative value means it pushed it lower.
Component Features: Within each example plot (e.g., '20-year mental illness'), individual features used to predict that specific meta-feature are listed vertically (e.g., 'Recent depression', 'Bipolar-1 PRS', 'Age stopped smoking').
Data Point Representation: Each dot on a plot represents one person in the study cohort (n=33,419). The color indicates the original value of the feature for that person (red=high, blue=low), and its position on the horizontal axis shows its SHAP value (impact on the meta-feature prediction). For example, in the '20-year mental illness' plot, high values (red dots) for 'Recent depression' tend to have positive SHAP values, increasing the predicted likelihood of mental illness.
Integrative Nature Highlighted: These examples demonstrate that the final CAD risk model incorporates information derived from predictions about seemingly unrelated conditions like mental health and sleep patterns, highlighting the integrative nature of the meta-prediction framework.

Scientific Validity

Enhanced Model Interpretability: Using SHAP plots to dissect the contribution of component features to the prediction of specific meta-features (especially non-CAD ones) enhances the interpretability of the complex meta-prediction model. It allows examination of why a certain meta-feature value was predicted.
Evidence for Complex Interactions: Demonstrating that predictions of non-cardiovascular outcomes (mental illness, sleep duration) derived from genetic and clinical data contribute to the final CAD risk prediction provides evidence for the model capturing complex, potentially pleiotropic relationships or shared risk pathways.
Support for Underlying Hypothesis: The inclusion of features like Polygenic Risk Scores (PRSs) for conditions like Bipolar disorder or Insomnia within these meta-feature predictions, and their subsequent influence on CAD risk, supports the 'omnigenic' hypothesis underlying the study design, where many genetic factors contribute indirectly to disease risk.
Validation of Meta-Feature Concept: These examples validate the meta-feature concept, showing that intermediate predictions serve as meaningful inputs to the final model, carrying information beyond the raw features alone.

Communication

Illustrative examples: Presenting specific examples of non-CAD meta-features (20-year mental illness, any-onset nonischemic CMP, present sleep duration) effectively illustrates the concept that factors beyond traditional cardiovascular risks contribute to the final CAD prediction within this framework.
Clarity of individual SHAP plots: The individual SHAP plots for these examples clearly show the distribution of SHAP values for component features (e.g., Bipolar-I PRS, age stopped smoking for mental illness prediction), mirroring the format of the main summary plot and aiding interpretation.
Consistency in presentation: The consistent use of color coding (red for high feature value, blue for low) and axis labeling (SHAP value, feature value) across all panels enhances coherence and readability.
Complementary information: These specific examples effectively complement the main SHAP summary plot by providing deeper insight into the composition and behavior of selected meta-features, particularly those less intuitively linked to CAD.
Clear plot titles: The titles for each example plot clearly state the meta-feature being explained (e.g., "20-year mental illness").

Fig. 4 | Identification of CAD risk subgroups and distinguishing features in...

Full Caption

Fig. 4 | Identification of CAD risk subgroups and distinguishing features in the UKBB population.

Figure/Table Image (Page 8)

First Reference in Text

Clustering of the incident cohort CAD cases revealed five distinct subgroups (Fig. 4a).

Description

Visualization Type: This panel (Fig. 4a) displays a heatmap, which is a graphical representation of data where values are depicted by color. It visualizes the results of grouping individuals who developed Coronary Artery Disease (CAD) during the study.
Clustering Method: The grouping was done using a statistical technique called hierarchical clustering. This method builds a hierarchy of clusters, grouping individuals based on similarity.
Basis for Clustering (SHAP Values): The similarity between individuals was determined by their SHAP values. SHAP (SHapley Additive exPlanations) values quantify how much each specific input feature (like age, blood pressure, genetic scores) contributed to the individual's predicted risk of CAD from the meta-prediction model. Clustering on SHAP values groups individuals who have similar reasons (feature contributions) for their predicted risk.
Heatmap Interpretation: The heatmap colors represent the correlation (statistical relationship) between the SHAP values of different predictive features across the individuals. Blocks of similar color indicate groups of individuals where the contributions of various features to their predicted risk are correlated in a similar way.
Identification of Subgroups: The analysis, as indicated by the reference text and the colored bar below the heatmap, identified five distinct subgroups among the CAD cases. These subgroups are labeled 'Lowest', 'Lower', 'Medium', 'Higher', 'Highest', likely corresponding to different underlying risk profiles or magnitudes.
Dendrograms: Dendrograms (tree-like diagrams) are shown along the top and left sides, illustrating the hierarchical structure produced by the clustering algorithm.

Scientific Validity

Novelty and Validity of Clustering on SHAP Values: Using SHAP values as the input for clustering is an innovative and valid approach. It allows for the identification of subgroups based on the underlying drivers of prediction (feature contributions) rather than just the input features themselves or the final predicted risk value, potentially revealing distinct etiological pathways or risk profiles captured by the model.
Appropriateness of Clustering Algorithm: Hierarchical clustering using Ward's linkage and Euclidean distance (as detailed in Methods) is a standard and appropriate technique for exploratory data analysis and identifying group structures.
Focus on Incident Cases: Clustering specifically on the incident CAD cases is a reasonable strategy to identify potentially different risk profiles among those who develop the disease, which can then be characterized and used to assign controls.
Determination of Cluster Number: Defining the number of clusters (five) via fixed-height tree cutting is a common heuristic but can be somewhat arbitrary. While visually supported by the heatmap structure, exploring the stability of these clusters or using quantitative metrics to determine the optimal number of clusters would enhance rigor.
Foundation for Subsequent Analysis: The identification of distinct subgroups based on model interpretation (SHAP values) provides a foundation for subsequent analyses exploring differential responses to interventions (as shown in Fig. 5), linking model interpretability to actionable insights.

Communication

Heatmap visualization: The heatmap visualization effectively conveys the presence of distinct blocks or patterns representing the identified subgroups based on SHAP value correlations.
Dendrogram clarity: The accompanying dendrograms clearly illustrate the hierarchical relationships derived from the clustering algorithm, showing how individuals and features are grouped.
Subgroup labeling: The color bar at the bottom explicitly labels the five distinct subgroups derived from the clustering, linking the visual patterns in the heatmap to the identified groups.
Interpretability limitations: While visually informative about structure, the heatmap itself requires subsequent analysis (like Fig. 4b, 4c) to understand the specific characteristics that define each subgroup; its standalone interpretability regarding why clusters differ is limited.
Color gradient use: The use of color gradients within the heatmap effectively represents the correlation strength between SHAP values for different predictors across individuals.

Fig. 5 | Benefit of clinical interventions in genetic risk and risk subgroups...

Full Caption

Fig. 5 | Benefit of clinical interventions in genetic risk and risk subgroups in the UKBB population.

Figure/Table Image (Page 9)

First Reference in Text

Similarly, the degree of absolute risk reduction achieved in each CAD subgroup by meeting these targets is presented in Fig. 5d-f,g-i, respectively.

Description

Purpose: Absolute Risk Reduction: These three graphs (panels d, e, and f) show the estimated absolute decrease in the 10-year risk of developing Coronary Artery Disease (CAD) for different groups of people when specific medical interventions are applied.
Population: Risk Subgroups: The groups of people shown are the five distinct CAD risk subgroups (identified in Figure 4 and represented by different colors) found within the UK Biobank population.
Interventions Simulated: Each panel focuses on a different intervention: panel d shows the effect of lowering LDL cholesterol ('bad' cholesterol) to various target levels (35, 55, 70, 100 mg/dl); panel e shows lowering HbA1c (a measure of long-term blood sugar control) to different targets (5%, 5.6%, 6%, 6.5%, 7%); panel f shows lowering systolic blood pressure (SBP) to various targets (100 to 160 mmHg).
Axes Interpretation: The vertical axis represents the percentage point reduction in absolute risk (e.g., a value of -4 means a 4% absolute reduction in risk). The horizontal axis shows the different target levels for the intervention.
Key Observation: Differential Benefit: The lines show that the amount of absolute risk reduction often differs between subgroups. For example, in panel d (LDL lowering), the highest-risk subgroup (red line) shows a much larger absolute risk reduction when targeting very low LDL levels (e.g., ~4% reduction going from LDL 100 to 35) compared to the lowest-risk subgroup (purple line), which shows very little absolute benefit.
Data Representation: The points on the graph represent the median risk reduction for each subgroup at each target level, and the error bars indicate the variability (standard error).

Scientific Validity

Validity of Simulation Approach: Simulating the effect of interventions by modifying the target analyte value within the trained prediction model and observing the change in predicted risk is a valid approach to estimate potential intervention benefits, leveraging the model's learned relationships.
Assessment of Heterogeneity of Benefit: Presenting results stratified by the SHAP-derived risk subgroups allows for the assessment of heterogeneity of treatment effect (as predicted by the model). This directly tests whether the identified subgroups exhibit differential benefits from standard interventions.
Clinical Relevance of Absolute Risk: Focusing on absolute risk reduction is clinically highly relevant, as it directly relates to the number needed to treat and the magnitude of benefit for an individual patient.
Biological Plausibility: The observed pattern, particularly for LDL lowering (panel d), where the highest-risk subgroup (which also has high genetic risk) derives the largest absolute benefit, aligns with biological plausibility and previous findings regarding statin benefits in high-PRS individuals.
Use of Standard Clinical Targets: The use of standard clinical targets for LDL, HbA1c, and SBP makes the simulation results interpretable in the context of current treatment guidelines.
Model-Based Prediction Caveat: While insightful, these are model-based predictions of benefit, not results from actual intervention trials within these subgroups. The validity relies on the accuracy and calibration of the underlying meta-prediction model.

Communication

Visualization of Absolute Risk Reduction: These panels (d, e, f) effectively visualize how the absolute amount of predicted risk reduction varies across the five previously identified CAD risk subgroups (color-coded) when simulating standard clinical interventions (LDL lowering, HbA1c lowering, SBP lowering).
Dose-Response Relationship: Plotting the median risk change with error bars (representing standard error) against different intervention target levels (e.g., target LDL levels of 35, 55, 70, 100 mg/dl) clearly illustrates the dose-response relationship within each subgroup.
Consistent Subgroup Coloring: The consistent color-coding of subgroups, linked back to Fig. 4, allows for easy comparison of intervention benefits across the different risk profiles.
Clarity via Separate Panels: Separating the interventions into three distinct panels (d for LDL, e for HbA1c, f for SBP) maintains clarity and avoids overcrowding.
Axis Labeling: The y-axis clearly labels 'Absolute risk change (%)', making the interpretation of the benefit straightforward.

Extended Data Fig. 1 | Feature importance and SHAP summary for 10-year...

Full Caption

Extended Data Fig. 1 | Feature importance and SHAP summary for 10-year prospective CAD risk prediction in the UK Biobank.

Figure/Table Image (Page 17)

First Reference in Text

The final 50 prioritized features included 13 directly measured features, 22 PRSs and 15 meta-features (Extended Data Fig. 1).

Description

Purpose: Feature Importance: This figure identifies the 50 most important factors ('features') used by the final machine learning model to predict the 10-year risk of developing Coronary Artery Disease (CAD) in the UK Biobank population.
Bar Plot (Overall Importance): It uses two types of visualizations. The left panel is a bar chart that ranks the features from most to least important based on their average impact on the model's prediction across all individuals. Importance is measured by the 'mean absolute SHAP value' - a higher bar means the feature generally had a larger impact (positive or negative) on the risk prediction.
SHAP Summary Plot (Detailed Impact): The right panel is a SHAP (SHapley Additive exPlanations) summary plot. This provides more detail than the bar chart. Each feature is listed vertically. For each feature, the horizontal spread of dots shows how much that feature influenced the prediction for each individual person in the test group (n=33,419). A dot's position indicates the SHAP value: positive values mean the feature increased the predicted risk, negative values mean it decreased the risk. The color of the dot indicates the feature's value for that person (red for high values, blue for low values).
Types of Features: The features shown include different types: directly measured clinical data (like cholesterol HDL ratio, systolic blood pressure - labeled 'biomarkers'), genetic risk scores (Polygenic Risk Scores or 'PRS' for CAD and other conditions), and 'meta-features' (which are themselves predictions from intermediate models, e.g., 'baseline Dx of anyonset revascularization (UFs)' - prediction of having had a revascularization procedure before the study start, based only on unmodifiable factors).
Top Ranked Features: The figure highlights that the most important features are a mix of these types, with meta-features related to prior cardiovascular events or procedures, key biomarkers, and specific CAD PRSs appearing high on the list.

Scientific Validity

Appropriateness of SHAP Method: SHAP values are a well-established and theoretically grounded method for explaining the output of complex machine learning models, particularly tree-based ensembles like XGBoost (used in this study). Using mean absolute SHAP values for ranking overall feature importance is standard practice.
Model Transparency: The identification of the top 50 features provides transparency into the final prediction model, showing which factors drive the risk estimates.
Support for Integrative Approach: The inclusion of diverse feature types (measured clinical data, multiple PRSs for different traits, derived meta-features) among the top 50 supports the study's integrative approach and the hypothesis that combining different data modalities improves prediction.
Clinical and Biological Plausibility: The specific features identified as most important (e.g., meta-features for revascularization/MI, HDL ratio, SBP, specific CAD PRSs) align well with established clinical and genetic knowledge of CAD risk factors, lending face validity to the model.
Directionality of Effects: The SHAP summary plot provides valuable insight into not just the magnitude but also the direction of feature effects (e.g., showing that higher HDL ratio generally lowers predicted risk, as expected).

Communication

Dual Visualization Approach: The figure effectively combines two complementary views: a bar plot showing the overall importance (mean absolute SHAP value) of each feature, ranking them clearly, and a detailed SHAP summary plot showing the distribution and direction of impact for each feature across all individuals.
SHAP Plot Clarity: The SHAP summary plot uses standard conventions (color for feature value, position for SHAP value) which are generally well-understood in the machine learning field, facilitating interpretation of feature effects (e.g., high values of a feature pushing risk prediction higher or lower).
Clear Feature Labeling: Features are clearly labeled on the y-axis, including specific identifiers for PRSs (Polygenic Risk Scores) and indicating whether meta-features are based on unmodifiable (UFs) or modifiable/unmodifiable factors (MUFs).
Feature Type Legend: A legend clearly distinguishes between different feature types (meta-feature, biomarkers, PRS) using color coding in the bar plot, adding another layer of information.
Appropriate Level of Detail: Presenting the top 50 features provides substantial detail about the model's composition without being overwhelming.

Extended Data Fig. 2 | Evaluating the calibration and predictive value of...

Full Caption

Extended Data Fig. 2 | Evaluating the calibration and predictive value of feature categories for the meta-prediction model in the UK Biobank.

Figure/Table Image (Page 18)

First Reference in Text

The final meta-prediction model was also found to be the most calibrated among the existing clinical risk scores, all recalibrated with the same cohort (Extended Data Fig. 2a).

Description

Purpose: Model Calibration Check: This graph (Panel a) is a calibration plot, designed to check how well the risk percentages predicted by different models match the actual percentage of people who experienced the event (developing Coronary Artery Disease, CAD) within the UK Biobank test cohort (n = 33,419).
Axes Definition: The horizontal axis shows the average predicted risk (from 0 to 1, or 0% to 100%) within different groups (usually deciles, or tenths, of predicted risk). The vertical axis shows the actual fraction (percentage) of people within that group who developed CAD.
Ideal Calibration: Ideally, if a model predicts a 10% risk for a group, about 10% of people in that group should actually develop CAD. In a perfectly calibrated model, the plotted points would fall exactly on the diagonal dashed line (labeled 'Perfectly calibrated').
Models Compared: The plot compares the study's 'meta-prediction' model against existing clinical scores (PCE, QRISK3, PREVENT). It also shows versions of these existing scores that were 'recalibrated' using the same study data, meaning they were adjusted to better fit this specific population.
Visual Result: Visually, the points for the 'meta-prediction' model (blue circles) appear closest to the diagonal line compared to the other models (both original and recalibrated), suggesting it is the best calibrated.
Brier Score Comparison: The legend provides Brier scores for each model. The Brier score is a single number summarizing both calibration and discrimination (accuracy); lower scores are better. The meta-prediction model has the lowest Brier score (0.0678), followed by the recalibrated clinical scores (around 0.077), while the original clinical scores have higher Brier scores (0.0775 to 0.0811), quantitatively supporting the visual impression of better calibration for the meta-prediction model.

Scientific Validity

Importance of Calibration Assessment: Calibration is a critical aspect of evaluating prediction models, especially for clinical risk prediction where the absolute predicted risk informs decisions. Assessing calibration via plots and Brier scores is standard and appropriate.
Comparison to Standard Scores: Comparing the new model against established clinical scores (PCE, QRISK3, PREVENT) provides necessary context regarding its performance relative to current standards.
Validity of Recalibration: Recalibrating the existing clinical scores using the same study cohort before comparison is methodologically sound. It ensures that any observed difference in calibration is not simply due to the comparator models being developed in different populations, thus providing a fairer assessment of the meta-prediction model's inherent calibration.
Use of Brier Score: The Brier score provides a proper scoring rule that assesses both calibration and discrimination simultaneously, offering a robust quantitative measure to complement the visual calibration plot.
Implication of Superior Calibration: The finding that the meta-prediction model demonstrates superior calibration suggests that its risk estimates are more reliable across the spectrum of predicted probabilities compared to the standard scores, even after recalibration, enhancing its potential clinical utility.

Communication

Standard Plot Format: The calibration plot format is standard and effectively communicates the agreement between predicted probabilities and observed event frequencies.
Direct Comparison: Plotting multiple models (meta-prediction, PCE, QRISK3, PREVENT, plus recalibrated versions) on the same axes allows for direct visual comparison of their calibration.
Reference Line Clarity: The inclusion of the diagonal line representing perfect calibration provides an immediate visual reference for assessing model performance.
Legend and Brier Score Inclusion: The legend clearly identifies each model and its corresponding marker/color. Including the Brier scores directly in the legend provides a concise quantitative summary alongside the visual plot.
Distinction of Recalibrated Scores: Distinguishing between the original and recalibrated versions of the clinical scores visually reinforces the importance of calibration within the specific study cohort.

Extended Data Fig. 3 | Comparative performance of CAD risk prediction models in...

Full Caption

Extended Data Fig. 3 | Comparative performance of CAD risk prediction models in the UK Biobank.

Figure/Table Image (Page 19)

Meta-prediction of coronary artery disease risk

Table of Contents

Overall Summary

Study Background and Main Findings

Research Impact and Future Directions

Critical Analysis and Recommendations

Section Analysis

Abstract

Key Aspects

Strengths

Suggestions for Improvement

Introduction

Key Aspects

Strengths

Suggestions for Improvement

Results

Key Aspects

Strengths

Suggestions for Improvement

Non-Text Elements