Benchmarking Accuracy and Generalizability of Four Graph Neural Networks Using Large In Vitro ADME Datasets from Different Chemical Spaces

In this work, we benchmark a variety of single‐ and multi‐task graph neural network (GNN) models against lower‐bar and higher‐bar traditional machine learning approaches employing human engineered molecular features. We consider four GNN variants – Graph Convolutional Network (GCN), Graph Attention...

Full description

Saved in:
Bibliographic Details
Published in:Molecular informatics 2022-08, Vol.41 (8), p.e2100321-n/a
Main Authors: Broccatelli, Fabio, Trager, Richard, Reutlinger, Michael, Karypis, George, Li, Mufei
Format: Article
Language:English
Subjects:
Accuracy
ADME
Benchmarks
Datasets
Deep learning
Fingerprints
graph neural network
Graph neural networks
in vitro assays
Laboratory tests
Machine learning
Message passing
multi-task learning
Neural networks
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this work, we benchmark a variety of single‐ and multi‐task graph neural network (GNN) models against lower‐bar and higher‐bar traditional machine learning approaches employing human engineered molecular features. We consider four GNN variants – Graph Convolutional Network (GCN), Graph Attention Network (GAT), Message Passing Neural Network (MPNN), and Attentive Fingerprint (AttentiveFP). So far deep learning models have been primarily benchmarked using lower‐bar traditional models solely based on fingerprints, while more realistic benchmarks employing fingerprints, whole‐molecule descriptors and predictions from other related endpoints (e. g., LogD7.4) appear to be scarce for industrial ADME datasets. In addition to time‐split test sets based on Genentech data, this study benefits from the availability of measurements from an external chemical space (Roche data). We identify GAT as a promising approach to implementing deep learning models. While all the deep learning models significantly outperform lower‐bar benchmark traditional models solely based on fingerprints, only GATs seem to offer a small but consistent improvement over higher‐bar benchmark traditional models. Finally, the accuracy of in vitro assays from different laboratories predicting the same experimental endpoints appears to be comparable with the accuracy of GAT single‐task models, suggesting that most of the observed error from the models is a function of the experimental error propagation.
ISSN:1868-1743
1868-1751
DOI:10.1002/minf.202100321