Anthropogenic biases in chemical reaction data hinder exploratory inorganic synthesis

• Published in: Nature (London) 2019-09, Vol.573 (7773), p.251-255
• Main Authors: Jia, Xiwen, Lynch, Allyson, Huang, Yuheng, Danielson, Matthew, Lang’at, Immaculate, Milder, Alexander, Ruby, Aaron E., Wang, Hao, Friedler, Sorelle A., Norquist, Alexander J., Schrier, Joshua
• Format: Article
• Language: English
• Subjects: 639/638/263/915; 639/638/549; 639/638/630; 639/705/1046; 706/648/697; Analysis; Anthropogenic factors; Artificial intelligence; Bias; Chemical reactions; Chemical synthesis; Chemistry Techniques, Synthetic - statistics & numerical data; Cognitive ability; Crystal structure; Crystallization; Data mining; Datasets; Design of experiments; Human beings; Human influences; Humanities and Social Sciences; Humans; Hydrothermal crystal growth; Influence; Influence on nature; Laboratory Personnel - psychology; Laboratory Personnel - statistics & numerical data; Learning algorithms; Letter; Machine Learning; Methods; multidisciplinary; Organic chemistry; Oxides; Pharmaceutical sciences; Popularity; Power law; Problem solving; Reagents; Research bias; Science; Science (multidisciplinary); Statistics; Success
• ISSN: 0028-0836; 1476-4687
• DOI: 10.1038/s41586-019-1540-5

Most chemical experiments are planned by human scientists and therefore are subject to a variety of human cognitive biases 1 , heuristics 2 and social influences 3 . These anthropogenic chemical reaction data are widely used to train machine-learning models 4 that are used to predict organic 5 and inorganic 6 , 7 syntheses. However, it is known that societal biases are encoded in datasets and are perpetuated in machine-learning models 8 . Here we identify as-yet-unacknowledged anthropogenic biases in both the reagent choices and reaction conditions of chemical reaction datasets using a combination of data mining and experiments. We find that the amine choices in the reported crystal structures of hydrothermal synthesis of amine-templated metal oxides 9 follow a power-law distribution in which 17% of amine reactants occur in 79% of reported compounds, consistent with distributions in social influence models 10 – 12 . An analysis of unpublished historical laboratory notebook records shows similarly biased distributions of reaction condition choices. By performing 548 randomly generated experiments, we demonstrate that the popularity of reactants or the choices of reaction conditions are uncorrelated to the success of the reaction. We show that randomly generated experiments better illustrate the range of parameter choices that are compatible with crystal formation. Machine-learning models that we train on a smaller randomized reaction dataset outperform models trained on larger human-selected reaction datasets, demonstrating the importance of identifying and addressing anthropogenic biases in scientific data. Human scientists make unrepresentative chemical reagent and reaction condition choices, and machine-learning algorithms trained on human-selected experiments are less capable of successfully predicting reaction outcomes than those trained on randomly generated experiments.