Chemist versus Machine: Traditional Knowledge versus Machine Learning Techniques

Abstract

In the past, traditional chemical heuristics have been very important for the discovery of new materials. Machine learning approaches have started to replace those chemical heuristics in recent years, and they offer new opportunities for materials science. Both approaches are very strongly interconnected.Classical chemical heuristics typically rely on less data than machine learning approaches. There are two different types of machine learning approaches in materials science: one relies on features inspired by classical chemical heuristics, the other one purely relies on relationships within the analyzed data.The growing amount of data offers an opportunity to test traditional chemical heuristics. In combination with machine learning techniques, also new, more data-driven chemical heuristics should be developed. Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together. Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together. Data science (see Glossary), artificial intelligence, and machine learning are nowadays present in all fields of science and technology, including chemistry and materials science. The impact of these techniques is expected to be very large, leading to a new path towards scientific discovery (sometimes called the fourth paradigm in science) [1.Agrawal A. Choudhary A. Perspective: materials informatics and big data: realization of the “fourth paradigm” of science in materials science.APL Mater. 2016; 4053208Crossref Scopus (425) Google Scholar,2.Hey T. et al.The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, 2009Google Scholar]. We will not review here all progress and future directions in the use of machine learning in materials science; we refer interested readers to recent reviews on this topic (e.g., [3.Schmidt J. et al.Recent advances and applications of machine learning in solid-state materials science.NPJ Comput. Mater. 2019; 5: 1-36Crossref Scopus (594) Google Scholar, 4.Butler K.T. et al.Machine learning for molecular and materials science.Nature. 2018; 559: 547-555Crossref PubMed Scopus (1218) Google Scholar, 5.Deringer V.L. et al.Machine learning interatomic potentials as emerging tools for materials science.Adv. Mater. 2019; 311902765Crossref Scopus (160) Google Scholar, 6.Schleder G.R. et al.From DFT to machine learning: recent approaches to materials science–a review.J. Phys. Mater. 2019; 2032001Crossref Scopus (225) Google Scholar]). Instead, we offer a personal perspective on how these new techniques, heavily relying on sophisticated algorithms and large data sets, compete, complement, challenge, and/or benefit from more traditional heuristic approaches. Since the early days of chemistry, scientists have looked for patterns from often limited sets of data. This led to many of today’s widely used chemical heuristics such as the periodic system of elements, electronegativities, and atomic radii. These heuristic models were built by combining the knowledge and creativity of the scientist with simplified physical pictures. For example, Pettifor introduced a completely new chemical scale that enabled separation of structure types of AB compounds within a 2D map. This scale was based on the size, valence, and electronegativity of the constituting atoms [7.Pettifor D.G. A chemical scale for crystal-structure maps.Solid State Commun. 1984; 51: 31-34Crossref Scopus (171) Google Scholar]. We will call this traditional approach classical or chemical heuristics throughout this article (Figure 1, Key Figure). Recently, the traditional heuristic approach has started to be replaced by machine learning techniques. This is not only due to improvements in machine learning methods that are now widely available through open source software but also to an ever-increasing amount of data available mainly through high-throughput ab initio computations [8.Jain A. et al.Commentary: the materials project: a materials genome approach to accelerating materials innovation.APL Mater. 2013; 1011002Crossref Scopus (3051) Google Scholar, 9.Draxl C. Scheffler M. The NOMAD laboratory: from data sharing to artificial intelligence.J. Phys. Mater. 2019; 2036001Crossref Scopus (83) Google Scholar, 10.Curtarolo S. et al.AFLOWLIB.ORG: a distributed materials properties repository from high-throughput ab initio calculations.Comput. Mater. Sci. 2012; 58: 227-235Crossref Scopus (572) Google Scholar, 11.Álvarez-Moreno M. et al.Managing the computational chemistry big data problem: the IoChem-BD platform.J. Chem. Inf. Model. 2015; 55: 95-103Crossref PubMed Scopus (258) Google Scholar]. These machine learning approaches can be divided into two categories. The first approach uses features often based on chemical heuristics (e.g., atomic radii or electronegativity) as inputs to machine learning techniques that can provide a relationship between these features and material properties. Here, machine learning brings new relationships to properties from traditional heuristic descriptors. For instance, machine learning has been used to find a mathematical relationship between well-known atomic features (e.g., atomic radii and ionization potential) to discriminate between wurtzite and rock salt forming binaries [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. The second approach relies only on rules derived from relationships within the data bypassing traditional chemical descriptors. For example, the probability of certain ions to replace each other has been derived in this manner [13.Hautier G. et al.Data mined ionic substitutions for the discovery of new compounds.Inorg. Chem. 2011; 50: 656-663Crossref PubMed Scopus (235) Google Scholar]. No physical/chemical feature such as atomic radius was implied in this study, only the idea that some atoms are more likely to replace other atoms than others. The three different approaches that include traditional chemical heuristics, the extraction of rules between established chemical features using machine learning, and purely data-driven approaches are summarized in Figure 1. The chemical heuristics approaches require less data but are inherently biased towards preconception that could turn out to be incorrect. Conversely, the purely data-driven approach could require data set sizes that are sometimes not available. Many of the heuristics that are still taught in general chemistry courses date back from at least a century ago. The concept of electronegativity was proposed by Avogadro and Berzelius in ~1809 [14.Jensen W.B. Electronegativity from Avogadro to Pauling: part 1: origins of the electronegativity concept.J. Chem. Educ. 1996; 73: 11Crossref Google Scholar], oxidation states by Wöhler (~1835) [15.Karen P. Oxidation state, a long-standing issue!.Angew. Chem. Int. Ed. 2015; 54: 4716-4726Crossref PubMed Scopus (62) Google Scholar], atomic radii by Loschmidt (~1866) [16.Rahm M. et al.Atomic and ionic radii of elements 1-96.Chem. Eur. J. 2016; 22: 14625-14632Crossref PubMed Scopus (134) Google Scholar], and the periodic table of elements dates 150 years back to Mendelejew [17.Mendelejew D. Über die Beziehungen der Eigenschaften zu den Atomgewichten der Elemente.Z. Chem. 1869; 12: 405-406Google Scholar]. Goldschmidt and Pauling derived rules on the stability of crystal structures more than 90 years ago [18.Pauling L. The principles determining the structure of complex ionic crystals.J. Am. Chem. Soc. 1929; 51: 1010Crossref Scopus (1394) Google Scholar,19.Goldschmidt V.M. Die Gesetze der Krystallochemie.Naturwissenschaften. 1926; 14: 477-485Crossref Scopus (1895) Google Scholar]. Without any doubt, chemical heuristics have been instrumental to advances in chemistry and materials sciences over the last 100 years. However, one must be careful to not use these heuristics blindly. Their historical importance should not exclude them from critical assessment. Rahm and colleagues have recently shown that two of our most fundamental chemical heuristics, the periodic table and electronegativities, can change drastically at high pressures [20.Rahm M. et al.Squeezing all elements in the periodic table: electron configuration and electronegativity of the atoms under compression.J. Am. Chem. Soc. 2019; 141: 10253-10271Crossref PubMed Scopus (73) Google Scholar]. Here, we face the common issue of extrapolation. For example, Li changes its valence and becomes a p group element at 300 gigapascal (GPa). K and heavier alkali metals become transitional metals. Furthermore, Na becomes the most electropositive s1 element. Thus, the structure of the periodic table and chemical reactivities will change completely for some elements with rising pressure. Similar changes should be expected for typical oxidation states. Chemical heuristics might have only limited transferability, needing to be adapted to allow for the exploration of more extreme conditions. Even in more common conditions, very well-established rules can turn out to be less powerful than expected when evaluated with modern techniques and larger data sets. Hautier and colleagues [21.George J. et al.The limited predictive power of the Pauling rules.Angew. Chem. Int. Ed. 2020; 59: 7569-7575Crossref PubMed Scopus (21) Google Scholar] have recently evaluated the predictive power of the Pauling rules, which connects the coordination environments of a crystal with its stability. For instance, the first Pauling rule links the preferred coordination environment of a cation to the cation–anion ratio (Figure 2A ). This analysis on oxides shows that Pauling’s first rule is only fulfilled for 66% of all tested local environments, with important deviations in alkali and alkali–earth chemistries, for instance (Figure 2A). Strikingly, and despite being a corner stone of solid-state chemistry, only 13% of a data set of 5000 oxides fulfil the four other Pauling rules [21.George J. et al.The limited predictive power of the Pauling rules.Angew. Chem. Int. Ed. 2020; 59: 7569-7575Crossref PubMed Scopus (21) Google Scholar]. This new assessment could only be performed using modern tools of information technology. First, a diverse set of oxide crystal structures had not only to be determined by crystallographers but also made readily available in databases such as the Inorganic Crystal Structure Database (ICSD), the or the Database et al.The 2016; Scopus Google Scholar, D. et al.Recent in the crystal structure crystal structure data and 2019; PubMed Scopus Google Scholar, S. et Database an of crystal PubMed Scopus Google Scholar]. Without these of the many of today’s data-driven not be Furthermore, tools for the of coordination environments had to be D. et analysis of coordination environments in Mater. Scopus Google D. et a and coordination environment 2020; Scopus Google Scholar] and a assessment of the Pauling rules had to be set [21.George J. et al.The limited predictive power of the Pauling rules.Angew. Chem. Int. Ed. 2020; 59: 7569-7575Crossref PubMed Scopus (21) Google Scholar]. In today’s of to information and is to that only crystal structures had been determined when Pauling proposed rules and that was available to the data Data and in the crystal structure 1996; PubMed Scopus Google Scholar]. In a of established rules, and have shown that a and Goldschmidt that the ionic radii of the ions a structure to the stability of has predictive power (Figure The of Sci. S. A. 2018; PubMed Scopus Google Scholar]. The Goldschmidt is based on the idea of a structure of ionic the idea of the and in their that were in the of based on the idea of a high stability of This can be used to with a high to the of structure available to and was larger than the one available to Goldschmidt in on this and 90 Since the of most chemical rules, the amount of data available and power have is the for the to test chemical heuristics more widely and This their in less common (e.g., high and/or and new rules the of heuristics but using modern machine learning techniques and larger data sets. the data in Figure 1, approach has been to use concepts from the classical heuristics as features or for machine learning of materials properties (Figure [3.Schmidt J. et al.Recent advances and applications of machine learning in solid-state materials science.NPJ Comput. Mater. 2019; 5: 1-36Crossref Scopus (594) Google Scholar]. The material is here by traditional physical and chemical features such as or atomic and machine learning models that these to materials properties are on large data sets. back to the of the Pauling rules, such approaches for instance, from (e.g., local environments or atomic but through machine learning for relationships that from these materials properties such as the and and properties have been machine in this [3.Schmidt J. et al.Recent advances and applications of machine learning in solid-state materials science.NPJ Comput. Mater. 2019; 5: 1-36Crossref Scopus (594) Google Scholar]. These can to crystal structure For instance, have shown that could be from and crystal structure including traditional heuristic such as atomic in the periodic Pettifor or (Figure et chemical can and of Mater. Scopus Google et properties of using machine 2020; Scopus Google of the for Machine The shows the for the of different properties. different types of are they are based on the chemical properties of the as by and based on the local atomic each with from et chemical can and of Mater. Scopus Google Scholar]. Chemical between rock salt and structures of AB materials as a of the 2D that was in [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. Here, the the compounds rock salt and structure within a 2D with from [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. The is under a Figure The shows the for the of different properties. different types of are they are based on the chemical properties of the as by and based on the local atomic each with from et chemical can and of Mater. Scopus Google Scholar]. Chemical between rock salt and structures of AB materials as a of the 2D that was in [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. Here, the the compounds rock salt and structure within a 2D with from [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. The is under a materials based on were the in the is a growing set of methods that very For example, structure have shown very T. Crystal for an and of material Rev. Lett. 2018; PubMed Scopus Google C. et as a machine learning for and Mater. 2019; Scopus Google Scholar]. For instance, crystal learning of different properties (e.g., or for a of structure types and T. Crystal for an and of material Rev. Lett. 2018; PubMed Scopus Google Scholar]. These crystal are based on a of atoms as and as within a information on the and that are based on chemical heuristics (e.g., in periodic table or are in and feature of this a is built that extraction of the of properties. In a and colleagues introduced to and within machine learning models of material properties C. et as a machine learning for and Mater. 2019; Scopus Google Scholar]. There is an of chemical features to use and have started to group them in and databases L. et an open source for materials data Mater. Sci. 2018; Scopus Google Scholar]). However, not all are to a chemical knowledge could (e.g., the Pauling rules could to on atomic radii and local environments to material is also to find the from data The approach is here to with many or features and the most This is a common in many machine learning methods [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google L.M. et physical for materials science by J. Phys. Scopus Google et a for the best in an of Rev. Mater. 2018; Scopus Google Scholar]. from and colleagues [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar] the importance of the by a of AB binaries between and rock salt the least and which in a large of typical atomic the which be the most predictive and established a relationship to be used for (Figure Here, they that atomic ionization electron and radii the probability of the valence or p show its were the most on a large of Figure also that the relationship between the and the is can also be with on crystal structure et al.Machine learning materials properties for 2020; Scholar]. The importance of feature for these has been shown to be critical when data sets are of data The approach of machine learning is to extract relationships between the data using the of chemical or For instance, one to the crystal structure and chemical elements on the to properties. This is as not require knowledge a of features or be but more data (Figure of such a data-driven approach from the of crystal structure on atomic radii and in the periodic have which ions can each other for more than 100 years. In recent years, a based on machine learning that can how likely certain can each other has been [13.Hautier G. et al.Data mined ionic substitutions for the discovery of new compounds.Inorg. Chem. 2011; 50: 656-663Crossref PubMed Scopus (235) Google Scholar]. the a probability the for two ions or elements to each No is made on chemical features in the periodic these The of between two ions is as a in Figure This was used and with to new or materials et for for 2018; Scopus Google Scholar, et a new of materials for Mater. 2012; Scopus Google Scholar, et of the Mater. 2019; PubMed Scopus Google Scholar] that were of those substitutions were and not on chemical For instance, the a with with and with to a new which was et for for 2018; Scopus Google Scholar]. data-driven models are traditional physical models is interatomic potentials V.L. et al.Machine learning interatomic potentials as emerging tools for materials science.Adv. Mater. 2019; 311902765Crossref Scopus (160) Google et the of the Rev. Lett. PubMed Scopus Google J. M. of Rev. Lett. PubMed Scopus Google Scholar]. such as potentials S. the of molecular the of of a Phys. Sci. Google Scholar] or models et al.The a review of and Sci. Scopus Google Scholar] have physical machine learning potentials these by and to crystal structure their In a they are purely data-driven with physical these a large set of data (e.g., from is There are types of these machine learning interatomic they by the used to the structure and the of are the potentials by and a of atomic is and the by are used et the of the Rev. Lett. PubMed Scopus Google J. M. of Rev. Lett. PubMed Scopus Google et chemical Rev. 2013; Scopus Google J. for Chem. Phys. 2011; PubMed Scopus Google Scholar]. The atomic with the of a local of atomic The by these potentials are of many atoms (e.g., C. et learning of the of of 2020; Scopus (21) Google T. et interatomic for structures and their in Rev. Mater. 2020; Scopus Google Scholar] or et interatomic for the change material Rev. 2012; Scopus Google V.L. G. Machine learning based interatomic for Rev. Scopus Google or accelerating the for new crystal structures V.L. et learning and of crystal 2018; PubMed Google Scholar]. is expected that they will impact many fields and they have been used in the of and materials V.L. and materials with Phys. 2020; Scopus Google J. et with high in Chem. Phys. 2020; PubMed Scopus Google Scholar]. these data-driven approaches are they not rely on that could turn out to be we should in that they could well-established chemical from a for materials the A. et of algorithms in for Mater. Sci. 2013; Scopus Google Scholar]. a a all was on stability but also and This was purely data-driven using heuristics (Figure best The performed on the data including from typical chemical the oxidation states in one material should to only materials with an of in the are to metals and the classical Goldschmidt is used for of the materials. The based on traditional chemical heuristics was as the algorithms (Figure the machine had been the or more relearning rules such as is the combination of data-driven and heuristics that to the best the two approaches of the relationships that might be from traditional knowledge (Figure of the approaches combination of the purely data-driven approach with heuristic features as and the to on the amount of traditional heuristic knowledge only atomic and information to the of more complex chemical used in the models on the data C. et as a machine learning for and Mater. 2019; Scopus Google Scholar]. Chemical heuristics and machine learning are in the to models patterns in data. of traditional chemical heuristics and machine learning only as we they on each Machine learning approaches can benefit from chemical heuristics as they provide features to materials (see The traditional chemical rules also provide for machine learning should be to not only their machine learning models by but also the and of artificial with traditional In our machine learning a only when traditional approaches. the of a new is to the rule of one can the in knowledge by the machine learning Chemical heuristics should also be challenged by machine learning and large-scale statistical The historical of a heuristic rule should not its critical assessment. In should that many traditional concepts and rules in chemistry have been on and be The of information an opportunity for to back to the and new rules based on large with the of machine learning the of materials science as from machine learning as from traditional chemical chemical heuristics are typically taught in chemistry courses in and are to our chemical Machine learning should allow for a to chemical as these traditional chemical of materials biased by traditional chemical compounds have been based on and traditional chemical heuristics. might be that materials also based on these are biased based on the use of traditional chemical heuristics their we that allow for classical chemical scientific progress based on machine learning only be made those approaches classical chemical heuristics. are to those approaches with each the of materials science as from machine learning as from traditional chemical chemical heuristics are typically taught in chemistry courses in and are to our chemical Machine learning should allow for a to chemical as these traditional chemical heuristics. of materials biased by traditional chemical compounds have been based on and traditional chemical heuristics. might be that materials also based on these are biased based on the use of traditional chemical heuristics their we that allow for classical chemical scientific progress based on machine learning only be made those approaches classical chemical heuristics. are to those approaches with each from the and under the We for on the Figure was with the of for of and 2011; Scopus Google Scholar]. of Machine learning can be a of artificial that in and a crystal the atoms their and the the between the and a built on of the crystal The of and which the feature of the crystal by the feature of on algorithms and principles to extract patterns from large of data. is to the fields of machine learning and data and these terms are sometimes used the stability of with structure types based on the ionic radii of ions constituting the crystal a with a in the of an which and of the in the algorithms that can large data sets and is an important of data science. a they the of to the local environments of and the those local environments are

Topics & Concepts