Evaluating the performance and robustness of LLMs in materials science Q&A and property predictions
Hongchen Wang, Kangming Li, Scott Ramsay, Yao Fehlis, Edward Kim, Jason Hattrick‐Simpers
Abstract
We evaluate LLM performance and robustness for materials science Q&A and property prediction. Prompt sensitivity and mode collapse reveal reliability concerns, providing informed skepticism and guiding more cautious adoption in scientific research.
Topics & Concepts
Robustness (evolution)Property (philosophy)EconomicsBiologyGeneticsEpistemologyPhilosophyGeneMachine Learning in Materials ScienceSoftware Engineering ResearchAdvanced Materials Characterization Techniques