How well do DeepSeek, ChatGPT, and Gemini respond to water science questions?
Seyed Hossein Hosseini, Ali Pourzangbar
Abstract
This study aims to evaluate the performance of three prominent LLMs, DeepSeek R1, ChatGPT-4o, and Gemini 2, in addressing key questions within four core fields of hydrology and water science: machine learning and optimization, remote sensing, flood modeling, and sediment transport. LLMs’ responses are systematically compared to benchmark responses derived from review articles in the respective fields. To assess the LLMs’ efficiency, a novel evaluation rubric is introduced in this study, incorporating four key criteria: relevancy, accuracy, authenticity, and novelty. Findings revealed that each model can address the core aspects of the benchmark questions. DeepSeek R1 achieved the highest overall scores in machine learning and optimization, flood modeling, and sediment transport, while ChatGPT-4o demonstrated superior performance in remote sensing. Notably, DeepSeek R1 and Gemini 2 exhibited the lowest response similarity in 95% of the evaluated questions, whereas ChatGPT-4o and Gemini 2 showed the highest similarity in 70% of cases.