Towards trustworthy artificial intelligence for decision-making: A lifecycle perspective on knowledge- and data-driven artificial intelligence systems
Emiel Miedema, Sabine Waschull, Christos Emmanouilidis
Abstract
Organisations increasingly use data-driven artificial intelligence (AI) systems in their decision-making processes. These AI systems may operate autonomously, support human decision-makers or increasingly act as collaborative team members. However, data-driven AI systems often function as black boxes, lacking interpretability. This poses a challenge in decision-making, as stakeholders involved in or impacted by the decision-making process frequently need to understand the rationale behind decisions. Moreover, data-driven AI systems operate without leveraging structured domain knowledge. As a result, data-driven AI systems may generate outputs that are misaligned with the decision context, objectives, or constraints, potentially leading to poor decisions or reduced trust in AI systems among users. Consequently, recent years have seen an increasing interest in integrating domain knowledge with data-driven AI. This is evident in neuro-symbolic AI, a subfield of AI that combines neural networks with symbolic AI. While this approach shows promise for enhancing the trustworthiness of AI systems in decision-making, the specific mechanisms by which domain knowledge integration contributes to dimensions of trustworthiness remain insufficiently explored. Therefore, this study reviews and integrates recent knowledge- and data-driven AI literature, along with relevant concepts for decision-making. Building on this foundation, it proposes a lifecycle framework for integrated knowledge- and data-driven AI systems for decision-making, and demonstrates its application through a healthcare application example. It further analyses the dimensions of trustworthiness for knowledge- and data-driven AI systems using the proposed lifecycle framework and application example. In doing so, this study advances the discourse on trustworthy AI for decision-making. • Literature review on knowledge- and data-driven AI systems for decision-making. • Introduces a lifecycle framework for knowledge- and data-driven AI systems. • Conceptual analysis of trustworthiness of knowledge- and data-driven AI systems.