The role of explainability and human intervention in AI decisions: jurisdictional and regulatory aspects

Abstract

As artificial intelligence (AI) systems are increasingly applied in critical domains such as healthcare, finance, and criminal justice, the need for explainability and meaningful human oversight has become a pressing legal and ethical concern. The inherent opacity of complex AI models, often described as the ‘black box problem,’ raises significant challenges in ensuring accountability, fairness, and due process when automated decisions impact individual rights and opportunities. This paper critically examines the legal frameworks governing AI explainability and human intervention across selected jurisdictions, including the European Union, the United States, the United Kingdom, Singapore, and Malaysia. It analyses the contrasting regulatory approaches, such as the enforceable rights under the EU AI Act and GDPR, compared with the more fragmented and sectoral models in the United States. The paper explores sector-specific instances of algorithmic discrimination in lending, diagnostic errors in healthcare, and wrongful arrests linked to facial recognition to illustrate the regulatory gaps and practical risks of insufficient transparency. It identifies key challenges such as limited enforcement mechanisms, legal ambiguity, the trade-off between accuracy and interpretability, and cross-jurisdictional inconsistencies. The paper concludes with policy recommendations that include strengthening statutory obligations for explainability, establishing algorithmic audit frameworks, developing transparency registries, and promoting greater international coordination in AI governance.

Topics & Concepts