Enhancing decision support in crop production: Analyzing conformal prediction for uncertainty quantification

Abstract

Assessing the confidence of machine learning models is increasingly important for enhancing decision support in digital agriculture. Addressing this challenge involves identifying and classifying sources of uncertainty and applying a suitable quantification method. To fully realize the potential of uncertainty quantification in digital agriculture, it is necessary to explore and evaluate different techniques, demonstrating their effectiveness in diverse scenarios. In this paper, we focus on the conformal prediction (CP) framework, specifically inductive conformal prediction (ICP), as a non-parametric tool for uncertainty quantification. Conformal prediction has gained a foothold for its multiple advantages, including being model-agnostic – requiring no retraining or changes to model architecture – and computationally efficient. Moreover, CP operates under a distribution-free framework, making no assumptions about the data, and provides calibrated uncertainty estimates where empirical errors match pre-defined theoretical levels. These properties make it particularly well-suited for agricultural tasks, where datasets often exhibit variability across regions and lack reliable ground truth labels. Through ablation studies, we comprehensively analyze ICP’s performance as a post-hoc approach for ResNet-18 and ViT-B/16, demonstrating that both achieve the required pre-defined error levels. We compare ICP performance against other methods in diverse agricultural tasks, including out-of-distribution detection and covariate shift. Our experiments highlight CP’s unique benefits, such as validity, efficiency, and low computational overhead, making it a promising approach for developing more reliable agricultural machine learning systems.

Topics & Concepts