Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey
Yang Gu, Hengyu You, Jian Cao, Muran Yu, Haoran Fan, Shiyou Qian
Abstract
Machine Learning (ML) workflows—spanning data preprocessing and feature engineering, model selection and hyperparameter optimization, and workflow evaluation—are increasingly embedded in complex software systems. Building these workflows manually demands substantial ML expertise, domain knowledge, and engineering effort. Automated ML (AutoML) frameworks address parts of this challenge but often suffer from constrained search spaces, limited adaptability, and low interpretability. Recent advances in Large Language Models (LLMs) have opened new opportunities to automate and enhance ML workflows by leveraging their capabilities in language understanding, reasoning, interaction, and code generation, posing new practical and theoretical challenges for software engineering (SE). This survey provides the first SE-oriented, stage-wise review of LLM-based ML workflow automation. We introduce a taxonomy covering all three workflow stages, systematically compare and analyze state-of-the-art methods, and synthesize both stage-specific and cross-stage trends. Our analysis yields SE-oriented implications, including the need for robust verification, quality management, context-aware deployment, and risk mitigation, alongside ensuring key quality attributes such as usability, modularity, traceability, and performance. The findings also call for adapting development models, rethinking lifecycle boundaries, and formalizing uncertainty handling to address the probabilistic and collaborative nature of LLM-assisted workflow generation. We further identify major open challenges and outline future research directions to guide the reliable and effective adoption of LLMs in ML workflow development. Our artifacts are publicly available at https://github.com/t-harden/LLM4AutoML .