Code Summarization without Direct Access to Code - Towards Exploring Federated LLMs for Software Engineering
Jahnavi Kumar, Sridhar Chimalakonda
Abstract
Software Engineering (SE) researchers are extensively applying Large Language Models (LLMs) to address challenges in SE tasks such as code clone detection, code summarization, and program comprehension. Despite promising results, LLMs have to be fine-tuned and customized with specific datasets for optimal performance. However, the proprietary nature of SE data, and the lack of LLMs trained on non-open source data is an open problem. While there exists work on applying Federated Learning (FL) for SE, integration of FL with LLMs for SE is unexplored. Hence, we propose a FedLLM for “code summarization” as developers spend more time in comprehending code. We setup a federated learning architecture and fine-tune LLM (Llama2 with 6.7B parameters) using Parameter Efficient Fine-Tuning (PEFT) for code summarization. We conducted our experiments on 40GB RAM GPU in an A100 architecture. Results show that FL-trained LLM is as effective as a centrally-trained one. We envision that leveraging non-open source data using FedLLM for SE could be an interesting research direction.