Modernizing Clemson University's Palmetto Cluster: Lessons Learned from 17 Years of HPC Administration
Asher Antao, James D. Burton, Douglas Dawson, Jill Gemmill, Zachary Gerstener, Ben Godfrey, Scott Groel, Zach Jordan, Becky Ligon, Dane Smith, Matthew Moersen, Sai Vinay Nandigam, Nicholas Hayden Rengier, Dennis Ryans, D. Hudson Smith, T. H. Wright
Abstract
Since its inception in 2007, the High Performance Computing (HPC) program at Clemson University has experienced significant growth. This paper aims to provide a comprehensive overview of the evolution and current capabilities of our HPC cluster, Palmetto, while also sharing the valuable lessons learned over 17 years of its administration. Specifically, we will discuss the application of these lessons to the design of our next-generation cluster, Palmetto 2, with a focus on enhancing data portability, improving networking, optimizing job scheduling, streamlining software management, and implementing more efficient user-support systems. It is our hope that the insights gained from our experiences will be beneficial to other cyberinfrastructure support groups seeking to modernize their HPC capabilities.