Jiffy
Anurag Khandelwal, Yupeng Tang, Rachit Agarwal, Aditya Akella, Ion Stoica
Abstract
Stateful serverless analytics can be enabled using a remote memory system for inter-task communication, and for storing and exchanging intermediate data. However, existing systems allocate memory resources at job granularity---jobs specify their memory demands at the time of the submission; and, the system allocates memory equal to the job's demand for the entirety of its lifetime. This leads to resource underutilization and/or performance degradation when intermediate data sizes vary during job execution.
Topics & Concepts
Stateful firewallComputer scienceGranularityTask (project management)Memory managementTask managementDistributed computingOperating systemComputer networkOverlayEconomicsNetwork packetManagementCloud Computing and Resource ManagementDistributed and Parallel Computing SystemsAdvanced Data Storage Technologies