Ditto: Efficient Serverless Analytics with Elastic Parallelism
Chao Jin, Zili Zhang, Xingyu Xiang, Songyun Zou, Gang Huang, Xuanzhe Liu, Xin Jin
Abstract
Serverless computing provides fine-grained resource elasticity for data analytics---a job can flexibly scale its resources for each stage, instead of sticking to a fixed pool of resources throughout its lifetime. Due to different data dependencies and different shuffling overheads caused by intra- and inter-server communication, the best degree of parallelism (DoP) for each stage varies based on runtime conditions.
Topics & Concepts
Computer scienceShufflingAnalyticsParallelism (grammar)Elasticity (physics)Parallel computingData analysisJob schedulerDistributed computingCloud computingOperating systemDatabaseData miningProgramming languageMaterials scienceComposite materialCloud Computing and Resource ManagementDistributed and Parallel Computing SystemsIoT and Edge/Fog Computing