Finding the right cloud configuration for analytics clusters
Muhammad Bilal, Marco Canini, Rodrigo Rodrigues
Abstract
Finding good cloud configurations for deploying a single distributed system is already a challenging task, and it becomes substantially harder when a data analytics cluster is formed by multiple distributed systems since the search space becomes exponentially larger. In particular, recent proposals for single system deployments rely on benchmarking runs that become prohibitively expensive as we shift to joint optimization of multiple systems, as users have to wait until the end of a long optimization run to start the production run of their job.
Topics & Concepts
Computer scienceCloud computingBenchmarkingAnalyticsDistributed computingCluster (spacecraft)Task (project management)Data scienceOperating systemEngineeringSystems engineeringBusinessMarketingCloud Computing and Resource ManagementData Stream Mining TechniquesIoT and Edge/Fog Computing