Achieving Microsecond-Scale Tail Latency Efficiently with Approximate Optimal Scheduling
Rishabh Iyer, Musa Unal, Marios Kogias, George Candea
Abstract
Datacenter applications expect microsecond-scale service times and tightly bound tail latency, with future workloads expected to be even more demanding. To address this challenge, state-of-the-art runtimes employ theoretically optimal scheduling policies, namely a single request queue and strict preemption.
Topics & Concepts
PreemptionComputer scienceScheduling (production processes)Latency (audio)MicrosecondQueueParallel computingProcessor schedulingDistributed computingComputer networkMathematical optimizationOperating systemScheduleMathematicsPhysicsTelecommunicationsAstronomyCloud Computing and Resource ManagementParallel Computing and Optimization TechniquesDistributed systems and fault tolerance