Efficient CUDA stream management for multi-DNN real-time inference on embedded GPUs
Weiguang Pang, Xiantong Luo, Kailun Chen, Dong Ji, Lei Qiao, Yi Wang
Topics & Concepts
Computer scienceCUDAInferenceScheduling (production processes)Deep neural networksGeneral-purpose computing on graphics processing unitsParallel computingComputer architectureArtificial neural networkDistributed computingComputer engineeringArtificial intelligenceOperating systemGraphicsOperations managementEconomicsAdvanced Neural Network ApplicationsIoT and Edge/Fog ComputingParallel Computing and Optimization Techniques