A novel GPU cluster scheduling algorithm for cloud computing: a power-grid-inspired predictive-optimization framework

Abstract
Efficient management of GPU resources in cloud computing is critical for maximizing cluster throughput and meeting latency-sensitive service-level objectives (SLOs). GPU workloads exhibit bursty arrivals, heterogeneous resource profiles, and performance interference under co-location—properties analogous to stochastic generation and volatile loads in power grids. Static or purely reactive schedulers suffer from resource fragmentation and unstable utilization, similar to grid instability caused by uncontrolled intermittency. This paper introduces a hierarchical GPU cluster scheduling algorithm inspired by power grid economic dispatch and automatic generation control (AGC). At each epoch, the predictor layer estimates future arrivals and runtimes via state-space filtering, paralleling grid load forecasting. The decision layer computes multi-objective task priorities and solves constrained resource allocation through a primal–dual scheme, akin to optimal power flow with stability margins. The feedback layer adapts priority weights via online learning from observed delays and SLA violations, inspired by AGC’s continuous frequency regulation. Implemented on an 8-GPU NVIDIA Tesla V100 cluster simulator, the proposed method improves mean GPU utilization by up to 30%, task throughput by 25%, and reduces average job completion time by 27.5% across diverse workload regimes. Ablations confirm that prediction, optimization, and feedback are all essential for stable, high-efficiency GPU scheduling—demonstrating effective cross-domain transfer from power system operation to data center resource management.