Pareto-optimized Content-hierarchical Adaptive Reinforcement Learning for QoE-driven Short-form Video Streaming

Abstract

The swipe-driven structure of short-form video creates a coupled control problem. The client must protect the playback of the current clip while deciding whether scarce bandwidth should be allocated to a subsequent clip that may not be viewed at all. Most adaptive bitrate (ABR) learners rely on a fixed scalar quality of experience (QoE) function, treating nominal bitrate as the primary proxy for the content value. This study presents a two-stage controller that separates feed-level prefetch allocation from chunk-level quality selection. Stage 1 converts early-swipe risk and playback state into a bounded prefetch budget. Stage 2 retains quality, rebuffering, smoothness, and wasted traffic as a four-dimensional reward and learns a preference-conditioned envelope Q-function. A content-aware utility ladder exposes the estimated value and measured byte cost of each candidate representation. A feasibility mask and a hysteresis filter constrain execution. Across four recorded network conditions, the controller achieves the highest official score among four evaluated systems: a four-trace mean QoE of 60.970 points and a 2.269 MB lower mean data wastage than the MPC baseline, which is equivalent to a 37.7% reduction. Under equal weight and tracewise normalization, its mean normalized streaming index is 0.980. A separate held-out evaluation of ten paired environment seeds reports a mean official score of 15.196 (95% CI: 11.324-19.068), while a 35-preference analysis yields a four-objective hypervolume of 0.433 (95% CI: 0.370-0.497) for the frozen stage 2 policy. These results highlight the potential of hierarchical reinforcement learning to improve user experience in swipe-driven video platforms.