Harmonizing Network Measurements with User Experience: Challenges, Innovations, and Solutions

In the modern digital age, understanding and accurately measuring network performance has become essential. Yet, the tools and methods we employ, particularly speed tests, have a double-edged sword. On one end, they provide crucial insights into network speeds and potential bottlenecks. On the other, they inadvertently impact the very user experience they aim to optimize. This dissertation investigates the intricate relationship between network measurements and user Quality of Experience (QoE). Our initial exploration reveals that standard speed tests, while diagnostic, can substantially degrade user QoE, extending download times and affecting video streaming quality. Such interruptions not only disrupt user activities but also highlight an inefficiency in our measurement methods. The data consumed in these tests, though essential for diagnostics, is discarded post-measurement, rendering it momentarily useful yet ultimately wasteful. In response to this challenge, we introduce rePurpose, a novel approach that reimagines speed tests. Instead of discarding the data post-measurement, rePurpose pre-stages video ads, integrating them within the existing advertising ecosystem. This not only offsets the data costs of speed tests but also enhances user QoE by reducing ad and video start-up delays. As the dissertation progresses, we further explore the concept of orchestrated network measurements, aiming to offer a more coordinated and holistic view of network performance. Through research questions and experimental designs, we highlight the need for scheduling and orchestration in network measurements, improving accuracy without compromising user experience. This dissertation highlights the critical balance between network performance measurements and user experience, offering solutions to bridge the existing gaps. Through research and analysis, we pave the way for a more harmonious coexistence between network diagnostics and digital user experiences.