流式处理
计算机科学
数据流挖掘
冗余(工程)
延迟(音频)
吞吐量
数据流
未压缩视频
实时计算
溪流
数据压缩
计算机网络
分布式计算
计算机硬件
数据挖掘
操作系统
视频处理
无线
人工智能
电信
视频跟踪
作者
Zujiang Yu,Feng Zhang,Hourun Li,Shuhao Zhang,Xiaoyong Du
标识
DOI:10.1109/icde55515.2023.00038
摘要
Stream processing prevails and SQL query on streams has become one of the most popular application scenarios. For example, in 2021, the global number of active IoT endpoints reaches 12.3 billion. Unfortunately, the increasing scale of data and strict user requests place much pressure on existing stream processing systems, requiring high processing throughput with low latency. To further improve the performance of current stream processing systems, we propose a compression-based stream processing engine, called CompressStreamDB, which enables adaptive fine-grained stream processing directly on compressed streams, without decompression. Particularly, CompressStreamDB involves eight compression methods targeting various data types in streams, and it also provides a cost model for dynamically selecting the appropriate compression methods. By exploring data redundancy among streams, CompressStreamDB not only saves space in data transmission between client and server, but also achieves high throughput with low latency in SQL query on stream processing. Our experimental results show that compared to the state-of-the-art stream processing system on uncompressed streams, CompressStreamDB achieves 3.24× throughput improvement and 66.0% lower latency on average. Besides, CompressStreamDB saves 66.8% space.
科研通智能强力驱动
Strongly Powered by AbleSci AI