大数据
事件(粒子物理)
数据处理
数据分析
数据挖掘
作者
Wei Cao,Yusong Gao,Feifei Li,Sheng Wang,Bingchen Lin,Ke Xu,Xiaojie Feng,Yucong Wang,Zhenjun Liu,Gejin Zhang
出处
期刊:International Conference on Management of Data
日期:2020-06-11
卷期号:: 739-753
标识
DOI:10.1145/3318464.3386136
摘要
With the increasing demand for real-time system monitoring and tracking in various contexts, the amount of time-stamped event data grows at an astonishing rate. Analytics on time-stamped events must be real time and the aggregated results need to be accurate even when data arrives out of order. Unfortunately, frequent occurrences of out-of-order data will significantly slow down the processing, and cause a large delay in the query response. Timon is a timestamped event database that aims to support aggregations and handle late arrivals both correctly (i.e., upholding the exactly-once semantics) and efficiently. Our insight is that a broad range of applications can be implemented with data structures and corresponding operators that satisfy associative and commutative properties. Records arriving after the low watermark are appended to Timon directly, allowing aggregations to be performed lazily. To improve query efficiency, Timon maintains a TS-LSM-Tree, which keeps the most recent data in memory and contains a time-partitioning tree on disk for high-volume data accumulated over long time span. Besides, Timon supports materialized aggregation views and correlation analysis across multiple streams. Timon has been successfully deployed at Alibaba Cloud and is a critical building block for Alibaba cloud's continuous monitoring and anomaly analysis infrastructure.
科研通智能强力驱动
Strongly Powered by AbleSci AI