基于 Apache Doris 构建 10 倍性价比的日志分析方案.pdf

1、基基于于 A Ap pa ac ch he e D Do or ri is s 构构建建 1 10 0 倍倍性性价价比比的的日日志志分分析析方方案案肖康飞轮科技 技术副总裁Apache Doris CommitterDoris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asi

2、a 2023目目录录1.日志存储分析场景需求2.基于 ES 的日志平台痛点3.基于 Doris 的新一代日志分析平台4.实践案例Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 20231 1日日志志存存储储分分析析场场景景需需求求Doris Summit As

3、ia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023日日志志存存储储分分析析的的典典型型应应用用场场景景降低安全风险 提升系统安全性网网络络安安全全支持业务分析 加速业务增长业业务务分分析析保障服务稳定 提升用户体验可可观观测测性性Doris Summit Asia 2023Doris

4、 Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023日日志志存存储储分分析析的的3 3V V存储规模大、存储周期长对存储成本敏感V Vo ol lu umme e -数数据据量量大大海量数据实时写入、低延迟可见实时交互式分析V Ve el lo oc ci it ty y -实实时时写写入入与与检检索索数据类

5、型多样，Text和JSONSchema EvolutionV Va ar ri ie et ty y -S Sc ch he emma a F Fr re ee eDoris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 20232 2基基于于 E ES S 的的日日志

6、志系系统统痛痛点点Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023Doris Summit Asia 2023基基于于 E ES S的的典典型型日日志志系系统统架架构构KafkaLogstashKibaba UIES DSL API RequestLogCollectoreg.filebeat数