5 Shopee -StarRocks 在 Shopee 的应用和实践.pdf

1、User Behavior AnalyticsAnti Fraud Rules EngineLog AnalysisReal-Time Data warehouseSupply Chain Logistics Report Video Quality AnalyticsApplication ObservabilityBusiness IntelligenceReal-Time Dashboard&AlertUser Behavior AnalyticsLogistics ReportLog PlatformReal-time DashboardData WarehouseAnti Fraud

2、 DetectionBusiness IntelligenceDQC TaskOffline AnalysisReal-time DashboardTime Series Data ProcessingMonitor&Alert SystemQuery is Slow&Extra ETL EffortQuery ScenarioTrino/SecondClickHouse/SecondStarRocks/SecondOLAP DB Improve timesPlain joinJoin by primary key(user_id)Filter by partition keyAll outp

3、ut columns are from left table54.082300+1.803StarRocks Speed up 30 times Plain joinJoin by primary key(user_id)and another key(shop_id)Filter by partition keyAll output columns are from left table34.868300+1.173StarRocks Speed up 30 times No JOIN but IN for ClickHouseFilter by partition keyAll outpu

4、t columns are from left table34.8682.0213.182ClickHouse Speed up 18 timesQuery EngineLakeHouseKey Results (Aug 2024)SR bypass rate:62%SR execution success rate:90.11%(vs.96.36%for Presto)SR queries:2.25 times faster on P90Success RateP90 exe time/second2.25 times faster StarRocks As Data Studio Query Engine Roll Out plan 24 Oct.800 core resource on live business24 Dec12000 core resource on live business24 Sept.500 core double run25 Feb.25000 core resource on live business estimate24 Nov2400 core resource on live business感谢观看！Thank you!关注公众号