广告实验是如何“欺骗”你的 —— 如何发掘真正的实验效力.pdf

1、2023 DataFunSummit广告实验是如何“欺骗”你的-如何发掘真正的实验效应段玮韬领英实验带头人领英目录IntroductionThe ChallengeExisting MethodsBudget SplitContents01 IntroductionThe worlds largest professional network143M+United States14M+Canada30M+Brazil11M+Mexico6M+Columbia4M+Chile6M+Argentina90M+Europe6M+South Africa1M+Kenya1M+Nigeria2M+Egy

2、pt2M+Saudi Arabia 1M+Israel6M+Turkey1M+Morocco45M+India36M+China1M+Hong Kong1M+Republic of Korea1M+Japan9M+Australia1M+New Zealand5M+Philippines9M+Indonesia3M+Malaysia2M+SingaporeStart a PostDigest Updates from My ConnectionsLand Your Next PlayKeep in Touch with Industry Trend60KSchools900MMembers26

3、MCompanies50KSkills15MOpen Jobs190BUpdates viewedLinkedIn Marketing Solution LinkedIn Marketing Solution helps advertisers reach the desired audienceSponsored Post AdAd02 Experimental Challenge&Misleading Results Strong Experiment CultureWe experiment on UI changes,relevance algorithms,backend chang

4、es,and even bug fixes.Advanced Experiment InfrastructureWe have start-of-the-art in-house platform to meet the growing need of experimentation30000+Metrics Computed1500+Daily Active Experiments40+TBMetric and Experiment Assignment Data ProcessedData is in Our DNA Marketplace experimentationSupply av

5、ailable ads slotsDemand advertiser platformMatching bidding,pacing and optimizationWe experiment on LinkedIn FeedLinkedIn Audience NetworkMemberCampaignAdvertiserExperimentation in Online PlatformsTREATMENTCONTROL$1/user$0.995/user+$0.005/user+0.5%liftExperimentation in Online PlatformsTREATMENTCONT

6、ROL$1/user$0.995/user+$0.005/user+0.5%liftWhen experiment ramped to 100%,we often observe a much smaller lift in overall revenue.Is A/B result cheating?Cannibalization Bias in Marketplace ExperimentationTREATMENTCONTROLReal revenue impact is no more than 0%Measured revenue impact in an experiment is