面向图数据分布外泛化的因果表示学习.pdf

1、DataFunSummit#2023面向图数据分布外泛化的因果表示学习陈永强-香港中文大学-博士研究生Yongqiang ChenCUHK,Tencent AI Lab2Towards Causal Representation Learning for Out-of-Distribution Generalization on Graphswith Yatao Bian,Yonggang Zhang,Kaiwen Zhou,Binghui Xie,Tongliang Liu,Bo Han,and James ChengOutOut-ofof-Distribution Generalizati

2、onDistribution GeneralizationModels trained with Empirical Risk Minimization(ERM)are often:-prone to spurious correlations-can hardly generalize to OOD data 3OutOut-ofof-Distribution GeneralizationDistribution GeneralizationThe goal of OOD generalization is:minf:XYmaxeEallLe(f)given a subset of trai

3、ning environments/domains ,where each corresponds to a dataset and a loss.Etr EallE EallDeLe4OutOut-ofof-Distribution GeneralizationDistribution GeneralizationLeveraging the Invariance Principle from causality,previous approaches aim to learn an invariant predictor:minf=w!eEtrLe(w ),s.t.w argminwLe(

4、w ),e Etr,that is simultaneously optimal across different environments/domains.(Peters et al.,2015;Arjovsky et al.,2019;Bottou et al.,2021;)5OutOut-ofof-Distribution GeneralizationDistribution Generalization(Peters et al.,2015;Arjovsky et al.,2019;Rosenfeld et al.,2021;Kamath et al.,2021;Ahuja et al

5、.,2021;)6OutOut-ofof-Distribution Generalization on GraphsDistribution Generalization on GraphsX78OutOut-ofof-Distribution Generalization on GraphsDistribution Generalization on Graphs(Knyazev et al.2019;Hu et al.,2020;Koh et al.,2021;Gui et al.,2022;Chen et al.,2022)A Graph Neural Network(GNN)makes

6、 predictions taking both structure-level and attribute-levelfeatures into account.9OutOut-ofof-Distribution Generalization on GraphsDistribution Generalization on Graphs(Knyazev et al.2019;Hu et al.,2020;Koh et al.,2021;Gui et al.,2022;Chen et al.,2022)OOD generalization on graphs is fundamentally m