读METASHFIT paper
简介
论文 title
METASHIFT: A DATASET OF DATASETS FOR EVALUAT-ING CONTEXTUAL DISTRIBUTION SHIFTS AND TRAIN-ING CONFLICTS
ICLR 2022 的paper，链接在这。
解决的问题：
In order to assess the reliability and fairness of a model, we therefore need to evaluate its performance and training behavior across heterogeneous types of data. However, the lack of well-structured datasets representing diverse data distributions makes systematic evaluation difficult.
贡献：
Our contributions: We present MetaShift as an important resource for studying the behavior of ML algorithms and training dynamics across data with heterogeneous contexts. Our methodology for constructing MetaShift can also be applied to other domains where metadata is available. We empirically evaluate the performance of different robust learning algorithms, showing that ERM performs well for modest shifts while no method is the clear winner for larger shifts. This finding suggests that domain generalization is an important and challenging task and that there's still a lot of room for new methods.
例子：
