From Synthetic to Real: Unsupervised Domain Adaptationfor Animal Pose Estimation

CVPR 2021 (Oral)

Chen Li Gim Hee Lee
Department of Computer Science, National University of Singapore

To solve the domain shift between synthetic and real data, existing works first generate pseudo labels with a model trained on synthetic data, and then gradually incorporate more pseudo labels into training according to the confidence score. However, these pseudo labels are inaccurate even with refinement techniques such as confidence-based filtering or geometry-based consistency. This will lead to degraed performance when used naively for training. In this work, we propose an online coarse-to-fine pseudo label update strategy to gradually replace the noisy pseudo labels with more accurate ones.


Animal pose estimation is an important field that has received increasing attention in the recent years. The main challenge for this task is the lack of labeled data. Existing works circumvent this problem with pseudo labels generated from data of other easily accessible domains such as synthetic data. However, these pseudo labels are noisyeven with consistency check or confidence-based filtering due to the domain shift in the data. To solve this problem, we design a multi-scale domain adaptation module(MDAM) to reduce the domain gap between the syntheticand real data. We further introduce an online coarse-to-fine pseudo label updating strategy. Specifically, we propose a self-distillation module in an inner coarse-update loop and a mean-teacher in an outer fine-update loop to generate new pseudo labels that gradually replace the old ones. Consequently, our model is able to learn from the old pseudo labels at the early stage, and gradually switch to the new pseudo labels to prevent overfitting in the later stage. We evaluate our approach on the TigDog and VisDA2019 datasets, where we outperform existing approaches by a large margin. We also demonstrate the generalization ability of our model by testing extensively on both unseen domains and unseen animal categories. Our code is available at the project website.

[Paper] [Code] [Video]


@InProceedings{Li_2021_CVPR, author = {Li, Chen and Lee, Gim Hee}, title = {From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2021}, pages = {1482-1491} }


Results on unseen domains and unseen animal categories

Webpage design borrowed from LASR