Noise May Contain Transferable Knowledge: Understanding Semi-supervised Heterogeneous Domain Adaptation from an Empirical Perspective
yyyaoyuan
Abstract
Semi-supervised heterogeneous domain adaptation (SHDA) addresses learning across domains with distinct feature representations and distributions, where source samples are labeled while most target samples are unlabeled, with only a small fraction labeled. Moreover, there is no one-to-one correspondence between source and target samples. Although various SHDA methods have been developed to tackle this problem, the nature of the knowledge transferred across heterogeneous domains remains unclear. This paper delves into this question from an empirical perspective. We conduct extensive experiments on about 330 SHDA tasks, employing two supervised learning methods and seven representative SHDA methods. Surprisingly, our observations indicate that both the category and feature information of source samples do not significantly impact the performance of the target domain. Additionally, noise drawn from simple distributions, when used as source samples, may contain transferable knowledge. Based on this insight, we perform a series of experiments to uncover the underlying principles of transferable knowledge in SHDA. Specifically, we design a unified Knowledge Transfer Framework (KTF) for SHDA. Based on the KTF, we find that the transferable knowledge in SHDA primarily stems from the transferability and discriminability of the source domain. Consequently, ensuring those properties in source samples, regardless of their origin (e.g., image, text, noise), can enhance the effectiveness of knowledge transfer in SHDA tasks. The codes and datasets are available at https://github.com/yyyaoyuan/SHDA.
Community
In this work, we investigate the nature of transferable knowledge in semi-supervised heterogeneous domain adaptation (SHDA) through extensive experiments on about 330 SHDA tasks. Surprisingly, our findings reveal that the category and feature information of source samples do not significantly impact the performance of the target domain. Also, even noise drawn from simple distributions may contain transferable knowledge. Based on those insights, we propose a unified Knowledge Transfer Framework (KTF). Based on the KTF, we find that the transferable knowledge in SHDA primarily stems from the transferability and discriminability of the source domain. Consequently, ensuring those properties in source samples, regardless of their origin (e.g., image, text, noise), can enhance the effectiveness of knowledge transfer in SHDA tasks.
We highlight the contributions of this paper as follows.
To the best of our knowledge, we are the first to empirically investigate the transferable knowledge in SHDA.
We observe that noise drawn from simple distributions as source samples may contain transferable knowledge, which has the potential to inspire more intriguing research.
Our observations indicate that the essence of transferable knowledge in SHDA primarily lies in the transferability and discriminability of the source domain, regardless of its origin (\tetit{e.g.}, image, text, and noise).
We open-source the codes and datasets used in this paper at https://github.com/yyyaoyuan/SHDA, including seven typical SHDA methods and several popular datasets, which, to our humble knowledge, is the first relatively comprehensive SHDA open-source repository.
We believe this work will interest researchers in domain adaptation and transfer learning, providing valuable insights and practical guidance for SHDA.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Propensity-driven Uncertainty Learning for Sample Exploration in Source-Free Active Domain Adaptation (2025)
- Aggregate to Adapt: Node-Centric Aggregation for Multi-Source-Free Graph Domain Adaptation (2025)
- Prototypical Distillation and Debiased Tuning for Black-box Unsupervised Domain Adaptation (2024)
- SWAT: Sliding Window Adversarial Training for Gradual Domain Adaptation (2025)
- Exploiting Aggregation and Segregation of Representations for Domain Adaptive Human Pose Estimation (2024)
- Source-free Semantic Regularization Learning for Semi-supervised Domain Adaptation (2025)
- Towards a Problem-Oriented Domain Adaptation Framework for Machine Learning (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper