language: en
tags:
- sentence-embeddings
- sentence-similarity
cambridgeltl/mirror-roberta-base-sentence-drophead
An unsupervised sentence encoder proposed by Liu et al. (2021), using drophead instead of dropout as feature space augmentation. The model is trained with unlabelled raw sentences, using roberta-base as the base model. Please use `[CLS]' as the representation of the input.
Citation
@inproceedings{
liu2021fast,
title={Fast, Effective and Self-Supervised: Transforming Masked LanguageModels into Universal Lexical and Sentence Encoders},
author={Liu, Fangyu and Vuli{\'c}, Ivan and Korhonen, Anna and Collier, Nigel},
booktitle={EMNLP 2021},
year={2021}
}