standardmodelbio/model-model-smb-mntp-llama-3.1-8b-v1

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders

LLM2Vec is a simple recipe to convert decoder-only LLMs into text encoders. It consists of 3 simple steps: 1) enabling bidirectional attention, 2) masked next token prediction, and 3) unsupervised contrastive learning. The model can be further fine-tuned to achieve state-of-the-art performance.

• Repository: https://github.com/McGill-NLP/llm2vec
• Paper: https://arxiv.org/abs/2404.05961

Usage

from llm2vec import LLM2Vec

model = LLM2Vec.from_pretrained(
    "standardmodelbio/model-model-smb-mntp-llama-3.1-8b-v1",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    max_length=4096,
    attn_implementation="flash_attention_2",
)

text = ["StandardModel"] * 8
embeddings = model.encode(text)
print(embeddings)
print(embeddings.shape)

"""
tensor([[ 1.1250,  0.7070, -0.1475,  ...,  0.8320,  0.2852, -0.3691],
        [ 1.1250,  0.7070, -0.1475,  ...,  0.8320,  0.2852, -0.3691],
        [ 1.1250,  0.7070, -0.1475,  ...,  0.8320,  0.2852, -0.3691],
        ...,
        [ 1.1250,  0.7070, -0.1475,  ...,  0.8320,  0.2852, -0.3691],
        [ 1.1250,  0.7070, -0.1475,  ...,  0.8320,  0.2852, -0.3691],
        [ 1.1250,  0.7070, -0.1475,  ...,  0.8320,  0.2852, -0.3691]])
torch.Size([8, 4096])
"""