README.md

metadata

tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:10837
  - loss:AttributeTripletLoss
base_model: Alibaba-NLP/gte-base-en-v1.5
widget:
  - source_sentence: '2000'
    sentences:
      - publication_date
      - Jennifer Hudson
      - February 01, 2001
      - author
  - source_sentence: 'Right to the Edge: Sydney to Tokyo by Any Means'
    sentences:
      - title
      - '9780263888300'
      - isbn_13
      - Rick Steves’ 2010 Spain
  - source_sentence: Empathy In Patient Care
    sentences:
      - publication_date
      - title
      - 'Creating a Web Site: The Missing Manual'
      - 30/09/2010
  - source_sentence: ': Wizards of the Coast'
    sentences:
      - ': Simon Pulse'
      - 10/10/2010
      - publisher
      - publication_date
  - source_sentence: ': 9780451223203'
    sentences:
      - ': 9780425221204'
      - 'The Diary of a Young Girl: The Definitive Edition'
      - title
      - isbn_13
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy
  - silhouette_cosine
  - silhouette_euclidean
model-index:
  - name: SentenceTransformer based on Alibaba-NLP/gte-base-en-v1.5
    results:
      - task:
          type: triplet
          name: Triplet
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: cosine_accuracy
            value: 1
            name: Cosine Accuracy
          - type: cosine_accuracy
            value: 0.9992526173591614
            name: Cosine Accuracy
      - task:
          type: silhouette
          name: Silhouette
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: silhouette_cosine
            value: 0.9388933777809143
            name: Silhouette Cosine
          - type: silhouette_euclidean
            value: 0.7954943776130676
            name: Silhouette Euclidean
          - type: silhouette_cosine
            value: 0.9398965239524841
            name: Silhouette Cosine
          - type: silhouette_euclidean
            value: 0.7953634262084961
            name: Silhouette Euclidean

SentenceTransformer based on Alibaba-NLP/gte-base-en-v1.5

This is a sentence-transformers model finetuned from Alibaba-NLP/gte-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Alibaba-NLP/gte-base-en-v1.5
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NewModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("albertus-sussex/veriscrape-book-test-sbert-bs128_lr5e-05_ep5_euclidean_snTrue_spFalse_hn3_fixuqmix_spl100")
# Run inference
sentences = [
    ': 9780451223203',
    ': 9780425221204',
    'The Diary of a Young Girl: The Definitive Edition',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	1.0

Silhouette

• Evaluated with veriscrape.training.SilhouetteEvaluator

Metric	Value
silhouette_cosine	0.9389
silhouette_euclidean	0.7955

Triplet

• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.9993

Silhouette

• Evaluated with veriscrape.training.SilhouetteEvaluator

Metric	Value
silhouette_cosine	0.9399
silhouette_euclidean	0.7954

Training Details

Training Dataset

Unnamed Dataset

• Size: 10,837 training samples
• Columns: anchor, positive, negative, pos_attr_name, and neg_attr_name
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative	pos_attr_name	neg_attr_name
  type	string	string	string	string	string
  details	min: 3 tokens mean: 7.31 tokens max: 47 tokens	min: 3 tokens mean: 7.4 tokens max: 29 tokens	min: 3 tokens mean: 6.97 tokens max: 23 tokens	min: 3 tokens mean: 3.72 tokens max: 5 tokens	min: 3 tokens mean: 3.83 tokens max: 5 tokens

• Samples:

  anchor	positive	negative	pos_attr_name	neg_attr_name
  Joanna Philbin	Kathy Reichs	Clarkson Potter	author	publisher
  9780297863120	9781841652160	Karen Rose	isbn_13	author
  David Zane Mairowitz	Jason Myers	9781407517384	author	isbn_13

• Loss: veriscrape.training.AttributeTripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 1,205 evaluation samples
• Columns: anchor, positive, negative, pos_attr_name, and neg_attr_name
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative	pos_attr_name	neg_attr_name
  type	string	string	string	string	string
  details	min: 3 tokens mean: 7.55 tokens max: 47 tokens	min: 3 tokens mean: 7.29 tokens max: 26 tokens	min: 3 tokens mean: 7.09 tokens max: 25 tokens	min: 3 tokens mean: 3.69 tokens max: 5 tokens	min: 3 tokens mean: 3.87 tokens max: 5 tokens

• Samples:

  anchor	positive	negative	pos_attr_name	neg_attr_name
  : 9781558580091	9780787988494	The Diary of a Young Girl: The Definitive Edition	isbn_13	title
  August 01, 2007	06/11/2008	: One World	publication_date	publisher
  Alan Morris	Juan Larrea	Penguin Books Ltd	author	publisher

• Loss: veriscrape.training.AttributeTripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• num_train_epochs: 5
• warmup_ratio: 0.1

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	cosine_accuracy	silhouette_cosine
-1	-1	-	-	0.4838	0.1721
1.0	85	0.9702	0.0570	0.9967	0.8609
2.0	170	0.0202	0.0015	1.0	0.9215
3.0	255	0.004	0.0027	1.0	0.9233
4.0	340	0.0023	0.0011	1.0	0.9372
5.0	425	0.001	0.0011	1.0	0.9389
-1	-1	-	-	0.9993	0.9399

Framework Versions

• Python: 3.10.16
• Sentence Transformers: 3.4.1
• Transformers: 4.45.2
• PyTorch: 2.5.1+cu124
• Accelerate: 1.5.2
• Datasets: 3.1.0
• Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

AttributeTripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}