BjarneNPO/BjarneNPO-01_09_2025_15_09_37

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m-v2.0

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v2.0. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-m-v2.0
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'GteModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("BjarneNPO-01_09_2025_15_09_37")
# Run inference
queries = [
    "Ein Vater taucht nicht auf bei den Eltern im Elternbeirat \r\n\r\nAu\u00dferdem auf die Kinder mit archivierten Angeh\u00f6rigen hingewiesen und ihr gezeigt",
]
documents = [
    'Weil er keinen Zugang zur EAPP hat, Außerdem auf die Kinder mit archivierten Angehörigen hingewiesen und ihr gezeigt wie sie das lösen kann',
    '1. Vorlage da. Userin auch gezeigt wie sie die verwanden kann\r\n2. Als Wunsch weitergegeben.',
    'In der Kinderliste haben Kinder gefehlt. Userin muss die Daten in der Kinderliste hinterlegen.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.4964,  0.1370, -0.0431]])

Evaluation

Metrics

Information Retrieval

• Dataset: Snowflake/snowflake-arctic-embed-m-v2.0
• Evaluated with scripts.InformationRetrievalEvaluatorCustom.InformationRetrievalEvaluatorCustom with these parameters:

  {
      "query_prompt_name": "query",
      "corpus_prompt_name": "query"
  }
  

Metric	Value
cosine_accuracy@1	0.1159
cosine_accuracy@3	0.6377
cosine_accuracy@5	0.7536
cosine_accuracy@10	0.8841
cosine_precision@1	0.1159
cosine_precision@3	0.3382
cosine_precision@5	0.3362
cosine_precision@10	0.271
cosine_recall@1	0.0152
cosine_recall@3	0.0813
cosine_recall@5	0.1281
cosine_recall@10	0.1965
cosine_ndcg@10	0.291
cosine_mrr@10	0.4015
cosine_map@100	0.1935

Training Details

Training Dataset

Unnamed Dataset

• Size: 72,349 training samples
• Columns: query and answer
• Approximate statistics based on the first 1000 samples:

  	query	answer
  type	string	string
  details	min: 6 tokens mean: 49.26 tokens max: 4339 tokens	min: 6 tokens mean: 28.66 tokens max: 238 tokens

• Samples:

  query	answer
  Nun ist die Monatsmeldung erfolgt, aber rote Ausrufezeichen tauchen auf.	Userin an das JA verwiesen, diese müssten ihr die Schloss-Monate zur Überarbeitung im Kibiz.web zurückgeben. Userin dazu empfohlen, die Kinder die nicht in kitaplus sind, aber in Kibiz.web - im KiBiz.web zu entfernen, wenn diese nicht vorhanden sind.
  Die Feiertage in den Stammdaten stimmen nicht.	Es besteht bereits ein Ticket dafür.
  Abrechnung kann nicht final freigegeben werden, es wird aber keiner Fehlermeldung angeziegt	im Hintergrund ist eine Fehlermeldung zu sehen. An Entwickler weitergeleitet.
  Korrektur vorgenommen.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• gradient_accumulation_steps: 4
• learning_rate: 4e-05
• lr_scheduler_type: cosine
• warmup_ratio: 0.08
• bf16: True
• tf32: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 4e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.08
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Snowflake/snowflake-arctic-embed-m-v2.0_cosine_ndcg@10
0.0177	10	2.222	-
0.0354	20	2.1657	-
0.0531	30	1.9611	-
0.0708	40	1.9064	-
0.0885	50	1.7658	-
0.1061	60	1.6493	-
0.1238	70	1.7006	-
0.1415	80	1.5747	-
0.1592	90	1.4265	-
0.1769	100	1.3506	-
0.1946	110	1.349	-
0.2123	120	1.2742	-
0.2300	130	1.1629	-
0.2477	140	1.173	-
0.2654	150	1.1773	-
0.2831	160	1.1101	-
0.3008	170	1.0419	-
0.3184	180	1.1084	-
0.3361	190	1.0667	-
0.3538	200	1.0174	-
0.3715	210	1.0776	-
0.3892	220	0.9883	-
0.4069	230	0.9751	-
0.4246	240	0.9467	-
0.4423	250	0.8335	-
0.4600	260	0.9196	-
0.4777	270	0.9604	-
0.4954	280	0.9193	-
0.5130	290	0.8227	-
0.5307	300	0.8538	-
0.5484	310	0.867	-
0.5661	320	0.8176	-
0.5838	330	0.8119	-
0.6015	340	0.8154	-
0.6192	350	0.8091	-
0.6369	360	0.7533	-
0.6546	370	0.7948	-
0.6723	380	0.792	-
0.6900	390	0.7766	-
0.7077	400	0.7901	-
0.7253	410	0.7517	-
0.7430	420	0.7825	-
0.7607	430	0.7635	-
0.7784	440	0.8115	-
0.7961	450	0.7359	-
0.8138	460	0.7824	-
0.8315	470	0.7623	-
0.8492	480	0.7299	-
0.8669	490	0.7406	-
0.8846	500	0.7614	-
0.9023	510	0.6883	-
0.9199	520	0.6846	-
0.9376	530	0.6769	-
0.9553	540	0.7502	-
0.9730	550	0.7281	-
0.9907	560	0.6141	-
1.0	566	-	0.2907
1.0071	570	0.6137	-
1.0248	580	0.5913	-
1.0425	590	0.5994	-
1.0602	600	0.5915	-
1.0778	610	0.594	-
1.0955	620	0.576	-
1.1132	630	0.5402	-
1.1309	640	0.5558	-
1.1486	650	0.5526	-
1.1663	660	0.5585	-
1.1840	670	0.5472	-
1.2017	680	0.5427	-
1.2194	690	0.5229	-
1.2371	700	0.5307	-
1.2548	710	0.5319	-
1.2724	720	0.5614	-
1.2901	730	0.5825	-
1.3078	740	0.5405	-
1.3255	750	0.5867	-
1.3432	760	0.57	-
1.3609	770	0.5272	-
1.3786	780	0.5566	-
1.3963	790	0.5371	-
1.4140	800	0.5084	-
1.4317	810	0.4834	-
1.4494	820	0.5608	-
1.4670	830	0.5439	-
1.4847	840	0.5198	-
1.5024	850	0.5187	-
1.5201	860	0.5295	-
1.5378	870	0.5075	-
1.5555	880	0.5612	-
1.5732	890	0.4302	-
1.5909	900	0.5036	-
1.6086	910	0.5535	-
1.6263	920	0.5629	-
1.6440	930	0.4835	-
1.6617	940	0.5806	-
1.6793	950	0.5547	-
1.6970	960	0.5008	-
1.7147	970	0.5518	-
1.7324	980	0.4992	-
1.7501	990	0.4895	-
1.7678	1000	0.4911	-
1.7855	1010	0.4759	-
1.8032	1020	0.5082	-
1.8209	1030	0.5329	-
1.8386	1040	0.468	-
1.8563	1050	0.4719	-
1.8739	1060	0.4937	-
1.8916	1070	0.4628	-
1.9093	1080	0.4668	-
1.9270	1090	0.4645	-
1.9447	1100	0.4933	-
1.9624	1110	0.5119	-
1.9801	1120	0.5162	-
1.9978	1130	0.4628	-
2.0	1132	-	0.2949
2.0142	1140	0.406	-
2.0318	1150	0.384	-
2.0495	1160	0.4111	-
2.0672	1170	0.3909	-
2.0849	1180	0.4097	-
2.1026	1190	0.4208	-
2.1203	1200	0.4042	-
2.1380	1210	0.3715	-
2.1557	1220	0.4217	-
2.1734	1230	0.381	-
2.1911	1240	0.3838	-
2.2088	1250	0.3797	-
2.2264	1260	0.3548	-
2.2441	1270	0.4092	-
2.2618	1280	0.4176	-
2.2795	1290	0.4105	-
2.2972	1300	0.3414	-
2.3149	1310	0.3782	-
2.3326	1320	0.3626	-
2.3503	1330	0.3897	-
2.3680	1340	0.4173	-
2.3857	1350	0.376	-
2.4034	1360	0.37	-
2.4211	1370	0.3827	-
2.4387	1380	0.3509	-
2.4564	1390	0.3725	-
2.4741	1400	0.3447	-
2.4918	1410	0.3882	-
2.5095	1420	0.3764	-
2.5272	1430	0.408	-
2.5449	1440	0.384	-
2.5626	1450	0.386	-
2.5803	1460	0.3892	-
2.5980	1470	0.3337	-
2.6157	1480	0.4048	-
2.6333	1490	0.3788	-
2.6510	1500	0.3814	-
2.6687	1510	0.3978	-
2.6864	1520	0.3505	-
2.7041	1530	0.4267	-
2.7218	1540	0.3549	-
2.7395	1550	0.3723	-
2.7572	1560	0.3883	-
2.7749	1570	0.3714	-
2.7926	1580	0.3275	-
2.8103	1590	0.3672	-
2.8280	1600	0.3915	-
2.8456	1610	0.3476	-
2.8633	1620	0.3962	-
2.8810	1630	0.4102	-
2.8987	1640	0.3944	-
2.9164	1650	0.4045	-
2.9341	1660	0.4133	-
2.9518	1670	0.3768	-
2.9695	1680	0.3845	-
2.9872	1690	0.4359	-
3.0	1698	-	0.2910

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.2
• PyTorch: 2.8.0+cu129
• Accelerate: 1.10.0
• Datasets: 3.6.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}