dnth/ssf-retriever-all-MiniLM-L6-v2

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dnth/ssf-retriever-all-MiniLM-L6-v2")
# Run inference
sentences = [
    "The Quality Manager (Fleet Management) approves the organisations quality management system (QMS) including quality assurance, quality control and inspection standards. He/She is accountable for internal and external quality audits, impact analyses and failure investigations to drive conformance of aircraft maintenance to procedures and standards prescribed by original equipment manufacturers (OEM), regulatory authorities and own organisation. He provides technical expertise for corrective and preventive actions for quality issues in aircraft parts and components. He endorses the certificate of conformance (CoC) for first article inspection (FAI). He approves updates to the technical library and Maintenance Organisation Exposition and sanctions alternative processes, tools and inspection technologies. He manages compliance with airworthiness and legislative requirements, and contributes to development of the organisation's standard operating procedures (SOPs), safety, health, risk and quality systems. He develops lean and sustainability practices, develops data analytics plans for strategic decision-making. He drives team performance to achieve business key performance indicators (KPIs) and leads talent recruitment and development plans. He possesses a quality, safety and risk compliance mindset, and is able to consider issues from multiple perspectives to make well-informed and effective decisions for the function. He demonstrates strong leadership, strategic thinking and organisational management skills to pre-empt quality risks in fleet management.",
    'Quality Assurance Manager in Aviation overseeing QMS, conducting audits, and ensuring compliance with OEM and regulatory standards for aircraft maintenance. Responsible for technical support on quality issues, endorsing CoC for FAI, and updating technical documentation. Develops SOPs and sustainability practices while driving team performance and leading recruitment efforts.',
    'Junior Quality Control Specialist in Healthcare responsible for monitoring compliance with safety regulations and conducting inspections of medical equipment. This role involves supporting the quality assurance team by documenting findings, assisting in audits, and implementing corrective actions for non-compliance. The specialist will update regulatory documentation and contribute to training programs for staff on quality standards and procedures.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6304, 0.0986],
#         [0.6304, 1.0000, 0.2345],
#         [0.0986, 0.2345, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 3,016 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 66 tokens mean: 170.03 tokens max: 403 tokens	min: 21 tokens mean: 69.6 tokens max: 180 tokens	min: 31 tokens mean: 84.68 tokens max: 217 tokens

• Samples:

  anchor	positive	negative
  The Manager (Flight Operations) plans, directs and coordinates flight operations in order to ensure flights run efficiently and safely in accordance with established rules and regulations. He/She is involved in the management of aircraft data systems and the administration of the Operations Control Centre (OCC). He develops plans and proposes changes to enhance flight control operations. During irregular operations, he is responsible for activating the emergency response plans and communicating contingency plans to the airport stakeholders and authorities. He identifies safety and/or security risks and develops plans to mitigate them. He also has overall accountability for assessment and selection of new team members. He forges partnerships and cultivates close working relationships with airport agencies, authorities, customers and vendors. The Manager (Flight Operations) has exceptional leadership and communication skills so as to lead and communicate with a wide range of stakeholders...	Flight Operations Manager oversees and coordinates all aspects of flight operations to ensure safety and efficiency, managing aircraft data systems and the Operations Control Centre. Responsible for developing operational plans, implementing emergency response strategies, and communicating with airport authorities and stakeholders. The role requires strong leadership, exceptional communication skills, and the ability to foster teamwork while managing safety risks and operational challenges.	Junior Safety Officer monitors and evaluates workplace safety protocols to ensure compliance with health regulations and standards. He/She is responsible for conducting safety audits, reporting hazards, and implementing safety training programs. The role involves collaborating with various departments to enhance safety measures, managing incident reports, and maintaining safety data systems. Strong communication skills and the ability to work under pressure are essential for effective coordination with staff and regulatory agencies.
  The Assistant Front Office Manager/Assistant Operations Manager/Duty Manager/Guest Relations Manager/Guest Services Manager manages the daily operations of the front office to ensure efficient functioning of the team in accordance with the department's operating procedures and service standards. He/She develops operations plans, monitors room inventory and collaborates with other departments on special guest arrangements. He also manages personalised reception services for VIP guests, implements loyalty programmes and identifies and resolves deviations and irregularities in service operations. To drive service excellence, he manages the team's service performance, oversees service recovery and interacts with guests to build guest relationships and gather service feedback. He also reviews systems and processes, comes up with new ideas and implements sustainability programmes. He manages operational risks through compliance management, crisis management and loss and risk prevention. His ...	Assistant Front Office Manager responsible for overseeing daily front office operations, ensuring team efficiency, developing operational plans, managing room inventory, and enhancing guest relations through personalized services and loyalty programs while maintaining service excellence and compliance.	Junior Risk Management Analyst needed to support the identification and assessment of potential risks within the financial sector. This role involves analyzing data to develop risk mitigation strategies, collaborating with various departments to ensure compliance with regulations, and preparing reports on risk assessments. The ideal candidate will have strong analytical skills, attention to detail, and the ability to work under pressure. Experience in financial modeling and risk analysis is preferred. This position requires working standard office hours with occasional overtime.
  The Ground Services Officer is responsible for the delivery of quality passenger services through providing assistance to passengers and addressing their queries. He/She communicates and monitors service standards of ground handling agents. He mentors and trains ground handling agents and ensures compliance with established standards. He follows up with passengers safety and security concerns and queries. In the event of flight disruptions or delays, he provides assistance to passengers. The Ground Services Officer is customer-centric and provides excellent service to the customers and passengers. He is comfortable with face-to-face interactions with the various stakeholders whilst working in shifts. He possesses adequate computer literacy in order to obtain real-time flight information and access the reservation and booking systems. Furthermore, he has good communication and interpersonal skills which allow him to work effectively in a team and execute expected service standards of th...	Ground Services Officer responsible for delivering exceptional passenger assistance and addressing inquiries while ensuring compliance with service standards. Mentors ground handling agents and manages safety and security concerns, providing support during flight disruptions. Strong customer service focus with excellent communication skills and computer literacy for real-time flight information access.	Junior Risk Management Analyst responsible for evaluating financial risks and developing mitigation strategies within the healthcare sector. The role involves analyzing data to identify potential threats, preparing risk assessment reports, and collaborating with cross-functional teams to enhance compliance with regulatory standards. Candidates should possess strong analytical skills and the ability to communicate findings effectively.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 754 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 754 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 57 tokens mean: 169.51 tokens max: 349 tokens	min: 23 tokens mean: 69.15 tokens max: 184 tokens	min: 31 tokens mean: 83.26 tokens max: 220 tokens

• Samples:

  anchor	positive	negative
  The Installation, Inspection and Servicing Senior Technical Officer conducts site inspections and investigations on gas equipment installations and proposes solutions to customers for gas installation non-compliance issues. He/She supervises contractors on day-to-day operations to ensure compliance with Codes of Practice, regulatory and, project requirements, and leads the servicing of gas installations and commissioning of gas appliances. He prepares technical specifications for tender contracts, and builds internal capabilities through on-the-job training. He works at customers' sites, including domestic, commercial and industrial buildings and is therefore required to have a high awareness of safety. He is meticulous, systematic, and a good problem solver, able to propose solutions to customers with gas pipe infringements. He is a good team player and interacts effectively with team members and customers.	Senior Technical Officer for Gas Installation Oversight and Compliance, responsible for conducting site inspections, supervising contractors, and providing solutions for gas installation issues while ensuring adherence to safety standards and regulatory requirements.	Junior Risk Management Analyst responsible for evaluating potential risks in financial operations, conducting audits, and developing mitigation strategies. This role involves preparing detailed reports on risk assessments, collaborating with various departments to ensure compliance with industry regulations, and implementing training programs to enhance team awareness of risk management practices.
  The Head Chef is the overall in-charge for the kitchens daily operations to guide preparation of food across all workstations. He/She supervises day-to-day operations of the kitchen and creates new recipes and menus to ensure continuous improvement. He implements customer service standards to enhance customer experience and leads the compliance of service, food hygiene, health and safety standards. Resilient and resourceful, he possesses mental resilience to operate in a high pressure environment, and is capable in communicating and working effectively with co-workers and suppliers. He is expected to stand for long periods and work in a fast-paced environment.	Head Chef responsible for managing daily kitchen operations, overseeing food preparation, and developing innovative recipes and menus to improve service quality. Ensures adherence to food hygiene, health, and safety standards while providing excellent customer service. Must be resilient, resourceful, and effective in communication with team members and suppliers, capable of thriving in a high-pressure, fast-paced setting.	Junior Sous Chef tasked with assisting in the overall management of kitchen operations, focusing on food preparation and maintaining high standards of service quality. Works under the Head Chef to implement menu changes and ensure compliance with health and safety regulations. Requires strong communication skills and the ability to work efficiently in a dynamic environment, while also standing for extended periods in a busy kitchen.
  The Senior Regulations and Trade Controls Analyst/Regulations and Trade Controls Analyst assumes the responsibility of coordinating trade regulatory requirements with working level stakeholders. He/She is also responsible for reviewing organisation's compliance with regulatory requirements. He assesses adequacy, effectiveness and efficiency of compliance with regulations and make appropriate recommendations. In addition, he is expected to communicate with regional colleagues to upkeep latest the regulatory standards and guidelines for organisational compliance on both a local and regional level. He possesses good communication and coordination skills with high attention to details.	Trade Compliance Analyst responsible for coordinating regulatory requirements with stakeholders, ensuring organizational compliance, assessing the effectiveness of regulations, and communicating with regional teams to maintain up-to-date standards and guidelines.	Junior Risk Management Associate tasked with evaluating the risk factors in organizational operations, collaborating with team members to implement risk mitigation strategies, and ensuring compliance with internal policies and industry standards while maintaining detailed documentation and reports.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• gradient_accumulation_steps: 2
• weight_decay: 0.1
• num_train_epochs: 10
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 2
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.1
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
0.5291	50	0.0583	0.0012
1.0529	100	0.0071	0.0008
1.5820	150	0.0046	0.0007
2.1058	200	0.0062	0.0009
2.6349	250	0.0043	0.0007
3.1587	300	0.002	0.0006
3.6878	350	0.0025	0.0005
4.2116	400	0.0041	0.0007
4.7407	450	0.0024	0.0005
5.2646	500	0.0022	0.0006
5.7937	550	0.0028	0.0014
6.3175	600	0.0024	0.0005
6.8466	650	0.0014	0.0006
7.3704	700	0.0026	0.0005
7.8995	750	0.0015	0.0004
8.4233	800	0.0018	0.0004
8.9524	850	0.0017	0.0004
9.4762	900	0.0029	0.0004
10.0	950	0.0013	0.0004

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.8
• Sentence Transformers: 5.0.0
• Transformers: 4.54.1
• PyTorch: 2.7.1+cu126
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}