dnth/ssf-retriever-modernbert-embed-base

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the ssf-train-valid dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dnth/ssf-retriever-modernbert-embed-base")
# Run inference
sentences = [
    'The Camera Operator executes the development of the visual look and style of the production. He/She is responsible for marking out the positions for camera equipment and production crew who are directly involved in the shoot. During the shoot, he is responsible for testing and operating the camera equipment to achieve the required shot composition while suggesting creative improvisations. He may be required to operate special-purpose cameras and camera equipment such as drones, Steadicam, Russian arm to capture visuals that may not be captured by standard camera-shooting processes. He may also operate 360 Cameras and other equipment required to capture live and recorded immersive content. He may also use electronic video and audio technologies in order to gather and present news. He is required to gather materials for either live transmission or recording, providing a representative account of events. He is responsible for the set-up and installation of broadcast equipment and manages the overall maintenance of sound, video and livestream recording equipment. He is also expected to format, edit and deliver recordings to the studio for events that were not streamed live. The work involves long hours of physically demanding tasks especially the capture of motion sequences, amidst high pressure. He is expected to operate in an outdoors environment and may be required to travel depending on the location of the shoot. He should have strong knowledge of camera equipment and camera operations. He should also possess the ability to visualise scenes and has the artistic vision to suggest improvisations to advised techniques of video capture. He ought to possess technical knowledge of immersive video capture and the artistic vision to realise the same in order to contribute to the development of immersive content. He is required to exhibit effective teamwork, be diplomatic and tactful when working with cast and crew.',
    'Camera Operator role focused on developing visual style and composition, responsible for positioning equipment, testing gear, and suggesting creative shots, including the use of drones and 360 cameras for immersive content capture. Requires strong technical skills in video and audio technologies and the ability to work in high-pressure outdoor environments.',
    'Junior Risk Analyst responsible for evaluating financial risks and developing strategies to mitigate potential losses. This role involves gathering and analyzing data, preparing reports on risk assessments, and collaborating with stakeholders to implement risk management solutions. Candidates should have strong analytical skills, attention to detail, and the ability to communicate complex information clearly. The position may require working long hours under tight deadlines, and familiarity with risk assessment tools and methodologies is essential.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.7646, -0.0565],
#         [ 0.7646,  1.0000, -0.0119],
#         [-0.0565, -0.0119,  1.0000]])

Training Details

Training Dataset

ssf-train-valid

• Dataset: ssf-train-valid at 591c937
• Size: 4,180 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 73 tokens mean: 181.92 tokens max: 349 tokens	min: 17 tokens mean: 59.85 tokens max: 182 tokens	min: 40 tokens mean: 81.65 tokens max: 150 tokens

• Samples:

  anchor	positive	negative
  The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.	Audit Assistant role focused on supporting audit processes, demonstrating understanding of audit principles, and enhancing skills to address straightforward challenges. Collaborates effectively within a team, adheres to professional ethics, and is committed to personal development and meticulous work.	Junior Financial Analyst responsible for conducting basic financial assessments and preparing reports under guidance. The role requires an understanding of financial principles, but focuses on data entry and basic analysis rather than complex audit tasks. The position demands teamwork and ethical standards, but is primarily concerned with budget tracking and financial forecasting in a retail environment.
  The Audit Senior Manager/Audit Manager manages a portfolio of engagements to deliver high quality audit services. He/she also provides leadership on audit engagements which includes client acceptance process, engagement planning, execution and finalisation of an audit engagement. He is fully accountable for the audit engagement and ensures that the engagement progress against budget and timeline is closely monitored. He also serves to develop and maintain long-term client relationships and value-add to the audit firm by identifying new business development opportunities. The Audit Senior Manager/Audit Manager reviews and provides key technical expertise to ensure the quality of audit work performed is in compliance with professional standards and requirements. He contributes towards continuous improvement in audit methodology and process. He will also assume a greater role in professional development activities such as training, staff recruitment and resource planning.	Audit Manager position responsible for overseeing multiple audit projects, ensuring high-quality service delivery, and leading audit teams through planning, execution, and completion. Focus on maintaining client relationships, monitoring engagement timelines and budgets, and identifying opportunities for business growth while ensuring compliance with audit standards.	Junior Financial Analyst needed to support the finance team in analyzing financial data and preparing reports. The role involves assisting in budgeting, forecasting, and financial modeling for various projects. The candidate will collaborate with different departments to ensure accurate financial planning and provide insights for decision-making. This position requires strong analytical skills and proficiency in financial software.
  The Audit Partner/Audit Director is a transformational leader who steers the organisation to achieve its business goals and objectives by formulating technical and strategic directions to drive change. He/She provides strategic vision and leadership to the organisation in order to develop and strengthen organisational capabilities and culture. The Audit Partner/Audit Director is expected to promote new ideas and business solutions that result in extended services to existing clients. He constantly seeks to expand client base and support business development activities. He also establishes consistent and rigorous quality and risk management processes and procedures. The Audit Partner/Audit Director uses a multitude of controls and procedures consisting professional, regulatory, business, economic, social and environmental conditions to manage risk exposure.	Audit Director with strategic leadership skills to enhance organisational capabilities and promote innovative solutions for client services and business development.	Junior Risk Analyst responsible for assessing potential risks and implementing mitigation strategies within the financial services sector. This role involves conducting risk assessments, preparing reports, and collaborating with teams to ensure compliance with regulatory requirements. The Junior Risk Analyst will also assist in developing risk management frameworks and monitoring risk exposure across various business units.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid

• Dataset: ssf-train-valid at 591c937
• Size: 1,045 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 72 tokens mean: 168.04 tokens max: 403 tokens	min: 20 tokens mean: 67.48 tokens max: 186 tokens	min: 40 tokens mean: 82.91 tokens max: 185 tokens

• Samples:

  anchor	positive	negative
  The Logistics Solutions and Implementation Director/Tailored Supply Chain Director/Channel Operations Director is responsible for managing the processes of business development and implementing custom-made or tailored end-to-end complex logistics solutions for customers, including managing post implementation optimisation. He/She is also responsible for managing logistics solutioning business resources. Resourceful and persuasive, he is required to manage resources and obtain buy-in from internal and external stakeholders. He is also expected to lead a department and make business decisions independently.	Logistics Solutions Director overseeing business development and tailored logistics implementations, focusing on end-to-end solutions and post-implementation optimization while managing resources and stakeholder engagement.	Junior Risk Management Analyst responsible for evaluating and mitigating risks within the financial services sector, conducting assessments, and collaborating with internal teams to enhance compliance and operational effectiveness. The role requires strong analytical skills and the ability to present findings to management.
  The Business Development Director/Country Route Development Director/Trade Lane Director/Freight Trade Director is responsible for developing new strategic business opportunities, client bases and managing business resources, including manpower and assets. He/She is also responsible for managing and engaging complex key accounts to develop trade development strategies and to develop strategic customer relationships. Resourceful and analytical, he is required to manage resources and obtain buy-in from internal and external stakeholders. He is also expected to lead a department and make business decisions independently.	Job opening for a Business Development Manager focused on establishing new strategic partnerships, enhancing client relationships, and overseeing operational resources, including team management and asset allocation. Ideal candidates are proactive and analytical, capable of engaging with key accounts and driving trade development initiatives while collaborating effectively with stakeholders.	Seeking a Junior Risk Management Analyst responsible for assessing potential risks within financial operations, analyzing data to identify trends, and developing mitigation strategies. This role requires collaboration with various departments to ensure compliance and protection of assets. Candidates should be detail-oriented and able to present findings to senior management for informed decision-making.
  The Business Development Manager/Sales and Marketing Manager/Vertical Sales Account Manager/Key Account Manager/Project Cargo Sales Manager/Route Development Manager/Trade Lane Manager is responsible for business development, managing large key accounts, marketing, sales of both broad based and niche logistics services including performing market research, prospecting, developing relationships with potential customers and meeting sales targets. He/She is also responsible for managing business resources, including manpower and internal assets. Resourceful and analytical, he is required to manage resources and obtain buy-in from internal and external stakeholders. He is also expected to lead teams and make business decisions independently.	Business Development Executive for logistics services focusing on key account management, sales growth, market research, prospecting new clients, and building relationships while achieving sales objectives. Responsible for managing operational resources and collaborating with stakeholders to drive business success.	Junior Financial Analyst in the healthcare sector tasked with conducting financial assessments, analyzing budget reports, and supporting the financial planning process. Responsible for preparing financial statements, assisting in audits, and collaborating with various departments to ensure accurate financial data management.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	9	0.2162	0.0133
2.0	18	0.0195	0.0095
3.0	27	0.0136	0.0080
4.0	36	0.0115	0.0074
5.0	45	0.0112	0.0074

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.8
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}