Fatin757/ssf-retriever-modernbert-v8

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the ssf-train-valid_v8 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid_v8

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatin757/ssf-retriever-modernbert-v8")
# Run inference
sentences = [
    'The Process Development/MS&T Engineer supports process development, monitoring and improvement activities for the biopharmaceuticals manufacturing facilities. He/She will analyse the critical material attributes of biopharmaceutical products, prepare Process Flow Diagrams (PFD), perform pilot tests and support technology transfer activities. He also assists in developing and updating Standard Operating Procedures (SOPs) for the manufacturing facility and supporting the delivery of associated training. The Process Development/MS&T Engineer should have deep understanding of the engineering and scientific concepts underlying the manufacture of the biopharmaceutical product and equipment involved in order to make significant contributions in determining how the product is made within the manufacturing facilities. The Process Development/MS&T Engineer should have a passion for innovation and continuous improvement and he applies this to his work, driving efficiency and improvement in new and existing manufacturing processes. He must be able work independently and exercise analytical and innovative thinking to analyse information, solve problems and improve existing methods and processes.',
    'The Process Development/MS&T Engineer is responsible for supporting process development, monitoring, and enhancement efforts within biopharmaceutical manufacturing operations. This role involves analyzing critical material attributes of biopharmaceutical products, creating Process Flow Diagrams (PFDs), conducting pilot-scale testing, and assisting with technology transfer activities. The engineer also contributes to the creation and revision of Standard Operating Procedures (SOPs) and helps deliver related training for manufacturing personnel. A strong grasp of the scientific and engineering principles related to biopharmaceutical production and equipment is essential, enabling the engineer to influence product manufacturing methods effectively. The Process Development/MS&T Engineer is driven by innovation and continuous improvement, applying analytical and creative thinking to optimize and refine manufacturing processes independently.',
    'The Senior Process Development Engineer leads a team to oversee large-scale manufacturing operations, focusing primarily on production scheduling and resource allocation for biopharmaceutical facilities. This role emphasizes managing personnel and coordinating cross-departmental communication rather than direct involvement in pilot testing or SOP development. The Senior Engineer typically handles budgeting and compliance reporting, with less emphasis on hands-on process innovation or detailed scientific analysis.\n\nThe Manufacturing Quality Assurance Engineer is tasked with ensuring compliance to regulatory standards and conducting audits within biopharmaceutical production lines. Responsibilities include reviewing batch records, investigating deviations, and implementing corrective actions. This position does not involve process flow diagram creation, pilot testing, or technology transfer activities but focuses instead on quality control and assurance processes.\n\nThe Process Development Engineer in a chemical manufacturing plant supports process optimization by analyzing raw material inputs and overseeing equipment maintenance schedules. The role entails preparing technical documentation and assisting with safety training but is centered on chemical production rather than biopharmaceutical processes, requiring different industry-specific knowledge and equipment expertise.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9210, 0.4243],
#         [0.9210, 1.0000, 0.5177],
#         [0.4243, 0.5177, 1.0000]])

Training Details

Training Dataset

ssf-train-valid_v8

• Dataset: ssf-train-valid_v8 at 1a48f71
• Size: 6,032 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 60 tokens mean: 171.21 tokens max: 403 tokens	min: 24 tokens mean: 91.44 tokens max: 255 tokens	min: 14 tokens mean: 99.88 tokens max: 378 tokens

• Samples:

  anchor	positive	negative
  The Operation Specialist supports plant operations by coordinating day-to-day production activities, as well as maintenance and turnaround schedules and activities, for production shift teams, so as to meet production plans and schedules. He/She supports the Site Incident Controller (SIC) during emergency response situations. The Operation Specialist contributes to plant operation improvements by working closely with the production, process engineering and discipline engineering teams to define and execute plant improvement projects, and by reviewing Standard Operating Procedures (SOPs) for the process area under his charge. He also supports the implementation of the Process Safety Management (PSM) framework for production activities, and ensures compliance with Workplace Safety and Health (WSH) and Environmental Management System (EMS) requirements across production teams. The Operation Specialist may work on either a rotating or day shift in the field. He works closely with other dep...	The Operation Specialist plays a key role in supporting plant operations by managing daily production tasks and coordinating maintenance and turnaround schedules for production shift teams to ensure production targets are met. This role assists the Site Incident Controller during emergencies and collaborates with production, process engineering, and discipline engineering teams to drive plant operation enhancements. The Operation Specialist is responsible for reviewing and updating Standard Operating Procedures for their process area, implementing the Process Safety Management framework for production activities, and ensuring adherence to Workplace Safety and Health and Environmental Management System standards. The position may require working on rotating or day shifts and demands strong problem-solving, organizational, communication, and interpersonal skills, along with the ability to work independently and liaise effectively with other departments.	The Operation Specialist in retail oversees daily store operations, manages inventory levels, and coordinates staff scheduling to meet sales targets. They support the store manager in handling customer complaints and assist with merchandising and promotional activities. This role requires excellent customer service skills, the ability to work in a fast-paced retail environment, and proficiency in point-of-sale systems. The Operation Specialist in software development coordinates project timelines, manages code deployment schedules, and supports the incident response team during system outages. They collaborate with software engineers and quality assurance teams to improve application performance and update technical documentation. This role requires strong coding skills, familiarity with agile methodologies, and effective communication with cross-functional teams. The Operation Specialist in hospitality manages event schedules, coordinates with catering and service staff, and ensures...
  The Senior Interchange Supervisor/Interchange Supervisor is responsible for supervising day-to-day bus interchange operations to provide efficient and reliable bus services to passengers. He/She monitors the regulating of bus services and redeployment of Bus Captains to ensure service reliability, and supervises the management of bus interchange facilities and security. He is responsible for liaising with vendors to carry out contract works and acts as the liaising officer for lost and found items. As a team leader, he supports the team in addressing passenger issues, allocates team duties, and manages team performance and development. He also prepares contingency plans for incident and/or accident management, operationalises procedures for compliance management, and proposes areas for continuous improvement. He is a resourceful individual with strong communication skills and is able to work collaboratively with others. He works on rotating shifts within the bus interchange and may be ...	The Senior Interchange Supervisor/Interchange Supervisor oversees daily operations at the bus interchange to ensure timely and dependable bus services for commuters. This role involves monitoring bus service regulation, reallocating Bus Captains to maintain service standards, and managing interchange facilities and security. The supervisor coordinates with vendors for contract-related tasks and handles lost and found items. As a team leader, they assist in resolving passenger concerns, assign duties, and oversee team performance and growth. They develop contingency plans for incidents or accidents, implement compliance procedures, and suggest improvements for operational efficiency. The position requires excellent communication skills, teamwork, and the flexibility to work rotating shifts, including weekends and public holidays.	The Senior Interchange Manager is responsible for developing strategic plans for multiple bus interchanges, overseeing long-term infrastructure projects, and managing vendor contracts at a corporate level. He/She leads cross-functional teams in transport policy development and focuses on regional service expansion rather than daily operations. The role requires extensive experience in transport planning and negotiation with governmental agencies. The Senior Bus Operations Controller monitors real-time bus fleet movements using advanced GPS systems and coordinates emergency responses but does not manage interchange facilities or passenger services directly. The Operations Supervisor for Rail Transit supervises train station staff, manages station security, and coordinates rail service disruptions, focusing exclusively on rail transport rather than bus services.
  The Deputy Workshop Manager supports the day-to-day workshop operations and the implementation of fleet maintenance activities to meet service requirements. He/She supports the coordination of workshop operations with other functional teams such as the Depot and Interchange Management, as well as the Bus Operations Control Centre (BOCC) to support the overall bus service operations. He supports fleet maintenance activities, implements improvement initiatives and conducts engineering studies by allocating required resources and coordination amongst different workshop sections. He also oversees the implementation of housekeeping practices, ensuring that quality logistic support is rendered to facilitate maintenance needs. He supports the management of workshop operating expenditures and forecasting of annual budgetary requirements to meet the workshop operations requirements. He has good knowledge of the bus service operations and is able coordinate effectively with internal and external...	The Deputy Workshop Manager is responsible for supporting daily workshop operations and executing fleet maintenance activities to fulfill service standards. This role involves coordinating workshop functions with teams such as Depot and Interchange Management and the Bus Operations Control Centre (BOCC) to ensure seamless bus service operations. The Deputy Manager allocates resources and coordinates across workshop sections to implement maintenance improvements and engineering studies. Additionally, they oversee housekeeping practices to provide quality logistical support for maintenance activities. They assist in managing workshop operating costs and forecasting budgets to sustain operational needs. With strong knowledge of bus service operations, the Deputy Workshop Manager effectively liaises with internal and external parties, demonstrates excellent supervisory abilities, and continuously pursues manpower and resource enhancements to support the organisation’s bus maintenance and s...	The Senior Maintenance Supervisor leads a team responsible for preventive and corrective maintenance of bus fleets, focusing on long-term asset reliability and compliance with safety regulations. This role collaborates closely with the Safety and Compliance Department and the Vehicle Inspection Unit to ensure adherence to statutory requirements. The supervisor manages workshop staffing schedules and oversees procurement of maintenance parts, while conducting performance audits and risk assessments. They also handle the preparation and monitoring of maintenance budgets, ensuring cost efficiency. With expertise in heavy vehicle systems, the Senior Maintenance Supervisor coordinates with external vendors and internal teams to optimize fleet availability and operational readiness. The Workshop Coordinator manages scheduling and logistics for multiple workshop sites, ensuring timely allocation of repair jobs and parts inventory management. They work with the Transport Planning and Scheduli...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid_v8

• Dataset: ssf-train-valid_v8 at 1a48f71
• Size: 1,508 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 58 tokens mean: 169.86 tokens max: 380 tokens	min: 25 tokens mean: 89.21 tokens max: 286 tokens	min: 15 tokens mean: 99.42 tokens max: 369 tokens

• Samples:

  anchor	positive	negative
  The Production Manager/Assistant Production Manager manages all technical aspects of the factory site, and keeps track of resources requirements. He/She plans the sequence of events from production to bringing the module from the factory to the construction site. He is responsible and able to work independently. He possess factory-based production knowledge and know-how and is able to coordinate the crew, supplies and equipment. He works on-site on a rotating or day-shift schedule.	The Production Manager/Assistant Production Manager oversees all technical operations within the factory premises and monitors resource needs. They organize the workflow from manufacturing to delivering the module to the construction location. They are accountable, capable of working autonomously, and have comprehensive factory production expertise. They coordinate personnel, materials, and machinery effectively while working on-site following a rotating or day-shift roster.	The Senior Production Supervisor leads a team responsible for quality control and safety compliance in the factory, focusing primarily on auditing processes rather than coordinating production schedules. The Construction Site Manager directs on-site activities including labor management and equipment allocation but does not engage with factory production or resource planning. The Manufacturing Operations Analyst uses data analytics to optimize production efficiency but does not participate in direct crew coordination or module transportation to construction sites.
  The Content and Experience Development Executive/Curator supports the curation of content aimed at delivering a meaningful and engaging experience for attractions visitors. This includes content creation, content improvement through research and maintaining the validity of the content over time. He/She may work in the capacity of an attractions subject matter expert, conservator, registrar or designer. He collaborates with operations, marketing and communications as well as sales departments to support attractions set-up, execute attractions experience, develop collaterals, visitor guidebooks and other audio-visual materials to enhance visitor experience and increase visitorship. Creative and resourceful, he develops engaging and informative content that effectively communicates exhibition and programme details to the organisation's target audience. He is also able to perform well, deliver under deadlines and leverage on existing communications and media technology to extend the influe...	Content curation, content creation, research skills, visitor experience development, subject matter expertise, collaboration with marketing and operations, exhibition communication, audio-visual material development, project management, interpersonal communication, mentoring, media technology utilization	Financial auditing, software programming, mechanical engineering, agricultural science, culinary arts, automotive repair, textile manufacturing, veterinary medicine
  The Installation, Inspection and Servicing Engineer plans for inspections of gas installations, reviews gas investigation findings and relevant documentation, and recommends servicing and/or rectification works required for gas installation issues. He/She oversees gas installation, and servicing works, and the commissioning of gas appliances. He manages the submissions of billings and meter statements, and reviews the technical specifications prepared for tender contracts. He/She oversees works performed by Licensed Gas Service Workers (LGSWs) to ensure compliance with Codes of Practice, regulatory and project requirements, and manages customers' feedback and requests for the installation, replacement and troubleshooting of gas appliances. To build internal capabilities,, he provides on-the-job training and analyses staffs strengths and areas of development. He supervises gas pipe works at customers' sites, including domestic, commercial and industrial buildings, and is therefore requi...	Gas installation, inspection planning, gas appliance commissioning, servicing and rectification, compliance with Codes of Practice, technical specification review, project management, customer feedback handling, on-the-job training, safety awareness in gas works, team leadership, collaboration with stakeholders.	Graphic design, social media marketing, culinary arts, fashion merchandising, creative writing, event planning, interior decorating, photography skills.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	12	0.2752	0.0157
2.0	24	0.0157	0.0068
3.0	36	0.0082	0.0045
4.0	48	0.0052	0.0041
5.0	60	0.0061	0.004

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.1
• Transformers: 4.56.2
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}