Fatin757/ssf-retriever-modernbert-v2

SentenceTransformer based on answerdotai/ModernBERT-base

This is a sentence-transformers model finetuned from answerdotai/ModernBERT-base on the ssf-train-valid_v3 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: answerdotai/ModernBERT-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid_v3

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatin757/ssf-retriever-modernbert-v2")
# Run inference
sentences = [
    "The Senior Anchor/Senior Presenter/Anchor/ Presenter - News delivers news stories to the broadcasting station's audience and is the public face or voice of the programmes broadcasted on various platforms. He/She is involved in the collection of news materials and is required to conduct research on stories and interview people who have accurate information on news events. He is also involved in the development and writing of content and is responsible for reviewing and editing materials written by other news reporters to ensure that the content is tailored to the target audience. He is required to host or co-host programmes by providing live commentaries and doing live interviews to create content that links closely to the stories. He often works from a studio and may be expected to travel in order to present news from remote locations in the field related to a particular major news event. He follows a fixed working schedule, but may be required to work at odd hours, including weekends, to cover important events. He should be an effective communicator with an understanding of news editorial process. He should ideally have a background in journalism or mass communications and possesses an understanding of daily newscast content and media ethics. He ought to be able to improvise and ad-lib in a live on-camera setting and be able to work well with others across a variety of situations.",
    "The Lead News Presenter is responsible for delivering engaging news stories to the audience of the broadcasting station, serving as the public face or voice of various programs aired across multiple platforms. This role involves gathering news materials, conducting thorough research, and interviewing credible sources to ensure accurate reporting on current events. The Lead News Presenter also plays a critical role in content development and writing, as well as reviewing and editing articles produced by other reporters to ensure alignment with the target audience's interests. Additionally, this position requires hosting or co-hosting programs, providing live commentary, and conducting live interviews to enhance storytelling. While primarily based in a studio, the Lead News Presenter may travel to report from remote locations for significant news events. The role follows a structured work schedule but may necessitate working irregular hours, including weekends, to cover key happenings. Strong communication skills and a solid understanding of the news editorial process are essential, along with a background in journalism or mass communications, familiarity with daily newscast content, and adherence to media ethics. The ability to improvise and ad-lib during live broadcasts while collaborating effectively with team members is also crucial.",
    "The Junior News Reporter is tasked with gathering and compiling news stories for the broadcasting station's audience, acting as a supportive figure in various programs aired on different platforms. This position includes the collection of news materials and conducting basic research on stories, as well as interviewing individuals who may provide information on news events. The Junior News Reporter assists in the writing and editing of content produced by senior reporters, ensuring it meets the audience's needs. This role may also involve supporting hosts during live programs by providing background information and conducting interviews to aid in storytelling. While primarily working from the studio, there may be occasional travel to gather information for specific news events. The Junior News Reporter typically follows a standard work schedule but might be called upon to work during odd hours, including weekends, to assist in covering significant events. Effective communication skills and a foundational understanding of media ethics are important, along with a basic knowledge of the news editorial process. The ability to work collaboratively with colleagues and adapt to various situations is also necessary.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9225, 0.4020],
#         [0.9225, 1.0000, 0.4397],
#         [0.4020, 0.4397, 1.0000]])

Training Details

Training Dataset

ssf-train-valid_v3

• Dataset: ssf-train-valid_v3 at 5d23d8e
• Size: 6,032 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 60 tokens mean: 168.86 tokens max: 403 tokens	min: 74 tokens mean: 165.4 tokens max: 318 tokens	min: 63 tokens mean: 139.63 tokens max: 253 tokens

• Samples:

  anchor	positive	negative
  The Brokerage Supervisor/ Freight Supervisor is responsible for liaising with customers, logistics operators and customs officials and supervising the custom clearance/freight forwarding operations to ensure goods are cleared through customs or quarantine in accordance with import and export laws and regulations. Analytical and systematic, he/she is required to supervise a freight operations team to execute operations in a timely manner to meet business and customers' requirements. He/She is also expected to work with internal and external stakeholders to accomplish his work.	The Logistics Operations Supervisor is tasked with coordinating communication between clients, transportation providers, and regulatory agencies while overseeing the customs clearance and freight forwarding processes. This role ensures compliance with all import and export regulations, facilitating the smooth transit of goods through customs and quarantine. The ideal candidate will possess strong analytical and organizational skills, leading a team of logistics professionals to execute operations efficiently and meet both business objectives and customer expectations. Collaboration with various internal and external partners is essential to successfully fulfill the responsibilities of this position.	The Freight Operations Coordinator is responsible for managing interactions with suppliers, transport companies, and regulatory bodies while overseeing the delivery processes to ensure shipments are dispatched in alignment with logistics standards. This role focuses on adherence to transportation guidelines and the timely execution of delivery schedules. The successful candidate should demonstrate excellent problem-solving abilities and be capable of directing a team of logistics personnel to achieve operational targets. Coordination with different departments and external partners is crucial for the effective management of this role.
  The Senior Quality Engineer evaluates and manages quality systems, tools and standards to meet business needs. He/She, as the subject matter expert, is required to identify risk areas, ensure the robustness of the risk control plans deployed for excursion free launch and conduct qualification and/or validation for new materials. He is responsible for developing a management system to ensure that operations meet both internal and external parties quality requirements. He has to take the lead in managing cross-functional teams in continuous improvement projects and assist in implementing process improvement projects. He plays an important role in organisational development through development of on-the-job training and mentoring of team leaders. The Senior Quality Engineer possesses an analytical mind and leadership skills to steer the team to perform their best and achieve the desired organisational outcomes.	The Quality Assurance Manager is responsible for assessing and overseeing quality systems, methodologies, and standards to align with business objectives. As an expert in the field, this individual will pinpoint areas of risk, ensure that effective risk management strategies are in place for successful product launches, and conduct thorough qualification and validation of new materials. The role involves creating a comprehensive quality management system to guarantee compliance with both internal standards and external regulations. Additionally, the Quality Assurance Manager will lead cross-functional teams in ongoing improvement initiatives and support the execution of process enhancement projects. This position is pivotal in fostering organizational growth through the development of on-the-job training programs and mentoring for team leaders. The ideal candidate will have strong analytical capabilities and leadership qualities to drive team performance and achieve key organizational ...	The Junior Quality Control Technician is tasked with performing routine inspections and testing of products to ensure compliance with quality standards. This role requires the technician to document findings and report any discrepancies to the senior staff. While they contribute to maintaining quality assurance processes, they are not responsible for developing management systems or leading cross-functional teams. Instead, their focus will be on executing established procedures and assisting in minor quality improvement tasks. The Junior Quality Control Technician will work under close supervision and will not engage in risk management or validation processes, limiting their involvement to basic quality checks and reporting.
  The Waste Process Engineer is responsible for designing, optimising and managing processes and systems for the efficient handling, treatment, transformation and disposal of waste, including electronic waste (e-waste) and plastic waste, for an organisation. He/She is focused on minimising waste generation, exploring new technologies for enhancing waste management efficiency and material recovery, and recommending improved waste management systems and processes within an organisation. Based on life-cycle analyses and evaluation of current waste streams, systems and waste-to-resource initiatives, he will develop and implement new processes, ensuring compliance with environmental regulations. The Waste Process Engineer must be meticulous, with an eye for detail and have strong analytical and research skills to stay up to date on best practices and circular economy strategies related to waste management. He is also expected to work collaboratively with cross-functional teams to promote su...	The Waste Management Engineer is tasked with the design, optimization, and oversight of processes and systems aimed at the effective handling, treatment, transformation, and disposal of various waste types, including electronic waste (e-waste) and plastics. This role emphasizes the reduction of waste generation and the exploration of innovative technologies to enhance waste management efficiency and material recovery. Additionally, the engineer will assess current waste streams and implement waste-to-resource initiatives based on thorough life-cycle analyses. Ensuring compliance with environmental regulations, the Waste Management Engineer will develop and execute new processes while collaborating with cross-functional teams to advocate for sustainable practices and advance the organization’s objectives within the Circular Economy.	The Environmental Compliance Officer is responsible for monitoring and enforcing adherence to environmental laws and regulations within an organization. This role focuses on assessing the impact of various operations on the environment and ensuring that all practices comply with legal standards. The officer will conduct regular audits, prepare reports, and provide training to staff about environmental policies. They will also work closely with regulatory agencies to maintain compliance and address any environmental concerns that may arise. Strong attention to detail and analytical skills are essential for this position, as is the ability to collaborate with various departments to ensure that the organization meets its sustainability goals.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid_v3

• Dataset: ssf-train-valid_v3 at 5d23d8e
• Size: 1,508 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 57 tokens mean: 166.64 tokens max: 349 tokens	min: 67 tokens mean: 162.89 tokens max: 301 tokens	min: 65 tokens mean: 139.7 tokens max: 263 tokens

• Samples:

  anchor	positive	negative
  The Manager - Standards and Practices (S&P) ensures that content delivered by the organisation complies with the regulatory requirements and censorship norms of the local territories where the content may be available. He/She also provides advisory ratings for the content based on the regulatory guidelines. He keeps abreast of the local, cultural and political norms and sensitivities to support the creation of content classification guidelines. The work involves coordinating internal and external processes for delivery within tight timelines. He is highly accountable for the organisation's brand and reputation given the sensitivities of content classification. He should be comfortable coordinating with internal and external stakeholders in order to balance the organisation's priorities with compliance to guidelines and norms. He should be effective at planning and organising. He should also be aware of the regulatory, political and cultural landscape and possess a keen eye for detail t...	The Content Compliance Manager plays a crucial role in ensuring that all materials produced by the organization meet the necessary regulatory standards and censorship requirements of the respective local markets. This individual will provide expert advisory ratings for content in alignment with established regulatory frameworks. Staying informed about local cultural and political dynamics is essential to aid in the development of content classification guidelines. The role requires effective coordination of both internal and external processes to ensure timely delivery while maintaining the integrity of the organization's brand and reputation. The ideal candidate will excel in stakeholder engagement, balancing organizational objectives with compliance mandates. Strong planning, organizational skills, and a meticulous attention to detail are vital for identifying compliance issues during content reviews. Proficiency in communication and stakeholder management is necessary for successful...	The Junior Risk Analyst is responsible for evaluating potential risks that may impact the organization’s operations and financial performance. This role involves conducting thorough assessments of various risk factors and preparing detailed reports on findings. The Junior Risk Analyst will collaborate with different departments to identify risk mitigation strategies and ensure that appropriate measures are in place. Strong analytical skills and attention to detail are essential for identifying potential vulnerabilities within the organization. The position requires effective communication with team members and management to discuss risk assessments and recommendations. The ideal candidate should also be comfortable working under tight deadlines while maintaining a high level of accuracy in their analyses.
  The Psychologist provides psychological services within multi-disciplinary settings, applying a wide array of psychometric assessments and treatments for clients with routine psychological conditions. He/She conducts psychoeducational programmes and training for other professionals. He is a resourceful, proactive and collaborative professional, and works in varied settings such as public and private institutions, hospitals, healthcare and voluntary welfare organisations. He works under supervision and in collaboration with other professionals in the course of his work.	The Clinical Psychologist delivers comprehensive psychological services in diverse multi-disciplinary environments, utilizing a broad spectrum of psychometric evaluations and therapeutic interventions for clients experiencing common psychological issues. This role involves designing and implementing psychoeducational programs and training sessions aimed at enhancing the skills of fellow professionals. The ideal candidate is resourceful, proactive, and excels in collaboration, working effectively across various settings including public and private institutions, hospitals, healthcare facilities, and voluntary welfare organizations. The Clinical Psychologist operates under supervision while engaging collaboratively with other professionals to ensure the best outcomes for clients.	The Clinical Psychologist conducts psychological assessments within corporate environments, focusing primarily on employee mental health and organizational behavior. This position emphasizes the development and implementation of workplace wellness programs and training for management teams. The successful candidate will be an innovative, self-motivated individual who thrives in collaborative settings, working closely with human resources and management to address workplace-related psychological issues. The Clinical Psychologist operates independently while coordinating with various departments to enhance employee well-being and productivity in a corporate context.
  The Planning Manager (Aircraft Engine / Component Maintenance) is responsible for strategising long-term supply chain management plans, optimising aircraft engine and component maintenance planning and resource management, and driving supplier capability and performance enhancement programmes. He/She develops vendor management strategies and leads technology application to strengthen sourcing, inventory and warehousing operations. He provides project management oversight for engine servicing operations and manages technical and programme reviews with customers and suppliers. He manages compliance with airworthiness and legislative requirements, and contributes to development of the organisation's standard operating procedures (SOPs), management systems, lean and sustainability practices, and data analytics plans for strategic decision-making. He drives team performance to achieve business key performance indicators (KPIs) and leads talent recruitment and development plans. He should de...	The Aircraft Maintenance Planning Specialist is tasked with formulating comprehensive long-term strategies for supply chain management, focusing on the optimization of maintenance planning for aircraft engines and components. This role involves enhancing supplier capabilities and performance through targeted programs. The specialist will develop effective vendor management strategies and leverage technology to improve sourcing, inventory control, and warehousing operations. Additionally, they will oversee project management for engine servicing activities and facilitate technical and program reviews with both customers and suppliers. Compliance with airworthiness regulations and legislative requirements is critical, as is contributing to the organization's standard operating procedures (SOPs), management systems, and data analytics initiatives for informed decision-making. The specialist will also drive team performance to meet key performance indicators (KPIs) and lead initiatives for...	The Aircraft Component Quality Assurance Coordinator is responsible for implementing quality control measures and ensuring compliance with industry standards in the maintenance of aircraft components. This role focuses on conducting inspections and audits to assess the performance of suppliers and service providers. The coordinator develops quality assurance strategies and applies technology to enhance inspection processes and documentation practices. They will manage quality-related projects and collaborate with engineering teams to address any compliance issues with regulatory requirements. Additionally, the coordinator contributes to the development of quality management systems and participates in data analysis for quality improvement initiatives. They are expected to drive team engagement to achieve quality performance metrics and support training and development programs for staff. Strong attention to detail, analytical skills, and the ability to work collaboratively across depar...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	189	0.3362	0.0096
2.0	378	0.0079	0.0056
3.0	567	0.0029	0.0042
4.0	756	0.0026	0.0037
5.0	945	0.0023	0.0032

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.0
• Transformers: 4.56.1
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}