Fatin757/ssf-retriever-modernbert-embed-base-V3

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the ssf-train-valid dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatin757/ssf-retriever-modernbert-embed-base-V3")
# Run inference
sentences = [
    "The Deck Officer (Special Limit) performs bridge navigation and deck watch duties, and voyage planning on board a ship operating within Singapores 'Special Limit' or about 30 nautical miles from Singapores port. He/She assists in search and rescue operations, and is responsible for maintaining the bridge navigational and communications, fire-fighting and life-saving equipment. He must pass a colour vision test and fulfil the requirements stipulated in the Standards of Training, Certification and Watchkeeping for Seafarers (STCW) issued by the International Maritime Organisation (IMO).",
    "The Navigation Officer (Special Limit) is responsible for conducting bridge navigation and overseeing deck watch operations while planning voyages aboard a vessel operating within Singapore's 'Special Limit', approximately 30 nautical miles from the port. This role involves assisting in search and rescue missions and ensuring the proper maintenance of navigational and communication equipment, as well as fire-fighting and life-saving apparatus. Candidates must pass a color vision test and meet the criteria outlined in the Standards of Training, Certification and Watchkeeping for Seafarers (STCW) as mandated by the International Maritime Organisation (IMO).",
    "The Deck Officer (Unlimited) carries out bridge navigation and deck watch responsibilities, along with voyage planning for a ship operating beyond Singapore's 'Special Limit'. This position includes participating in search and rescue operations and is tasked with ensuring the upkeep of navigational systems, communication devices, and emergency equipment. Applicants are required to pass a color vision assessment and adhere to the standards specified in the International Maritime Organisation (IMO) guidelines for maritime personnel.\n\n## Reason\nThe negative description presents a Deck Officer (Unlimited) role, which operates beyond the 'Special Limit' as opposed to within it, indicating a different scope of responsibility and operational context. The focus on an 'Unlimited' capacity suggests a broader range of duties and navigation responsibilities compared to the original role.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9717, 0.2962],
#         [0.9717, 1.0000, 0.3348],
#         [0.2962, 0.3348, 1.0000]])

Training Details

Training Dataset

ssf-train-valid

• Dataset: ssf-train-valid at dc3bb19
• Size: 6,032 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 64 tokens mean: 168.21 tokens max: 256 tokens	min: 70 tokens mean: 162.17 tokens max: 256 tokens	min: 73 tokens mean: 175.32 tokens max: 256 tokens

• Samples:

  anchor	positive	negative
  The Assistant Research Director works role in the field of social work research. He/She has expertise and experience in domains under social work research in order to assist in supervising advance research designs, methods, collection and analysis of data, project management and collaborations with external organisations. He leads the formulation of systemic, collaborative research, integration of research findings to social service, fund management, administrative and operational functions, and strategic foreign analysis with professionals. He is also responsible for advising external organisations and related ministries on social work-related protocols and programmes. A highly experienced researcher who is committed, service-orientated and possesses the willingness to mentor, the Assistant Research Director works in academic settings. He also works in collaboration with other agencies and ministries and academic institutions in the course of his work.	The Associate Director of Social Work Research plays a pivotal role in advancing the field of social work through comprehensive research initiatives. This individual possesses extensive expertise in various aspects of social work research, enabling them to oversee complex research designs, methodologies, and data analysis. They are responsible for managing projects and fostering collaborations with external entities, ensuring that research findings are effectively integrated into social services. Additionally, the Associate Director handles fund management, administrative tasks, and strategic analyses in collaboration with professionals in the field. They also provide guidance to external organizations and government ministries regarding social work protocols and programs. A dedicated and experienced researcher, the Associate Director is committed to service excellence and mentoring others, working closely with academic institutions and other agencies throughout their career.	The Senior Director of Social Services oversees the operations within community service organizations. This leader has substantial experience in managing various social service programs and is responsible for ensuring effective service delivery and compliance with regulations. They lead the development of community outreach initiatives, program evaluations, and inter-agency collaborations. The Senior Director also manages budgets, administrative functions, and strategic planning with a focus on enhancing service quality. They provide support to local government bodies and community organizations regarding social service practices and initiatives. A seasoned professional in the field, the Senior Director is dedicated to community engagement and workforce development, often working with different stakeholders to improve service outcomes. ## Reason The negative description focuses on a different role within social services rather than research, emphasizing operational management and comm...
  The Senior Assistant Engineer/Assistant Engineer (Automatic Fare Collection) is responsible for supervising his/her team in performing preventive and corrective maintenance work on Automatic Fare Collection (AFC) systems. His duties also include proposing workflow improvements to improve the reliability of the AFC systems. He also manages teams performance in achieving established Key Performance Indicators (KPIs), as well as facilitating the work of external contractors. He is required to carry out his duties in the depot, workshop and/or at various train stations during train operating hours. He is meticulous, analytical, conducts hi work and leads his team in a systematic approach to resolve technical issues and challenges.	The Lead Systems Engineer (Automatic Fare Collection) oversees a team dedicated to executing preventive and corrective maintenance on Automatic Fare Collection (AFC) systems. This role involves recommending workflow enhancements to boost the reliability of AFC systems. Additionally, the Lead Systems Engineer is responsible for managing team performance to meet established Key Performance Indicators (KPIs) and coordinating the efforts of external contractors. The position requires working in depots, workshops, and various train stations during operational hours. An ideal candidate is detail-oriented, analytical, and employs a systematic approach to lead the team in addressing technical challenges effectively.	The Junior Systems Analyst (Automatic Payment Processing) assists in the execution of routine maintenance tasks related to Automatic Payment Processing systems. This position involves implementing minor adjustments to enhance system performance. The Junior Systems Analyst also tracks team progress towards achieving set performance metrics and collaborates with external vendors. The role requires presence in offices and service centers during operational hours. A successful candidate is organized, detail-focused, and supports the team in troubleshooting system issues and operational hurdles. ## Reason The negative description outlines the role of a Junior Systems Analyst in Automatic Payment Processing, which differs from the original role focused on Automatic Fare Collection systems. The responsibilities and scope are distinct, emphasizing a lower seniority level and a different function within a related domain.
  The Senior Engineer/Engineer (Permanent Way and Civil Structure) leads multiple teams in performing preventive and corrective maintenance on tracks, railway reserves and buildings. He/She is accountable for planning the maintenance work activities, providing technical advice to team members as well as supervising complex issues pertaining to fault analysis and testing of permanent ways and civil structures. He is also involved in the engagement and management of external contractors and ensuring the achievement of operating standards and quality standards. He is required to work in shifts and carries out his duties at various rail premises such as on train tracks, in train tunnels and at various train stations. He has a strong understanding of civil and structural design and is methodical in approaching engineering challenges. He is a team player with good interpersonal skills and is able to demonstrate strong supervisory and leadership skills to implement work processes and systems to...	The Lead Civil Engineer for Rail Infrastructure is responsible for overseeing multiple teams dedicated to the preventive and corrective maintenance of railway tracks, reserves, and associated structures. This role entails planning maintenance activities, offering technical guidance to team members, and addressing complex issues related to fault analysis and testing of civil infrastructure. The Lead Civil Engineer also manages external contractors to ensure compliance with operating and quality standards. The position requires shift work across various rail facilities, including train tracks, tunnels, and stations. A deep understanding of civil and structural design is essential, along with a methodical approach to engineering challenges. The ideal candidate is a collaborative team player with excellent interpersonal skills and proven supervisory and leadership abilities to effectively implement work processes that meet operational needs.	The Junior Civil Engineer for Urban Development assists in various projects related to the maintenance and construction of urban infrastructure, such as roads, bridges, and public facilities. This position focuses on supporting senior engineers in the planning and execution of maintenance tasks while providing basic technical assistance. The Junior Civil Engineer engages with contractors to facilitate project execution but is not primarily responsible for ensuring quality standards. The role does not involve shift work and is typically based in an office environment, with occasional site visits. A foundational knowledge of civil engineering principles is required, and the candidate should exhibit teamwork skills and a willingness to learn from experienced engineers to contribute to project success. ## Reason The negative description is for a Junior Civil Engineer in Urban Development, which differs from the Senior Engineer role in the anchor due to its focus on urban infrastructure ra...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid

• Dataset: ssf-train-valid at dc3bb19
• Size: 1,508 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 58 tokens mean: 167.27 tokens max: 256 tokens	min: 61 tokens mean: 161.99 tokens max: 256 tokens	min: 79 tokens mean: 175.94 tokens max: 256 tokens

• Samples:

  anchor	positive	negative
  The Chief Executive Officer/Chief Operating Officer/Managing Director/General Manager/President defines the long-term strategic direction to grow the business in line with the organisations overall vision, mission and values. He/She translates broad goals into achievable steps, anticipates and stays ahead of trends, and takes advantage of business opportunities. He represents the organisation with customers, investors, and business partners, and holds responsibility for fostering a culture of workplace safety and health and adherence to industry quality standards. He inspires the organisation towards achieving business goals and fulfilling the vision, mission and values by striving for continuous improvement, driving innovation and equipping the organisation to embrace change. He possesses excellent analytical, problem-solving and leadership skills and is an effective people leader.	The Chief Executive Officer (CEO) is responsible for establishing the long-term strategic vision to enhance the growth of the organization in alignment with its core values and mission. This role involves translating overarching objectives into actionable plans, proactively identifying emerging trends, and capitalizing on new business opportunities. The CEO serves as the primary representative of the organization to clients, investors, and partners, while also ensuring a commitment to workplace safety, health, and compliance with industry quality standards. By fostering a culture of continuous improvement and innovation, the CEO motivates the organization to achieve its goals and fulfill its vision. Strong analytical, problem-solving, and leadership capabilities are essential, as well as the ability to effectively lead and inspire teams.	The Chief Executive Officer (CEO) of a Non-Profit Organization is tasked with defining the long-term strategic initiatives to enhance community outreach in line with the organization’s mission and values. This position involves translating general objectives into specific community programs, predicting and responding to social trends, and leveraging partnerships for funding opportunities. The CEO represents the organization to stakeholders, donors, and community leaders, while being responsible for promoting a culture of volunteer engagement and compliance with regulatory standards. By driving community involvement and fostering innovation, the CEO encourages the organization to meet its outreach goals and fulfill its mission. Exceptional communication, problem-solving, and leadership skills are crucial, along with the ability to effectively manage a diverse group of volunteers. ## Reason The negative description differs from the anchor by focusing on a non-profit context rather than ...
  The Assistant Technical Superintendent monitors ship operations and evaluates technical aspects of vessels for maintenance needs. He/She collaborates with vessel operators to develop the proper technical repair plans to address identified maintenance needs, and supervises maintenance procedures to ensure compliance with port rules and regulations, as well as international codes and regulations, including the International Maritime Organisation (IMO) code, International Labour Organisation (ILO) regulations, the International Safety Management (ISM) code, International Ship and Port Facility Security (ISPS) code, Maritime Labour Convention (MLC) regulations, and relevant ISO standards. He is also in-charge of crew-level administration matters. He is flexible and possesses strong initiative and good communication skills	The Marine Technical Supervisor is responsible for overseeing ship operations and assessing the technical requirements for vessel maintenance. This role involves working closely with vessel operators to create effective technical repair strategies that address maintenance issues. The Marine Technical Supervisor ensures that all maintenance activities adhere to port regulations and international standards, including the codes set forth by the International Maritime Organisation (IMO), International Labour Organisation (ILO), International Safety Management (ISM), International Ship and Port Facility Security (ISPS), Maritime Labour Convention (MLC), and applicable ISO standards. Additionally, this position includes managing crew-related administrative tasks. The ideal candidate should demonstrate flexibility, strong initiative, and excellent communication skills.	The Junior Marine Safety Officer is tasked with ensuring compliance with safety regulations and conducting safety drills on board vessels. This role focuses on monitoring safety protocols and providing training to crew members to maintain a safe working environment. The Junior Marine Safety Officer will also be responsible for reporting safety incidents and recommending improvements to safety procedures. This position requires attention to detail and the ability to communicate effectively with crew members. However, it does not involve technical evaluations of vessel maintenance or management of repair plans.
  Make-up and/or Hair Artists are responsible for applying make-up and hairstyles for cast before and during a performance to capture their visual appearance in line with the desired look and vision of the production as outlined by the make-up and hair design plans. This may include the application of both cosmetic and special effects make-up. They are responsible for translating the vision for each cast into their physical appearance through effective make-up and hairstyles. Make-up and/or Hair Artists need to be aware of factors such as production lighting that may impact the appearance of make-up and hair. They should also consult with cast on any skincare concerns or allergic precautions and be able to cater to cast of all age groups, genders and racial/ethnic backgrounds. In productions where cast are responsible for their own make-up and hair, Make-up and/or Hair Artists may provide additional support and assistance. Make-up and/or Hair Artists are typically present in larger venue...	The Makeup and Hair Designer is tasked with creating and applying makeup and hairstyles for performers prior to and during shows, ensuring their visual presentation aligns with the artistic vision of the production as outlined in the design plans. This includes the use of both cosmetic and special effects makeup. The designer is responsible for translating the creative vision into the performers' physical appearances through skillful application of makeup and hairstyles. They must consider elements such as production lighting that can affect the final look and engage in discussions with performers regarding any skincare issues or allergies. The role requires adaptability to work with individuals of all ages, genders, and diverse backgrounds. In larger productions, the Makeup and Hair Designer typically operates within a dedicated team, while in smaller settings, these responsibilities may be shared with other production staff.	The Makeup and Hair Coordinator is responsible for overseeing the application of makeup and hairstyles for models during photo shoots to ensure their appearance meets the specific aesthetic requirements of the campaign as defined by the creative team. This may involve the use of both traditional and avant-garde makeup techniques. The coordinator is tasked with interpreting the creative brief into the models' looks through precise makeup and hairstyling. They must take into account factors such as camera lighting that could influence the makeup and hair appearance. The role includes consulting with models about any skin sensitivities or allergies and requires versatility in working with models of various ages, genders, and cultural backgrounds. In large-scale campaigns, the Makeup and Hair Coordinator may lead a team of artists, while in smaller projects, they may personally handle the application of makeup and hair. ## Reason The negative description presents a Makeup and Hair Coordin...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 3e-05
• num_train_epochs: 10
• lr_scheduler_type: cosine
• warmup_ratio: 0.15
• bf16: True
• tf32: False
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 3e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.15
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	12	0.1692	0.0140
2.0	24	0.0151	0.0043
3.0	36	0.0079	0.0032
4.0	48	0.0059	0.0027
5.0	60	0.0052	0.0025
6.0	72	0.0041	0.0024
7.0	84	0.006	0.0024
8.0	96	0.0044	0.0024
9.0	108	0.0053	0.0023
10.0	120	0.0044	0.0023

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.4
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.1
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}