pj-mathematician/JobSkillBGE-large-en-v1.5-v2

Job-Skill matching fintuned BAAI/bge-large-en-v1.5 (v2)

Top performing model on TalentCLEF 2025 Task B. Use it for job title <-> skill set matching

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-large-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("pj-mathematician/JobSkillBGE-large-en-v1.5-v2")
# Run inference
sentences = [
    "An insulation supervisor, regardless of the specific type of insulation material or installation area, requires strong project management skills, knowledge of building codes and safety regulations, and expertise in insulation techniques to oversee the installation process effectively and ensure quality standards are met.\n['insulation supervisor', 'supervisor of installation of insulating materials', 'supervisor of insulation materials installation', 'supervisor of installation of insulation', 'solid wall insulation installation supervisor', 'insulation installers supervisor', 'cavity wall insulation installation supervisor', 'loft insulation installation supervisor']",
    "The skill of installing insulation material is primarily required by job roles such as insulation workers, HVAC technicians, and construction specialists, who are responsible for improving energy efficiency and thermal comfort in buildings by correctly fitting and fixing insulation materials in various structures.\n['install insulation material', 'insulate structure', 'fix insulation', 'insulation material installation', 'installation of insulation material', 'fitting insulation', 'insulating structure', 'installing insulation material', 'fixing insulation', 'fit insulation']",
    "Job roles such as Food Safety Inspector, Public Health Officer, and Environmental Health Specialist require the skill of taking action on food safety violations to ensure compliance with health regulations and maintain public safety standards.\n['take action on food safety violations', 'invoke action on food safety violations', 'agree action on food safety violations', 'pursue action on food safety violations', 'determine action on food safety violations']",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Dataset: full_en
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7204
cosine_accuracy@20	1.0
cosine_accuracy@50	1.0
cosine_accuracy@100	1.0
cosine_accuracy@150	1.0
cosine_accuracy@200	1.0
cosine_precision@1	0.7204
cosine_precision@20	0.4916
cosine_precision@50	0.387
cosine_precision@100	0.3046
cosine_precision@150	0.2575
cosine_precision@200	0.225
cosine_recall@1	0.0102
cosine_recall@20	0.1316
cosine_recall@50	0.2509
cosine_recall@100	0.3866
cosine_recall@150	0.4822
cosine_recall@200	0.5556
cosine_ndcg@1	0.7204
cosine_ndcg@20	0.5307
cosine_ndcg@50	0.4445
cosine_ndcg@100	0.4348
cosine_ndcg@150	0.4782
cosine_ndcg@200	0.5218
cosine_mrr@1	0.7204
cosine_mrr@20	0.8362
cosine_mrr@50	0.8362
cosine_mrr@100	0.8362
cosine_mrr@150	0.8362
cosine_mrr@200	0.8362
cosine_map@1	0.7204
cosine_map@20	0.335
cosine_map@50	0.2334
cosine_map@100	0.2057
cosine_map@150	0.2211
cosine_map@200	0.2387
cosine_map@500	0.288

Training Details

Training Dataset

Unnamed Dataset

• Size: 114,699 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 78 tokens mean: 144.94 tokens max: 354 tokens	min: 51 tokens mean: 114.13 tokens max: 274 tokens

• Samples:

  anchor	positive
  A technical director or any of its synonyms requires a strong blend of technical expertise and leadership skills, including the ability to oversee technical operations, manage teams, and ensure the successful execution of technical projects while maintaining operational efficiency and innovation. ['technical director', 'technical and operations director', 'head of technical', 'director of technical arts', 'head of technical department', 'technical supervisor', 'technical manager']	Job roles that require promoting health and safety include occupational health and safety specialists, safety managers, and public health educators, all of whom work to ensure safe and healthy environments in workplaces and communities. ['promote health and safety', 'promote importance of health and safety', 'promoting health and safety', 'advertise health and safety']
  A technical director or any of its synonyms requires a strong blend of technical expertise and leadership skills, including the ability to oversee technical operations, manage teams, and ensure the successful execution of technical projects while maintaining operational efficiency and innovation. ['technical director', 'technical and operations director', 'head of technical', 'director of technical arts', 'head of technical department', 'technical supervisor', 'technical manager']	Job roles that require organizing rehearsals include directors, choreographers, and conductors in theater, dance, and music ensembles, who must efficiently plan and schedule practice sessions to prepare performers for a successful final performance. ['organise rehearsals', 'organise rehearsal', 'organize rehearsals', 'plan rehearsals', 'arrange rehearsals', 'organising rehearsals', 'schedule rehearsals']
  A technical director or any of its synonyms requires a strong blend of technical expertise and leadership skills, including the ability to oversee technical operations, manage teams, and ensure the successful execution of technical projects while maintaining operational efficiency and innovation. ['technical director', 'technical and operations director', 'head of technical', 'director of technical arts', 'head of technical department', 'technical supervisor', 'technical manager']	Job roles such as Health and Safety Managers, Environmental Health Officers, and Risk Management Specialists often require the skill of negotiating health and safety issues with third parties to ensure compliance and protection standards are met across different organizations and sites. ['negotiate health and safety issues with third parties', 'agree with third parties on health and safety', 'negotiate issues on health and safety with third parties', 'negotiate with third parties on health and safety issues', 'negotiate health and safety matters with third parties']

• Loss: CachedGISTEmbedLoss with these parameters:

  {'guide': SentenceTransformer(
    (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
    (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
    (2): Normalize()
  ), 'temperature': 0.01, 'mini_batch_size': 32, 'margin_strategy': 'absolute', 'margin': 0.0}
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• gradient_accumulation_steps: 2
• num_train_epochs: 5
• warmup_ratio: 0.05
• log_on_each_node: False
• fp16: True
• dataloader_num_workers: 4
• ddp_find_unused_parameters: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 2
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.05
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: False
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: True
• dataloader_num_workers: 4
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: True
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	full_en_cosine_ndcg@200
-1	-1	-	0.4795
0.0022	1	10.6462	-
0.2232	100	4.5115	-
0.4464	200	2.9237	0.5255
0.6696	300	2.5327	-
0.8929	400	2.3451	0.5305
1.1161	500	1.9882	-
1.3393	600	1.7738	0.5240
1.5625	700	1.7365	-
1.7857	800	1.6932	0.5251
2.0089	900	1.6184	-
2.2321	1000	1.285	0.5254
2.4554	1100	1.2651	-
2.6786	1200	1.2739	0.5238
2.9018	1300	1.2625	-
3.125	1400	1.0726	0.5251
3.3482	1500	0.9606	-
3.5714	1600	0.9594	0.5214
3.7946	1700	0.954	-
4.0179	1800	0.9264	0.5239
4.2411	1900	0.7486	-
4.4643	2000	0.7424	0.5218
4.6875	2100	0.7127	-
4.9107	2200	0.7129	0.5218

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 4.1.0
• Transformers: 4.51.2
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}