--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:25743 - loss:MultipleNegativesRankingLoss base_model: Lajavaness/bilingual-embedding-small widget: - source_sentence: Luciano Hang da HAVAN, send 200 oxygen cylinders to Manaus in their planes. attitude of man of true, bravo! O sentences: - The photo of Pedro Sánchez «enjoying while Gran Canaria burns» - Havan owner Luciano Hang donated 200 oxygen cylinders to Manaus - Video of the show in Shanghai staged by robots made in China - source_sentence: '"PERSEVERANCE" SEND THE FIRST COLOR IMAGES FROM THE SURFACE OF MARS' sentences: - If an election has 51% of the votes cast, the election is annulled. - This video shows Indian Air Force attack helicopters flying over Pangong Lake in Ladakh. - The first video images of Mars from the Perseverance rover - source_sentence: SPEECH BY PEDRO CASTILLO, IT WAS BASED ON THE HATE OF SPAIN OF A PAST PRE-HISPANIC THAT I ONLY KNOW EXPLAINS FROM THE MOST ABSOLUTE IGNORANCE AND STUPIDITY" KING OF SPAINIn fact, between the president of Colombia, Duque, and the king of Spain, the most regretful of having come to the inauguration of the clown with a hat is the latter. sentences: - Felipe VI said that Pedro Castillo's speech is explained from ignorance and stupidity - '"Population poorly tolerated quarantine and social distancing measures during the Spanish flu, when the first deconfinement took place, abandoning all precautionary measures"' - Genuine photo of Philippine lawmaker Sarah Elago supporting mandatory military training in schools - source_sentence: Australia Day has nothing to do with Captain Cook or Botany Bay The Landing of Captain Cook at the site of Sydney happened on the 28th of April 1770 - NOT on the 26th of January 1770. The first fleet arrived in Australia on 18 January 1788 and landed at Botany Bay on 20 January 1788. AUSTRALIA DAY CELEBRATES THE DAY ALL AUSTRALIANS STOPPED BEING BRITISH CITIZENS AND BECAME AUSTRALIAN CITIZENS IN 1949. Facts about Australia Day Our Education system and the popular press is not competently advising our children !! Twisting the truth a bit. Don't expect the media to educate you, that's not part of their agenda. Australia Day does not celebrate the arrival of the first fleet or the invasion of anything. The First Fleet arrived in Botany Bay on the 18th of January. However, Captain Cook's landing was included in Australia Day celebrations as a reminder of a significant historical event. Since the extravagant bicentenary celebrations of 1988, when Sydney-siders decided Captain Cook's landing should become the focus of the Australia Day commemoration, the importance of this date for all Australians has begun to fade. Now, a generation later, it's all but lost. This is because our politicians and educators have not been doing a good job promoting the day. Our politicians have not been advertising the real reason for Australia Day, and our educators have not been teaching our children the importance of the 26th of January to all Australians. The media, as usual, is happy to twist the truth for the sake of controversy. In recent years, the media has helped fan the flames of discontent among the Aboriginal community. Many are now so offended by what they see as a celebration of the beginning of the darkest days of Aboriginal history, they want the date changed. Various local Councils are seeking to remove themselves from Australia Day celebrations, even refusing to participate in citizenship ceremonies, and calls are going out to have Australia Day on a different day. The big question is, why has the Government allowed this misconception to continue? Captain Cook didn't land on the 26th of January. So changing the date of any celebration of Captain Cook's landing would not have any impact on Australia Day, but maybe it would clear the way for the truth about Australia Day. The reality is, the Aborigines in this country suffered under the hands of British colonialism. This is as much Australia's history as the landing of the first fleet, and both should be remembered, equally. Both should be taught, side by side, in our schools. Australians of today reject what was done under British governance to the Aborigines. We reject what was done under British governance to the Irish and many other cultures around the world. So, after the horrors of WWII, we decided to fix it. We became our own people. On the 26th of January 1949, the Australian nationality came into existence when the Nationality and Citizenship Act 1948 was enacted. That was the day we were first called Australians and allowed to travel with Passports as Australians. Under the Nationality Act 1920 (Cth), all Aborigines and Torres Strait Islanders born after January 1, 1921, gained the status of British subjects. In 1949, therefore, they automatically became Australian citizens under the Nationality and Citizenship Act 1948. Before that special date, all people living in Australia, including Aborigines born after 1921, were called 'British Subjects' and forced to travel on British Passports and fight in British wars. We all became Australians on the same day! This is why we celebrate Australia Day on the 26th of January! This was the day Australians became free to make our own decisions about which wars we would fight and how our citizens would be treated. It was the day Aborigines were declared Australians. Until this date, Aborigines were not protected by law. For the first time since Cook's landing, this new Act gave Aboriginal Australians by inference and precedent the full protection of Australian law. Because of this Act, the government became free to help Aborigines, and since that day much has been done to assist Aboriginal Australians, including saying 'sorry' for the previous atrocities done before this law came into being. This was a great day for all Australians! This is why the 26th of January is the day new Australians receive their citizenship. It is a day which celebrates the implementation of the Nationality and Citizenship Act of 1948 - the Act which gave freedom and protection to the first Australians and gives all Australians, old and new, the right to live under the protection of Australian Law, united as one nation. Now, isn't that cause for celebration? Education is key! There is a great need for education on the real reason we celebrate Australia Day on the 26th of January. This reason needs to be advertised and taught in schools. We all need to remember this one very special day in Australia's history, when freedom came to all Australians. What was achieved that day is something for which all Australians can be proud! We need to remember both the good and the bad in our history, but the emphasis must be the freedom and unity all Australians now have, because of what was done on the 26th of January 1949, to allow all of us to live without fear in a land of peace. Isn't it time all Australians were taught the real reason we celebrate Australia Day on Jan 26th? sentences: - Australia Day is commemorated on the day when Australian citizenship law passed - Sri Lankan Defense Secretary Kamal Gunarathne praised ex-minister Rishad Bathiudeen in speech - The corona virus is possibly caused due to the use of rhino horn - source_sentence: This is how the buildings moved this morning in Mexico City sentences: - The video captures the earthquake of June 23, 2020 - Photo shows bombing in Gaza in January 2022 - Photo shows former South African president Jacob Zuma in prison pipeline_tag: sentence-similarity library_name: sentence-transformers --- # SentenceTransformer based on Lajavaness/bilingual-embedding-small This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Lajavaness/bilingual-embedding-small](https://huggingface.co/Lajavaness/bilingual-embedding-small). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Lajavaness/bilingual-embedding-small](https://huggingface.co/Lajavaness/bilingual-embedding-small) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 384 dimensions - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BilingualModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference sentences = [ 'This is how the buildings moved this morning in Mexico City', 'The video captures the earthquake of June 23, 2020', 'Photo shows bombing in Gaza in January 2022', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 25,743 training samples * Columns: sentence_0, sentence_1, and label * Approximate statistics based on the first 1000 samples: | | sentence_0 | sentence_1 | label | |:--------|:------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:--------------------------------------------------------------| | type | string | string | float | | details | | | | * Samples: | sentence_0 | sentence_1 | label | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------|:-----------------| | best music k.m KOSE CELLIE HINS GUINOT SKIN CARE KWhat people fear most is not being physically disabled, but giving up on themselves. There are still many beautiful things in life to aspire to! This stunning performance, known as the American spirit, brought tears to the eyes of 10,000 spectators. Male dancer Babo has been blind since childhood due to a fire in his home. In order to protect him, his mother held him tightly in her arms and jumped from the 7th floor. The mother died as a result, and the little baby became blind due to bleeding from the fundus. His mother was an ice skater before he died, and Babo also had a soft spot for ice skating. Although he couldn't see anything, he still pursued dance enthusiastically. He danced the famous tango "La Cumparsita" with his partner at the World Figure Skating Championships in Helsinki! 1. His ears are like bats that can measure the sound and distance around him. 2. The female dancer is very amazing. She danced with him and led him for... | Performance by a blind American ice dancer | 1.0 | | Photo from 2016. "Good" times when health was "fine" and the press did not report anything about. Bunch of Hypocrites...Let's go fight my people... . left right not army above all | Photo of a hospital in 2016. Good times when health was "good" and the press didn't report anything about it | 1.0 | | Haifa Oh Tel Aviv-Yafo Oh N WEST BANK Jerusalem is GAZA STRIPE Be'er Sheva Israel 65 65 35 35 15 M5 10 40Google and Apple maps have officially removed Palestine from the World Maps. Today Palestine was erased from the maps tomorrow Palestine will be erased from the world. PUT PALESTINE BACK ON THE MAP. Please unite now Pakistanio. Enemy is very strong if we are divided. Think just about Pakistan. Support each other, support Pakistan and support your leadership. | Google and Apple removed Palestine from its maps | 1.0 | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `num_train_epochs`: 1 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: no - `prediction_loss_only`: True - `per_device_train_batch_size`: 8 - `per_device_eval_batch_size`: 8 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1 - `num_train_epochs`: 1 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin
### Training Logs | Epoch | Step | Training Loss | |:------:|:----:|:-------------:| | 0.1554 | 500 | 0.1021 | | 0.3108 | 1000 | 0.0732 | | 0.4661 | 1500 | 0.0781 | | 0.6215 | 2000 | 0.0762 | | 0.7769 | 2500 | 0.0763 | | 0.9323 | 3000 | 0.0739 | | 0.1554 | 500 | 0.0474 | | 0.3108 | 1000 | 0.0478 | | 0.4661 | 1500 | 0.0558 | | 0.6215 | 2000 | 0.0542 | | 0.7769 | 2500 | 0.0457 | | 0.9323 | 3000 | 0.0395 | ### Framework Versions - Python: 3.11.11 - Sentence Transformers: 3.4.1 - Transformers: 4.48.3 - PyTorch: 2.5.1+cu124 - Accelerate: 1.3.0 - Datasets: 3.3.1 - Tokenizers: 0.21.0 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```