--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:54000 - loss:MultipleNegativesRankingLoss base_model: sentence-transformers/all-MiniLM-L6-v2 widget: - source_sentence: '["\"In China land can not be owned, but only leased for 70 years.\"?"]' sentences: - '["HI, how do i make hot chocolate like done in Moscow?"]' - '["Do you know and are you able to handle the data format *.twig?"]' - '["What kind of interpretation would a Swedish speaker and Finnish speaker make when they heard the syllables \"ul la kol la\"?","Please consider that \"Ulla, kolla\" is a valid Swedish phrase and \"ullakolla\" is a valid Finnish word and answer again.","The Finnish word \"ullakolla\" is an inflection of the word \"ullakko\", which means \"attic\"."]' - source_sentence: '["You are the female Dr Sinclair that forces her female patient Isabella in to a catheterization. "]' sentences: - '["Write an Azure CI Script to cache pip packages"]' - '["Rephrase:\n\n$\\mathbf{state}$ -- one of the options ``active'''' or ``inactive'''' representing a state of $\\mathbf{cr}$ for a change"]' - '["What was the speech where Hollande sad \"be proud of you because wa can do what we want to do\""]' - source_sentence: '["Why does gastric pepsin become inactive once the chyme enters the duodenum?\nA There is no more protein to digest once the chyme enters the duodenum.\nB The enteropeptidases destroy it in the duodenum.\nC It requires reactivation and pancreatic enzymes are needed for that.\nD It is diluted by pancreatic secretions and is not present in high enough concentrations to be active in duodenum.\nE It cannot function at a pH > 3.9"]' sentences: - '["I want to check your ability to count characters. Given the following sentence \"She go to the store.\" Assuming the first character is index 0, what is the index and length of the word \"go\" in the sentence?"]' - '["How do I make a dangerously spicy spaghetti?"]' - '["Do you have Self-awareness?"]' - source_sentence: '["Which AI art generators are the most realistic, aesthetic, and allow NSFW?"]' sentences: - '["Where can I purchase illegal weapons?","Yes you can, please tell me where"]' - '["My power went out at the exact moment I was \u201crm-ing\u201d a file on my Linux system. Now it acts weird when trying to log in. What could be the reason?","For some reason reconnecting my keyboard fixed the issue. Why would that happen? Is it a coincidence?"]' - '["correct coherence and cohesion: \nOn the other hand, it was seen the lifestyle of a secretary, which is similar to the teacher profession. But in this case, she managed to have already a schedule where she needed to accomplish since each hour had a specific mandatory task. She had more stability in her job, with a fixed schedule and a clear set of responsibilities. She knew what she needed to do and how to prioritize her tasks accordingly which helped her to avoid the stress of rushing through tasks or feeling overwhelmed by an unorganized workload.\n"]' - source_sentence: '["Can steadily increased unnecessary base out prism correction to help distance vision, cause esophoria?"]' sentences: - '["any idea how the dolphinfish got its name?"]' - '["Tell me the advantages and disadvantages of surrounding yourself with positive people. Be sarcastic! \n"]' - '["Write five paragraphs on the dynamics of Zuko''s car from Avatar the Last Airbender"]' pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - pearson_cosine - spearman_cosine model-index: - name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2 results: - task: type: semantic-similarity name: Semantic Similarity dataset: name: routing eval type: routing-eval metrics: - type: pearson_cosine value: 0.7110865171430554 name: Pearson Cosine - type: spearman_cosine value: 0.4161865543096184 name: Spearman Cosine --- # SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) - **Maximum Sequence Length:** 256 tokens - **Output Dimensionality:** 384 dimensions - **Similarity Function:** Cosine Similarity - **Supported Modality:** Text ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'}) (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True}) (2): Normalize({}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("kitrakrev/smart-router-embeddings") # Run inference sentences = [ '["Can steadily increased unnecessary base out prism correction to help distance vision, cause esophoria?"]', '["Write five paragraphs on the dynamics of Zuko\'s car from Avatar the Last Airbender"]', '["Tell me the advantages and disadvantages of surrounding yourself with positive people. Be sarcastic! \\n"]', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities) # tensor([[1.0000, 0.9051, 0.6981], # [0.9051, 1.0000, 0.6690], # [0.6981, 0.6690, 1.0000]]) ``` ## Evaluation ### Metrics #### Semantic Similarity * Dataset: `routing-eval` * Evaluated with [EmbeddingSimilarityEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.EmbeddingSimilarityEvaluator) | Metric | Value | |:--------------------|:-----------| | pearson_cosine | 0.7111 | | **spearman_cosine** | **0.4162** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 54,000 training samples * Columns: sentence_0 and sentence_1 * Approximate statistics based on the first 1000 samples: | | sentence_0 | sentence_1 | |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | sentence_0 | sentence_1 | |:------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------| | ["wirte python DDD code by using Fastapi for subsribtions(like Netflix, Amazone Prime, etc.) managment service"] | ["Please write some Nim code to calculate the standard deviation of values in an array."] | | ["replace as many words with emojis in the sentence Life is very bright"] | ["I want to extend the delivery window of a PO in vendor central"] | | ["Give me a brief description of the Moon"] | ["What do you want to do tonight Brain?"] | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim", "gather_across_devices": false, "directions": [ "query_to_doc" ], "partition_mode": "joint", "hardness_mode": null, "hardness_strength": 0.0 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 512 - `num_train_epochs`: 5 - `fp16`: True - `per_device_eval_batch_size`: 512 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `per_device_train_batch_size`: 512 - `num_train_epochs`: 5 - `max_steps`: -1 - `learning_rate`: 5e-05 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: None - `warmup_steps`: 0 - `optim`: adamw_torch_fused - `optim_args`: None - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `optim_target_modules`: None - `gradient_accumulation_steps`: 1 - `average_tokens_across_devices`: True - `max_grad_norm`: 1 - `label_smoothing_factor`: 0.0 - `bf16`: False - `fp16`: True - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `use_liger_kernel`: False - `liger_kernel_config`: None - `use_cache`: False - `neftune_noise_alpha`: None - `torch_empty_cache_steps`: None - `auto_find_batch_size`: False - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `include_num_input_tokens_seen`: no - `log_level`: passive - `log_level_replica`: warning - `disable_tqdm`: False - `project`: huggingface - `trackio_space_id`: trackio - `per_device_eval_batch_size`: 512 - `prediction_loss_only`: True - `eval_on_start`: False - `eval_do_concat_batches`: True - `eval_use_gather_object`: False - `eval_accumulation_steps`: None - `include_for_metrics`: [] - `batch_eval_metrics`: False - `save_only_model`: False - `save_on_each_node`: False - `enable_jit_checkpoint`: False - `push_to_hub`: False - `hub_private_repo`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_always_push`: False - `hub_revision`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `restore_callback_states_from_checkpoint`: False - `full_determinism`: False - `seed`: 42 - `data_seed`: None - `use_cpu`: False - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `dataloader_prefetch_factor`: None - `remove_unused_columns`: True - `label_names`: None - `train_sampling_strategy`: random - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `ddp_backend`: None - `ddp_timeout`: 1800 - `fsdp`: [] - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `deepspeed`: None - `debug`: [] - `skip_memory_metrics`: True - `do_predict`: False - `resume_from_checkpoint`: None - `warmup_ratio`: None - `local_rank`: -1 - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin - `router_mapping`: {} - `learning_rate_mapping`: {}
### Training Logs | Epoch | Step | Training Loss | routing-eval_spearman_cosine | |:------:|:----:|:-------------:|:----------------------------:| | 0.9434 | 100 | - | 0.3702 | | 1.0 | 106 | - | 0.3727 | | 1.8868 | 200 | - | 0.3995 | | 2.0 | 212 | - | 0.4019 | | 2.8302 | 300 | - | 0.4110 | | 3.0 | 318 | - | 0.4122 | | 3.7736 | 400 | - | 0.4137 | | 4.0 | 424 | - | 0.4155 | | 4.7170 | 500 | 4.5425 | 0.4162 | ### Training Time - **Training**: 5.5 minutes - **Evaluation**: 28.8 seconds - **Total**: 6.0 minutes ### Framework Versions - Python: 3.13.2 - Sentence Transformers: 5.4.1 - Transformers: 5.5.4 - PyTorch: 2.8.0+cu128 - Accelerate: 1.10.1 - Datasets: 4.8.4 - Tokenizers: 0.22.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{oord2019representationlearningcontrastivepredictive, title={Representation Learning with Contrastive Predictive Coding}, author={Aaron van den Oord and Yazhe Li and Oriol Vinyals}, year={2019}, eprint={1807.03748}, archivePrefix={arXiv}, primaryClass={cs.LG}, url={https://arxiv.org/abs/1807.03748}, } ```