update code instructions and intro

README.md

@@ -3,12 +3,18 @@ license: apache-2.0
 ---
 
 # Model Description
-Mellum-base-4B is the first open-source installation of LLMs for code-related tasks by JetBrains. 
-The model is trained specifically for code completion task on >3 trillion tokens with 8192 context window on N programming languages. 
-We employed LLaMA-like architecture in total with 4B parameters without using Grouped Query Attention, which makes it convenient for both efficient in inference in cloud (e.g. with vLLM) and fast local inference (e.g. with llama.cpp or Ollama).
-Mellum was trained with AMP using bf16 precision, and the same bf16 version is uploaded to HuggingFace for public usage.
-It is designed for professional developer tooling integration (e.g., intelligent suggestions in IDEs), AI code assistants, and research applications in code understanding and generation. 
-Published model is a base model meaning that it does not excel in down-stream tasks, however it is fully suitable for SFT/RL fine-tuning.
+Mellum-base-4B is JetBrains' first open-source large language model (LLM) optimized for code-related tasks.
+
+Trained on over 4 trillion tokens with a context window of 8192 tokens across multiple programming languages, Mellum-base-4B is tailored specifically for code completion. 
+The model follows a LLaMA-style architecture with 4 billion parameters and does not use Grouped Query Attention (GQA), making it efficient for both cloud inference (e.g., via vLLM) and local deployment (e.g., using llama.cpp or Ollama).
+
+Mellum was trained using Automatic Mixed Precision (AMP) with bf16 precision. 
+The uploaded version on Hugging Face retains the bf16 format for public use.
+
+Designed for integration into professional developer tooling (e.g., intelligent code suggestions in IDEs), AI-powered coding assistants, and research on code understanding and generation, Mellum is also well-suited for educational applications and fine-tuning experiments.
+
+This release includes a base model, and some SFT models as well. 
+Keep in mind that base model is not fine-tuned for downstream tasks out-of-the-box, however, it is fully capable of supporting supervised fine-tuning (SFT) and reinforcement learning (RL) for adaptation to specific applications.
 
 # Training Data
 - Total Training Tokens: ~4.2 trillion tokens
@@ -16,12 +22,12 @@ Published model is a base model meaning that it does not excel in down-stream ta
 
 # Training Details
 - Context Window: 8,192 tokens
-- Optimization: Standard language modeling objective adapted for code completion and infilling.
+- Optimization: Standard language modeling objective.
 - Hardware: Cluster of 256 x H200 NVIDIA GPUs with Infiniband
 - Training Duration: ~20 days
 
 # Benchmarks
-In addition to the base model scores, we are providing scores for a Mellum fine-tuned for Python to provide model’s users with some feeling about potential capabilities.
+In addition to the base model scores, we are providing scores for a Mellum fine-tuned for Python to provide model’s users with some estimation about potential capabilities.
 
 ## RepoBench
 - Type: single-line
@@ -59,30 +65,77 @@ Java Subset:
 | Mellum-4b-sft-python  | 80.45%      | 48.19%     | 37.68%      |
 | Mellum-4b-base        | 66.21%      | 38.52%     | 29.70%      |
 
-# Intended Use
-- Integration into IDEs and code editors for powering code completion.
-- Research into code generation, AI pair programming, and infilling techniques.
-- Educational scenarios for code models fine-tuning.
-
 # Limitations
 - Biases: May reflect biases present in public codebases. For example it will likely produce code which is similar in style to the open-source repositories.
 - Security: Code suggestions should not be assumed to be secure or free of vulnerabilities.
 
 # Sample Usage
 Here’s an example of how to run and sample from the model:
-python
-CopyEdit
-*TODO: Insert sample code here*
+
+```python
+import json
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+example = """
+import sys
+import os
+import time
+
+sys.path.append(os.getcwd())
+
+from cluster.prepare_data import get_headers_pairs_list, write_dist_matrix
+from cluster.token_edit_distance import get_distance_matrix
+
+if len(sys.argv) < 3:
+    print(
+        "Too few arguments. You should provide: \n1. dataset_filename" +
+        "\n2. output_data_filename"
+    )
+    sys.exit()
+
+start = time.perf_counter()
+dataset_filename_ = sys.argv[1]
+output_data_filename_ = sys.argv[2]
+
+headers_pairs = get_headers_pairs_list(dataset_filename_, verbose=True)
+
+dist_matrix, max_dist = get_distance_matrix(
+    list(map(lambda x: x[1], headers_pairs)),
+    verbose=True
+)
+
+write_dist_matrix(dist_matrix, max_dist, output_data_filename_, verbose=True)
+
+end = time.perf_counter()
+"""
+
+tokenizer = AutoTokenizer.from_pretrained('mellum-base-4b')
+model = AutoModelForCausalLM.from_pretrained('mellum-base-4b')
+encoded_input = tokenizer(example, return_tensors='pt', return_token_type_ids=False)
+input_len = len(encoded_input["input_ids"][0])
+out = model.generate(
+    **encoded_input,
+    max_new_tokens=100,
+    num_beams=1,
+    pad_token_id=tokenizer.eos_token_id,
+    eos_token_id=tokenizer.eos_token_id,
+)
+print("### Context")
+print(tokenizer.decode(out[0][:input_len]))
+print("### Prediction")
+print(tokenizer.decode(out[0][input_len:]))
+```
 
 # Citation
 If you use this model, please cite:
-bibtex
-CopyEdit
-@misc{jetbrains_code_completion_llm,
-  title={Mellum},
-  author={JetBrains},
+
+```bibtex
+@misc{mellum-base-4b,
+  title={Mellum base 4B},
+  author={Nikita Pavlichenko, Iurii Nazarov, Ivan Dolgov, Julia Reshetnikova, Ekaterina Garanina, Karol Lasocki, Sergei Boitsov, Dariia Karaeva, Ivan Bondyrev, Maksim Sheptyakov, Dmitry Ustalov, Nikita Abramov, Olga Kolomyttseva, Kseniia Lysaniuk, Ilia Zavidnyi, Anton Semenkin, Uladzislau Sazanovich},
   year={2025},
 }
+```
 
 # Contact
-
+For questions, collaborations and requests reach us out via [email protected]