Update README.md

README.md

@@ -14,15 +14,15 @@ model-index:
       name: Text Classification
     metrics:
     - type: loss
-      value: 0.06214376166462898
+      value: 0.061837393790483475
     - type: mse
-      value: 0.06214376166462898
+      value: 0.061837393790483475
       name: Validation Mean Squared Error
     - type: r2
-      value: 0.5705611109733582
+      value: 0.5726782083511353
       name: Validation R-Squared
     - type: mae
-      value: 0.1838676631450653
+      value: 0.183049738407135
       name: Validation Mean Absolute Error
 language:
 - en
@@ -33,57 +33,68 @@ This model utilizes the [Distilroberta base](https://huggingface.co/distilrobert
 
 ## Model description
 
-The model evaluates the query for completeness and grammatical correctness, providing a score between 0 and 1, where 1 indicates correctness.
+A regression head has been appended to the DistilRoBERTa model to tailor it for a regression task. This additional component is crucial and needs to be loaded alongside the base model during inference to ensure accurate predictions. The model evaluates the query for completeness and grammatical correctness, providing a score between 0 and 1, where 1 indicates correctness.
 
 ## Usage
 
 Inference API has been disabled as this is a regression task, not a text classification task, and HuggingFace does not provide a pipeline for regression tasks.
 
 ```python
-# Sentences
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+tokenizer = AutoTokenizer.from_pretrained("AdamCodd/distilroberta-query-wellformedness")
+
+class RegressionModel(torch.nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.model = AutoModelForSequenceClassification.from_pretrained("AdamCodd/distilroberta-query-wellformedness")
+        self.regression_head = torch.nn.Linear(self.model.config.hidden_size, 1)
+
+    def forward(self, input_ids, attention_mask, **kwargs):
+        outputs = self.model.base_model(input_ids=input_ids, attention_mask=attention_mask)
+        rating = self.regression_head(outputs.last_hidden_state[:, 0, :])
+        rating = torch.sigmoid(rating)
+        return rating.squeeze()
+
+regression_model = RegressionModel()
+# Do not forget to set the correct path to load the regression head
+regression_model.regression_head.load_state_dict(torch.load(r"path_to_the_regression_head.pth"))
+regression_model.eval()
+# Examples
 sentences = [
-    "The cat and dog in the yard.",  # Incorrect - It should be "The cat and dog are in the yard."
-    "she don't like apples.",  # Incorrect - It should be "She doesn't like apples."
-    "Is rain sunny days sometimes?",  # Incorrect - It should be "Do sunny days sometimes have rain?"
-    "She enjoys reading books and playing chess.",  # Correct
-    "How many planets are there in our solar system?"  # Correct
+    "The cat and dog in the yard.",
+    "she don't like apples.",
+    "Is rain sunny days sometimes?",
+    "She enjoys reading books and playing chess.",
+    "How many planets are there in our solar system?"
 ]
 
-# Tokenizing the sentences
 inputs = tokenizer(sentences, truncation=True, padding=True, return_tensors='pt')
 
-# Getting the model's predictions
-with torch.no_grad():  # Disabling gradient calculation as we are only doing inference
-    model.eval()  # Setting the model to evaluation mode
-    predicted_ratings = model(
-        input_ids=inputs['input_ids'], 
-        attention_mask=inputs['attention_mask']
-    )
+with torch.no_grad():
+    outputs = regression_model(input_ids=inputs['input_ids'], attention_mask=inputs['attention_mask'])
 
-# The predicted_ratings is a tensor, so we'll convert it to a list of standard Python numbers
-predicted_ratings = predicted_ratings.squeeze().tolist()
-
-# Printing the predicted ratings
-for i, rating in enumerate(predicted_ratings):
+predictions = outputs.tolist()
+for i, rating in enumerate(predictions):
     print(f'Sentence: {sentences[i]}')
     print(f'Predicted Rating: {rating}\n')
 ```
 Output:
 ```
 Sentence: The cat and dog in the yard.
-Predicted Rating: 0.3482873737812042
+Predicted Rating: 0.20011138916015625
 
 Sentence: she don't like apples.
-Predicted Rating: 0.07787154614925385
+Predicted Rating: 0.08289700001478195
 
 Sentence: Is rain sunny days sometimes?
-Predicted Rating: 0.19854165613651276
+Predicted Rating: 0.20011138916015625
 
 Sentence: She enjoys reading books and playing chess.
-Predicted Rating: 0.9327691793441772
+Predicted Rating: 0.8915354013442993
 
 Sentence: How many planets are there in our solar system?
-Predicted Rating: 0.9746372103691101
+Predicted Rating: 0.974799394607544
 ```
 
 ## Training and evaluation data
@@ -101,7 +112,7 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: AdamW with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- lr_scheduler_warmup_steps: 450
+- lr_scheduler_warmup_steps: 400
 - num_epochs: 5
 
 ### Training results
@@ -109,10 +120,10 @@ The following hyperparameters were used during training:
 Metrics: Mean Squared Error, R-Squared, Mean Absolute Error
 
 ```
-'test_loss': 0.06214376166462898,
-'test_mse': 0.06214376166462898,
-'test_r2': 0.5705611109733582,
-'test_mae': 0.1838676631450653
+'test_loss': 0.061837393790483475,
+'test_mse': 0.061837393790483475,
+'test_r2': 0.5726782083511353,
+'test_mae': 0.183049738407135
 ```
 
 ### Framework versions