AmelieSchreiber commited on
Commit
cf09ded
·
1 Parent(s): f3daf6c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -2
README.md CHANGED
@@ -47,20 +47,70 @@ structural models mentioned above, meaning the model may return some false posit
47
  ## Overfitting Issues
48
 
49
  ```python
50
- ({'accuracy': 0.9908574638195745,
51
  'precision': 0.7748830511095647,
52
  'recall': 0.9862043939282111,
53
  'f1': 0.8678649909611492,
54
  'auc': 0.9886039823329382,
55
  'mcc': 0.8699396085712834},
56
- {'accuracy': 0.9486280975482552,
57
  'precision': 0.40980984516603186,
58
  'recall': 0.827004864790918,
59
  'f1': 0.5480444772577421,
60
  'auc': 0.890196425388581,
61
  'mcc': 0.560633448203768})
62
  ```
 
63
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
64
 
65
  ## Running Inference
66
 
 
47
  ## Overfitting Issues
48
 
49
  ```python
50
+ Train: ({'accuracy': 0.9908574638195745,
51
  'precision': 0.7748830511095647,
52
  'recall': 0.9862043939282111,
53
  'f1': 0.8678649909611492,
54
  'auc': 0.9886039823329382,
55
  'mcc': 0.8699396085712834},
56
+ Test: {'accuracy': 0.9486280975482552,
57
  'precision': 0.40980984516603186,
58
  'recall': 0.827004864790918,
59
  'f1': 0.5480444772577421,
60
  'auc': 0.890196425388581,
61
  'mcc': 0.560633448203768})
62
  ```
63
+ Let's analyze the train and test metrics one by one:
64
 
65
+ ### **1. Accuracy**
66
+ - **Train**: 99.09%
67
+ - **Test**: 94.86%
68
+
69
+ The accuracy is notably high in both training and test datasets, indicating that the model makes correct predictions a significant majority of the time. The high accuracy on the test dataset signifies good generalization capabilities.
70
+
71
+ ### **2. Precision**
72
+ - **Train**: 77.49%
73
+ - **Test**: 41.00%
74
+
75
+ While the precision is quite good in the training dataset, it sees a decrease in the test dataset. This suggests that a substantial proportion of the instances that the model predicts as positive are actually negative, which could potentially lead to a higher false-positive rate.
76
+
77
+ ### **3. Recall**
78
+ - **Train**: 98.62%
79
+ - **Test**: 82.70%
80
+
81
+ The recall is impressive in both the training and test datasets, indicating that the model is able to identify a large proportion of actual positive instances correctly. A high recall in the test dataset suggests that the model maintains its sensitivity in identifying positive cases when generalized to unseen data.
82
+
83
+ ### **4. F1-Score**
84
+ - **Train**: 86.79%
85
+ - **Test**: 54.80%
86
+
87
+ The F1-score, which is the harmonic mean of precision and recall, is good in the training dataset but sees a decrease in the test dataset. The decrease in the F1-score from training to testing suggests a worsened balance between precision and recall in the unseen data, largely due to a decrease in precision.
88
+
89
+ ### **5. AUC (Area Under the ROC Curve)**
90
+ - **Train**: 98.86%
91
+ - **Test**: 89.02%
92
+
93
+ The AUC is quite high in both the training and test datasets, indicating that the model has a good capability to distinguish between the positive and negative classes. A high AUC in the test dataset further suggests that the model generalizes well to unseen data.
94
+
95
+ ### **6. MCC (Matthews Correlation Coefficient)**
96
+ - **Train**: 86.99%
97
+ - **Test**: 56.06%
98
+
99
+ The MCC, a balanced metric which takes into account true and false positives and negatives, is good in the training set but decreases in the test set. This suggests a diminished quality of binary classifications on the test dataset compared to the training dataset.
100
+
101
+ ### **Overall Analysis**
102
+
103
+ - **Generalization**: The metrics reveal that the model has a good generalization capability, as indicated by the high accuracy, recall, and AUC on the test dataset.
104
+
105
+ - **Precision-Recall Trade-off**: The model maintains a high recall but experiences a dip in precision in the test dataset, leading to a lower F1-score. It indicates a tendency to predict more false positives, which might require tuning to balance precision and recall optimally.
106
+
107
+ - **Improvement Suggestions**:
108
+ - **Precision Improvement**: Focus on strategies to improve precision, such as feature engineering or experimenting with different classification thresholds.
109
+ - **Hyperparameter Tuning**: Engaging in hyperparameter tuning might assist in enhancing the model's performance on unseen data.
110
+ - **Complexity Reduction**: Consider reducing the model's complexity to prevent potential overfitting and improve generalization.
111
+ - **Class Imbalance**: If the dataset has a class imbalance, techniques such as resampling or utilizing class weights might be beneficial.
112
+
113
+ In conclusion, the model performs well on the training dataset and maintains a reasonably good performance on the test dataset, demonstrating a solid generalization capability. However, the decrease in certain metrics like precision and F1-score in the test dataset compared to the training dataset indicates room for improvement to optimize the model further for unseen data. It would be advantageous to enhance precision without significantly compromising recall to achieve a more harmonious balance between the two.
114
 
115
  ## Running Inference
116