Added Adversarial model.

README.md

@@ -0,0 +1,118 @@
+---
+tags:
+- vision
+- clip
+- fine-tuned
+- PatchCamelyon
+- medical-imaging
+license: apache-2.0
+library_name: transformers
+model_type: clip_vision_model
+datasets:
+- 1aurent/PatchCamelyon
+---
+
+# CLIP ViT Base Patch32 Fine-Tuned on PatchCamelyon (PCAM)
+
+## Overview
+This repository contains a model trained on adversarial data of the [CLIP ViT Base Patch32 finetuned](https://huggingface.co/lens-ai/clip-vit-base-patch32_pcam_finetuned) model on the [PatchCamelyon (PCAM)](https://huggingface.co/datasets/1aurent/PatchCamelyon) dataset and also on [PatchCamelyon Adversarial(PCAM)](https://huggingface.co/datasets/lens-ai/adversarial_pcam) dataset.The model is optimized for histopathological image classification.
+
+## Model Description
+
+- **Model Type:** CLIP Vision Transformer (ViT-B/32) with classification head
+- **Task:** Binary classification of histopathological images
+- **Training Data:** PatchCamelyon dataset
+- **Input:** RGB images of size 224x224 pixels
+- **Output:** Binary classification (cancer/non-cancer)
+
+## Base Model Details
+- **Base Model**: `openai/clip-vit-base-patch32_pcam_finetuned`
+- **Fine-tuned for**: Medical image classification (tumor vs. non-tumor)
+
+- **Base Model Evaluation Results Summary**:
+    - **Clean Accuracy: 86.30%**
+    - **PGD**:
+        **Success Rate: 50.10%**
+        **Average L2 Distance: 12.0844**
+    - **FGSM**:
+         **Success Rate: 44.14%**
+         **Average L2 Distance: 12.0957**
+    - **DeepFool**:
+         **Success Rate: 81.64%**
+         **Average L2 Distance: 224.6645**
+
+- **Adversarial Model Evaluation Results**:
+    - **clean_accuracy**: 86.7218017578125,
+    - **epochs**: 5
+    - **attacks**
+        **PGD** 
+            **success_rate**: 17.87109375,
+            **avg_l2_dist**: 12.093187361955643
+        **FGSM**: 
+            **success_rate**: 17.3828125,
+            **avg_l2_dist**: 12.09616070985794
+        **DeepFool**:
+            **success_rate: 35.62109375,**
+            **avg_l2_dist: 234.12759065628052**
+
+- **Hardware**: Trained on GPU-A100
+
+## Usage
+### Installation
+Ensure you have `transformers`, `torch`, and `safetensors` installed:
+```bash
+pip install transformers torch safetensors
+```
+
+```python
+from transformers import CLIPVisionConfig, CLIPVisionModel, CLIPFeatureExtractor
+import torch
+from torch import nn
+
+class PCamClassifier(nn.Module):
+    def __init__(self, config_dict):
+        super().__init__()
+        self.config = CLIPVisionConfig(**config_dict)
+        self.vision_model = CLIPVisionModel(self.config)
+        self.classifier = nn.Linear(self.config.hidden_size, 2)
+
+    def forward(self, pixel_values):
+        outputs = self.vision_model(pixel_values)
+        return self.classifier(outputs.pooler_output)
+
+# Load model
+config_dict = {
+    "_name_or_path": "openai/clip-vit-base-patch32",
+    "architectures": ["CLIPVisionModel"],
+    "attention_dropout": 0.0,
+    "dropout": 0.0,
+    "hidden_act": "quick_gelu",
+    "hidden_size": 768,
+    "image_size": 224,
+    "initializer_factor": 1.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "layer_norm_eps": 1e-05,
+    "model_type": "clip_vision_model",
+    "num_attention_heads": 12,
+    "num_channels": 3,
+    "num_hidden_layers": 12,
+    "patch_size": 32,
+    "projection_dim": 512,
+    "torch_dtype": "float32"
+}
+
+# Initialize model
+model = PCamClassifier(config_dict)
+model.load_state_dict(torch.load('best_enhanced_pcam_model.pt'))
+
+## Evaluation
+We plan to release additional metrics, including robustness evaluation with adversarial attacks in future updates.
+
+## License
+This model is released under the MIT License.
+
+## Contact
+For any questions, please reach out to **Venkata Tej** at [LensAI](https://huggingface.co/lens-ai).
+
+

best_enhanced_pcam_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1996c94be4f041dd20cc8aa2684fb53b33925656b24192478a57a82ec59084d1
+size 1049756882

config.json

@@ -0,0 +1,20 @@
+{
+    "_name_or_path": "lens-ai/clip-vit-base-patch32_pcam_finetuned",
+    "architectures": ["CLIPVisionModel"],
+    "attention_dropout": 0.0,
+    "dropout": 0.0,
+    "hidden_act": "quick_gelu",
+    "hidden_size": 768,
+    "image_size": 224,
+    "initializer_factor": 1.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "layer_norm_eps": 1e-05,
+    "model_type": "clip_vision_model",
+    "num_attention_heads": 12,
+    "num_channels": 3,
+    "num_hidden_layers": 12,
+    "patch_size": 32,
+    "projection_dim": 512,
+    "torch_dtype": "float32"
+}

results.json

@@ -0,0 +1,13 @@
+{
+    "clean_accuracy": 86.7218017578125,
+    "attacks": {
+        "PGD": {
+            "success_rate": 17.87109375,
+            "avg_l2_dist": 12.093187361955643
+        },
+        "FGSM": {
+            "success_rate": 17.3828125,
+            "avg_l2_dist": 12.09616070985794
+        }
+    }
+}