Upload 4 files

README.md

@@ -0,0 +1,73 @@
+title: Medical Analysis System
+emoji: 🤖
+colorFrom: blue
+colorTo: purple
+sdk: gradio
+sdk_version: 5.20.1
+app_file: app.py
+pinned: false
+
+# MediSync: Multi-Modal Medical Analysis System
+
+MediSync is an AI-powered healthcare solution that combines X-ray image analysis with patient report text processing to provide comprehensive medical insights.
+
+## Features
+
+- **X-ray Image Analysis**: Detects abnormalities in chest X-rays using pre-trained vision models from Hugging Face.
+- **Medical Report Processing**: Extracts key information from patient reports using NLP models.
+- **Multi-modal Integration**: Combines insights from both image and text data for more accurate diagnosis suggestions.
+- **User-friendly Interface**: Simple web interface for uploading images and reports.
+
+## Project Structure
+
+```
+mediSync/
+├── app.py                    # Main application with Gradio interface
+├── models/
+│   ├── image_analyzer.py     # X-ray image analysis module
+│   ├── text_analyzer.py      # Medical report text analysis module
+│   └── multimodal_fusion.py  # Fusion of image and text insights
+├── utils/
+│   ├── preprocessing.py      # Data preprocessing utilities
+│   └── visualization.py      # Result visualization utilities
+├── data/
+│   └── sample/               # Sample data for testing
+└── tests/                    # Unit tests
+```
+
+## Setup Instructions
+
+1. Clone this repository:
+```bash
+git clone [repository-url]
+cd MediSync
+```
+
+2. Install dependencies:
+```bash
+pip install -r requirements.txt
+```
+
+3. Run the application:
+```bash
+python app.py
+```
+
+4. Access the web interface at `http://localhost:7860`
+
+## Models Used
+
+- **X-ray Analysis**: facebook/deit-base-patch16-224-medical-cxr
+- **Medical Text Analysis**: medicalai/ClinicalBERT
+- **Additional Support Models**: Medical question answering and entity recognition models
+
+## Use Cases
+
+- Preliminary screening of chest X-rays
+- Cross-validation of radiologist reports
+- Educational tool for medical students
+- Research tool for studying correlation between visual findings and written reports
+
+## Note
+
+This system is designed as a support tool and should not replace professional medical diagnosis. Always consult with healthcare professionals for medical decisions. 

mediSync.log

@@ -0,0 +1,80 @@
+2025-03-10 22:44:41,854 - __main__ - INFO - Starting MediSync setup
+2025-03-10 22:44:41,854 - __main__ - INFO - Checking for sample data
+2025-03-10 22:44:42,626 - __main__ - INFO - No sample images found. Downloading...
+2025-03-10 22:44:42,626 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/53448173/322830a37f0fa0852773ca2db3e8d8_big_gallery.jpeg to D:\Multi Model AI Problem Solving\mediSync\data\sample\normal_chest_xray.jpg
+2025-03-10 22:44:43,532 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/53448173/322830a37f0fa0852773ca2db3e8d8_big_gallery.jpeg: HTTP Error 404: Not Found
+2025-03-10 22:44:43,532 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/52465460/e4d8791bd7502ab72af8d9e5c322db_big_gallery.jpg to D:\Multi Model AI Problem Solving\mediSync\data\sample\pneumonia_xray.jpg
+2025-03-10 22:44:43,988 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/52465460/e4d8791bd7502ab72af8d9e5c322db_big_gallery.jpg: HTTP Error 404: Not Found
+2025-03-10 22:44:43,988 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/556520/cf17c05750adb04b2a6e23afb47c7d_big_gallery.jpg to D:\Multi Model AI Problem Solving\mediSync\data\sample\cardiomegaly_xray.jpg
+2025-03-10 22:44:44,605 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/556520/cf17c05750adb04b2a6e23afb47c7d_big_gallery.jpg: HTTP Error 404: Not Found
+2025-03-10 22:44:44,605 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/19972291/41eed1a2cdad06d26c3f415a6ed65a_big_gallery.jpeg to D:\Multi Model AI Problem Solving\mediSync\data\sample\nodule_xray.jpg
+2025-03-10 22:44:45,107 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/19972291/41eed1a2cdad06d26c3f415a6ed65a_big_gallery.jpeg: HTTP Error 404: Not Found
+2025-03-10 22:44:45,107 - mediSync.utils.download_samples - INFO - Downloaded 0 out of 4 images
+2025-03-10 22:44:45,107 - mediSync.utils.download_samples - INFO - Created sample info file: D:\Multi Model AI Problem Solving\mediSync\data\sample\sample_info.txt
+2025-03-10 22:44:45,107 - __main__ - INFO - Launching MediSync application
+2025-03-10 22:44:48,918 - httpx - INFO - HTTP Request: GET https://api.gradio.app/gradio-messaging/en "HTTP/1.1 200 OK"
+2025-03-10 22:45:02,402 - mediSync.app - INFO - Initializing MediSync application
+2025-03-10 22:45:03,005 - httpx - INFO - HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
+2025-03-10 22:45:03,294 - httpx - INFO - HTTP Request: GET http://127.0.0.1:7860/gradio_api/startup-events "HTTP/1.1 200 OK"
+2025-03-10 22:45:03,722 - httpx - INFO - HTTP Request: HEAD http://127.0.0.1:7860/ "HTTP/1.1 200 OK"
+2025-03-10 22:46:47,162 - mediSync.app - INFO - Loading models...
+2025-03-10 22:46:47,162 - models.multimodal_fusion - INFO - Using device: cpu
+2025-03-10 22:46:47,178 - models.image_analyzer - INFO - Using device: cpu
+2025-03-10 22:46:47,391 - models.image_analyzer - ERROR - Failed to load model: facebook/deit-base-patch16-224-medical-cxr is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
+If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
+2025-03-10 22:46:47,391 - models.multimodal_fusion - ERROR - Failed to initialize image analyzer: facebook/deit-base-patch16-224-medical-cxr is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
+If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
+2025-03-10 22:46:47,391 - models.text_analyzer - INFO - Using device: cpu
+2025-03-10 22:46:47,490 - models.text_analyzer - ERROR - Failed to load NER model: samrawal/bert-base-uncased_medical-ner is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
+If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
+2025-03-10 22:46:58,283 - models.text_analyzer - INFO - Successfully loaded classifier model: medicalai/ClinicalBERT
+2025-03-10 22:46:58,283 - models.multimodal_fusion - INFO - Successfully initialized text analyzer
+2025-03-10 22:46:58,283 - mediSync.app - INFO - Models loaded successfully
+2025-03-10 22:46:58,379 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 22:46:58,379 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 22:46:58,379 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 22:46:58,379 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 22:47:44,547 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp19zqepvi\upload.png
+2025-03-10 22:47:55,128 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 22:47:55,128 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 22:48:20,220 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpowd6qwty\upload.png
+2025-03-10 22:55:22,019 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 22:55:22,019 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 22:55:22,025 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 22:55:22,025 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 22:55:35,606 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpipzmsxch\upload.png
+2025-03-10 22:55:45,116 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 22:55:45,116 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:01:32,009 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpo88f1uaw\upload.png
+2025-03-10 23:05:24,892 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpnckk4w27\upload.png
+2025-03-10 23:05:45,332 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpxab_m30m\upload.png
+2025-03-10 23:06:01,387 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpcit78i2e\upload.png
+2025-03-10 23:06:04,157 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpivli_abx\upload.png
+2025-03-10 23:11:47,429 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp4cpwvo5h\upload.png
+2025-03-10 23:22:27,118 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:22:27,118 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:22:27,118 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:22:27,118 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:24:22,378 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:24:22,378 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:24:22,378 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:24:22,378 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:24:35,252 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp947ie5e1\upload.png
+2025-03-10 23:24:37,554 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:24:37,554 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:24:37,554 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:24:37,554 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:24:58,770 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 23:24:58,770 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:25:25,652 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 23:25:25,652 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:27:06,100 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:27:06,100 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:27:06,100 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:27:06,100 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:27:46,183 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:27:46,183 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:27:46,183 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:27:46,183 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:28:15,394 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 23:28:15,394 - models.text_analyzer - WARNING - NER model not available

requirements.txt

@@ -0,0 +1,11 @@
+torch>=2.0.0
+transformers>=4.30.0
+datasets>=2.12.0
+pillow>=9.0.0
+matplotlib>=3.7.0
+pandas>=2.0.0
+numpy>=1.24.0
+scikit-learn>=1.2.0
+gradio>=4.0.0
+huggingface_hub>=0.16.0
+opencv-python>=4.7.0 

run.py

@@ -0,0 +1,71 @@
+"""
+MediSync: Multi-Modal Medical Analysis System - Runner Script
+============================================================
+This script downloads sample data and launches the MediSync application.
+"""
+
+import logging
+import sys
+from pathlib import Path
+
+# Set up logging
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+    handlers=[logging.StreamHandler(), logging.FileHandler("mediSync.log")],
+)
+logger = logging.getLogger(__name__)
+
+
+def main():
+    """
+    Main function to set up and run the MediSync application.
+    """
+    logger.info("Starting MediSync setup")
+
+    # Add the current directory to Python path
+    current_dir = Path(__file__).resolve().parent
+    sys.path.append(str(current_dir))
+
+    # Download sample data if needed
+    try:
+        logger.info("Checking for sample data")
+        from mediSync.utils.download_samples import (
+            create_sample_info_file,
+            download_sample_images,
+        )
+
+        # Check if sample directory exists and contains images
+        sample_dir = current_dir / "mediSync" / "data" / "sample"
+        sample_dir.mkdir(parents=True, exist_ok=True)
+
+        image_files = list(sample_dir.glob("*.jpg")) + list(sample_dir.glob("*.png"))
+
+        if not image_files:
+            logger.info("No sample images found. Downloading...")
+            download_sample_images()
+            create_sample_info_file()
+        else:
+            logger.info(f"Found {len(image_files)} existing sample images")
+
+    except Exception as e:
+        logger.error(f"Error setting up sample data: {e}")
+        print(f"Warning: Could not set up sample data: {e}")
+
+    # Launch the application
+    try:
+        logger.info("Launching MediSync application")
+        from mediSync.app import create_interface
+
+        create_interface()
+
+    except Exception as e:
+        logger.error(f"Error launching application: {e}")
+        print(f"Error: Failed to launch MediSync application: {e}")
+        return 1
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())