Amarthya7 commited on
Commit
681725a
Β·
verified Β·
1 Parent(s): 560c35c

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -109
README.md DELETED
@@ -1,109 +0,0 @@
1
- ---
2
- title: MediSync - Multi-Modal Medical Analysis System
3
- emoji: 🩺
4
- colorFrom: blue
5
- colorTo: purple
6
- sdk: gradio
7
- sdk_version: 5.20.1
8
- app_file: app.py
9
- pinned: false
10
- ---
11
-
12
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
13
-
14
- # MediSync: Multi-Modal Medical Analysis System
15
-
16
- MediSync is an AI-powered healthcare solution that combines X-ray image analysis with patient report text processing to provide comprehensive medical insights.
17
-
18
- Introduction
19
- MediSync is a multi-modal AI system that combines X-ray image analysis with medical report text processing to provide comprehensive medical insights. By leveraging state-of-the-art deep learning models for both vision and language understanding, MediSync can:
20
-
21
- Analyze chest X-ray images to detect abnormalities
22
- Extract key clinical information from medical reports
23
- Fuse insights from both modalities for enhanced diagnosis support
24
- Provide comprehensive visualization of analysis results
25
- This AI system demonstrates the power of multi-modal fusion in the healthcare domain, where integrating information from multiple sources can lead to more robust and accurate analyses.
26
-
27
- System Architecture
28
- MediSync follows a modular architecture with three main components:
29
-
30
- Image Analysis Module: Processes X-ray images using pre-trained vision models
31
- Text Analysis Module: Analyzes medical reports using NLP models
32
- Multimodal Fusion Module: Combines insights from both modalities
33
- The system uses the following high-level workflow:
34
-
35
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
36
- β”‚ X-ray Image β”‚
37
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜
38
- β”‚
39
- β–Ό
40
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
41
- β”‚ Preprocessing │───▢│ Image Analysis │───▢│ β”‚
42
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
43
- β”‚ Multimodal β”‚
44
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Fusion │───▢ Results
45
- β”‚ Medical Report │───▢│ Text Analysis │───▢│ β”‚
46
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚
47
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
48
-
49
-
50
- ## Features
51
-
52
- - **X-ray Image Analysis**: Detects abnormalities in chest X-rays using pre-trained vision models from Hugging Face.
53
- - **Medical Report Processing**: Extracts key information from patient reports using NLP models.
54
- - **Multi-modal Integration**: Combines insights from both image and text data for more accurate diagnosis suggestions.
55
- - **User-friendly Interface**: Simple web interface for uploading images and reports.
56
-
57
- ## Project Structure
58
-
59
- ```
60
- mediSync/
61
- β”œβ”€β”€ app.py # Main application with Gradio interface
62
- β”œβ”€β”€ models/
63
- β”‚ β”œβ”€β”€ image_analyzer.py # X-ray image analysis module
64
- β”‚ β”œβ”€β”€ text_analyzer.py # Medical report text analysis module
65
- β”‚ └── multimodal_fusion.py # Fusion of image and text insights
66
- β”œβ”€β”€ utils/
67
- β”‚ β”œβ”€β”€ preprocessing.py # Data preprocessing utilities
68
- β”‚ └── visualization.py # Result visualization utilities
69
- β”œβ”€β”€ data/
70
- β”‚ └── sample/ # Sample data for testing
71
- └── tests/ # Unit tests
72
- ```
73
-
74
- ## Setup Instructions
75
-
76
- 1. Clone this repository:
77
- ```bash
78
- git clone [repository-url]
79
- cd MediSync
80
- ```
81
-
82
- 2. Install dependencies:
83
- ```bash
84
- pip install -r requirements.txt
85
- ```
86
-
87
- 3. Run the application:
88
- ```bash
89
- python app.py
90
- ```
91
-
92
- 4. Access the web interface at `http://localhost:7860`
93
-
94
- ## Models Used
95
-
96
- - **X-ray Analysis**: facebook/deit-base-patch16-224-medical-cxr
97
- - **Medical Text Analysis**: medicalai/ClinicalBERT
98
- - **Additional Support Models**: Medical question answering and entity recognition models
99
-
100
- ## Use Cases
101
-
102
- - Preliminary screening of chest X-rays
103
- - Cross-validation of radiologist reports
104
- - Educational tool for medical students
105
- - Research tool for studying correlation between visual findings and written reports
106
-
107
- ## Note
108
-
109
- This system is designed as a support tool and should not replace professional medical diagnosis. Always consult with healthcare professionals for medical decisions.