Upload 6 files

DOCUMENTATION.md

@@ -0,0 +1,335 @@
+# MediSync: Multi-Modal Medical Analysis System
+
+## Comprehensive Technical Documentation
+
+### Table of Contents
+1. [Introduction](#introduction)
+2. [System Architecture](#system-architecture)
+3. [Installation](#installation)
+4. [Usage](#usage)
+5. [Core Components](#core-components)
+6. [Model Details](#model-details)
+7. [API Reference](#api-reference)
+8. [Extending the System](#extending-the-system)
+9. [Troubleshooting](#troubleshooting)
+10. [References](#references)
+
+---
+
+## Introduction
+
+MediSync is a multi-modal AI system that combines X-ray image analysis with medical report text processing to provide comprehensive medical insights. By leveraging state-of-the-art deep learning models for both vision and language understanding, MediSync can:
+
+- Analyze chest X-ray images to detect abnormalities
+- Extract key clinical information from medical reports
+- Fuse insights from both modalities for enhanced diagnosis support
+- Provide comprehensive visualization of analysis results
+
+This AI system demonstrates the power of multi-modal fusion in the healthcare domain, where integrating information from multiple sources can lead to more robust and accurate analyses.
+
+## System Architecture
+
+MediSync follows a modular architecture with three main components:
+
+1. **Image Analysis Module**: Processes X-ray images using pre-trained vision models
+2. **Text Analysis Module**: Analyzes medical reports using NLP models
+3. **Multimodal Fusion Module**: Combines insights from both modalities
+
+The system uses the following high-level workflow:
+
+```
+                      ┌─────────────────┐
+                      │    X-ray Image  │
+                      └────────┬────────┘
+                               │
+                               ▼
+┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
+│  Preprocessing  │───▶│  Image Analysis │───▶│                 │
+└─────────────────┘    └─────────────────┘    │                 │
+                                               │   Multimodal    │
+┌─────────────────┐    ┌─────────────────┐    │     Fusion      │───▶ Results
+│ Medical Report  │───▶│  Text Analysis  │───▶│                 │
+└─────────────────┘    └─────────────────┘    │                 │
+                                               └─────────────────┘
+```
+
+## Installation
+
+### Prerequisites
+- Python 3.8 or higher
+- Pip package manager
+
+### Setup Instructions
+
+1. Clone the repository:
+```bash
+git clone [repository-url]
+cd mediSync
+```
+
+2. Install dependencies:
+```bash
+pip install -r requirements.txt
+```
+
+3. Download sample data:
+```bash
+python -m mediSync.utils.download_samples
+```
+
+## Usage
+
+### Running the Application
+
+To launch the MediSync application with the Gradio interface:
+
+```bash
+python run.py
+```
+
+This will:
+1. Download sample data if not already present
+2. Initialize the application
+3. Launch the Gradio web interface
+
+### Web Interface
+
+MediSync provides a user-friendly web interface with three main tabs:
+
+1. **Multimodal Analysis**: Upload an X-ray image and enter a medical report for combined analysis
+2. **Image Analysis**: Upload an X-ray image for image-only analysis
+3. **Text Analysis**: Enter a medical report for text-only analysis
+
+### Command Line Usage
+
+You can also use the core components directly from Python:
+
+```python
+from mediSync.models import XRayImageAnalyzer, MedicalReportAnalyzer, MultimodalFusion
+
+# Initialize models
+fusion_model = MultimodalFusion()
+
+# Analyze image and text
+results = fusion_model.analyze("path/to/image.jpg", "Medical report text...")
+
+# Get explanation
+explanation = fusion_model.get_explanation(results)
+print(explanation)
+```
+
+## Core Components
+
+### Image Analysis Module
+
+The `XRayImageAnalyzer` class is responsible for analyzing X-ray images:
+
+- Uses the DeiT (Data-efficient image Transformers) model fine-tuned on chest X-rays
+- Detects abnormalities and classifies findings
+- Provides confidence scores and primary findings
+
+Key methods:
+- `analyze(image_path)`: Analyzes an X-ray image
+- `get_explanation(results)`: Generates a human-readable explanation
+
+### Text Analysis Module
+
+The `MedicalReportAnalyzer` class processes medical report text:
+
+- Extracts medical entities (conditions, treatments, tests)
+- Assesses severity level
+- Extracts key findings
+- Suggests follow-up actions
+
+Key methods:
+- `extract_entities(text)`: Extracts medical entities
+- `assess_severity(text)`: Determines severity level
+- `extract_findings(text)`: Extracts key clinical findings
+- `suggest_followup(text, entities, severity)`: Suggests follow-up actions
+- `analyze(text)`: Performs comprehensive analysis
+
+### Multimodal Fusion Module
+
+The `MultimodalFusion` class combines insights from both modalities:
+
+- Calculates agreement between image and text analyses
+- Determines confidence-weighted findings
+- Provides comprehensive severity assessment
+- Merges follow-up recommendations
+
+Key methods:
+- `analyze_image(image_path)`: Analyzes image only
+- `analyze_text(text)`: Analyzes text only
+- `analyze(image_path, report_text)`: Performs multimodal analysis
+- `get_explanation(fused_results)`: Generates comprehensive explanation
+
+## Model Details
+
+### X-ray Analysis Model
+
+- **Model**: facebook/deit-base-patch16-224-medical-cxr
+- **Architecture**: Data-efficient image Transformer (DeiT)
+- **Training Data**: Chest X-ray datasets
+- **Input Size**: 224x224 pixels
+- **Output**: Classification probabilities for various conditions
+
+### Medical Text Analysis Models
+
+- **Entity Recognition Model**: samrawal/bert-base-uncased_medical-ner
+- **Classification Model**: medicalai/ClinicalBERT
+- **Architecture**: BERT-based transformer models
+- **Training Data**: Medical text and reports
+
+## API Reference
+
+### XRayImageAnalyzer
+
+```python
+from mediSync.models import XRayImageAnalyzer
+
+# Initialize
+analyzer = XRayImageAnalyzer(model_name="facebook/deit-base-patch16-224-medical-cxr")
+
+# Analyze image
+results = analyzer.analyze("path/to/image.jpg")
+
+# Get explanation
+explanation = analyzer.get_explanation(results)
+```
+
+### MedicalReportAnalyzer
+
+```python
+from mediSync.models import MedicalReportAnalyzer
+
+# Initialize
+analyzer = MedicalReportAnalyzer()
+
+# Analyze report
+results = analyzer.analyze("Medical report text...")
+
+# Access specific components
+entities = results["entities"]
+severity = results["severity"]
+findings = results["findings"]
+recommendations = results["followup_recommendations"]
+```
+
+### MultimodalFusion
+
+```python
+from mediSync.models import MultimodalFusion
+
+# Initialize
+fusion = MultimodalFusion()
+
+# Multimodal analysis
+results = fusion.analyze("path/to/image.jpg", "Medical report text...")
+
+# Get explanation
+explanation = fusion.get_explanation(results)
+```
+
+## Extending the System
+
+### Adding New Models
+
+To add a new image analysis model:
+
+1. Create a new class that follows the same interface as `XRayImageAnalyzer`
+2. Update the `MultimodalFusion` class to use your new model
+
+```python
+class NewXRayModel:
+    def __init__(self, model_name, device=None):
+        # Initialize your model
+        pass
+        
+    def analyze(self, image_path):
+        # Implement analysis logic
+        return results
+        
+    def get_explanation(self, results):
+        # Generate explanation
+        return explanation
+```
+
+### Custom Preprocessing
+
+You can extend the preprocessing utilities in `utils/preprocessing.py` for custom data preparation:
+
+```python
+def my_custom_preprocessor(image_path, **kwargs):
+    # Implement custom preprocessing
+    return processed_image
+```
+
+### Visualization Extensions
+
+To add new visualization options, extend the utilities in `utils/visualization.py`:
+
+```python
+def my_custom_visualization(results, **kwargs):
+    # Create custom visualization
+    return figure
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Model Loading Errors**
+   - Ensure you have a stable internet connection for downloading models
+   - Check that you have sufficient disk space
+   - Try specifying a different model checkpoint
+
+2. **Image Processing Errors**
+   - Ensure images are in a supported format (JPEG, PNG)
+   - Check that the image is a valid X-ray image
+   - Try preprocessing the image manually using the utility functions
+
+3. **Performance Issues**
+   - For faster inference, use a GPU if available
+   - Reduce image resolution if processing is too slow
+   - Use the text-only analysis for quicker results
+
+### Logging
+
+MediSync uses Python's logging module for debug information:
+
+```python
+import logging
+logging.basicConfig(level=logging.DEBUG)
+```
+
+Log files are saved to `mediSync.log` in the application directory.
+
+## References
+
+### Datasets
+
+- [MIMIC-CXR](https://physionet.org/content/mimic-cxr/2.0.0/): Large dataset of chest radiographs with reports
+- [ChestX-ray14](https://www.nih.gov/news-events/news-releases/nih-clinical-center-provides-one-largest-publicly-available-chest-x-ray-datasets-scientific-community): NIH dataset of chest X-rays
+
+### Papers
+
+- He, K., et al. (2020). "Vision Transformers for Medical Image Analysis"
+- Irvin, J., et al. (2019). "CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison"
+- Johnson, A.E.W., et al. (2019). "MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs"
+
+### Tools and Libraries
+
+- [Hugging Face Transformers](https://huggingface.co/docs/transformers/index)
+- [PyTorch](https://pytorch.org/)
+- [Gradio](https://gradio.app/)
+
+---
+
+## License
+
+This project is licensed under the MIT License - see the LICENSE file for details.
+
+## Acknowledgments
+
+- The development of MediSync was inspired by recent advances in multi-modal learning in healthcare.
+- Special thanks to the open-source community for providing pre-trained models and tools. 

README.md

@@ -1,374 +1,63 @@
----
-title: MediSync
-emoji: 🩺
-colorFrom: blue
-colorTo: indigo
-sdk: gradio
-sdk_version: 5.20.1
-app_file: app.py
-pinned: false
----
-
-
-# MediSync: Multi-Modal Medical Analysis System
-
-MediSync is an AI-powered healthcare solution that combines X-ray image analysis with patient report text processing to provide comprehensive medical insights.
-
-## Features
-
-- **X-ray Image Analysis**: Detects abnormalities in chest X-rays using pre-trained vision models from Hugging Face.
-- **Medical Report Processing**: Extracts key information from patient reports using NLP models.
-- **Multi-modal Integration**: Combines insights from both image and text data for more accurate diagnosis suggestions.
-- **User-friendly Interface**: Simple web interface for uploading images and reports.
-
-## Project Structure
-
-```
-mediSync/
-├── app.py                    # Main application with Gradio interface
-├── models/
-│   ├── image_analyzer.py     # X-ray image analysis module
-│   ├── text_analyzer.py      # Medical report text analysis module
-│   └── multimodal_fusion.py  # Fusion of image and text insights
-├── utils/
-│   ├── preprocessing.py      # Data preprocessing utilities
-│   └── visualization.py      # Result visualization utilities
-├── data/
-│   └── sample/               # Sample data for testing
-└── tests/                    # Unit tests
-```
-# MediSync: Multi-Modal Medical Analysis System
-
-## Comprehensive Technical Documentation
-
-### Table of Contents
-1. [Introduction](#introduction)
-2. [System Architecture](#system-architecture)
-3. [Installation](#installation)
-4. [Usage](#usage)
-5. [Core Components](#core-components)
-6. [Model Details](#model-details)
-7. [API Reference](#api-reference)
-8. [Extending the System](#extending-the-system)
-9. [Troubleshooting](#troubleshooting)
-10. [References](#references)
-
----
-
-## Introduction
-
-MediSync is a multi-modal AI system that combines X-ray image analysis with medical report text processing to provide comprehensive medical insights. By leveraging state-of-the-art deep learning models for both vision and language understanding, MediSync can:
-
-- Analyze chest X-ray images to detect abnormalities
-- Extract key clinical information from medical reports
-- Fuse insights from both modalities for enhanced diagnosis support
-- Provide comprehensive visualization of analysis results
-
-This AI system demonstrates the power of multi-modal fusion in the healthcare domain, where integrating information from multiple sources can lead to more robust and accurate analyses.
-
-## System Architecture
-
-MediSync follows a modular architecture with three main components:
-
-1. **Image Analysis Module**: Processes X-ray images using pre-trained vision models
-2. **Text Analysis Module**: Analyzes medical reports using NLP models
-3. **Multimodal Fusion Module**: Combines insights from both modalities
-
-The system uses the following high-level workflow:
-
-```
-                      ┌─────────────────┐
-                      │    X-ray Image  │
-                      └────────┬────────┘
-                               │
-                               ▼
-┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
-│  Preprocessing  │───▶│  Image Analysis │───▶│                 │
-└─────────────────┘    └─────────────────┘    │                 │
-                                               │   Multimodal    │
-┌─────────────────┐    ┌─────────────────┐    │     Fusion      │───▶ Results
-│ Medical Report  │───▶│  Text Analysis  │───▶│                 │
-└─────────────────┘    └─────────────────┘    │                 │
-                                               └─────────────────┘
-```
-
-## Installation
-
-### Prerequisites
-- Python 3.8 or higher
-- Pip package manager
-
-### Setup Instructions
-
-1. Clone the repository:
-```bash
-git clone [repository-url]
-cd mediSync
-```
-
-2. Install dependencies:
-```bash
-pip install -r requirements.txt
-```
-
-3. Download sample data:
-```bash
-python -m mediSync.utils.download_samples
-```
-
-## Usage
-
-### Running the Application
-
-To launch the MediSync application with the Gradio interface:
-
-```bash
-python run.py
-```
-
-This will:
-1. Download sample data if not already present
-2. Initialize the application
-3. Launch the Gradio web interface
-
-### Web Interface
-
-MediSync provides a user-friendly web interface with three main tabs:
-
-1. **Multimodal Analysis**: Upload an X-ray image and enter a medical report for combined analysis
-2. **Image Analysis**: Upload an X-ray image for image-only analysis
-3. **Text Analysis**: Enter a medical report for text-only analysis
-
-### Command Line Usage
-
-You can also use the core components directly from Python:
-
-```python
-from mediSync.models import XRayImageAnalyzer, MedicalReportAnalyzer, MultimodalFusion
-
-# Initialize models
-fusion_model = MultimodalFusion()
-
-# Analyze image and text
-results = fusion_model.analyze("path/to/image.jpg", "Medical report text...")
-
-# Get explanation
-explanation = fusion_model.get_explanation(results)
-print(explanation)
-```
-
-## Core Components
-
-### Image Analysis Module
-
-The `XRayImageAnalyzer` class is responsible for analyzing X-ray images:
-
-- Uses the DeiT (Data-efficient image Transformers) model fine-tuned on chest X-rays
-- Detects abnormalities and classifies findings
-- Provides confidence scores and primary findings
-
-Key methods:
-- `analyze(image_path)`: Analyzes an X-ray image
-- `get_explanation(results)`: Generates a human-readable explanation
-
-### Text Analysis Module
-
-The `MedicalReportAnalyzer` class processes medical report text:
-
-- Extracts medical entities (conditions, treatments, tests)
-- Assesses severity level
-- Extracts key findings
-- Suggests follow-up actions
-
-Key methods:
-- `extract_entities(text)`: Extracts medical entities
-- `assess_severity(text)`: Determines severity level
-- `extract_findings(text)`: Extracts key clinical findings
-- `suggest_followup(text, entities, severity)`: Suggests follow-up actions
-- `analyze(text)`: Performs comprehensive analysis
-
-### Multimodal Fusion Module
-
-The `MultimodalFusion` class combines insights from both modalities:
-
-- Calculates agreement between image and text analyses
-- Determines confidence-weighted findings
-- Provides comprehensive severity assessment
-- Merges follow-up recommendations
-
-Key methods:
-- `analyze_image(image_path)`: Analyzes image only
-- `analyze_text(text)`: Analyzes text only
-- `analyze(image_path, report_text)`: Performs multimodal analysis
-- `get_explanation(fused_results)`: Generates comprehensive explanation
-
-## Model Details
-
-### X-ray Analysis Model
-
-- **Model**: facebook/deit-base-patch16-224-medical-cxr
-- **Architecture**: Data-efficient image Transformer (DeiT)
-- **Training Data**: Chest X-ray datasets
-- **Input Size**: 224x224 pixels
-- **Output**: Classification probabilities for various conditions
-
-### Medical Text Analysis Models
-
-- **Entity Recognition Model**: samrawal/bert-base-uncased_medical-ner
-- **Classification Model**: medicalai/ClinicalBERT
-- **Architecture**: BERT-based transformer models
-- **Training Data**: Medical text and reports
-
-## API Reference
-
-### XRayImageAnalyzer
-
-```python
-from mediSync.models import XRayImageAnalyzer
-
-# Initialize
-analyzer = XRayImageAnalyzer(model_name="facebook/deit-base-patch16-224-medical-cxr")
-
-# Analyze image
-results = analyzer.analyze("path/to/image.jpg")
-
-# Get explanation
-explanation = analyzer.get_explanation(results)
-```
-
-### MedicalReportAnalyzer
-
-```python
-from mediSync.models import MedicalReportAnalyzer
-
-# Initialize
-analyzer = MedicalReportAnalyzer()
-
-# Analyze report
-results = analyzer.analyze("Medical report text...")
-
-# Access specific components
-entities = results["entities"]
-severity = results["severity"]
-findings = results["findings"]
-recommendations = results["followup_recommendations"]
-```
-
-### MultimodalFusion
-
-```python
-from mediSync.models import MultimodalFusion
-
-# Initialize
-fusion = MultimodalFusion()
-
-# Multimodal analysis
-results = fusion.analyze("path/to/image.jpg", "Medical report text...")
-
-# Get explanation
-explanation = fusion.get_explanation(results)
-```
-
-## Extending the System
-
-### Adding New Models
-
-To add a new image analysis model:
-
-1. Create a new class that follows the same interface as `XRayImageAnalyzer`
-2. Update the `MultimodalFusion` class to use your new model
-
-```python
-class NewXRayModel:
-    def __init__(self, model_name, device=None):
-        # Initialize your model
-        pass
-        
-    def analyze(self, image_path):
-        # Implement analysis logic
-        return results
-        
-    def get_explanation(self, results):
-        # Generate explanation
-        return explanation
-```
-
-### Custom Preprocessing
-
-You can extend the preprocessing utilities in `utils/preprocessing.py` for custom data preparation:
-
-```python
-def my_custom_preprocessor(image_path, **kwargs):
-    # Implement custom preprocessing
-    return processed_image
-```
-
-### Visualization Extensions
-
-To add new visualization options, extend the utilities in `utils/visualization.py`:
-
-```python
-def my_custom_visualization(results, **kwargs):
-    # Create custom visualization
-    return figure
-```
-
-## Troubleshooting
-
-### Common Issues
-
-1. **Model Loading Errors**
-   - Ensure you have a stable internet connection for downloading models
-   - Check that you have sufficient disk space
-   - Try specifying a different model checkpoint
-
-2. **Image Processing Errors**
-   - Ensure images are in a supported format (JPEG, PNG)
-   - Check that the image is a valid X-ray image
-   - Try preprocessing the image manually using the utility functions
-
-3. **Performance Issues**
-   - For faster inference, use a GPU if available
-   - Reduce image resolution if processing is too slow
-   - Use the text-only analysis for quicker results
-
-### Logging
-
-MediSync uses Python's logging module for debug information:
-
-```python
-import logging
-logging.basicConfig(level=logging.DEBUG)
-```
-
-Log files are saved to `mediSync.log` in the application directory.
-
-## References
-
-### Datasets
-
-- [MIMIC-CXR](https://physionet.org/content/mimic-cxr/2.0.0/): Large dataset of chest radiographs with reports
-- [ChestX-ray14](https://www.nih.gov/news-events/news-releases/nih-clinical-center-provides-one-largest-publicly-available-chest-x-ray-datasets-scientific-community): NIH dataset of chest X-rays
-
-### Papers
-
-- He, K., et al. (2020). "Vision Transformers for Medical Image Analysis"
-- Irvin, J., et al. (2019). "CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison"
-- Johnson, A.E.W., et al. (2019). "MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs"
-
-### Tools and Libraries
-
-- [Hugging Face Transformers](https://huggingface.co/docs/transformers/index)
-- [PyTorch](https://pytorch.org/)
-- [Gradio](https://gradio.app/)
-
----
-
-## License
-
-This project is licensed under the MIT License - see the LICENSE file for details.
-
-## Acknowledgments
-
-- The development of MediSync was inspired by recent advances in multi-modal learning in healthcare.
-- Special thanks to the open-source community for providing pre-trained models and tools.
+# MediSync: Multi-Modal Medical Analysis System
+
+[![Open In HF Spaces](https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-sm.svg)](https://huggingface.co/spaces/MediSync)
+
+## Overview
+
+MediSync is an AI-powered healthcare solution that combines X-ray image analysis with patient report text processing to provide comprehensive medical insights. This demo showcases how multiple AI models can work together to provide a more complete analysis than either could alone.
+
+## Features
+
+- **X-ray Image Analysis**: Detects abnormalities in chest X-rays using pre-trained vision models
+- **Medical Report Processing**: Extracts key information from patient reports using NLP models
+- **Multi-modal Integration**: Combines insights from both image and text data for more accurate diagnosis suggestions
+- **Image Enhancement**: Provides image enhancement options for better visualization
+
+## How to Use This Space
+
+1. **Upload an X-ray Image**: Click on the upload field to add your chest X-ray image
+2. **Enter Medical Report Text**: Type or paste the corresponding radiologist's report
+3. **Choose Analysis Type**:
+   - **Multimodal Analysis**: Combines both image and text data
+   - **Image Analysis**: Focuses only on the X-ray image
+   - **Text Analysis**: Extracts key information from the medical report
+
+## Models Used
+
+- **X-ray Analysis**: facebook/deit-base-patch16-224-medical-cxr
+- **Medical Text Analysis**: medicalai/ClinicalBERT
+- **Additional Support Models**: Medical entity recognition models
+
+## Example Inputs
+
+The app comes with example inputs you can use to test its functionality:
+- Sample chest X-rays with varying conditions
+- Example medical reports with different findings
+
+## Important Disclaimer
+
+This system is designed as a support tool and should NOT replace professional medical diagnosis. It is intended for educational and research purposes only. Always consult with healthcare professionals for medical decisions.
+
+## Technical Details
+
+MediSync is built with:
+- Gradio for the user interface
+- PyTorch and Transformers for the model backends
+- Advanced visualization utilities for result interpretation
+
+## Citation & Credits
+
+If you use MediSync in your research, please cite:
+
+```
+@software{MediSync2023,
+  author = {MediSync Team},
+  title = {MediSync: Multi-Modal Medical Analysis System},
+  year = {2023},
+  url = {https://huggingface.co/spaces/MediSync}
+}
+```
+
+## License
+
+This project is released under the MIT License. 

app.py

@@ -0,0 +1,57 @@
+"""
+MediSync: Multi-Modal Medical Analysis System - Hugging Face Spaces Entry Point
+==============================================================================
+This is the main entry point for the Hugging Face Space deployment.
+"""
+
+import logging
+import os
+import sys
+from pathlib import Path
+
+# Configure logging
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+    handlers=[logging.StreamHandler()],
+)
+logger = logging.getLogger(__name__)
+
+# Add the current directory to Python path
+current_dir = Path(__file__).resolve().parent
+sys.path.append(str(current_dir))
+
+# Pre-download models (important for Hugging Face Spaces)
+try:
+    logger.info("Pre-downloading models for caching...")
+    from transformers import AutoModel, AutoModelForImageClassification, AutoTokenizer
+
+    # Set cache directory (HF Spaces specific)
+    os.environ["TRANSFORMERS_CACHE"] = os.path.join(os.getcwd(), "model_cache")
+    os.makedirs(os.environ["TRANSFORMERS_CACHE"], exist_ok=True)
+
+    # Medical vision model
+    vision_model_name = "facebook/deit-base-patch16-224-medical-cxr"
+    logger.info(f"Downloading vision model: {vision_model_name}")
+    AutoModelForImageClassification.from_pretrained(vision_model_name)
+
+    # Clinical BERT model
+    text_model_name = "medicalai/ClinicalBERT"
+    logger.info(f"Downloading text model: {text_model_name}")
+    AutoModel.from_pretrained(text_model_name)
+    AutoTokenizer.from_pretrained(text_model_name)
+
+    logger.info("All models downloaded and cached successfully!")
+except Exception as e:
+    logger.warning(f"Warning: Could not pre-download models: {e}")
+
+# Import and run the MediSync application
+from mediSync.app import create_interface
+
+# Create and launch the interface
+logger.info("Creating and launching the interface...")
+interface = create_interface()
+
+# For Hugging Face Spaces
+if __name__ == "__main__":
+    interface.launch()

mediSync.log

@@ -0,0 +1,80 @@
+2025-03-10 22:44:41,854 - __main__ - INFO - Starting MediSync setup
+2025-03-10 22:44:41,854 - __main__ - INFO - Checking for sample data
+2025-03-10 22:44:42,626 - __main__ - INFO - No sample images found. Downloading...
+2025-03-10 22:44:42,626 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/53448173/322830a37f0fa0852773ca2db3e8d8_big_gallery.jpeg to D:\Multi Model AI Problem Solving\mediSync\data\sample\normal_chest_xray.jpg
+2025-03-10 22:44:43,532 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/53448173/322830a37f0fa0852773ca2db3e8d8_big_gallery.jpeg: HTTP Error 404: Not Found
+2025-03-10 22:44:43,532 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/52465460/e4d8791bd7502ab72af8d9e5c322db_big_gallery.jpg to D:\Multi Model AI Problem Solving\mediSync\data\sample\pneumonia_xray.jpg
+2025-03-10 22:44:43,988 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/52465460/e4d8791bd7502ab72af8d9e5c322db_big_gallery.jpg: HTTP Error 404: Not Found
+2025-03-10 22:44:43,988 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/556520/cf17c05750adb04b2a6e23afb47c7d_big_gallery.jpg to D:\Multi Model AI Problem Solving\mediSync\data\sample\cardiomegaly_xray.jpg
+2025-03-10 22:44:44,605 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/556520/cf17c05750adb04b2a6e23afb47c7d_big_gallery.jpg: HTTP Error 404: Not Found
+2025-03-10 22:44:44,605 - mediSync.utils.download_samples - INFO - Downloading https://prod-images-static.radiopaedia.org/images/19972291/41eed1a2cdad06d26c3f415a6ed65a_big_gallery.jpeg to D:\Multi Model AI Problem Solving\mediSync\data\sample\nodule_xray.jpg
+2025-03-10 22:44:45,107 - mediSync.utils.download_samples - ERROR - Error downloading https://prod-images-static.radiopaedia.org/images/19972291/41eed1a2cdad06d26c3f415a6ed65a_big_gallery.jpeg: HTTP Error 404: Not Found
+2025-03-10 22:44:45,107 - mediSync.utils.download_samples - INFO - Downloaded 0 out of 4 images
+2025-03-10 22:44:45,107 - mediSync.utils.download_samples - INFO - Created sample info file: D:\Multi Model AI Problem Solving\mediSync\data\sample\sample_info.txt
+2025-03-10 22:44:45,107 - __main__ - INFO - Launching MediSync application
+2025-03-10 22:44:48,918 - httpx - INFO - HTTP Request: GET https://api.gradio.app/gradio-messaging/en "HTTP/1.1 200 OK"
+2025-03-10 22:45:02,402 - mediSync.app - INFO - Initializing MediSync application
+2025-03-10 22:45:03,005 - httpx - INFO - HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
+2025-03-10 22:45:03,294 - httpx - INFO - HTTP Request: GET http://127.0.0.1:7860/gradio_api/startup-events "HTTP/1.1 200 OK"
+2025-03-10 22:45:03,722 - httpx - INFO - HTTP Request: HEAD http://127.0.0.1:7860/ "HTTP/1.1 200 OK"
+2025-03-10 22:46:47,162 - mediSync.app - INFO - Loading models...
+2025-03-10 22:46:47,162 - models.multimodal_fusion - INFO - Using device: cpu
+2025-03-10 22:46:47,178 - models.image_analyzer - INFO - Using device: cpu
+2025-03-10 22:46:47,391 - models.image_analyzer - ERROR - Failed to load model: facebook/deit-base-patch16-224-medical-cxr is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
+If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
+2025-03-10 22:46:47,391 - models.multimodal_fusion - ERROR - Failed to initialize image analyzer: facebook/deit-base-patch16-224-medical-cxr is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
+If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
+2025-03-10 22:46:47,391 - models.text_analyzer - INFO - Using device: cpu
+2025-03-10 22:46:47,490 - models.text_analyzer - ERROR - Failed to load NER model: samrawal/bert-base-uncased_medical-ner is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
+If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
+2025-03-10 22:46:58,283 - models.text_analyzer - INFO - Successfully loaded classifier model: medicalai/ClinicalBERT
+2025-03-10 22:46:58,283 - models.multimodal_fusion - INFO - Successfully initialized text analyzer
+2025-03-10 22:46:58,283 - mediSync.app - INFO - Models loaded successfully
+2025-03-10 22:46:58,379 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 22:46:58,379 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 22:46:58,379 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 22:46:58,379 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 22:47:44,547 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp19zqepvi\upload.png
+2025-03-10 22:47:55,128 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 22:47:55,128 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 22:48:20,220 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpowd6qwty\upload.png
+2025-03-10 22:55:22,019 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 22:55:22,019 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 22:55:22,025 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 22:55:22,025 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 22:55:35,606 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpipzmsxch\upload.png
+2025-03-10 22:55:45,116 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 22:55:45,116 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:01:32,009 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpo88f1uaw\upload.png
+2025-03-10 23:05:24,892 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpnckk4w27\upload.png
+2025-03-10 23:05:45,332 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpxab_m30m\upload.png
+2025-03-10 23:06:01,387 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpcit78i2e\upload.png
+2025-03-10 23:06:04,157 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmpivli_abx\upload.png
+2025-03-10 23:11:47,429 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp4cpwvo5h\upload.png
+2025-03-10 23:22:27,118 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:22:27,118 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:22:27,118 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:22:27,118 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:24:22,378 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:24:22,378 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:24:22,378 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:24:22,378 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:24:35,252 - mediSync.app - INFO - Enhancing image: C:\Users\amart\AppData\Local\Temp\tmp947ie5e1\upload.png
+2025-03-10 23:24:37,554 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:24:37,554 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:24:37,554 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:24:37,554 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:24:58,770 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 23:24:58,770 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:25:25,652 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 23:25:25,652 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:27:06,100 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:27:06,100 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:27:06,100 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:27:06,100 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:27:46,183 - mediSync.app - INFO - Performing multimodal analysis
+2025-03-10 23:27:46,183 - models.multimodal_fusion - WARNING - Image analyzer not available
+2025-03-10 23:27:46,183 - models.text_analyzer - WARNING - NER model not available
+2025-03-10 23:27:46,183 - models.multimodal_fusion - INFO - Agreement score between modalities: 0.25
+2025-03-10 23:28:15,394 - mediSync.app - INFO - Analyzing medical report text
+2025-03-10 23:28:15,394 - models.text_analyzer - WARNING - NER model not available

requirements.txt

@@ -0,0 +1,11 @@
+torch>=2.0.0
+transformers>=4.30.0
+datasets>=2.12.0
+pillow>=9.0.0
+matplotlib>=3.7.0
+pandas>=2.0.0
+numpy>=1.24.0
+scikit-learn>=1.2.0
+gradio>=4.0.0
+huggingface_hub>=0.16.0
+opencv-python>=4.7.0 

space.yml

@@ -0,0 +1,9 @@
+title: MediSync - Multi-Modal Medical Analysis System
+emoji: 🩺
+colorFrom: blue
+colorTo: indigo
+sdk: gradio
+sdk_version: 4.0.0
+app_file: app.py
+pinned: false
+license: mit