--- title: MedSketch AI emoji: 🏆 colorFrom: indigo colorTo: pink sdk: streamlit sdk_version: 1.44.1 app_file: app.py pinned: false short_description: Medical Image --- # MedSketch AI – Advanced Clinical Diagram Generator 🖼️ **MedSketch AI** is a web application built with Streamlit that leverages cutting-edge AI models (like OpenAI's DALL-E 3 via the GPT-4o API endpoint access) to generate medical diagrams and illustrations from text prompts. It allows users to specify styles, associate metadata, perform batch generation, annotate the results, and export annotations. [![Streamlit App](https://static.streamlit.io/badges/streamlit_badge_black_white.svg)](https://your-deployed-app-url.com) [Insert Screenshot/GIF of the App Here - Highly Recommended!] *A visual demonstration of MedSketch AI in action.* --- ## ✨ Features * **AI-Powered Generation:** Create medical diagrams using models like OpenAI's DALL-E 3 (accessed via API). (Placeholder for Stable Diffusion LoRA integration). * **Style Presets & Customization:** Apply predefined styles (Anatomical, H&E, IHC) or define custom styles. Control stylization strength. * **Batch Processing:** Generate multiple diagrams simultaneously by entering one prompt per line. * **Metadata Association:** Tag generated images with optional Patient ID, Region of Interest (ROI), and UMLS/SNOMED CT codes. * **Interactive Annotation:** Draw annotations (freehand) directly onto the generated images using `streamlit-drawable-canvas`. * **Session History:** Keep track of generated images and their associated metadata within the current session. * **Annotation Export:** Download all annotations made during the session as a structured JSON file, including associated metadata and generation details. * **Robust Error Handling:** Provides informative feedback on API errors or other issues. * **Configurable:** Easy setup using Streamlit Secrets or environment variables for API keys. * **Clear History:** Option to clear the session history and annotations. ## Prerequisites * **Python:** Version 3.8 or higher recommended. * **pip:** Python package installer. * **Git:** For cloning the repository. * **OpenAI API Key:** You need an API key from OpenAI to use the DALL-E 3 / GPT-4o generation features. ## 🚀 Installation & Setup 1. **Clone the Repository:** ```bash git clone https://github.com/your-username/medsketch-ai.git # Replace with your repo URL cd medsketch-ai ``` 2. **Create `requirements.txt`:** Create a file named `requirements.txt` in the project root with the following content: ```txt streamlit openai streamlit-drawable-canvas Pillow requests ``` 3. **Install Dependencies:** ```bash pip install -r requirements.txt ``` 4. **Configure OpenAI API Key:** You **must** provide your OpenAI API key. There are two primary methods: * **a) Streamlit Secrets (Recommended for Deployment):** * Create a directory named `.streamlit` in your project root if it doesn't exist. * Inside `.streamlit`, create a file named `secrets.toml`. * Add your API key to `secrets.toml`: ```toml # .streamlit/secrets.toml OPENAI_API_KEY="sk-YourSecretOpenAI_ApiKeyGoesHere" ``` * **Important:** Ensure `.streamlit/secrets.toml` is added to your `.gitignore` file to prevent accidentally committing your secret key. * **b) Environment Variable (Good for Local Development):** * Set the `OPENAI_API_KEY` environment variable in your terminal session: * **Linux/macOS:** ```bash export OPENAI_API_KEY='sk-YourSecretOpenAI_ApiKeyGoesHere' ``` * **Windows (Command Prompt):** ```bash set OPENAI_API_KEY=sk-YourSecretOpenAI_ApiKeyGoesHere ``` * **Windows (PowerShell):** ```bash $env:OPENAI_API_KEY='sk-YourSecretOpenAI_ApiKeyGoesHere' ``` * The application will automatically look for this environment variable if the Streamlit secret is not found. ## ▶️ Running the Application Once the dependencies are installed and the API key is configured, run the Streamlit app from your project's root directory: ```bash streamlit run app.py Use code with caution. Markdown Your default web browser should automatically open to the application's URL (usually http://localhost:8501). 📖 Usage Guide Configure Settings (Sidebar): Select Model: Choose between "GPT-4o (API)" (uses DALL-E 3) or the placeholder "Stable Diffusion LoRA". Select Preset Style: Choose a visual style like "Anatomical Diagram", "H&E Histology", etc., or select "Custom" and enter your own style description. Stylization Strength: Adjust the slider to control how strongly the style influences the output (this is conceptually passed in the prompt). (Optional) Metadata: Enter relevant Patient/Case ID, ROI, or UMLS/SNOMED codes. These will be associated with the generated images in the history and export. Enter Prompts (Main Area): In the text area, describe the medical diagram(s) you want to generate. For batch generation, enter one prompt per line. Generate: Click the "🚀 Generate Diagram(s)" button. View Results: Generated images will appear below the button, organized in columns. Each result includes the image, the prompt used, and a download button (⬇️ Download PNG). Annotate (Optional): Below each image, an annotation canvas (✏️ Annotate:) is provided. Use your mouse to draw directly on the image (default is freehand red lines). Annotations are automatically saved to the session state. Review History & Export Annotations (Bottom Section): The "📚 Session History & Annotations" section appears once generations are complete. It lists the prompts used, model/style settings, and associated metadata for each generated item. You can expand each item to view the raw JSON data of any annotations made. Click "⬇️ Export All Annotations (JSON)" to download a JSON file containing all annotations from the current session, enriched with metadata and generation details. Clear History (Sidebar): Use the "⚠️ Clear History & Annotations" button in the sidebar to reset the session. 🛠️ Technology Stack Framework: Streamlit AI Generation: OpenAI API (DALL-E 3) Annotation: streamlit-drawable-canvas Image Handling: Pillow (PIL Fork) API Requests: requests (for image download if using URL format) Language: Python 💡 Future Enhancements (Roadmap) Implement actual Stable Diffusion LoRA model integration. Support for additional AI image generation models. More advanced annotation tools (shapes, text, colors). Ability to load/edit existing annotations. Improved image storage/retrieval in session state (potentially using caching or temporary files). User accounts and persistent storage (beyond session). More sophisticated prompt engineering assistance. 🙏 Contributing Contributions are welcome! If you have suggestions for improvements or find a bug, please feel free to: Open an issue to discuss the change or report the bug. Fork the repository, make your changes, and submit a pull request. Please ensure your code follows basic Python best practices and includes documentation where necessary. 📜 License This project is licensed under the MIT License - see the LICENSE.txt file for details. (You should create a LICENSE.txt file in your repository containing the text of the MIT License or your chosen license). **To make this README complete:** 1. **Replace Placeholders:** Update `https://github.com/your-username/medsketch-ai.git` and `https://your-deployed-app-url.com` with your actual URLs. 2. **Add Screenshot/GIF:** Capture a compelling visual of your app and embed it where indicated. This significantly improves understanding. 3. **Create `LICENSE.txt`:** Add a file named `LICENSE.txt` to your repository containing the full text of the MIT license (or whichever license you choose). You can easily find standard license text online (e.g., choosealicense.com). 4. **Commit `requirements.txt`:** Make sure the `requirements.txt` file described is actually created and committed to your repository. Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference