README.md

---
title: Click2mask
license: other
license_name: MIT License with Commons Clause Restriction
emoji: 🔵
colorFrom: indigo
colorTo: yellow
sdk: gradio
sdk_version: 5.27.1
app_file: app.py
pinned: false
short_description: Official Demo for Click2Mask (AAAI 2025)
---

# [AAAI 2025] Click2Mask: Local Editing with Dynamic Mask Generation

Official Hugging Face demo for ["Click2Mask: Local Editing with Dynamic Mask Generation"](https://omeregev.github.io/click2mask/) (AAAI 2025)

**Paper by:** [Omer Regev](https://www.linkedin.com/in/omeregev/), [Omri Avrahami](https://omriavrahami.com/), [Dani Lischinski](https://www.cs.huji.ac.il/~danix/)

<a href="https://omeregev.github.io/click2mask/">
  <img src="https://img.shields.io/badge/Website-blue?style=flat&logo=github" alt="Website">
</a>
<a href="https://github.com/omeregev/click2mask">
  <img src="https://img.shields.io/badge/GitHub-Code-blue?style=flat&logo=github" alt="GitHub Code">
</a>
<a href="https://colab.research.google.com/github/omeregev/click2mask/blob/main/demo.ipynb">
  <img src="https://img.shields.io/badge/Colab-Demo-orange?style=flat&logo=googlecolab" alt="Colab">
</a>
<a href="https://arxiv.org/abs/2409.08272">
  <img src="https://img.shields.io/badge/arXiv-2409.08272-b31b1b?style=flat&logo=arxiv" alt="arXiv">
</a>
<a href="https://omeregev.github.io/click2mask/static/paper/Click2Mask.pdf">
  <img src="https://img.shields.io/badge/Paper-PDF-red?style=flat&logo=adobe" alt="Paper PDF">
</a>
<a href="https://youtu.be/A0ZEVTm9SLw?si=_coDIWRXa8Wo-2na">
  <img src="https://img.shields.io/badge/Video-YouTube-red?style=flat&logo=youtube" alt="YouTube Video">
</a>
<br><br>

<img src="https://raw.githubusercontent.com/omeregev/click2mask/main/imgs/teaser.gif" alt="Click2Mask Teaser"/>

Given an image, a <b>Click</b>, and a prompt for an added object, a **Mask** is generated dynamically, 
simultaneously with the object generation throughout the diffusion process.

Current methods rely on existing objects/segments, or user effort (masks/detailed text), 
to localize object additions. Our approach enables free-form editing, 
where the manipulated area is not well-defined, using just a <b>Click</b> for localization.

## How to Use This Demo

1. **Upload an image** or load example with the button below
2. **Click on the image** where you want to add an object
3. **Enter a text prompt** describing what you want to add
4. **Click Generate** to see your edited image with the dynamically generated mask

> **Note:** This Space provides a simplified demo interface. For full functionality including batch processing and CLI tools (plus this Gradio interface), use the [Colab notebook](https://colab.research.google.com/github/omeregev/click2mask/blob/main/demo.ipynb) or [GitHub](https://github.com/omeregev/click2mask).
## Examples

<img src="https://raw.githubusercontent.com/omeregev/click2mask/main/imgs/results.jpg" alt="Click2Mask Results" width=900/>

## Comparison with SOTA Methods
<img src="https://raw.githubusercontent.com/omeregev/click2mask/main/imgs/compare.png" alt="Comparison" width=900/>

## Citation

```bibtex
@misc{regev2024click2masklocaleditingdynamic,
      title={Click2Mask: Local Editing with Dynamic Mask Generation}, 
      author={Omer Regev and Omri Avrahami and Dani Lischinski},
      year={2024},
      eprint={2409.08272},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2409.08272}, 
}
```

## Acknowledgements

This code is based on [Blended Latent Diffusion](https://github.com/omriav/blended-latent-diffusion/tree/master) and [Stable Diffusion](https://github.com/CompVis/stable-diffusion).