Add pipeline tag, abstract, code link, and relevant images (#1)

- Add pipeline tag, abstract, code link, and relevant images (1778272aae6d07bc178ca3d3a1a491b576b6767e)


Co-authored-by: Niels Rogge <[email protected]>

README.md

@@ -1,32 +1,43 @@
 ---
-library_name: diffusers
-license: mit
 base_model:
 - black-forest-labs/FLUX.1-dev
+library_name: diffusers
+license: mit
+pipeline_tag: text-to-image
 ---
 
+# FLUX.1-dev-PrefGRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
 
+This model is trained using [Pref-GRPO](https://codegoat24.github.io/UnifiedReward/Pref-GRPO) on the training dataset of [UniGenBench](https://github.com/CodeGoat24/UniGenBench).
 
+## Abstract
+Recent advancements highlight the importance of GRPO-based reinforcement learning methods and benchmarking in enhancing text-to-image (T2I) generation. However, current methods using pointwise reward models (RM) for scoring generated images are susceptible to reward hacking. We reveal that this happens when minimal score differences between images are amplified after normalization, creating illusory advantages that drive the model to over-optimize for trivial gains, ultimately destabilizing the image generation process. To address this, we propose Pref-GRPO, a pairwise preference reward-based GRPO method that shifts the optimization objective from score maximization to preference fitting, ensuring more stable training. In Pref-GRPO, images are pairwise compared within each group using preference RM, and the win rate is used as the reward signal. Extensive experiments demonstrate that PREF-GRPO differentiates subtle image quality differences, providing more stable advantages and mitigating reward hacking. Additionally, existing T2I benchmarks are limited by coarse evaluation criteria, hindering comprehensive model assessment. To solve this, we introduce UniGenBench, a unified T2I benchmark comprising 600 prompts across 5 main themes and 20 subthemes. It evaluates semantic consistency through 10 primary and 27 sub-criteria, leveraging MLLM for benchmark construction and evaluation. Our benchmarks uncover the strengths and weaknesses of both open and closed-source T2I models and validate the effectiveness of Pref-GRPO.
 
-## FLUX.1-dev-PrefGRPO
-
-This model is trained using [Pref-GRPO](https://codegoat24.github.io/UnifiedReward/Pref-GRPO) on the training dataset of [UniGenBench](https://github.com/CodeGoat24/UniGenBench).
+## Model Overview
+Here are the main components of the Pref-GRPO pipeline:
+<div align="center">
+    <img src="https://github.com/CodeGoat24/Pref-GRPO/raw/main/assets/pref_grpo_pipeline.png" alt="Pref-GRPO Pipeline" width="100%"/>
+</div>
 
+Pref-GRPO addresses the issue of reward hacking in text-to-image generation:
+<div align="center">
+    <img src="https://github.com/CodeGoat24/Pref-GRPO/raw/main/assets/pref_grpo_reward_hacking.png" alt="Pref-GRPO Reward Hacking" width="100%"/>
+</div>
 
 For further details, please refer to the following resources:
 - 📰 Paper: https://arxiv.org/pdf/2508.20751
 - 🪐 Project Page: https://codegoat24.github.io/UnifiedReward/Pref-GRPO
+- 💻 Code: https://github.com/CodeGoat24/Pref-GRPO
 - 🤗 UniGenBench: https://github.com/CodeGoat24/UniGenBench
 - 🤗 Leaderboard: https://huggingface.co/spaces/CodeGoat24/UniGenBench_Leaderboard
 - 👋 Point of Contact: [Yibin Wang](https://codegoat24.github.io)
 
-
 ### Quick Start
-~~~python
+```python
 pip install -U diffusers
-~~~
+```
 
-~~~python
+```python
 import torch
 from diffusers import FluxPipeline
 
@@ -45,12 +56,11 @@ image = pipe(
 ).images[0]
 image.save("flux-dev.png")
 
-~~~
-
+```
 
 ## Citation
 
-```
+```bibtex
 @article{Pref-GRPO&UniGenBench,
   title={Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning},
   author={Wang, Yibin and Li, Zhimin and Zang, Yuhang and Zhou, Yujie and Bu, Jiazi and Wang, Chunyu and Lu, Qinglin, and Jin, Cheng and Wang, Jiaqi},