File size: 4,120 Bytes
c71839b a4ef927 6e4be07 a4ef927 6e4be07 a4ef927 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 |
---
title: MADGuard AI Explorer
emoji: 🧠
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: "4.44.0"
app_file: app.py
pinned: false
---
# 🧠 MADGuard AI Explorer
A diagnostic Gradio tool to simulate feedback loops in Retrieval-Augmented Generation (RAG) pipelines and detect **Model Autophagy Disorder (MAD)** risks.
---
## 🛠️ Tool Description
- Toggle between **real** and **synthetic** input sources
- Visualize pipeline feedback loops with **Graphviz**
- Analyze training data via:
- Type-Token Ratio (TTR)
- Cosine Similarity
- Composite MAD Risk Score
---
## 🚀 Run It Locally
```bash
git clone <your-repo-url>
cd madguard
pip install -r requirements.txt
python app.py
```
Then open [http://127.0.0.1:7860](http://127.0.0.1:7860) in your browser.
---
## 🌐 Deploy on Hugging Face Spaces
1. Create a new Space (select **Gradio** as the SDK)
2. Upload:
- `app.py`
- `requirements.txt`
- All files in the `visuals/` folder
3. Hugging Face builds the app and gives you a public URL
---
<details>
<summary>📚 Research Background</summary>
### 📄 Self-consuming LLMs: How and When Models Feed Themselves – Santurkar et al., 2023
This paper introduces and explores **Model Autophagy Disorder (MAD)** — showing that large language models trained on their own outputs tend to lose performance and accumulate error over time.
**MADGuard implements several of the paper’s proposed detection strategies:**
| Research Recommendation | MADGuard Implementation |
| ------------------------------------------- | ----------------------------------------- |
| Lexical redundancy analysis | ✅ via Type-Token Ratio (TTR) |
| Embedding-based similarity scoring | ✅ via SentenceTransformers + cosine |
| Warning system for feedback loop risk | ✅ risk score (Low / Medium / High) |
| Distinguishing real vs. synthetic inputs | ❌ not implemented (user-controlled only) |
| Multi-round retraining degradation tracking | ❌ not yet supported |
> “MADGuard AI Explorer is inspired by key findings from this research, aligning with early warnings and pipeline hygiene practices recommended in their work.”
📎 [Read Full Paper on arXiv](https://arxiv.org/abs/2307.01850)
</details>
---
<details>
<summary>👥 Who Is It For?</summary>
- **AI/ML Engineers**: Prevent model collapse due to training on synthetic outputs
- **MLOps Professionals**: Pre-retraining diagnostics
- **AI Researchers**: Study model feedback loops
- **Responsible AI Teams**: Audit data pipelines for ethical AI
### Why Use It?
- Avoid data contamination
- Ensure model freshness
- Support data-centric decisions
- Provide audit-ready diagnostics
</details>
---
<details>
<summary>🧱 Limitations & Future Plans</summary>
### 🔸 Current Limitations
| Area | Missing Element |
| ------------------- | ----------------------------------------- |
| Multi-batch Uploads | No history or comparative dataset support |
| Real/Synthetic Tag | No auto-tagging or provenance logging |
| Visual Analytics | No charts, timelines, or embeddings view |
| Custom Thresholds | Fixed MAD score weightings |
| Provenance Tracking | No metadata or source history logging |
### 🔮 Future Plans
- 📊 Batch evaluations with historical trendlines
- 🧠 RAG framework integration (e.g., LangChain)
- 🧩 Live evaluation API endpoint
- 🔒 Source tracking and audit trails
- 🧾 Exportable audit/compliance reports
</details>
---
<details>
<summary>📄 More Details</summary>
### 🔍 Features Recap
- Simulates feedback loops in RAG pipelines
- Visualizes flow using Graphviz
- Accepts `.csv` or `.json` data
- Calculates TTR, cosine similarity, MAD score
- Classifies risk (Low / Medium / High)
- Offers human-readable suggestions
- Based on: [Santurkar et al., 2023 – arXiv:2307.01850](https://arxiv.org/abs/2307.01850)
### 📜 License
MIT License (see [LICENSE](LICENSE))
</details>
---
|