---
title: EvoTransformer Demo
emoji: 🧬
colorFrom: pink
colorTo: green
sdk: gradio
app_file: app.py
pinned: false
license: mit
sdk_version: 5.36.2
---

# 🧬 EvoTransformer Demo

Welcome to the official demo of **EvoTransformer** — an evolving Transformer architecture built to adapt itself *during training* using principles inspired by evolutionary algorithms.

This project showcases a lightweight, in-training neural architecture search (NAS) system that mutates key traits such as:
- Number of layers  
- Attention heads  
- Feed-forward dimension  
- Dropout  
- Memory module toggle  

> 📍 Developed by **Dr. Heman Mohabeer**, Intelligent Africa Ltd  
> 📤 Submitted to JMLR 2025 | 🌍 Built from **Mauritius**

---

## 🚀 Try It Live

Use the Gradio interface to simulate architectural evolution across generations.  
Visualize how traits adapt — and get a simulated accuracy + parameter estimate.

---

## 📊 Behind the Scenes

EvoTransformer includes:
- Genetic operators: mutation, crossover (demo limited to mutation)
- Structural traits representation
- Online evolution loop
- Lightweight scoring and parameter estimation

This demo is a simplified, live-running version of the full EvoTransformer system submitted for peer review.

---

## 📚 Citation

```bibtex
@misc{mohabeer2024evotransformer,
  title={EvoTransformer: In-Training Evolution of Transformer Architectures for Adaptive and Efficient NLP},
  author={Heman Mohabeer},
  year={2024},
  note={Hugging Face Demo},
  url={https://huggingface.co/spaces/HemanM/EvoTransformer-Demo}
}

---

## 🔗 Links

- 📄 [JMLR Submission PDF (coming soon)]()
- 🧠 [Colab Notebook (in progress)]()
- 📘 [More from Dr. Heman Mohabeer](https://linkedin.com/in/hemanmohabeer)

---

## 📜 License

MIT License — feel free to use, fork, and build upon this demo.