Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
@@ -1,13 +1,73 @@
|
|
1 |
---
|
2 |
title: EvoTransformer Demo
|
3 |
-
emoji:
|
4 |
colorFrom: pink
|
5 |
colorTo: green
|
6 |
sdk: gradio
|
7 |
-
sdk_version: 5.36.2
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
license: mit
|
11 |
---
|
12 |
|
13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
title: EvoTransformer Demo
|
3 |
+
emoji: π§¬
|
4 |
colorFrom: pink
|
5 |
colorTo: green
|
6 |
sdk: gradio
|
|
|
7 |
app_file: app.py
|
8 |
pinned: false
|
9 |
license: mit
|
10 |
---
|
11 |
|
12 |
+
# 𧬠EvoTransformer Demo
|
13 |
+
|
14 |
+
Welcome to the official demo of **EvoTransformer** β an evolving Transformer architecture built to adapt itself *during training* using principles inspired by evolutionary algorithms.
|
15 |
+
|
16 |
+
This project showcases a lightweight, in-training neural architecture search (NAS) system that mutates key traits such as:
|
17 |
+
- Number of layers
|
18 |
+
- Attention heads
|
19 |
+
- Feed-forward dimension
|
20 |
+
- Dropout
|
21 |
+
- Memory module toggle
|
22 |
+
|
23 |
+
> π Developed by **Dr. Heman Mohabeer**, Intelligent Africa Ltd
|
24 |
+
> π€ Submitted to JMLR 2025 | π Built from **Mauritius**
|
25 |
+
|
26 |
+
---
|
27 |
+
|
28 |
+
## π Try It Live
|
29 |
+
|
30 |
+
Use the Gradio interface to simulate architectural evolution across generations.
|
31 |
+
Visualize how traits adapt β and get a simulated accuracy + parameter estimate.
|
32 |
+
|
33 |
+
---
|
34 |
+
|
35 |
+
## π Behind the Scenes
|
36 |
+
|
37 |
+
EvoTransformer includes:
|
38 |
+
- Genetic operators: mutation, crossover (demo limited to mutation)
|
39 |
+
- Structural traits representation
|
40 |
+
- Online evolution loop
|
41 |
+
- Lightweight scoring and parameter estimation
|
42 |
+
|
43 |
+
This demo is a simplified, live-running version of the full EvoTransformer system submitted for peer review.
|
44 |
+
|
45 |
+
---
|
46 |
+
|
47 |
+
## π Citation
|
48 |
+
|
49 |
+
```bibtex
|
50 |
+
@misc{mohabeer2024evotransformer,
|
51 |
+
title={EvoTransformer: In-Training Evolution of Transformer Architectures for Adaptive and Efficient NLP},
|
52 |
+
author={Heman Mohabeer},
|
53 |
+
year={2024},
|
54 |
+
note={Hugging Face Demo},
|
55 |
+
url={https://huggingface.co/spaces/HemanM/EvoTransformer-Demo}
|
56 |
+
}
|
57 |
+
|
58 |
+
---
|
59 |
+
|
60 |
+
## π Links
|
61 |
+
|
62 |
+
- π [JMLR Submission PDF (coming soon)]()
|
63 |
+
- π§ [Colab Notebook (in progress)]()
|
64 |
+
- π [More from Dr. Heman Mohabeer](https://linkedin.com/in/hemanmohabeer)
|
65 |
+
|
66 |
+
---
|
67 |
+
|
68 |
+
## π License
|
69 |
+
|
70 |
+
MIT License β feel free to use, fork, and build upon this demo.
|
71 |
+
|
72 |
+
|
73 |
+
|