Update README.md
Browse files
README.md
CHANGED
|
@@ -56,8 +56,8 @@ Bright diagonal line: Each position matches itself perfectly.
|
|
| 56 |
Wider bright bands: Positions can "see" farther (good for long dependencies) but can be noisy.
|
| 57 |
Narrow bands: More focus on nearby positions (good for local patterns)
|
| 58 |
|
| 59 |
-
|
| 60 |
-
|
| 61 |
|
| 62 |
Static 10k theta is perfectly fine for a text model but probably not for a NLP ai.
|
| 63 |
|
|
@@ -78,7 +78,8 @@ The theoretical foundation:
|
|
| 78 |
- Varying the rotation frequency based on pitch creates a more speech-aware positional encoding
|
| 79 |
|
| 80 |
---
|
| 81 |
-
|
|
|
|
| 82 |
|
| 83 |
## The F0-Conditioned Rotation Mechanism
|
| 84 |
|
|
|
|
| 56 |
Wider bright bands: Positions can "see" farther (good for long dependencies) but can be noisy.
|
| 57 |
Narrow bands: More focus on nearby positions (good for local patterns)
|
| 58 |
|
| 59 |
+
<img width="470" alt="cc" src="https://github.com/user-attachments/assets/28d00fc5-2676-41ed-a971-e4d857af43f8" />
|
| 60 |
+
<img width="470" alt="cc2" src="https://github.com/user-attachments/assets/9089e806-966b-41aa-8793-bee03a6e6be1" />
|
| 61 |
|
| 62 |
Static 10k theta is perfectly fine for a text model but probably not for a NLP ai.
|
| 63 |
|
|
|
|
| 78 |
- Varying the rotation frequency based on pitch creates a more speech-aware positional encoding
|
| 79 |
|
| 80 |
---
|
| 81 |
+
|
| 82 |
+
<img width="470" alt="cc2" src="https://github.com/user-attachments/assets/d52a48b1-8717-4d29-9452-cfdf43c92fe8" />
|
| 83 |
|
| 84 |
## The F0-Conditioned Rotation Mechanism
|
| 85 |
|