dimitarpg13 commited on
Commit
6ae0deb
·
verified ·
1 Parent(s): 0099be8

Rename Fock-Attention to 'Fock Attention (MLP V_theta)', fix stale pre-causal-leak-fix 9.30 PPL comparisons, add V_theta-vs-exchange-mechanism insight

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -294,8 +294,8 @@ model.eval()
294
  |---|---|---|---|
295
  | Matched Attention (baseline) | **7.81** | 19.5M | -- |
296
  | [Hybrid SPLM+Attn](https://huggingface.co/dimitarpg13/semsimula-hybrid-splm) | **8.50** | ~19.0M | +0.69 |
297
- | [Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) | **9.30** | 17.4M | +1.49 |
298
- | [Fock Attention](https://huggingface.co/dimitarpg13/semsimula-fock-attention) | **9.42** | 16.7M | +1.61 |
299
  | **Multi-Xi PARFLM** (this model) | **12.06** | 17.6M | +4.25 |
300
  | [Multi-Xi SPLM](https://huggingface.co/dimitarpg13/semsimula-splm-multixi) | **11.51** | 16.5M | +3.70 |
301
 
@@ -316,7 +316,7 @@ This model is part of the **Semantic Simulation SPLM family**:
316
  | Fock-PARFLM v2.1 (depth-cond. isotropic Gaussian) | Bounded multi-context + pairwise + Fock | TinyStories | 16.33 | [semsimula-fock-parflm-depthcond-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-depthcond-vtheta) |
317
  | Fock-PARFLM v2.1 (depth-cond. anisotropic Gaussian + fock-reg) | Bounded, ellipsoidal multi-context + pairwise + Fock | TinyStories | 9.04 | [semsimula-fock-parflm-anisogaussian-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta) |
318
  | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, first-order/Fock-G1) | Same as above, gradient-flow integrator | TinyStories | 8.95 | [semsimula-fock-parflm-anisogaussian-vtheta-fock-g1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-fock-g1) |
319
- | Fock-Attention | Fock + attention | TinyStories | 9.42 | [semsimula-fock-attention](https://huggingface.co/dimitarpg13/semsimula-fock-attention) |
320
  | Hybrid SPLM+Attn | Attention + SPLM refinement | TinyStories | 8.50 | [semsimula-hybrid-splm](https://huggingface.co/dimitarpg13/semsimula-hybrid-splm) |
321
  | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=384) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 278.27 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep) |
322
  | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=768) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 326.97 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep) |
 
294
  |---|---|---|---|
295
  | Matched Attention (baseline) | **7.81** | 19.5M | -- |
296
  | [Hybrid SPLM+Attn](https://huggingface.co/dimitarpg13/semsimula-hybrid-splm) | **8.50** | ~19.0M | +0.69 |
297
+ | [Fock Attention (MLP V_theta)](https://huggingface.co/dimitarpg13/semsimula-fock-attention) | **9.42** | 16.7M | +1.61 |
298
+ | [Fock-PARFLM v2.1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm) | **9.70** | 17.4M | +1.89 |
299
  | **Multi-Xi PARFLM** (this model) | **12.06** | 17.6M | +4.25 |
300
  | [Multi-Xi SPLM](https://huggingface.co/dimitarpg13/semsimula-splm-multixi) | **11.51** | 16.5M | +3.70 |
301
 
 
316
  | Fock-PARFLM v2.1 (depth-cond. isotropic Gaussian) | Bounded multi-context + pairwise + Fock | TinyStories | 16.33 | [semsimula-fock-parflm-depthcond-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-depthcond-vtheta) |
317
  | Fock-PARFLM v2.1 (depth-cond. anisotropic Gaussian + fock-reg) | Bounded, ellipsoidal multi-context + pairwise + Fock | TinyStories | 9.04 | [semsimula-fock-parflm-anisogaussian-vtheta](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta) |
318
  | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, first-order/Fock-G1) | Same as above, gradient-flow integrator | TinyStories | 8.95 | [semsimula-fock-parflm-anisogaussian-vtheta-fock-g1](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-fock-g1) |
319
+ | Fock Attention (MLP V_theta) | Fock + attention | TinyStories | 9.42 | [semsimula-fock-attention](https://huggingface.co/dimitarpg13/semsimula-fock-attention) |
320
  | Hybrid SPLM+Attn | Attention + SPLM refinement | TinyStories | 8.50 | [semsimula-hybrid-splm](https://huggingface.co/dimitarpg13/semsimula-hybrid-splm) |
321
  | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=384) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 278.27 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep) |
322
  | Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=768) | Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis | OpenWebText | 326.97 (best of 8, 3K-step sweep) | [semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep](https://huggingface.co/dimitarpg13/semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep) |