add ressources 1 (#32)

- add ressources 1 (31d2d7654c86abf91f87a59a26e5ade4a44a1825)


Co-authored-by: Elie Bakouch <[email protected]>

src/index.html

@@ -2464,6 +2464,11 @@
             <a href="https://arxiv.org/abs/2312.11805"><strong>Gemini</strong></a>
             <p>Presents Google's multimodal model architecture capable of processing text, images, audio, and video inputs.</p>
         </div>
+              
+        <div>
+            <a href="https://arxiv.org/abs/2407.21783"><strong>Llama 3</strong></a>
+            <p>The Llama 3 Herd of Models</p>
+        </div>
 
         <div>
             <a href="https://arxiv.org/abs/2412.19437v1"><strong>DeepSeek-V3</strong></a>
@@ -2472,7 +2477,6 @@
 
 
         <h3>Training Frameworks</h3>
-
         <div>
             <a href="https://github.com/facebookresearch/fairscale/tree/main"><strong>FairScale</strong></a>
             <p>PyTorch extension library for large-scale training, offering various parallelism and optimization techniques.</p>
@@ -2525,6 +2529,11 @@
             <p>Comprehensive guide to understanding and optimizing GPU memory usage in PyTorch.</p>
         </div>
 
+        <div>
+            <a href="https://huggingface.co/blog/train_memory"><strong>Memory profiling walkthrough on a simple example</strong></a>
+            <p>Visualize and understand GPU memory in PyTorch.</p>
+        </div>
+
         <div>
             <a href="https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html"><strong>TensorBoard Profiler Tutorial</strong></a>
             <p>Guide to using TensorBoard's profiling tools for PyTorch models.</p>
@@ -2586,6 +2595,11 @@
             <a href="https://arxiv.org/abs/1710.03740"><strong>Mixed precision training</strong></a>
             <p>Introduces mixed precision training techniques for deep learning models.</p>
         </div>
+              
+        <div>
+            <a href="https://main-horse.github.io/posts/visualizing-6d/"><strong>@main_horse blog</strong></a>
+            <p>Visualizing 6D Mesh Parallelism</p>
+        </div>
 
         <h3>Hardware</h3>
 
@@ -2603,6 +2617,11 @@
             <a href="https://www.semianalysis.com/p/100000-h100-clusters-power-network"><strong>Semianalysis - 100k H100 cluster</strong></a>
             <p>Analysis of large-scale H100 GPU clusters and their implications for AI infrastructure.</p>
         </div>
+              
+        <div>
+            <a href="https://modal.com/gpu-glossary/readme"><strong>Modal GPU Glossary </strong></a>
+            <p>CUDA docs for human</p>
+        </div>
 
         <h3>Others</h3>
 
@@ -2630,6 +2649,37 @@
             <a href="https://www.harmdevries.com/post/context-length/"><strong>Harm's blog for long context</strong></a>
             <p>Investigation into long context training in terms of data and training cost.</p>
         </div>
+              
+        <div>
+            <a href="https://www.youtube.com/@GPUMODE/videos"><strong>GPU Mode</strong></a>
+            <p>A GPU reading group and community.</p>
+        </div>
+              
+        <div>
+            <a href="https://youtube.com/playlist?list=PLvtrkEledFjqOLuDB_9FWL3dgivYqc6-3&si=fKWPotx8BflLAUkf"><strong>EleutherAI Youtube channel</strong></a>
+            <p>ML Scalability & Performance Reading Group</p>
+        </div>
+              
+        <div>
+            <a href="https://jax-ml.github.io/scaling-book/"><strong>Google Jax Scaling book</strong></a>
+            <p>How to Scale Your Model</p>
+        </div>
+              
+        <div>
+            <a href="https://github.com/facebookresearch/capi/blob/main/fsdp.py"><strong>@fvsmassa & @TimDarcet FSDP</strong></a>
+            <p>Standalone ~500 LoC FSDP implementation</p>
+        </div>
+
+        <div>
+            <a href="https://www.thonking.ai/"><strong>thonking.ai</strong></a>
+            <p>Some of Horace He blogpost</p>
+        </div>
+              
+        <div>
+            <a href="https://gordicaleksa.medium.com/eli5-flash-attention-5c44017022ad"><strong>Aleksa's ELI5 Flash Attention</strong></a>
+            <p>Easy explanation of Flash Attention</p>
+        </div>
+        
         
         <h2>Appendix</h2>