| Stage 1 is not as interesting for scalability, and this guide focuses on stages 2 and 3. The zero_optimization configuration contains all the options for what to enable and how to configure them. For a more detailed explanation of each parameter, take a look at the DeepSpeed Configuration JSON reference. | |
| DeepSpeed doesn’t validate parameter names and any typos fallback on the parameter's default setting. |