README_WEIGHTS.md

SousChef-v1 Weight File Documentation

New Fields in config.json

• model_type: Specifies the model type, which is updated to souschef_v1 in this release.
• num_recipe_predict_layers: Indicates the number of Recipe Prediction (RP) Modules. The open-sourced SousChef-v1 weights include 2 RP Modules.
• quantization_config: Describes the configuration for FP8 quantization.

---

Weight Structure Overview

The SousChef-v1 weight file consists of two main components: Main Model Weights and RP Modules.

1. Main Model Weights

• Composition:
  • Input/output embedding layers and a complete set of 48 Transformer hidden layers.
• Parameter Count:
  • Total parameters: 250B
  • Activation parameters: 20.3B (including 1.2B for Embedding and 1.1B for the output Head).

Structural Details

• Embedding Layer:
  • model.embed_tokens.weight
• Transformer Hidden Layers:
  • model.layers.0 to model.layers.47, totaling num_hidden_layers layers.
• Output Layer:
  • model.norm.weight
  • lm_head.weight

2. Recipe Prediction (RP) Modules

• Composition:
  • Additional RP Modules defined by the num_recipe_predict_layers field. In this model, the value is set to 2.
• Parameter Count:
  • Parameters: 10.5B unique parameters, excluding the shared 1.2B Embedding and 1.1B output Head.
  • Activation parameters: 3.2B (including the shared 1.2B Embedding and 1.1B output Head).

Structural Details

• embed_tokens: Shares parameters with the Embedding layer of the Main Model weights.
• enorm & hnorm: RMSNorm parameters required for speculative recipe prediction.
• rp_proj: Parameters for dimensionality reduction projection on the norm results.
• Additional Transformer Hidden Layers:
  • model.layers.48.self_attn & mlp to model.layers.49.self_attn & mlp (structure identical to the Main Model hidden layers).
• shared_head: Shares parameters with the output Head of the Main Model weights.

---

Loading Rules

• Main Model Weights: Loaded via the num_hidden_layers parameter in config.json.
• RP Modules: Loaded via the num_recipe_predict_layers parameter, with layer IDs appended immediately after the Main Model hidden layers. For example:
  • If num_hidden_layers = 48 and num_recipe_predict_layers = 2, the RP Module's layer IDs are 48 and 49.

---

FP8 Weight Documentation

SousChef-v1 natively supports FP8 weight format with 128x128 block scaling.

FP8 Configuration

The FP8 weight file introduces a quantization_config field to describe the quantization method. Below is an example configuration:

"quantization_config": {
  "activation_scheme": "dynamic",
  "fmt": "e4m3",
  "quant_method": "fp8",
  "weight_block_size": [128, 128]
}

• Quantization Format:
  • Format type: fp8 and e4m3 (corresponding to torch.float8_e4m3fn).
  • Weight block size: 128x128.
• Activation Quantization Scheme:
  • Utilizes dynamic activation quantization (dynamic).

Dequantization Method

The FP8 weight file includes a weight_scale_inv field, which stores the dequantization scale for each weight block.

• Storage Format: float32 Tensor, stored alongside the weight data.
• Dequantization Formula:
  • If the weight block is not aligned to 128, it is zero-padded to 128 before calculating the scale. After quantization, the padded portion is removed.
  • The dequantization process is performed as: (128x128 weight block) * weight_scale_inv.

Through dequantization of the FP8 weights, runtime operations enable online quantization at a granularity of per-token-per-128-channel.

---