Image-Text-to-Text
Transformers
Safetensors
qwen3_5
qwen3.5
Merge
omnimerge
task-arithmetic
code
conversational
Instructions to use ManniX-ITA/Qwen3.5-4B-MicroCoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ManniX-ITA/Qwen3.5-4B-MicroCoder with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="ManniX-ITA/Qwen3.5-4B-MicroCoder") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("ManniX-ITA/Qwen3.5-4B-MicroCoder") model = AutoModelForMultimodalLM.from_pretrained("ManniX-ITA/Qwen3.5-4B-MicroCoder", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ManniX-ITA/Qwen3.5-4B-MicroCoder with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ManniX-ITA/Qwen3.5-4B-MicroCoder" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ManniX-ITA/Qwen3.5-4B-MicroCoder", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/ManniX-ITA/Qwen3.5-4B-MicroCoder
- SGLang
How to use ManniX-ITA/Qwen3.5-4B-MicroCoder with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ManniX-ITA/Qwen3.5-4B-MicroCoder" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ManniX-ITA/Qwen3.5-4B-MicroCoder", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ManniX-ITA/Qwen3.5-4B-MicroCoder" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ManniX-ITA/Qwen3.5-4B-MicroCoder", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use ManniX-ITA/Qwen3.5-4B-MicroCoder with Docker Model Runner:
docker model run hf.co/ManniX-ITA/Qwen3.5-4B-MicroCoder
Add files using upload-large-folder tool
Browse files- .caliber/learning/current-session.jsonl +7 -0
- .caliber/learning/state.json +6 -0
- .gitattributes +1 -0
- chat_template.jinja +88 -0
- config.json +114 -0
- model-00001-of-00002.safetensors +3 -0
- model-00002-of-00002.safetensors +3 -0
- model.safetensors.index.json +745 -0
- processor_config.json +63 -0
- tokenizer.json +3 -0
- tokenizer_config.json +300 -0
.caliber/learning/current-session.jsonl
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"timestamp":"2026-05-04T06:49:50.467Z","session_id":"4208dc59-08bc-4b1e-8d4e-8ec3407fd4ac","hook_event_name":"PostToolUse","tool_name":"Bash","tool_input":{"command":"kill 52848 2>/dev/null; sleep 2; pgrep -af \"hf upload\" | grep -v claude | head; rm -rf /srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith/.caliber /srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith/.merge_done; ls -la /srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith/ | head","description":"Kill upload, clean dir"},"tool_response":{"stdout":"total 9121028\ndrwxr-sr-x 3 root users 323 May 4 08:49 .\ndrwxr-sr-x 39 root users 4096 May 4 01:44 ..\ndrwxr-sr-x 3 root users 33 May 4 07:23 .cache\n-rw-r--r-- 1 root users 4047 May 1 14:24 chat_template.jinja\n-rw-r--r-- 1 root users 3471 May 1 14:24 config.json\n-rw-r--r-- 1 root users 5376584768 May 3 18:45 model-00001-of-00002.safetensors\n-rw-r--r-- 1 root users 3943243384 May 3 18:49 model-00002-of-00002.safetensors\n-rw-r--r-- 1 root users 67340 May 3 18:50 model.safetensors.index.json\n-rw-r--r-- 1 root users 1300 May 1 14:35 processor_config.json","stderr":"","interrupted":false,"isImage":false,"noOutputExpected":false},"tool_use_id":"toolu_018GzyrD2A6eTH8SFXNAsXe4","cwd":"/srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith"}
|
| 2 |
+
{"timestamp":"2026-05-04T06:50:01.952Z","session_id":"4208dc59-08bc-4b1e-8d4e-8ec3407fd4ac","hook_event_name":"PostToolUse","tool_name":"Bash","tool_input":{"command":"cd /srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith; nohup hf upload-large-folder --repo-type=model --no-bars --no-report ManniX-ITA/Qwen3.5-4B-MicroCoder . > /srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/logs/microcoder_weights_upload.log 2>&1 & echo \"PID=$!\"; disown; sleep 3; pgrep -af \"hf upload-large-folder\" | grep -v claude | head -2","description":"Restart upload, clean output"},"tool_response":{"stdout":"PID=57784\n57784 /usr/bin/python3 /usr/local/bin/hf upload-large-folder --repo-type=model --no-bars --no-report ManniX-ITA/Qwen3.5-4B-MicroCoder .","stderr":"","interrupted":false,"isImage":false,"noOutputExpected":false},"tool_use_id":"toolu_016duUfqqtzKAvy7rpBHGJt7","cwd":"/srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith"}
|
| 3 |
+
{"timestamp":"2026-05-04T07:02:41.683Z","session_id":"4208dc59-08bc-4b1e-8d4e-8ec3407fd4ac","hook_event_name":"UserPromptSubmit","prompt_content":"Diving in now in which sense? why you stopped then? are you waiting for something?","cwd":"/srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith"}
|
| 4 |
+
{"timestamp":"2026-05-04T07:02:47.668Z","session_id":"4208dc59-08bc-4b1e-8d4e-8ec3407fd4ac","hook_event_name":"PostToolUse","tool_name":"Bash","tool_input":{"command":"mkdir -p /shared/dev/omnimergekit/recipes/microcoder_4b /srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/pruned","description":"Create output dirs"},"tool_response":{"stdout":"","stderr":"","interrupted":false,"isImage":false,"noOutputExpected":true},"tool_use_id":"toolu_011c2n8bsYtgkKBPuG5qk834","cwd":"/srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith"}
|
| 5 |
+
{"timestamp":"2026-05-04T07:10:04.049Z","session_id":"4208dc59-08bc-4b1e-8d4e-8ec3407fd4ac","hook_event_name":"PostToolUse","tool_name":"Write","tool_input":{"file_path":"/shared/dev/omnimergekit/recipes/microcoder_4b/prune_local_heal.py","content":"#!/usr/bin/env python3\n\"\"\"\nprune_local_heal.py — Qwen3.5 hybrid-arch attention head pruning with\n backprop-free local healing (W_O lstsq refit).\n\nTargets the Qwen3.5 hybrid architecture which mixes:\n - full_attention layers (Qwen3_5Attention with gated query, GQA 16:4, head_dim=256)\n - linear_attention layers (Qwen3_5GatedDeltaNet, 32 v_heads / 16 k_heads, head_dim=128)\n\nTwo prune levers:\n --prune-full-frac : fraction of Q heads to drop in full_attention layers\n --prune-linear-frac : fraction of V heads to drop in linear_attention layers\n\nPhases:\n 0. Capture per-layer o_proj-input (full_attn) and out_proj-input (linear_attn)\n on a calibration set — these are the targets for healing.\n 1. Score per-head importance via Michel-style gradient on per-head scale α=1.\n 2. For each layer, in forward order, on the *current* (already-modified) model:\n a. Re-capture pre-proj input\n b. Zero rows for dropped heads in q/k/v / in_proj_qkv (and corresponding\n rows in z/b/a/conv1d/A_log/dt_bias for linear_attn)\n c. lstsq refit the kept columns of o_proj / out_proj so the post-proj\n output matches the original target\n\nThe model stays full-size on disk (mask-style); a separate physical-resize\nstep converts to a smaller model after eval validation. Mask-pruning is what\nlstsq refit operates on, and it preserves output quality identically — only\ninference compute differs.\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nimport json\nimport os\nimport time\nfrom pathlib import Path\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\n\ndef log(msg: str):\n print(f\"[{time.strftime('%H:%M:%S')}] {msg}\", flush=True)\n\n\n# ---------- calibration ----------\n\nCALIB_TEXTS = [\n # Code-leaning (HumanEval/MBPP-style prompts) + a few prose to keep general-purpose signal\n \"from typing import List\\n\\n\\ndef has_close_elements(numbers: List[float], threshold: float) -> bool:\\n \\\"\\\"\\\"Check if there are any two numbers in the list that are closer than threshold.\\\"\\\"\\\"\\n for i, a in enumerate(numbers):\\n for j, b in enumerate(numbers):\\n if i != j and abs(a - b) < threshold:\\n return True\\n return False\\n\",\n \"def fibonacci(n: int) -> int:\\n \\\"\\\"\\\"Return the n-th Fibonacci number.\\\"\\\"\\\"\\n if n < 2:\\n return n\\n a, b = 0, 1\\n for _ in range(n - 1):\\n a, b = b, a + b\\n return b\\n\",\n \"import re\\n\\n\\ndef is_palindrome(s: str) -> bool:\\n s = re.sub(r'[^A-Za-z0-9]', '', s).lower()\\n return s == s[::-1]\\n\",\n \"from collections import Counter\\n\\n\\ndef most_common_word(text: str) -> str:\\n words = text.lower().split()\\n return Counter(words).most_common(1)[0][0]\\n\",\n \"def quicksort(arr):\\n if len(arr) <= 1: return arr\\n pivot = arr[len(arr)//2]\\n return quicksort([x for x in arr if x < pivot]) + [x for x in arr if x == pivot] + quicksort([x for x in arr if x > pivot])\\n\",\n \"def merge_sorted(a, b):\\n \\\"\\\"\\\"Merge two pre-sorted lists into a sorted list.\\\"\\\"\\\"\\n out, i, j = [], 0, 0\\n while i < len(a) and j < len(b):\\n if a[i] <= b[j]: out.append(a[i]); i += 1\\n else: out.append(b[j]); j += 1\\n out.extend(a[i:]); out.extend(b[j:])\\n return out\\n\",\n \"import heapq\\n\\n\\ndef k_largest(nums, k):\\n return heapq.nlargest(k, nums)\\n\",\n \"class Stack:\\n def __init__(self): self._items = []\\n def push(self, x): self._items.append(x)\\n def pop(self): return self._items.pop() if self._items else None\\n def peek(self): return self._items[-1] if self._items else None\\n def __len__(self): return len(self._items)\\n\",\n \"def binary_search(arr, target):\\n lo, hi = 0, len(arr) - 1\\n while lo <= hi:\\n mid = (lo + hi) // 2\\n if arr[mid] == target: return mid\\n if arr[mid] < target: lo = mid + 1\\n else: hi = mid - 1\\n return -1\\n\",\n \"import json, sys\\n\\n\\ndef sum_field(path: str, field: str) -> float:\\n with open(path) as f:\\n rows = json.load(f)\\n return sum(r.get(field, 0) for r in rows)\\n\",\n \"The mitochondrion is a double-membrane-bound organelle found in most eukaryotic cells. They generate most of the cell's supply of adenosine triphosphate (ATP), used as a source of chemical energy.\",\n \"In linear algebra, the determinant is a scalar value that is a function of the entries of a square matrix. It allows characterizing some properties of the matrix and the linear map represented by it.\",\n]\n\n\ndef build_calib(tokenizer, target_tokens: int, device: str) -> torch.Tensor:\n \"\"\"Concat CALIB_TEXTS, tokenize, take first target_tokens. Returns (1, T) on device.\"\"\"\n text = \"\\n\\n\".join(CALIB_TEXTS)\n while True:\n ids = tokenizer(text, return_tensors=\"pt\").input_ids\n if ids.shape[1] >= target_tokens:\n break\n text = text + \"\\n\\n\" + text # double until enough\n ids = ids[:, :target_tokens].to(device)\n log(f\"calib: {ids.shape[1]} tokens (vocab id range {ids.min().item()}..{ids.max().item()})\")\n return ids\n\n\n# ---------- model layout helpers ----------\n\ndef get_layers(model):\n \"\"\"Return decoder layers list regardless of nesting.\"\"\"\n if hasattr(model, \"model\") and hasattr(model.model, \"layers\"):\n return model.model.layers\n if hasattr(model, \"model\") and hasattr(model.model, \"language_model\"):\n return model.model.language_model.layers\n if hasattr(model, \"language_model\"):\n return model.language_model.layers\n raise RuntimeError(\"could not locate decoder layers\")\n\n\ndef text_config(model):\n cfg = model.config\n if hasattr(cfg, \"text_config\"):\n return cfg.text_config\n return cfg\n\n\ndef layer_kind(model, idx: int) -> str:\n \"\"\"'full_attention' or 'linear_attention'.\"\"\"\n return text_config(model).layer_types[idx]\n\n\ndef is_full(layer) -> bool:\n return hasattr(layer, \"self_attn\")\n\n\ndef is_linear(layer) -> bool:\n return hasattr(layer, \"linear_attn\")\n\n\n# ---------- phase 0: capture targets ----------\n\ndef phase0_capture(model, calib_ids):\n \"\"\"\n Forward the un-modified model and capture, per attention-bearing layer:\n - full_attn: input to o_proj (the gated, concatenated head output, shape [B, T, n_q*head_dim])\n - linear_attn: input to out_proj (shape [B, T, value_dim])\n These are the targets the lstsq refit will try to reproduce post-prune.\n \"\"\"\n layers = get_layers(model)\n targets = {} # layer_idx -> tensor (cpu)\n handles = []\n\n for L, layer in enumerate(layers):\n if is_full(layer):\n proj = layer.self_attn.o_proj\n elif is_linear(layer):\n proj = layer.linear_attn.out_proj\n else:\n continue\n\n def make_hook(idx):\n def hook(module, inputs, output):\n # input is the tensor fed into o_proj/out_proj; capture and detach\n x = inputs[0].detach().to(torch.float32).cpu()\n targets[idx] = x\n return hook\n handles.append(proj.register_forward_pre_hook(lambda m, inp, idx=L: targets.__setitem__(idx, inp[0].detach().to(torch.float32).cpu())))\n\n log(f\"phase0: registered {len(handles)} pre-hooks; running forward...\")\n with torch.no_grad():\n model(calib_ids)\n\n for h in handles:\n h.remove()\n log(f\"phase0: captured {len(targets)} layer targets\")\n return targets\n\n\n# ---------- phase 1: per-head importance via Michel-style gradient ----------\n\nclass FullAttnHeadGate(nn.Module):\n \"\"\"\n Wraps Qwen3_5Attention to insert a per-Q-head scaling factor α.\n α multiplies the gated, concatenated head output before o_proj. After\n fwd+bwd, |grad α| at α=1 ranks Q-head importance.\n \"\"\"\n def __init__(self, attn, num_q_heads: int, head_dim: int):\n super().__init__()\n self.attn = attn\n self.alpha = nn.Parameter(torch.ones(num_q_heads))\n self.num_q_heads = num_q_heads\n self.head_dim = head_dim\n self._orig_o_proj_forward = attn.o_proj.forward\n\n def patched_o_proj(x: torch.Tensor) -> torch.Tensor:\n # x: [B, T, n_q*head_dim]. Reshape to [B, T, n_q, head_dim], scale, flatten.\n B, T, _ = x.shape\n x_h = x.view(B, T, self.num_q_heads, self.head_dim)\n x_h = x_h * self.alpha.view(1, 1, -1, 1)\n x = x_h.view(B, T, -1)\n return self._orig_o_proj_forward(x)\n\n attn.o_proj.forward = patched_o_proj\n\n def restore(self):\n self.attn.o_proj.forward = self._orig_o_proj_forward\n\n\nclass LinearAttnHeadGate(nn.Module):\n \"\"\"\n Wraps Qwen3_5GatedDeltaNet to insert a per-V-head scaling factor α.\n α scales the value-dim signal feeding out_proj (post-norm, pre-projection).\n \"\"\"\n def __init__(self, lin_attn, num_v_heads: int, head_v_dim: int):\n super().__init__()\n self.lin_attn = lin_attn\n self.alpha = nn.Parameter(torch.ones(num_v_heads))\n self.num_v_heads = num_v_heads\n self.head_v_dim = head_v_dim\n self._orig_out_proj_forward = lin_attn.out_proj.forward\n\n def patched_out_proj(x: torch.Tensor) -> torch.Tensor:\n # x: [B, T, value_dim] = [B, T, n_v*head_v_dim]\n B, T, _ = x.shape\n x_h = x.view(B, T, self.num_v_heads, self.head_v_dim)\n x_h = x_h * self.alpha.view(1, 1, -1, 1)\n x = x_h.view(B, T, -1)\n return self._orig_out_proj_forward(x)\n\n lin_attn.out_proj.forward = patched_out_proj\n\n def restore(self):\n self.lin_attn.out_proj.forward = self._orig_out_proj_forward\n\n\ndef phase1_importance(model, calib_ids):\n \"\"\"\n Insert α per attention-bearing layer, fwd+bwd a single CE loss (next-token),\n return |grad α| per head, per layer.\n \"\"\"\n cfg = text_config(model)\n layers = get_layers(model)\n n_q_full = cfg.num_attention_heads\n head_dim_full = cfg.head_dim\n n_v_lin = cfg.linear_num_value_heads\n head_v_dim_lin = cfg.linear_value_head_dim\n\n gates = {} # layer_idx -> gate\n for L, layer in enumerate(layers):\n if is_full(layer):\n g = FullAttnHeadGate(layer.self_attn, n_q_full, head_dim_full)\n g.alpha.requires_grad_(True)\n gates[L] = g\n elif is_linear(layer):\n g = LinearAttnHeadGate(layer.linear_attn, n_v_lin, head_v_dim_lin)\n g.alpha.requires_grad_(True)\n gates[L] = g\n\n # All non-α params: no grad\n for p in model.parameters():\n p.requires_grad_(False)\n for g in gates.values():\n g.alpha.requires_grad_(True)\n\n log(f\"phase1: gates installed on {len(gates)} layers; computing CE+bwd...\")\n model.train() # so dropout=0 still respects training-mode reqgrads — but model has dropout=0\n # Use cross-entropy on shifted next-token prediction\n out = model(calib_ids)\n logits = out.logits # [1, T, V]\n shift_logits = logits[:, :-1, :].contiguous().to(torch.float32)\n shift_labels = calib_ids[:, 1:].contiguous()\n loss = F.cross_entropy(shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1))\n log(f\"phase1: CE loss = {loss.item():.4f}\")\n loss.backward()\n\n importance = {}\n for L, g in gates.items():\n imp = g.alpha.grad.detach().abs().to(torch.float32).cpu()\n importance[L] = imp\n g.restore()\n model.eval()\n log(f\"phase1: importance computed for {len(importance)} layers\")\n return importance\n\n\n# ---------- phase 2: per-layer prune + lstsq refit ----------\n\n@torch.no_grad()\ndef prune_full_attention_inplace(layer, keep_q_heads, target_olin_input, model, calib_ids, device):\n \"\"\"\n Mask-prune full_attention layer:\n - Zero out q_proj rows for dropped Q heads (both q-half and gate-half).\n - Optionally drop matching KV heads if a whole GQA group is dropped.\n - lstsq-refit o_proj cols for kept Q heads on `target_olin_input`\n (the original o_proj input that we want to reproduce at this layer).\n\n target_olin_input: [B, T, n_q * head_dim] — tensor on cpu fp32.\n \"\"\"\n cfg = text_config(model)\n n_q = cfg.num_attention_heads\n n_kv = cfg.num_key_value_heads\n head_dim = cfg.head_dim\n group_size = n_q // n_kv # 16/4 = 4\n\n keep_q = sorted(int(i) for i in keep_q_heads)\n drop_q = [i for i in range(n_q) if i not in set(keep_q)]\n log(f\" full L?: keep_q={keep_q}, drop_q={drop_q}\")\n\n sa = layer.self_attn\n\n # Zero q_proj rows for dropped heads. q_proj is shape (n_q*head_dim*2, hidden).\n # The 2× factor: each head gets [q_h | gate_h]. Need to zero both halves.\n # Layout per head h of size 2*head_dim: rows [h*2*head_dim : (h+1)*2*head_dim]\n qw = sa.q_proj.weight.data # (n_q*head_dim*2, hidden)\n for h in drop_q:\n qw[h * 2 * head_dim : (h + 1) * 2 * head_dim, :] = 0\n # Optional KV pruning: drop whole groups\n keep_kv_groups = sorted({h // group_size for h in keep_q})\n drop_kv_groups = [g for g in range(n_kv) if g not in set(keep_kv_groups)]\n if drop_kv_groups:\n log(f\" full L?: drop KV groups {drop_kv_groups}\")\n for g in drop_kv_groups:\n sa.k_proj.weight.data[g * head_dim : (g + 1) * head_dim, :] = 0\n sa.v_proj.weight.data[g * head_dim : (g + 1) * head_dim, :] = 0\n\n # lstsq refit on o_proj\n # o_proj: y = W_O @ x where x = [B, T, n_q * head_dim], y = [B, T, hidden].\n # We want to find W_O' such that W_O' @ x' ≈ y_target, where x' has dropped-head\n # cols zeroed (since we zeroed q_proj rows above, those head cols of x are now 0).\n # But target was captured pre-prune. The refit uses the ACTUAL post-prune x,\n # which we get by re-running the layer's forward up to o_proj.\n #\n # Approach: re-forward the model with the zeroed q_proj, hook o_proj's input\n # at this layer, then solve lstsq(x_actual, y_target) for new W_O.\n # We do this in the caller (one re-fwd per layer is wasteful; instead we\n # re-forward once at the start of phase 2 after each prune).\n # Here we just receive x_actual via the caller after re-fwd.\n return # caller does the lstsq with re-captured x\n\n\n@torch.no_grad()\ndef prune_linear_attention_inplace(layer, keep_v_heads, model):\n \"\"\"\n Mask-prune linear_attention (gated DeltaNet):\n - Zero rows of in_proj_qkv corresponding to dropped V heads (value portion only).\n - Zero rows of in_proj_z, in_proj_b, in_proj_a for dropped V heads.\n - Zero conv1d depthwise channels for the value rows.\n - Zero entries of A_log, dt_bias for dropped V heads.\n - K-head dropping enabled when whole groups are pruned.\n The lstsq refit of out_proj is done by the caller after re-forward.\n \"\"\"\n cfg = text_config(model)\n n_v = cfg.linear_num_value_heads\n n_k = cfg.linear_num_key_heads\n head_v = cfg.linear_value_head_dim\n head_k = cfg.linear_key_head_dim\n key_dim = n_k * head_k\n value_dim = n_v * head_v\n\n keep_v = sorted(int(i) for i in keep_v_heads)\n drop_v = [i for i in range(n_v) if i not in set(keep_v)]\n log(f\" lin L?: keep_v={keep_v[:8]}{'...' if len(keep_v)>8 else ''}, drop_v={drop_v}\")\n\n la = layer.linear_attn\n\n # in_proj_qkv: (2*key_dim + value_dim, hidden). Layout: [Q | K | V] along output.\n # Value section starts at 2*key_dim and goes to 2*key_dim + value_dim.\n iqkv = la.in_proj_qkv.weight.data\n v_off = 2 * key_dim\n for h in drop_v:\n iqkv[v_off + h * head_v : v_off + (h + 1) * head_v, :] = 0\n # in_proj_z: (value_dim, hidden) — same layout as the V section\n iz = la.in_proj_z.weight.data\n for h in drop_v:\n iz[h * head_v : (h + 1) * head_v, :] = 0\n # in_proj_b, in_proj_a: (n_v, hidden)\n for h in drop_v:\n la.in_proj_b.weight.data[h, :] = 0\n la.in_proj_a.weight.data[h, :] = 0\n # conv1d: depthwise, (conv_dim, 1, kernel). conv_dim = 2*key_dim + value_dim\n cw = la.conv1d.weight.data # (conv_dim, 1, kernel)\n for h in drop_v:\n cw[v_off + h * head_v : v_off + (h + 1) * head_v, :, :] = 0\n # A_log, dt_bias: (n_v,)\n for h in drop_v:\n la.A_log.data[h] = 0 # log(1) = 0 means decay rate 1 — but unused since v rows are zeroed\n la.dt_bias.data[h] = 0\n\n # K-head group pruning if whole group dropped\n v_per_k = n_v // n_k # 32/16 = 2\n # Each k-head h_k corresponds to v-heads [h_k*v_per_k, (h_k+1)*v_per_k]\n drop_k_groups = []\n for h_k in range(n_k):\n v_in_group = list(range(h_k * v_per_k, (h_k + 1) * v_per_k))\n if all(v in drop_v for v in v_in_group):\n drop_k_groups.append(h_k)\n if drop_k_groups:\n log(f\" lin L?: drop K groups {drop_k_groups}\")\n for h_k in drop_k_groups:\n # K section in iqkv: [key_dim : 2*key_dim], offset 0 is Q\n for kk in range(2): # Q and K share dims layout [Q | K]\n base = kk * key_dim\n iqkv[base + h_k * head_k : base + (h_k + 1) * head_k, :] = 0\n cw[base + h_k * head_k : base + (h_k + 1) * head_k, :, :] = 0\n return\n\n\n@torch.no_grad()\ndef lstsq_refit_o_proj(proj: nn.Linear, x_actual: torch.Tensor, y_target: torch.Tensor, ridge: float = 1e-3):\n \"\"\"\n Solve W_new @ x_actual ≈ y_target via ridge-regularized least squares,\n write into proj.weight (shape: [out_features, in_features]).\n\n x_actual: [B, T, in_features] (post-prune o_proj input)\n y_target: [B, T, out_features] (pre-prune o_proj output target = W_orig @ x_orig)\n\n Note: y_target was captured as o_proj INPUT pre-prune. But we want post-proj\n output identity. Two options:\n (a) reproduce the post-projection output: y_target_out = W_orig @ x_orig_input.\n x_actual is the new o_proj input (with zeroed-head dims). lstsq finds W\n such that W @ x_actual ≈ W_orig @ x_orig_input.\n (b) reproduce the pre-prune o_proj input by adjusting kept cols: solve\n x_actual @ M ≈ x_orig_input. Not what we want.\n We use (a). The caller passes y_target = W_orig @ x_orig_input, computed\n by running the original projection on the captured input.\n \"\"\"\n assert x_actual.dim() == 3 and y_target.dim() == 3\n in_f = proj.in_features\n out_f = proj.out_features\n X = x_actual.reshape(-1, in_f).to(torch.float32)\n Y = y_target.reshape(-1, out_f).to(torch.float32)\n # Solve W (out_f x in_f): minimize ||X W^T - Y||^2 + ridge * ||W||^2\n # Normal equations: (X^T X + ridge I) W^T = X^T Y\n XtX = X.t() @ X\n XtY = X.t() @ Y\n XtX += ridge * torch.eye(in_f, dtype=XtX.dtype, device=XtX.device)\n W_t = torch.linalg.solve(XtX, XtY) # (in_f, out_f)\n W_new = W_t.t().contiguous() # (out_f, in_f)\n # Sanity: residual norm\n Y_hat = X @ W_t\n resid = (Y_hat - Y).pow(2).mean().sqrt().item()\n target_norm = Y.pow(2).mean().sqrt().item()\n rel = resid / max(target_norm, 1e-9)\n log(f\" lstsq refit: rel_resid={rel:.4f}, target_rms={target_norm:.4f}\")\n proj.weight.data.copy_(W_new.to(proj.weight.dtype).to(proj.weight.device))\n\n\n# ---------- main ----------\n\n@torch.no_grad()\ndef capture_proj_input_at_layer(model, calib_ids, layer_idx: int) -> torch.Tensor:\n \"\"\"Forward the (possibly modified) model and capture the o_proj/out_proj input at one specific layer.\"\"\"\n layer = get_layers(model)[layer_idx]\n if is_full(layer):\n proj = layer.self_attn.o_proj\n elif is_linear(layer):\n proj = layer.linear_attn.out_proj\n else:\n raise RuntimeError(\"layer has no attention\")\n\n captured = {}\n def hook(m, inp):\n captured[\"x\"] = inp[0].detach().to(torch.float32).cpu()\n h = proj.register_forward_pre_hook(hook)\n try:\n model(calib_ids)\n finally:\n h.remove()\n return captured[\"x\"]\n\n\ndef main():\n ap = argparse.ArgumentParser()\n ap.add_argument(\"--model-path\", required=True)\n ap.add_argument(\"--output\", required=True)\n ap.add_argument(\"--prune-full-frac\", type=float, default=0.0,\n help=\"fraction of Q heads to drop in full_attention layers\")\n ap.add_argument(\"--prune-linear-frac\", type=float, default=0.0,\n help=\"fraction of V heads to drop in linear_attention layers\")\n ap.add_argument(\"--calib-tokens\", type=int, default=2048)\n ap.add_argument(\"--device\", default=\"cuda\")\n ap.add_argument(\"--ridge\", type=float, default=1e-3)\n ap.add_argument(\"--smoke\", action=\"store_true\",\n help=\"only prune layer 3 (full) and/or layer 0 (linear) — sanity check\")\n args = ap.parse_args()\n\n log(f\"loading {args.model_path}\")\n tok = AutoTokenizer.from_pretrained(args.model_path)\n model = AutoModelForCausalLM.from_pretrained(args.model_path, torch_dtype=torch.bfloat16)\n model = model.to(args.device)\n\n # Disable use_cache: hooks easier and we never call generate during pruning\n model.config.use_cache = False\n if hasattr(model.config, \"text_config\"):\n model.config.text_config.use_cache = False\n model.eval()\n layers = get_layers(model)\n log(f\"model loaded: {len(layers)} layers; cfg full_heads={text_config(model).num_attention_heads} lin_v_heads={text_config(model).linear_num_value_heads}\")\n\n calib_ids = build_calib(tok, args.calib_tokens, args.device)\n\n # Phase 0: capture original o_proj/out_proj INPUT per layer (cpu fp32)\n targets = phase0_capture(model, calib_ids)\n # Convert each target to o_proj OUTPUT (the real target for healing)\n # by running it through the ORIGINAL projection weights, in fp32.\n log(\"phase0: computing pre-prune o_proj OUTPUTs as healing targets\")\n target_outputs = {}\n for L, x in targets.items():\n layer = layers[L]\n proj = layer.self_attn.o_proj if is_full(layer) else layer.linear_attn.out_proj\n W = proj.weight.data.to(torch.float32).cpu() # (out, in)\n # x: [B, T, in], y = x @ W^T: [B, T, out]\n y = x @ W.t()\n target_outputs[L] = y\n log(f\"phase0: {len(target_outputs)} target outputs prepared\")\n\n # Phase 1: importance\n importance = phase1_importance(model, calib_ids)\n\n # Phase 2: per-layer prune + lstsq refit, in forward order\n cfg = text_config(model)\n n_q_full = cfg.num_attention_heads\n n_v_lin = cfg.linear_num_value_heads\n\n smoke_layers = set()\n if args.smoke:\n # Pick first full layer and first linear layer\n for L, layer in enumerate(layers):\n if is_full(layer) and not any(is_full(layers[Li]) for Li in smoke_layers):\n smoke_layers.add(L)\n if is_linear(layer) and not any(is_linear(layers[Li]) for Li in smoke_layers):\n smoke_layers.add(L)\n if len(smoke_layers) >= 2:\n break\n log(f\"SMOKE mode: only pruning layers {sorted(smoke_layers)}\")\n\n n_drop_full = int(round(n_q_full * args.prune_full_frac))\n n_drop_lin = int(round(n_v_lin * args.prune_linear_frac))\n log(f\"phase2: drop {n_drop_full}/{n_q_full} full Q-heads per layer, {n_drop_lin}/{n_v_lin} linear V-heads per layer\")\n\n pruned_layers = []\n for L, layer in enumerate(layers):\n if args.smoke and L not in smoke_layers:\n continue\n if is_full(layer) and n_drop_full > 0:\n imp = importance[L]\n keep_idx = imp.argsort(descending=True)[: n_q_full - n_drop_full].tolist()\n log(f\"layer {L} [full]: importance min={imp.min():.3e} max={imp.max():.3e} mean={imp.mean():.3e}\")\n prune_full_attention_inplace(layer, keep_idx, None, model, calib_ids, args.device)\n # Re-capture o_proj input post-prune at this layer, then lstsq refit\n x_actual = capture_proj_input_at_layer(model, calib_ids, L)\n y_target = target_outputs[L]\n lstsq_refit_o_proj(layer.self_attn.o_proj, x_actual, y_target, ridge=args.ridge)\n pruned_layers.append((L, \"full\"))\n elif is_linear(layer) and n_drop_lin > 0:\n imp = importance[L]\n keep_idx = imp.argsort(descending=True)[: n_v_lin - n_drop_lin].tolist()\n log(f\"layer {L} [lin]: importance min={imp.min():.3e} max={imp.max():.3e} mean={imp.mean():.3e}\")\n prune_linear_attention_inplace(layer, keep_idx, model)\n x_actual = capture_proj_input_at_layer(model, calib_ids, L)\n y_target = target_outputs[L]\n lstsq_refit_o_proj(layer.linear_attn.out_proj, x_actual, y_target, ridge=args.ridge)\n pruned_layers.append((L, \"lin\"))\n\n # Final CE check\n with torch.no_grad():\n out = model(calib_ids)\n logits = out.logits\n shift_logits = logits[:, :-1, :].contiguous().to(torch.float32)\n shift_labels = calib_ids[:, 1:].contiguous()\n ce_post = F.cross_entropy(shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1)).item()\n log(f\"final CE on calibration: {ce_post:.4f}\")\n\n # Save\n out_dir = Path(args.output)\n out_dir.mkdir(parents=True, exist_ok=True)\n log(f\"saving to {out_dir}\")\n model.save_pretrained(out_dir)\n tok.save_pretrained(out_dir)\n\n # Drop a manifest of what we did\n manifest = {\n \"source\": str(args.model_path),\n \"prune_full_frac\": args.prune_full_frac,\n \"prune_linear_frac\": args.prune_linear_frac,\n \"n_drop_full\": n_drop_full,\n \"n_drop_linear\": n_drop_lin,\n \"calib_tokens\": args.calib_tokens,\n \"ridge\": args.ridge,\n \"smoke\": args.smoke,\n \"pruned_layers\": pruned_layers,\n \"final_ce\": ce_post,\n }\n (out_dir / \"prune_manifest.json\").write_text(json.dumps(manifest, indent=2))\n log(\"done\")\n\n\nif __name__ == \"__main__\":\n main()\n"},"tool_response":{"_truncated":"{\"type\":\"create\",\"filePath\":\"/shared/dev/omnimergekit/recipes/microcoder_4b/prune_local_heal.py\",\"content\":\"#!/usr/bin/env python3\\n\\\"\\\"\\\"\\nprune_local_heal.py — Qwen3.5 hybrid-arch attention head pruning with\\n backprop-free local healing (W_O lstsq refit).\\n\\nTargets the Qwen3.5 hybrid architecture which mixes:\\n - full_attention layers (Qwen3_5Attention with gated query, GQA 16:4, head_dim=256)\\n - linear_attention layers (Qwen3_5GatedDeltaNet, 32 v_heads / 16 k_heads, head_dim=128)\\n\\nTwo prune levers:\\n --prune-full-frac : fraction of Q heads to drop in full_attention layers\\n --prune-linear-frac : fraction of V heads to drop in linear_attention layers\\n\\nPhases:\\n 0. Capture per-layer o_proj-input (full_attn) and out_proj-input (linear_attn)\\n on a calibration set — these are the targets for healing.\\n 1. Score per-head importance via Michel-style gradient on per-head scale α=1.\\n 2. For each layer, in forward order, on the *current* (already-modified) model:\\n a. Re-capture pre-proj input\\n b. Zero rows for dropped heads in q/k/v / in_proj_qkv (and corresponding\\n rows in z/b/a/conv1d/A_log/dt_bias for linear_attn)\\n c. lstsq refit the kept columns of o_proj / out_proj so the post-proj\\n output matches the original target\\n\\nThe model stays full-size on disk (mask-style); a separate physical-resize\\nstep converts to a smaller model after eval validation. Mask-pruning is what\\nlstsq refit operates on, and it preserves output quality identically — only\\ninference compute differs.\\n\\\"\\\"\\\"\\nfrom __future__ import annotations\\n\\nimport argparse\\nimport json\\nimport os\\nimport time\\nfrom pathlib import Path\\n\\nimport torch\\nimport torch.nn as nn\\nimport torch.nn.functional as F\\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\\n\\n\\ndef log(msg: str):\\n print(f\\\"[{time.strftime('%H:%M:%S')}] {msg}\\\", flush=True)\\n\\n\\n# ---------- calibration ----------\\n\\nCALIB_TEXTS = [\\n # Co"},"tool_use_id":"toolu_01HVaYqJUYe7SbK3zE8iSdue","cwd":"/srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith"}
|
| 6 |
+
{"timestamp":"2026-05-04T07:10:10.273Z","session_id":"4208dc59-08bc-4b1e-8d4e-8ec3407fd4ac","hook_event_name":"PostToolUse","tool_name":"Edit","tool_input":{"file_path":"/shared/dev/omnimergekit/recipes/microcoder_4b/prune_local_heal.py","old_string":"import argparse\nimport json\nimport os\nimport time","new_string":"import argparse\nimport json\nimport time","replace_all":false},"tool_response":{"_truncated":"{\"filePath\":\"/shared/dev/omnimergekit/recipes/microcoder_4b/prune_local_heal.py\",\"oldString\":\"import argparse\\nimport json\\nimport os\\nimport time\",\"newString\":\"import argparse\\nimport json\\nimport time\",\"originalFile\":\"#!/usr/bin/env python3\\n\\\"\\\"\\\"\\nprune_local_heal.py — Qwen3.5 hybrid-arch attention head pruning with\\n backprop-free local healing (W_O lstsq refit).\\n\\nTargets the Qwen3.5 hybrid architecture which mixes:\\n - full_attention layers (Qwen3_5Attention with gated query, GQA 16:4, head_dim=256)\\n - linear_attention layers (Qwen3_5GatedDeltaNet, 32 v_heads / 16 k_heads, head_dim=128)\\n\\nTwo prune levers:\\n --prune-full-frac : fraction of Q heads to drop in full_attention layers\\n --prune-linear-frac : fraction of V heads to drop in linear_attention layers\\n\\nPhases:\\n 0. Capture per-layer o_proj-input (full_attn) and out_proj-input (linear_attn)\\n on a calibration set — these are the targets for healing.\\n 1. Score per-head importance via Michel-style gradient on per-head scale α=1.\\n 2. For each layer, in forward order, on the *current* (already-modified) model:\\n a. Re-capture pre-proj input\\n b. Zero rows for dropped heads in q/k/v / in_proj_qkv (and corresponding\\n rows in z/b/a/conv1d/A_log/dt_bias for linear_attn)\\n c. lstsq refit the kept columns of o_proj / out_proj so the post-proj\\n output matches the original target\\n\\nThe model stays full-size on disk (mask-style); a separate physical-resize\\nstep converts to a smaller model after eval validation. Mask-pruning is what\\nlstsq refit operates on, and it preserves output quality identically — only\\ninference compute differs.\\n\\\"\\\"\\\"\\nfrom __future__ import annotations\\n\\nimport argparse\\nimport json\\nimport os\\nimport time\\nfrom pathlib import Path\\n\\nimport torch\\nimport torch.nn as nn\\nimport torch.nn.functional as F\\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\\n\\n\\ndef log(msg: str):\\n print(f\\\"[{time.s"},"tool_use_id":"toolu_01WasHPd6eJF6aCAPkUAhTSw","cwd":"/srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith"}
|
| 7 |
+
{"timestamp":"2026-05-04T07:10:15.622Z","session_id":"4208dc59-08bc-4b1e-8d4e-8ec3407fd4ac","hook_event_name":"PostToolUse","tool_name":"Edit","tool_input":{"file_path":"/shared/dev/omnimergekit/recipes/microcoder_4b/prune_local_heal.py","old_string":" key_dim = n_k * head_k\n value_dim = n_v * head_v\n\n keep_v = sorted(int(i) for i in keep_v_heads)","new_string":" key_dim = n_k * head_k\n\n keep_v = sorted(int(i) for i in keep_v_heads)","replace_all":false},"tool_response":{"_truncated":"{\"filePath\":\"/shared/dev/omnimergekit/recipes/microcoder_4b/prune_local_heal.py\",\"oldString\":\" key_dim = n_k * head_k\\n value_dim = n_v * head_v\\n\\n keep_v = sorted(int(i) for i in keep_v_heads)\",\"newString\":\" key_dim = n_k * head_k\\n\\n keep_v = sorted(int(i) for i in keep_v_heads)\",\"originalFile\":\"#!/usr/bin/env python3\\n\\\"\\\"\\\"\\nprune_local_heal.py — Qwen3.5 hybrid-arch attention head pruning with\\n backprop-free local healing (W_O lstsq refit).\\n\\nTargets the Qwen3.5 hybrid architecture which mixes:\\n - full_attention layers (Qwen3_5Attention with gated query, GQA 16:4, head_dim=256)\\n - linear_attention layers (Qwen3_5GatedDeltaNet, 32 v_heads / 16 k_heads, head_dim=128)\\n\\nTwo prune levers:\\n --prune-full-frac : fraction of Q heads to drop in full_attention layers\\n --prune-linear-frac : fraction of V heads to drop in linear_attention layers\\n\\nPhases:\\n 0. Capture per-layer o_proj-input (full_attn) and out_proj-input (linear_attn)\\n on a calibration set — these are the targets for healing.\\n 1. Score per-head importance via Michel-style gradient on per-head scale α=1.\\n 2. For each layer, in forward order, on the *current* (already-modified) model:\\n a. Re-capture pre-proj input\\n b. Zero rows for dropped heads in q/k/v / in_proj_qkv (and corresponding\\n rows in z/b/a/conv1d/A_log/dt_bias for linear_attn)\\n c. lstsq refit the kept columns of o_proj / out_proj so the post-proj\\n output matches the original target\\n\\nThe model stays full-size on disk (mask-style); a separate physical-resize\\nstep converts to a smaller model after eval validation. Mask-pruning is what\\nlstsq refit operates on, and it preserves output quality identically — only\\ninference compute differs.\\n\\\"\\\"\\\"\\nfrom __future__ import annotations\\n\\nimport argparse\\nimport json\\nimport time\\nfrom pathlib import Path\\n\\nimport torch\\nimport torch.nn as nn\\nimport torch.nn.functional as F\\nfrom transformers impor"},"tool_use_id":"toolu_01Eo49sKnQyNLedwvVR8Ygqk","cwd":"/srv/dev-disk-by-uuid-f8b1803e-334f-4f4b-af3b-f802bb6883c5/backup_models/4b_phase1/merged/v2i-jv-base-task-arith"}
|
.caliber/learning/state.json
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"sessionId": "4208dc59-08bc-4b1e-8d4e-8ec3407fd4ac",
|
| 3 |
+
"eventCount": 7,
|
| 4 |
+
"lastAnalysisTimestamp": null,
|
| 5 |
+
"lastAnalysisEventCount": 0
|
| 6 |
+
}
|
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- if tools %}
|
| 2 |
+
{{- '<|im_start|>system\n' }}
|
| 3 |
+
{%- if messages[0].role == 'system' %}
|
| 4 |
+
{{- messages[0].content + '\n\n' }}
|
| 5 |
+
{%- endif %}
|
| 6 |
+
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 7 |
+
{%- for tool in tools %}
|
| 8 |
+
{{- "\n" }}
|
| 9 |
+
{{- tool | tojson }}
|
| 10 |
+
{%- endfor %}
|
| 11 |
+
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 12 |
+
{%- else %}
|
| 13 |
+
{%- if messages[0].role == 'system' %}
|
| 14 |
+
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
| 15 |
+
{%- endif %}
|
| 16 |
+
{%- endif %}
|
| 17 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
| 18 |
+
{%- for message in messages[::-1] %}
|
| 19 |
+
{%- set index = (messages|length - 1) - loop.index0 %}
|
| 20 |
+
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
| 21 |
+
{%- set ns.multi_step_tool = false %}
|
| 22 |
+
{%- set ns.last_query_index = index %}
|
| 23 |
+
{%- endif %}
|
| 24 |
+
{%- endfor %}
|
| 25 |
+
{%- for message in messages %}
|
| 26 |
+
{%- if message.content is string %}
|
| 27 |
+
{%- set content = message.content %}
|
| 28 |
+
{%- else %}
|
| 29 |
+
{%- set content = '' %}
|
| 30 |
+
{%- endif %}
|
| 31 |
+
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
| 32 |
+
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 33 |
+
{%- elif message.role == "assistant" %}
|
| 34 |
+
{%- set reasoning_content = '' %}
|
| 35 |
+
{%- if message.reasoning_content is string %}
|
| 36 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 37 |
+
{%- else %}
|
| 38 |
+
{%- if '</think>' in content %}
|
| 39 |
+
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
| 40 |
+
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
| 41 |
+
{%- endif %}
|
| 42 |
+
{%- endif %}
|
| 43 |
+
{%- if loop.index0 > ns.last_query_index %}
|
| 44 |
+
{%- if loop.last or (not loop.last and reasoning_content) %}
|
| 45 |
+
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
| 46 |
+
{%- else %}
|
| 47 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 48 |
+
{%- endif %}
|
| 49 |
+
{%- else %}
|
| 50 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 51 |
+
{%- endif %}
|
| 52 |
+
{%- if message.tool_calls %}
|
| 53 |
+
{%- for tool_call in message.tool_calls %}
|
| 54 |
+
{%- if (loop.first and content) or (not loop.first) %}
|
| 55 |
+
{{- '\n' }}
|
| 56 |
+
{%- endif %}
|
| 57 |
+
{%- if tool_call.function %}
|
| 58 |
+
{%- set tool_call = tool_call.function %}
|
| 59 |
+
{%- endif %}
|
| 60 |
+
{{- '<tool_call>\n{"name": "' }}
|
| 61 |
+
{{- tool_call.name }}
|
| 62 |
+
{{- '", "arguments": ' }}
|
| 63 |
+
{%- if tool_call.arguments is string %}
|
| 64 |
+
{{- tool_call.arguments }}
|
| 65 |
+
{%- else %}
|
| 66 |
+
{{- tool_call.arguments | tojson }}
|
| 67 |
+
{%- endif %}
|
| 68 |
+
{{- '}\n</tool_call>' }}
|
| 69 |
+
{%- endfor %}
|
| 70 |
+
{%- endif %}
|
| 71 |
+
{{- '<|im_end|>\n' }}
|
| 72 |
+
{%- elif message.role == "tool" %}
|
| 73 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 74 |
+
{{- '<|im_start|>user' }}
|
| 75 |
+
{%- endif %}
|
| 76 |
+
{{- '\n<tool_response>\n' }}
|
| 77 |
+
{{- content }}
|
| 78 |
+
{{- '\n</tool_response>' }}
|
| 79 |
+
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
| 80 |
+
{{- '<|im_end|>\n' }}
|
| 81 |
+
{%- endif %}
|
| 82 |
+
{%- endif %}
|
| 83 |
+
{%- endfor %}
|
| 84 |
+
{%- if add_generation_prompt %}
|
| 85 |
+
{{- '<|im_start|>assistant
|
| 86 |
+
<think>
|
| 87 |
+
' }}
|
| 88 |
+
{%- endif %}
|
config.json
ADDED
|
@@ -0,0 +1,114 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Qwen3_5ForConditionalGeneration"
|
| 4 |
+
],
|
| 5 |
+
"torch_dtype": "bfloat16",
|
| 6 |
+
"eos_token_id": 248046,
|
| 7 |
+
"image_token_id": 248056,
|
| 8 |
+
"model_name": "unsloth/Qwen3.5-4B",
|
| 9 |
+
"model_type": "qwen3_5",
|
| 10 |
+
"pad_token_id": 248055,
|
| 11 |
+
"text_config": {
|
| 12 |
+
"attention_bias": false,
|
| 13 |
+
"attention_dropout": 0.0,
|
| 14 |
+
"attn_output_gate": true,
|
| 15 |
+
"bos_token_id": null,
|
| 16 |
+
"torch_dtype": "bfloat16",
|
| 17 |
+
"eos_token_id": 248044,
|
| 18 |
+
"full_attention_interval": 4,
|
| 19 |
+
"head_dim": 256,
|
| 20 |
+
"hidden_act": "silu",
|
| 21 |
+
"hidden_size": 2560,
|
| 22 |
+
"initializer_range": 0.02,
|
| 23 |
+
"intermediate_size": 9216,
|
| 24 |
+
"layer_types": [
|
| 25 |
+
"linear_attention",
|
| 26 |
+
"linear_attention",
|
| 27 |
+
"linear_attention",
|
| 28 |
+
"full_attention",
|
| 29 |
+
"linear_attention",
|
| 30 |
+
"linear_attention",
|
| 31 |
+
"linear_attention",
|
| 32 |
+
"full_attention",
|
| 33 |
+
"linear_attention",
|
| 34 |
+
"linear_attention",
|
| 35 |
+
"linear_attention",
|
| 36 |
+
"full_attention",
|
| 37 |
+
"linear_attention",
|
| 38 |
+
"linear_attention",
|
| 39 |
+
"linear_attention",
|
| 40 |
+
"full_attention",
|
| 41 |
+
"linear_attention",
|
| 42 |
+
"linear_attention",
|
| 43 |
+
"linear_attention",
|
| 44 |
+
"full_attention",
|
| 45 |
+
"linear_attention",
|
| 46 |
+
"linear_attention",
|
| 47 |
+
"linear_attention",
|
| 48 |
+
"full_attention",
|
| 49 |
+
"linear_attention",
|
| 50 |
+
"linear_attention",
|
| 51 |
+
"linear_attention",
|
| 52 |
+
"full_attention",
|
| 53 |
+
"linear_attention",
|
| 54 |
+
"linear_attention",
|
| 55 |
+
"linear_attention",
|
| 56 |
+
"full_attention"
|
| 57 |
+
],
|
| 58 |
+
"linear_conv_kernel_dim": 4,
|
| 59 |
+
"linear_key_head_dim": 128,
|
| 60 |
+
"linear_num_key_heads": 16,
|
| 61 |
+
"linear_num_value_heads": 32,
|
| 62 |
+
"linear_value_head_dim": 128,
|
| 63 |
+
"mamba_ssm_dtype": "float32",
|
| 64 |
+
"max_position_embeddings": 262144,
|
| 65 |
+
"mlp_only_layers": [],
|
| 66 |
+
"model_type": "qwen3_5_text",
|
| 67 |
+
"mtp_num_hidden_layers": 1,
|
| 68 |
+
"mtp_use_dedicated_embeddings": false,
|
| 69 |
+
"num_attention_heads": 16,
|
| 70 |
+
"num_hidden_layers": 32,
|
| 71 |
+
"num_key_value_heads": 4,
|
| 72 |
+
"pad_token_id": null,
|
| 73 |
+
"partial_rotary_factor": 0.25,
|
| 74 |
+
"rms_norm_eps": 1e-06,
|
| 75 |
+
"rope_parameters": {
|
| 76 |
+
"mrope_interleaved": true,
|
| 77 |
+
"mrope_section": [
|
| 78 |
+
11,
|
| 79 |
+
11,
|
| 80 |
+
10
|
| 81 |
+
],
|
| 82 |
+
"partial_rotary_factor": 0.25,
|
| 83 |
+
"rope_theta": 10000000,
|
| 84 |
+
"rope_type": "default"
|
| 85 |
+
},
|
| 86 |
+
"tie_word_embeddings": true,
|
| 87 |
+
"use_cache": true,
|
| 88 |
+
"vocab_size": 248320
|
| 89 |
+
},
|
| 90 |
+
"tie_word_embeddings": true,
|
| 91 |
+
"unsloth_fixed": true,
|
| 92 |
+
"unsloth_version": "2026.3.4",
|
| 93 |
+
"use_cache": false,
|
| 94 |
+
"video_token_id": 248057,
|
| 95 |
+
"vision_config": {
|
| 96 |
+
"deepstack_visual_indexes": [],
|
| 97 |
+
"depth": 24,
|
| 98 |
+
"torch_dtype": "bfloat16",
|
| 99 |
+
"hidden_act": "gelu_pytorch_tanh",
|
| 100 |
+
"hidden_size": 1024,
|
| 101 |
+
"in_channels": 3,
|
| 102 |
+
"initializer_range": 0.02,
|
| 103 |
+
"intermediate_size": 4096,
|
| 104 |
+
"model_type": "qwen3_5",
|
| 105 |
+
"num_heads": 16,
|
| 106 |
+
"num_position_embeddings": 2304,
|
| 107 |
+
"out_hidden_size": 2560,
|
| 108 |
+
"patch_size": 16,
|
| 109 |
+
"spatial_merge_size": 2,
|
| 110 |
+
"temporal_patch_size": 2
|
| 111 |
+
},
|
| 112 |
+
"vision_end_token_id": 248054,
|
| 113 |
+
"vision_start_token_id": 248053
|
| 114 |
+
}
|
model-00001-of-00002.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:45d2fec62c40a19c4b439671b0abaa55cec317bd39e6ee32760507a77e041d61
|
| 3 |
+
size 5376584768
|
model-00002-of-00002.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:15de8b5eb052e291abb1fdf903642224fc9270417b0ee18025e4a902a150cb62
|
| 3 |
+
size 3943243384
|
model.safetensors.index.json
ADDED
|
@@ -0,0 +1,745 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"metadata": {
|
| 3 |
+
"total_size": 9319828152
|
| 4 |
+
},
|
| 5 |
+
"weight_map": {
|
| 6 |
+
"model.language_model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
| 7 |
+
"model.language_model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 8 |
+
"model.language_model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 9 |
+
"model.language_model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 10 |
+
"model.language_model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 11 |
+
"model.language_model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 12 |
+
"model.language_model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 13 |
+
"model.language_model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 14 |
+
"model.language_model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 15 |
+
"model.language_model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 16 |
+
"model.language_model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 17 |
+
"model.language_model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 18 |
+
"model.language_model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 19 |
+
"model.language_model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 20 |
+
"model.language_model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 21 |
+
"model.language_model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 22 |
+
"model.language_model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 23 |
+
"model.language_model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 24 |
+
"model.language_model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 25 |
+
"model.language_model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 26 |
+
"model.language_model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 27 |
+
"model.language_model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 28 |
+
"model.language_model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 29 |
+
"model.language_model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 30 |
+
"model.language_model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 31 |
+
"model.language_model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 32 |
+
"model.language_model.layers.21.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 33 |
+
"model.language_model.layers.21.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 34 |
+
"model.language_model.layers.21.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 35 |
+
"model.language_model.layers.22.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 36 |
+
"model.language_model.layers.22.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 37 |
+
"model.language_model.layers.22.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 38 |
+
"model.language_model.layers.23.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 39 |
+
"model.language_model.layers.23.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 40 |
+
"model.language_model.layers.23.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 41 |
+
"model.language_model.layers.31.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 42 |
+
"model.language_model.layers.31.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 43 |
+
"model.language_model.layers.31.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 44 |
+
"model.language_model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 45 |
+
"model.language_model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 46 |
+
"model.language_model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 47 |
+
"model.language_model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 48 |
+
"model.language_model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 49 |
+
"model.language_model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 50 |
+
"model.language_model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 51 |
+
"model.language_model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 52 |
+
"model.language_model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 53 |
+
"model.language_model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 54 |
+
"model.language_model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 55 |
+
"model.language_model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 56 |
+
"mtp.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 57 |
+
"mtp.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 58 |
+
"mtp.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 59 |
+
"model.language_model.layers.29.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 60 |
+
"model.language_model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 61 |
+
"model.language_model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 62 |
+
"model.language_model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 63 |
+
"model.language_model.layers.30.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 64 |
+
"model.language_model.layers.30.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 65 |
+
"model.language_model.layers.30.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 66 |
+
"model.language_model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 67 |
+
"model.language_model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 68 |
+
"model.language_model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 69 |
+
"model.language_model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 70 |
+
"model.language_model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 71 |
+
"model.language_model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 72 |
+
"model.language_model.layers.24.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 73 |
+
"model.language_model.layers.24.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 74 |
+
"model.language_model.layers.24.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 75 |
+
"model.language_model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 76 |
+
"model.language_model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 77 |
+
"model.language_model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 78 |
+
"model.language_model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 79 |
+
"model.language_model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 80 |
+
"model.language_model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 81 |
+
"model.language_model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 82 |
+
"model.language_model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 83 |
+
"model.language_model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 84 |
+
"model.language_model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 85 |
+
"model.language_model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 86 |
+
"model.language_model.layers.20.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 87 |
+
"model.language_model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 88 |
+
"model.language_model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 89 |
+
"model.language_model.layers.25.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 90 |
+
"model.language_model.layers.25.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 91 |
+
"model.language_model.layers.25.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
| 92 |
+
"model.language_model.layers.26.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
| 93 |
+
"model.language_model.layers.26.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
| 94 |
+
"model.language_model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
| 95 |
+
"model.language_model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
| 96 |
+
"model.language_model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
| 97 |
+
"model.language_model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
| 98 |
+
"model.language_model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
| 99 |
+
"model.language_model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
| 100 |
+
"model.language_model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
| 101 |
+
"model.language_model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
| 102 |
+
"model.language_model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
| 103 |
+
"model.language_model.layers.0.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
| 104 |
+
"model.language_model.layers.0.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
| 105 |
+
"model.language_model.layers.0.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
| 106 |
+
"model.language_model.layers.10.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 107 |
+
"model.language_model.layers.11.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 108 |
+
"model.language_model.layers.18.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 109 |
+
"model.language_model.layers.19.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 110 |
+
"model.language_model.layers.2.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 111 |
+
"model.language_model.layers.12.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 112 |
+
"model.language_model.layers.13.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 113 |
+
"model.language_model.layers.7.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 114 |
+
"model.language_model.layers.8.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 115 |
+
"model.language_model.layers.22.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 116 |
+
"model.language_model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 117 |
+
"model.language_model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 118 |
+
"model.language_model.layers.4.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 119 |
+
"model.language_model.layers.14.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 120 |
+
"model.language_model.layers.15.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 121 |
+
"model.language_model.layers.9.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 122 |
+
"mtp.layers.0.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 123 |
+
"model.language_model.layers.3.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 124 |
+
"model.language_model.layers.30.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 125 |
+
"model.language_model.layers.16.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 126 |
+
"model.language_model.layers.17.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 127 |
+
"model.language_model.layers.24.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 128 |
+
"model.language_model.layers.25.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 129 |
+
"model.language_model.layers.5.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 130 |
+
"model.language_model.layers.6.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 131 |
+
"model.language_model.layers.20.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 132 |
+
"model.language_model.layers.21.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 133 |
+
"model.language_model.layers.26.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 134 |
+
"model.language_model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
| 135 |
+
"model.language_model.layers.28.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 136 |
+
"model.language_model.layers.29.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 137 |
+
"model.language_model.layers.0.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 138 |
+
"model.language_model.layers.1.linear_attn.in_proj_qkv.weight": "model-00002-of-00002.safetensors",
|
| 139 |
+
"model.visual.merger.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 140 |
+
"mtp.fc.weight": "model-00002-of-00002.safetensors",
|
| 141 |
+
"model.language_model.layers.10.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 142 |
+
"model.language_model.layers.10.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 143 |
+
"model.language_model.layers.11.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 144 |
+
"model.language_model.layers.18.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 145 |
+
"model.language_model.layers.18.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 146 |
+
"model.language_model.layers.19.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 147 |
+
"model.language_model.layers.2.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 148 |
+
"model.language_model.layers.12.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 149 |
+
"model.language_model.layers.12.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 150 |
+
"model.language_model.layers.13.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 151 |
+
"model.language_model.layers.13.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 152 |
+
"model.language_model.layers.7.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 153 |
+
"model.language_model.layers.8.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 154 |
+
"model.language_model.layers.8.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 155 |
+
"model.language_model.layers.21.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 156 |
+
"model.language_model.layers.22.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 157 |
+
"model.language_model.layers.22.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 158 |
+
"model.language_model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 159 |
+
"model.language_model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 160 |
+
"model.language_model.layers.4.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 161 |
+
"model.language_model.layers.4.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 162 |
+
"model.language_model.layers.14.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 163 |
+
"model.language_model.layers.14.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 164 |
+
"model.language_model.layers.15.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 165 |
+
"model.language_model.layers.9.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 166 |
+
"model.language_model.layers.9.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 167 |
+
"mtp.layers.0.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 168 |
+
"model.language_model.layers.3.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 169 |
+
"model.language_model.layers.30.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 170 |
+
"model.language_model.layers.30.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 171 |
+
"model.language_model.layers.16.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 172 |
+
"model.language_model.layers.16.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 173 |
+
"model.language_model.layers.17.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 174 |
+
"model.language_model.layers.17.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 175 |
+
"model.language_model.layers.24.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 176 |
+
"model.language_model.layers.24.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 177 |
+
"model.language_model.layers.25.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 178 |
+
"model.language_model.layers.25.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 179 |
+
"model.language_model.layers.5.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 180 |
+
"model.language_model.layers.5.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 181 |
+
"model.language_model.layers.6.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 182 |
+
"model.language_model.layers.6.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 183 |
+
"model.language_model.layers.2.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 184 |
+
"model.language_model.layers.20.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 185 |
+
"model.language_model.layers.20.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 186 |
+
"model.language_model.layers.21.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 187 |
+
"model.language_model.layers.26.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 188 |
+
"model.language_model.layers.26.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 189 |
+
"model.language_model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
| 190 |
+
"model.language_model.layers.28.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 191 |
+
"model.language_model.layers.28.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 192 |
+
"model.language_model.layers.29.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 193 |
+
"model.language_model.layers.29.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 194 |
+
"model.language_model.layers.0.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 195 |
+
"model.language_model.layers.0.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 196 |
+
"model.language_model.layers.1.linear_attn.in_proj_z.weight": "model-00002-of-00002.safetensors",
|
| 197 |
+
"model.language_model.layers.1.linear_attn.out_proj.weight": "model-00002-of-00002.safetensors",
|
| 198 |
+
"model.visual.merger.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 199 |
+
"model.visual.blocks.0.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 200 |
+
"model.visual.blocks.0.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 201 |
+
"model.visual.blocks.1.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 202 |
+
"model.visual.blocks.1.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 203 |
+
"model.visual.blocks.10.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 204 |
+
"model.visual.blocks.10.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 205 |
+
"model.visual.blocks.11.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 206 |
+
"model.visual.blocks.11.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 207 |
+
"model.visual.blocks.12.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 208 |
+
"model.visual.blocks.12.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 209 |
+
"model.visual.blocks.13.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 210 |
+
"model.visual.blocks.13.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 211 |
+
"model.visual.blocks.14.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 212 |
+
"model.visual.blocks.14.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 213 |
+
"model.visual.blocks.15.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 214 |
+
"model.visual.blocks.15.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 215 |
+
"model.visual.blocks.16.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 216 |
+
"model.visual.blocks.16.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 217 |
+
"model.visual.blocks.17.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 218 |
+
"model.visual.blocks.17.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 219 |
+
"model.visual.blocks.18.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 220 |
+
"model.visual.blocks.18.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 221 |
+
"model.visual.blocks.19.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 222 |
+
"model.visual.blocks.19.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 223 |
+
"model.visual.blocks.2.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 224 |
+
"model.visual.blocks.2.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 225 |
+
"model.visual.blocks.20.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 226 |
+
"model.visual.blocks.20.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 227 |
+
"model.visual.blocks.21.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 228 |
+
"model.visual.blocks.21.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 229 |
+
"model.visual.blocks.22.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 230 |
+
"model.visual.blocks.22.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 231 |
+
"model.visual.blocks.23.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 232 |
+
"model.visual.blocks.23.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 233 |
+
"model.visual.blocks.3.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 234 |
+
"model.visual.blocks.3.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 235 |
+
"model.visual.blocks.4.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 236 |
+
"model.visual.blocks.4.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 237 |
+
"model.visual.blocks.5.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 238 |
+
"model.visual.blocks.5.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 239 |
+
"model.visual.blocks.6.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 240 |
+
"model.visual.blocks.6.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 241 |
+
"model.visual.blocks.7.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 242 |
+
"model.visual.blocks.7.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 243 |
+
"model.visual.blocks.8.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 244 |
+
"model.visual.blocks.8.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 245 |
+
"model.visual.blocks.9.mlp.linear_fc1.weight": "model-00002-of-00002.safetensors",
|
| 246 |
+
"model.visual.blocks.9.mlp.linear_fc2.weight": "model-00002-of-00002.safetensors",
|
| 247 |
+
"model.visual.blocks.0.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 248 |
+
"model.visual.blocks.1.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 249 |
+
"model.visual.blocks.10.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 250 |
+
"model.visual.blocks.11.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 251 |
+
"model.visual.blocks.12.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 252 |
+
"model.visual.blocks.13.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 253 |
+
"model.visual.blocks.14.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 254 |
+
"model.visual.blocks.15.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 255 |
+
"model.visual.blocks.16.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 256 |
+
"model.visual.blocks.17.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 257 |
+
"model.visual.blocks.18.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 258 |
+
"model.visual.blocks.19.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 259 |
+
"model.visual.blocks.2.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 260 |
+
"model.visual.blocks.20.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 261 |
+
"model.visual.blocks.21.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 262 |
+
"model.visual.blocks.22.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 263 |
+
"model.visual.blocks.23.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 264 |
+
"model.visual.blocks.3.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 265 |
+
"model.visual.blocks.4.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 266 |
+
"model.visual.blocks.5.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 267 |
+
"model.visual.blocks.6.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 268 |
+
"model.visual.blocks.7.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 269 |
+
"model.visual.blocks.8.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 270 |
+
"model.visual.blocks.9.attn.qkv.weight": "model-00002-of-00002.safetensors",
|
| 271 |
+
"model.language_model.layers.11.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 272 |
+
"model.language_model.layers.11.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 273 |
+
"model.language_model.layers.19.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 274 |
+
"model.language_model.layers.19.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 275 |
+
"model.language_model.layers.7.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 276 |
+
"model.language_model.layers.7.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 277 |
+
"model.language_model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 278 |
+
"model.language_model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 279 |
+
"model.language_model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 280 |
+
"model.language_model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 281 |
+
"model.language_model.layers.15.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 282 |
+
"model.language_model.layers.15.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 283 |
+
"mtp.layers.0.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 284 |
+
"mtp.layers.0.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 285 |
+
"model.language_model.layers.3.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 286 |
+
"model.language_model.layers.3.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 287 |
+
"model.language_model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
| 288 |
+
"model.language_model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
| 289 |
+
"model.visual.pos_embed.weight": "model-00002-of-00002.safetensors",
|
| 290 |
+
"model.visual.patch_embed.proj.weight": "model-00002-of-00002.safetensors",
|
| 291 |
+
"model.visual.blocks.0.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 292 |
+
"model.visual.blocks.1.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 293 |
+
"model.visual.blocks.10.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 294 |
+
"model.visual.blocks.11.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 295 |
+
"model.visual.blocks.12.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 296 |
+
"model.visual.blocks.13.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 297 |
+
"model.visual.blocks.14.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 298 |
+
"model.visual.blocks.15.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 299 |
+
"model.visual.blocks.16.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 300 |
+
"model.visual.blocks.17.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 301 |
+
"model.visual.blocks.18.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 302 |
+
"model.visual.blocks.19.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 303 |
+
"model.visual.blocks.2.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 304 |
+
"model.visual.blocks.20.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 305 |
+
"model.visual.blocks.21.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 306 |
+
"model.visual.blocks.22.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 307 |
+
"model.visual.blocks.23.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 308 |
+
"model.visual.blocks.3.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 309 |
+
"model.visual.blocks.4.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 310 |
+
"model.visual.blocks.5.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 311 |
+
"model.visual.blocks.6.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 312 |
+
"model.visual.blocks.7.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 313 |
+
"model.visual.blocks.8.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 314 |
+
"model.visual.blocks.9.attn.proj.weight": "model-00002-of-00002.safetensors",
|
| 315 |
+
"model.language_model.layers.10.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 316 |
+
"model.language_model.layers.10.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 317 |
+
"model.language_model.layers.18.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 318 |
+
"model.language_model.layers.18.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 319 |
+
"model.language_model.layers.2.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 320 |
+
"model.language_model.layers.2.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 321 |
+
"model.language_model.layers.12.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 322 |
+
"model.language_model.layers.12.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 323 |
+
"model.language_model.layers.13.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 324 |
+
"model.language_model.layers.13.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 325 |
+
"model.language_model.layers.8.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 326 |
+
"model.language_model.layers.8.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 327 |
+
"model.language_model.layers.22.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 328 |
+
"model.language_model.layers.22.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 329 |
+
"model.language_model.layers.4.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 330 |
+
"model.language_model.layers.4.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 331 |
+
"model.language_model.layers.14.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 332 |
+
"model.language_model.layers.14.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 333 |
+
"model.language_model.layers.9.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 334 |
+
"model.language_model.layers.9.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 335 |
+
"model.language_model.layers.30.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 336 |
+
"model.language_model.layers.30.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 337 |
+
"model.language_model.layers.16.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 338 |
+
"model.language_model.layers.16.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 339 |
+
"model.language_model.layers.17.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 340 |
+
"model.language_model.layers.17.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 341 |
+
"model.language_model.layers.24.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 342 |
+
"model.language_model.layers.24.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 343 |
+
"model.language_model.layers.25.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 344 |
+
"model.language_model.layers.25.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 345 |
+
"model.language_model.layers.5.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 346 |
+
"model.language_model.layers.5.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 347 |
+
"model.language_model.layers.6.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 348 |
+
"model.language_model.layers.6.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 349 |
+
"model.language_model.layers.20.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 350 |
+
"model.language_model.layers.20.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 351 |
+
"model.language_model.layers.21.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 352 |
+
"model.language_model.layers.21.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 353 |
+
"model.language_model.layers.26.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 354 |
+
"model.language_model.layers.26.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 355 |
+
"model.language_model.layers.28.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 356 |
+
"model.language_model.layers.28.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 357 |
+
"model.language_model.layers.29.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 358 |
+
"model.language_model.layers.29.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 359 |
+
"model.language_model.layers.0.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 360 |
+
"model.language_model.layers.0.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 361 |
+
"model.language_model.layers.1.linear_attn.in_proj_b.weight": "model-00002-of-00002.safetensors",
|
| 362 |
+
"model.language_model.layers.1.linear_attn.in_proj_a.weight": "model-00002-of-00002.safetensors",
|
| 363 |
+
"model.language_model.layers.10.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 364 |
+
"model.language_model.layers.2.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 365 |
+
"model.language_model.layers.12.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 366 |
+
"model.language_model.layers.13.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 367 |
+
"model.language_model.layers.8.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 368 |
+
"model.language_model.layers.9.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 369 |
+
"model.language_model.layers.22.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 370 |
+
"model.language_model.layers.4.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 371 |
+
"model.language_model.layers.5.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 372 |
+
"model.language_model.layers.14.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 373 |
+
"model.language_model.layers.16.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 374 |
+
"model.language_model.layers.30.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 375 |
+
"model.language_model.layers.17.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 376 |
+
"model.language_model.layers.18.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 377 |
+
"model.language_model.layers.24.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 378 |
+
"model.language_model.layers.25.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 379 |
+
"model.language_model.layers.6.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 380 |
+
"model.language_model.layers.20.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 381 |
+
"model.language_model.layers.21.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 382 |
+
"model.language_model.layers.26.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 383 |
+
"model.language_model.layers.28.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 384 |
+
"model.language_model.layers.29.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 385 |
+
"model.language_model.layers.0.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 386 |
+
"model.language_model.layers.1.linear_attn.conv1d.weight": "model-00002-of-00002.safetensors",
|
| 387 |
+
"model.visual.blocks.0.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 388 |
+
"model.visual.blocks.1.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 389 |
+
"model.visual.blocks.10.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 390 |
+
"model.visual.blocks.11.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 391 |
+
"model.visual.blocks.12.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 392 |
+
"model.visual.blocks.13.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 393 |
+
"model.visual.blocks.14.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 394 |
+
"model.visual.blocks.15.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 395 |
+
"model.visual.blocks.16.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 396 |
+
"model.visual.blocks.17.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 397 |
+
"model.visual.blocks.18.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 398 |
+
"model.visual.blocks.19.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 399 |
+
"model.visual.blocks.2.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 400 |
+
"model.visual.blocks.20.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 401 |
+
"model.visual.blocks.21.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 402 |
+
"model.visual.blocks.22.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 403 |
+
"model.visual.blocks.23.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 404 |
+
"model.visual.blocks.3.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 405 |
+
"model.visual.blocks.4.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 406 |
+
"model.visual.blocks.5.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 407 |
+
"model.visual.blocks.6.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 408 |
+
"model.visual.blocks.7.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 409 |
+
"model.visual.blocks.8.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 410 |
+
"model.visual.blocks.9.mlp.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 411 |
+
"model.visual.merger.linear_fc1.bias": "model-00002-of-00002.safetensors",
|
| 412 |
+
"model.visual.blocks.0.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 413 |
+
"model.visual.blocks.1.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 414 |
+
"model.visual.blocks.10.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 415 |
+
"model.visual.blocks.11.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 416 |
+
"model.visual.blocks.12.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 417 |
+
"model.visual.blocks.13.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 418 |
+
"model.visual.blocks.14.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 419 |
+
"model.visual.blocks.15.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 420 |
+
"model.visual.blocks.16.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 421 |
+
"model.visual.blocks.17.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 422 |
+
"model.visual.blocks.18.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 423 |
+
"model.visual.blocks.19.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 424 |
+
"model.visual.blocks.2.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 425 |
+
"model.visual.blocks.20.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 426 |
+
"model.visual.blocks.21.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 427 |
+
"model.visual.blocks.22.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 428 |
+
"model.visual.blocks.23.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 429 |
+
"model.visual.blocks.3.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 430 |
+
"model.visual.blocks.4.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 431 |
+
"model.visual.blocks.5.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 432 |
+
"model.visual.blocks.6.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 433 |
+
"model.visual.blocks.7.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 434 |
+
"model.visual.blocks.8.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 435 |
+
"model.visual.blocks.9.attn.qkv.bias": "model-00002-of-00002.safetensors",
|
| 436 |
+
"model.language_model.layers.10.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 437 |
+
"model.language_model.layers.10.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 438 |
+
"model.language_model.layers.11.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 439 |
+
"model.language_model.layers.11.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 440 |
+
"model.language_model.layers.18.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 441 |
+
"model.language_model.layers.19.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 442 |
+
"model.language_model.layers.19.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 443 |
+
"model.language_model.layers.2.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 444 |
+
"model.language_model.layers.12.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 445 |
+
"model.language_model.layers.12.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 446 |
+
"model.language_model.layers.13.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 447 |
+
"model.language_model.layers.13.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 448 |
+
"model.language_model.layers.7.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 449 |
+
"model.language_model.layers.8.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 450 |
+
"model.language_model.layers.8.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 451 |
+
"model.language_model.layers.9.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 452 |
+
"model.language_model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 453 |
+
"model.language_model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 454 |
+
"model.language_model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 455 |
+
"model.language_model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 456 |
+
"model.language_model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 457 |
+
"model.language_model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 458 |
+
"model.language_model.layers.4.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 459 |
+
"model.language_model.layers.4.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 460 |
+
"model.language_model.layers.14.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 461 |
+
"model.language_model.layers.14.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 462 |
+
"model.language_model.layers.15.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 463 |
+
"model.language_model.layers.15.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 464 |
+
"model.language_model.layers.16.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 465 |
+
"model.language_model.layers.9.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 466 |
+
"mtp.layers.0.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 467 |
+
"mtp.layers.0.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 468 |
+
"mtp.norm.weight": "model-00002-of-00002.safetensors",
|
| 469 |
+
"mtp.pre_fc_norm_embedding.weight": "model-00002-of-00002.safetensors",
|
| 470 |
+
"mtp.pre_fc_norm_hidden.weight": "model-00002-of-00002.safetensors",
|
| 471 |
+
"model.language_model.layers.3.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 472 |
+
"model.language_model.layers.3.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 473 |
+
"model.language_model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 474 |
+
"model.language_model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 475 |
+
"model.language_model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 476 |
+
"model.language_model.layers.16.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 477 |
+
"model.language_model.layers.17.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 478 |
+
"model.language_model.layers.17.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 479 |
+
"model.language_model.layers.18.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 480 |
+
"model.language_model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 481 |
+
"model.language_model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 482 |
+
"model.language_model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 483 |
+
"model.language_model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 484 |
+
"model.language_model.layers.5.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 485 |
+
"model.language_model.layers.5.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 486 |
+
"model.language_model.layers.6.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 487 |
+
"model.language_model.layers.6.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 488 |
+
"model.language_model.layers.7.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 489 |
+
"model.language_model.layers.2.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 490 |
+
"model.language_model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 491 |
+
"model.language_model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 492 |
+
"model.language_model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 493 |
+
"model.language_model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 494 |
+
"model.language_model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 495 |
+
"model.language_model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 496 |
+
"model.language_model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 497 |
+
"model.language_model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 498 |
+
"model.language_model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 499 |
+
"model.language_model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 500 |
+
"model.language_model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 501 |
+
"model.language_model.layers.0.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 502 |
+
"model.language_model.layers.0.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 503 |
+
"model.language_model.layers.1.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 504 |
+
"model.language_model.layers.1.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
| 505 |
+
"model.language_model.norm.weight": "model-00002-of-00002.safetensors",
|
| 506 |
+
"model.visual.merger.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 507 |
+
"model.visual.blocks.0.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 508 |
+
"model.visual.blocks.0.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 509 |
+
"model.visual.blocks.0.norm1.bias": "model-00002-of-00002.safetensors",
|
| 510 |
+
"model.visual.blocks.0.norm1.weight": "model-00002-of-00002.safetensors",
|
| 511 |
+
"model.visual.blocks.0.norm2.bias": "model-00002-of-00002.safetensors",
|
| 512 |
+
"model.visual.blocks.0.norm2.weight": "model-00002-of-00002.safetensors",
|
| 513 |
+
"model.visual.blocks.1.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 514 |
+
"model.visual.blocks.1.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 515 |
+
"model.visual.blocks.1.norm1.bias": "model-00002-of-00002.safetensors",
|
| 516 |
+
"model.visual.blocks.1.norm1.weight": "model-00002-of-00002.safetensors",
|
| 517 |
+
"model.visual.blocks.1.norm2.bias": "model-00002-of-00002.safetensors",
|
| 518 |
+
"model.visual.blocks.1.norm2.weight": "model-00002-of-00002.safetensors",
|
| 519 |
+
"model.visual.blocks.10.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 520 |
+
"model.visual.blocks.10.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 521 |
+
"model.visual.blocks.10.norm1.bias": "model-00002-of-00002.safetensors",
|
| 522 |
+
"model.visual.blocks.10.norm1.weight": "model-00002-of-00002.safetensors",
|
| 523 |
+
"model.visual.blocks.10.norm2.bias": "model-00002-of-00002.safetensors",
|
| 524 |
+
"model.visual.blocks.10.norm2.weight": "model-00002-of-00002.safetensors",
|
| 525 |
+
"model.visual.blocks.11.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 526 |
+
"model.visual.blocks.11.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 527 |
+
"model.visual.blocks.11.norm1.bias": "model-00002-of-00002.safetensors",
|
| 528 |
+
"model.visual.blocks.11.norm1.weight": "model-00002-of-00002.safetensors",
|
| 529 |
+
"model.visual.blocks.11.norm2.bias": "model-00002-of-00002.safetensors",
|
| 530 |
+
"model.visual.blocks.11.norm2.weight": "model-00002-of-00002.safetensors",
|
| 531 |
+
"model.visual.blocks.12.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 532 |
+
"model.visual.blocks.12.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 533 |
+
"model.visual.blocks.12.norm1.bias": "model-00002-of-00002.safetensors",
|
| 534 |
+
"model.visual.blocks.12.norm1.weight": "model-00002-of-00002.safetensors",
|
| 535 |
+
"model.visual.blocks.12.norm2.bias": "model-00002-of-00002.safetensors",
|
| 536 |
+
"model.visual.blocks.12.norm2.weight": "model-00002-of-00002.safetensors",
|
| 537 |
+
"model.visual.blocks.13.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 538 |
+
"model.visual.blocks.13.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 539 |
+
"model.visual.blocks.13.norm1.bias": "model-00002-of-00002.safetensors",
|
| 540 |
+
"model.visual.blocks.13.norm1.weight": "model-00002-of-00002.safetensors",
|
| 541 |
+
"model.visual.blocks.13.norm2.bias": "model-00002-of-00002.safetensors",
|
| 542 |
+
"model.visual.blocks.13.norm2.weight": "model-00002-of-00002.safetensors",
|
| 543 |
+
"model.visual.blocks.14.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 544 |
+
"model.visual.blocks.14.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 545 |
+
"model.visual.blocks.14.norm1.bias": "model-00002-of-00002.safetensors",
|
| 546 |
+
"model.visual.blocks.14.norm1.weight": "model-00002-of-00002.safetensors",
|
| 547 |
+
"model.visual.blocks.14.norm2.bias": "model-00002-of-00002.safetensors",
|
| 548 |
+
"model.visual.blocks.14.norm2.weight": "model-00002-of-00002.safetensors",
|
| 549 |
+
"model.visual.blocks.15.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 550 |
+
"model.visual.blocks.15.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 551 |
+
"model.visual.blocks.15.norm1.bias": "model-00002-of-00002.safetensors",
|
| 552 |
+
"model.visual.blocks.15.norm1.weight": "model-00002-of-00002.safetensors",
|
| 553 |
+
"model.visual.blocks.15.norm2.bias": "model-00002-of-00002.safetensors",
|
| 554 |
+
"model.visual.blocks.15.norm2.weight": "model-00002-of-00002.safetensors",
|
| 555 |
+
"model.visual.blocks.16.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 556 |
+
"model.visual.blocks.16.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 557 |
+
"model.visual.blocks.16.norm1.bias": "model-00002-of-00002.safetensors",
|
| 558 |
+
"model.visual.blocks.16.norm1.weight": "model-00002-of-00002.safetensors",
|
| 559 |
+
"model.visual.blocks.16.norm2.bias": "model-00002-of-00002.safetensors",
|
| 560 |
+
"model.visual.blocks.16.norm2.weight": "model-00002-of-00002.safetensors",
|
| 561 |
+
"model.visual.blocks.17.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 562 |
+
"model.visual.blocks.17.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 563 |
+
"model.visual.blocks.17.norm1.bias": "model-00002-of-00002.safetensors",
|
| 564 |
+
"model.visual.blocks.17.norm1.weight": "model-00002-of-00002.safetensors",
|
| 565 |
+
"model.visual.blocks.17.norm2.bias": "model-00002-of-00002.safetensors",
|
| 566 |
+
"model.visual.blocks.17.norm2.weight": "model-00002-of-00002.safetensors",
|
| 567 |
+
"model.visual.blocks.18.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 568 |
+
"model.visual.blocks.18.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 569 |
+
"model.visual.blocks.18.norm1.bias": "model-00002-of-00002.safetensors",
|
| 570 |
+
"model.visual.blocks.18.norm1.weight": "model-00002-of-00002.safetensors",
|
| 571 |
+
"model.visual.blocks.18.norm2.bias": "model-00002-of-00002.safetensors",
|
| 572 |
+
"model.visual.blocks.18.norm2.weight": "model-00002-of-00002.safetensors",
|
| 573 |
+
"model.visual.blocks.19.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 574 |
+
"model.visual.blocks.19.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 575 |
+
"model.visual.blocks.19.norm1.bias": "model-00002-of-00002.safetensors",
|
| 576 |
+
"model.visual.blocks.19.norm1.weight": "model-00002-of-00002.safetensors",
|
| 577 |
+
"model.visual.blocks.19.norm2.bias": "model-00002-of-00002.safetensors",
|
| 578 |
+
"model.visual.blocks.19.norm2.weight": "model-00002-of-00002.safetensors",
|
| 579 |
+
"model.visual.blocks.2.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 580 |
+
"model.visual.blocks.2.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 581 |
+
"model.visual.blocks.2.norm1.bias": "model-00002-of-00002.safetensors",
|
| 582 |
+
"model.visual.blocks.2.norm1.weight": "model-00002-of-00002.safetensors",
|
| 583 |
+
"model.visual.blocks.2.norm2.bias": "model-00002-of-00002.safetensors",
|
| 584 |
+
"model.visual.blocks.2.norm2.weight": "model-00002-of-00002.safetensors",
|
| 585 |
+
"model.visual.blocks.20.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 586 |
+
"model.visual.blocks.20.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 587 |
+
"model.visual.blocks.20.norm1.bias": "model-00002-of-00002.safetensors",
|
| 588 |
+
"model.visual.blocks.20.norm1.weight": "model-00002-of-00002.safetensors",
|
| 589 |
+
"model.visual.blocks.20.norm2.bias": "model-00002-of-00002.safetensors",
|
| 590 |
+
"model.visual.blocks.20.norm2.weight": "model-00002-of-00002.safetensors",
|
| 591 |
+
"model.visual.blocks.21.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 592 |
+
"model.visual.blocks.21.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 593 |
+
"model.visual.blocks.21.norm1.bias": "model-00002-of-00002.safetensors",
|
| 594 |
+
"model.visual.blocks.21.norm1.weight": "model-00002-of-00002.safetensors",
|
| 595 |
+
"model.visual.blocks.21.norm2.bias": "model-00002-of-00002.safetensors",
|
| 596 |
+
"model.visual.blocks.21.norm2.weight": "model-00002-of-00002.safetensors",
|
| 597 |
+
"model.visual.blocks.22.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 598 |
+
"model.visual.blocks.22.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 599 |
+
"model.visual.blocks.22.norm1.bias": "model-00002-of-00002.safetensors",
|
| 600 |
+
"model.visual.blocks.22.norm1.weight": "model-00002-of-00002.safetensors",
|
| 601 |
+
"model.visual.blocks.22.norm2.bias": "model-00002-of-00002.safetensors",
|
| 602 |
+
"model.visual.blocks.22.norm2.weight": "model-00002-of-00002.safetensors",
|
| 603 |
+
"model.visual.blocks.23.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 604 |
+
"model.visual.blocks.23.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 605 |
+
"model.visual.blocks.23.norm1.bias": "model-00002-of-00002.safetensors",
|
| 606 |
+
"model.visual.blocks.23.norm1.weight": "model-00002-of-00002.safetensors",
|
| 607 |
+
"model.visual.blocks.23.norm2.bias": "model-00002-of-00002.safetensors",
|
| 608 |
+
"model.visual.blocks.23.norm2.weight": "model-00002-of-00002.safetensors",
|
| 609 |
+
"model.visual.blocks.3.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 610 |
+
"model.visual.blocks.3.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 611 |
+
"model.visual.blocks.3.norm1.bias": "model-00002-of-00002.safetensors",
|
| 612 |
+
"model.visual.blocks.3.norm1.weight": "model-00002-of-00002.safetensors",
|
| 613 |
+
"model.visual.blocks.3.norm2.bias": "model-00002-of-00002.safetensors",
|
| 614 |
+
"model.visual.blocks.3.norm2.weight": "model-00002-of-00002.safetensors",
|
| 615 |
+
"model.visual.blocks.4.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 616 |
+
"model.visual.blocks.4.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 617 |
+
"model.visual.blocks.4.norm1.bias": "model-00002-of-00002.safetensors",
|
| 618 |
+
"model.visual.blocks.4.norm1.weight": "model-00002-of-00002.safetensors",
|
| 619 |
+
"model.visual.blocks.4.norm2.bias": "model-00002-of-00002.safetensors",
|
| 620 |
+
"model.visual.blocks.4.norm2.weight": "model-00002-of-00002.safetensors",
|
| 621 |
+
"model.visual.blocks.5.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 622 |
+
"model.visual.blocks.5.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 623 |
+
"model.visual.blocks.5.norm1.bias": "model-00002-of-00002.safetensors",
|
| 624 |
+
"model.visual.blocks.5.norm1.weight": "model-00002-of-00002.safetensors",
|
| 625 |
+
"model.visual.blocks.5.norm2.bias": "model-00002-of-00002.safetensors",
|
| 626 |
+
"model.visual.blocks.5.norm2.weight": "model-00002-of-00002.safetensors",
|
| 627 |
+
"model.visual.blocks.6.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 628 |
+
"model.visual.blocks.6.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 629 |
+
"model.visual.blocks.6.norm1.bias": "model-00002-of-00002.safetensors",
|
| 630 |
+
"model.visual.blocks.6.norm1.weight": "model-00002-of-00002.safetensors",
|
| 631 |
+
"model.visual.blocks.6.norm2.bias": "model-00002-of-00002.safetensors",
|
| 632 |
+
"model.visual.blocks.6.norm2.weight": "model-00002-of-00002.safetensors",
|
| 633 |
+
"model.visual.blocks.7.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 634 |
+
"model.visual.blocks.7.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 635 |
+
"model.visual.blocks.7.norm1.bias": "model-00002-of-00002.safetensors",
|
| 636 |
+
"model.visual.blocks.7.norm1.weight": "model-00002-of-00002.safetensors",
|
| 637 |
+
"model.visual.blocks.7.norm2.bias": "model-00002-of-00002.safetensors",
|
| 638 |
+
"model.visual.blocks.7.norm2.weight": "model-00002-of-00002.safetensors",
|
| 639 |
+
"model.visual.blocks.8.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 640 |
+
"model.visual.blocks.8.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 641 |
+
"model.visual.blocks.8.norm1.bias": "model-00002-of-00002.safetensors",
|
| 642 |
+
"model.visual.blocks.8.norm1.weight": "model-00002-of-00002.safetensors",
|
| 643 |
+
"model.visual.blocks.8.norm2.bias": "model-00002-of-00002.safetensors",
|
| 644 |
+
"model.visual.blocks.8.norm2.weight": "model-00002-of-00002.safetensors",
|
| 645 |
+
"model.visual.blocks.9.attn.proj.bias": "model-00002-of-00002.safetensors",
|
| 646 |
+
"model.visual.blocks.9.mlp.linear_fc2.bias": "model-00002-of-00002.safetensors",
|
| 647 |
+
"model.visual.blocks.9.norm1.bias": "model-00002-of-00002.safetensors",
|
| 648 |
+
"model.visual.blocks.9.norm1.weight": "model-00002-of-00002.safetensors",
|
| 649 |
+
"model.visual.blocks.9.norm2.bias": "model-00002-of-00002.safetensors",
|
| 650 |
+
"model.visual.blocks.9.norm2.weight": "model-00002-of-00002.safetensors",
|
| 651 |
+
"model.visual.merger.norm.bias": "model-00002-of-00002.safetensors",
|
| 652 |
+
"model.visual.merger.norm.weight": "model-00002-of-00002.safetensors",
|
| 653 |
+
"model.visual.patch_embed.proj.bias": "model-00002-of-00002.safetensors",
|
| 654 |
+
"model.language_model.layers.10.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 655 |
+
"model.language_model.layers.11.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 656 |
+
"model.language_model.layers.11.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 657 |
+
"model.language_model.layers.18.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 658 |
+
"model.language_model.layers.19.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 659 |
+
"model.language_model.layers.19.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 660 |
+
"model.language_model.layers.2.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 661 |
+
"model.language_model.layers.12.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 662 |
+
"model.language_model.layers.13.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 663 |
+
"model.language_model.layers.7.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 664 |
+
"model.language_model.layers.7.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 665 |
+
"model.language_model.layers.8.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 666 |
+
"model.language_model.layers.22.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 667 |
+
"model.language_model.layers.23.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 668 |
+
"model.language_model.layers.31.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 669 |
+
"model.language_model.layers.31.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 670 |
+
"model.language_model.layers.4.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 671 |
+
"model.language_model.layers.14.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 672 |
+
"model.language_model.layers.15.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 673 |
+
"model.language_model.layers.15.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 674 |
+
"model.language_model.layers.9.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 675 |
+
"mtp.layers.0.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 676 |
+
"mtp.layers.0.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 677 |
+
"model.language_model.layers.3.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 678 |
+
"model.language_model.layers.3.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 679 |
+
"model.language_model.layers.30.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 680 |
+
"model.language_model.layers.16.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 681 |
+
"model.language_model.layers.17.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 682 |
+
"model.language_model.layers.23.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 683 |
+
"model.language_model.layers.24.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 684 |
+
"model.language_model.layers.25.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 685 |
+
"model.language_model.layers.5.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 686 |
+
"model.language_model.layers.6.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 687 |
+
"model.language_model.layers.20.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 688 |
+
"model.language_model.layers.21.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 689 |
+
"model.language_model.layers.26.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 690 |
+
"model.language_model.layers.27.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
| 691 |
+
"model.language_model.layers.27.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
| 692 |
+
"model.language_model.layers.28.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 693 |
+
"model.language_model.layers.29.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 694 |
+
"model.language_model.layers.0.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 695 |
+
"model.language_model.layers.1.linear_attn.norm.weight": "model-00002-of-00002.safetensors",
|
| 696 |
+
"model.language_model.layers.10.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 697 |
+
"model.language_model.layers.12.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 698 |
+
"model.language_model.layers.2.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 699 |
+
"model.language_model.layers.13.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 700 |
+
"model.language_model.layers.14.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 701 |
+
"model.language_model.layers.8.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 702 |
+
"model.language_model.layers.9.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 703 |
+
"model.language_model.layers.22.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 704 |
+
"model.language_model.layers.4.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 705 |
+
"model.language_model.layers.5.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 706 |
+
"model.language_model.layers.16.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 707 |
+
"model.language_model.layers.30.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 708 |
+
"model.language_model.layers.17.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 709 |
+
"model.language_model.layers.18.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 710 |
+
"model.language_model.layers.24.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 711 |
+
"model.language_model.layers.25.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 712 |
+
"model.language_model.layers.6.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 713 |
+
"model.language_model.layers.20.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 714 |
+
"model.language_model.layers.21.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 715 |
+
"model.language_model.layers.26.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 716 |
+
"model.language_model.layers.28.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 717 |
+
"model.language_model.layers.29.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 718 |
+
"model.language_model.layers.0.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 719 |
+
"model.language_model.layers.1.linear_attn.A_log": "model-00002-of-00002.safetensors",
|
| 720 |
+
"model.language_model.layers.10.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 721 |
+
"model.language_model.layers.2.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 722 |
+
"model.language_model.layers.12.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 723 |
+
"model.language_model.layers.13.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 724 |
+
"model.language_model.layers.8.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 725 |
+
"model.language_model.layers.9.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 726 |
+
"model.language_model.layers.22.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 727 |
+
"model.language_model.layers.4.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 728 |
+
"model.language_model.layers.5.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 729 |
+
"model.language_model.layers.14.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 730 |
+
"model.language_model.layers.16.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 731 |
+
"model.language_model.layers.30.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 732 |
+
"model.language_model.layers.17.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 733 |
+
"model.language_model.layers.18.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 734 |
+
"model.language_model.layers.24.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 735 |
+
"model.language_model.layers.25.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 736 |
+
"model.language_model.layers.6.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 737 |
+
"model.language_model.layers.20.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 738 |
+
"model.language_model.layers.21.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 739 |
+
"model.language_model.layers.26.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 740 |
+
"model.language_model.layers.28.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 741 |
+
"model.language_model.layers.29.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 742 |
+
"model.language_model.layers.0.linear_attn.dt_bias": "model-00002-of-00002.safetensors",
|
| 743 |
+
"model.language_model.layers.1.linear_attn.dt_bias": "model-00002-of-00002.safetensors"
|
| 744 |
+
}
|
| 745 |
+
}
|
processor_config.json
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"image_processor": {
|
| 3 |
+
"data_format": "channels_first",
|
| 4 |
+
"do_convert_rgb": true,
|
| 5 |
+
"do_normalize": true,
|
| 6 |
+
"do_rescale": true,
|
| 7 |
+
"do_resize": true,
|
| 8 |
+
"image_mean": [
|
| 9 |
+
0.5,
|
| 10 |
+
0.5,
|
| 11 |
+
0.5
|
| 12 |
+
],
|
| 13 |
+
"image_processor_type": "Qwen2VLImageProcessorFast",
|
| 14 |
+
"image_std": [
|
| 15 |
+
0.5,
|
| 16 |
+
0.5,
|
| 17 |
+
0.5
|
| 18 |
+
],
|
| 19 |
+
"merge_size": 2,
|
| 20 |
+
"patch_size": 16,
|
| 21 |
+
"resample": 3,
|
| 22 |
+
"rescale_factor": 0.00392156862745098,
|
| 23 |
+
"size": {
|
| 24 |
+
"longest_edge": 16777216,
|
| 25 |
+
"shortest_edge": 65536
|
| 26 |
+
},
|
| 27 |
+
"temporal_patch_size": 2
|
| 28 |
+
},
|
| 29 |
+
"processor_class": "Qwen3VLProcessor",
|
| 30 |
+
"video_processor": {
|
| 31 |
+
"data_format": "channels_first",
|
| 32 |
+
"default_to_square": true,
|
| 33 |
+
"do_convert_rgb": true,
|
| 34 |
+
"do_normalize": true,
|
| 35 |
+
"do_rescale": true,
|
| 36 |
+
"do_resize": true,
|
| 37 |
+
"do_sample_frames": true,
|
| 38 |
+
"fps": 2,
|
| 39 |
+
"image_mean": [
|
| 40 |
+
0.5,
|
| 41 |
+
0.5,
|
| 42 |
+
0.5
|
| 43 |
+
],
|
| 44 |
+
"image_std": [
|
| 45 |
+
0.5,
|
| 46 |
+
0.5,
|
| 47 |
+
0.5
|
| 48 |
+
],
|
| 49 |
+
"max_frames": 768,
|
| 50 |
+
"merge_size": 2,
|
| 51 |
+
"min_frames": 4,
|
| 52 |
+
"patch_size": 16,
|
| 53 |
+
"resample": 3,
|
| 54 |
+
"rescale_factor": 0.00392156862745098,
|
| 55 |
+
"return_metadata": false,
|
| 56 |
+
"size": {
|
| 57 |
+
"longest_edge": 25165824,
|
| 58 |
+
"shortest_edge": 4096
|
| 59 |
+
},
|
| 60 |
+
"temporal_patch_size": 2,
|
| 61 |
+
"video_processor_type": "Qwen3VLVideoProcessor"
|
| 62 |
+
}
|
| 63 |
+
}
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cb5c6475e730b6520efd128ddcd4788c4f0046621f19a2662ef97075d172b385
|
| 3 |
+
size 19989621
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,300 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"audio_bos_token": "<|audio_start|>",
|
| 4 |
+
"audio_eos_token": "<|audio_end|>",
|
| 5 |
+
"audio_token": "<|audio_pad|>",
|
| 6 |
+
"backend": "tokenizers",
|
| 7 |
+
"bos_token": null,
|
| 8 |
+
"clean_up_tokenization_spaces": false,
|
| 9 |
+
"eos_token": "<|im_end|>",
|
| 10 |
+
"errors": "replace",
|
| 11 |
+
"image_token": "<|image_pad|>",
|
| 12 |
+
"is_local": false,
|
| 13 |
+
"model_max_length": 262144,
|
| 14 |
+
"model_specific_special_tokens": {
|
| 15 |
+
"audio_bos_token": "<|audio_start|>",
|
| 16 |
+
"audio_eos_token": "<|audio_end|>",
|
| 17 |
+
"audio_token": "<|audio_pad|>",
|
| 18 |
+
"image_token": "<|image_pad|>",
|
| 19 |
+
"video_token": "<|video_pad|>",
|
| 20 |
+
"vision_bos_token": "<|vision_start|>",
|
| 21 |
+
"vision_eos_token": "<|vision_end|>"
|
| 22 |
+
},
|
| 23 |
+
"pad_token": "<|vision_pad|>",
|
| 24 |
+
"padding_side": "right",
|
| 25 |
+
"pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
|
| 26 |
+
"processor_class": "Qwen3VLProcessor",
|
| 27 |
+
"split_special_tokens": false,
|
| 28 |
+
"tokenizer_class": "TokenizersBackend",
|
| 29 |
+
"unk_token": null,
|
| 30 |
+
"video_token": "<|video_pad|>",
|
| 31 |
+
"vision_bos_token": "<|vision_start|>",
|
| 32 |
+
"vision_eos_token": "<|vision_end|>",
|
| 33 |
+
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0].role == 'system' %}\n {{- messages[0].content + '\\n\\n' }}\n {%- endif %}\n {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == \"user\" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n {%- if message.content is string %}\n {%- set content = message.content %}\n {%- else %}\n {%- set content = '' %}\n {%- endif %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is string %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- else %}\n {%- if '</think>' in content %}\n {%- set reasoning_content = content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n {%- set content = content.split('</think>')[-1].lstrip('\\n') %}\n {%- endif %}\n {%- endif %}\n {%- if loop.index0 > ns.last_query_index %}\n {%- if loop.last or (not loop.last and reasoning_content) %}\n {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- if message.tool_calls %}\n {%- for tool_call in message.tool_calls %}\n {%- if (loop.first and content) or (not loop.first) %}\n {{- '\\n' }}\n {%- endif %}\n {%- if tool_call.function %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {%- if tool_call.arguments is string %}\n {{- tool_call.arguments }}\n {%- else %}\n {{- tool_call.arguments | tojson }}\n {%- endif %}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\n<think>\n' }}\n{%- endif %}",
|
| 34 |
+
"added_tokens_decoder": {
|
| 35 |
+
"248044": {
|
| 36 |
+
"content": "<|endoftext|>",
|
| 37 |
+
"lstrip": false,
|
| 38 |
+
"normalized": false,
|
| 39 |
+
"rstrip": false,
|
| 40 |
+
"single_word": false,
|
| 41 |
+
"special": true
|
| 42 |
+
},
|
| 43 |
+
"248045": {
|
| 44 |
+
"content": "<|im_start|>",
|
| 45 |
+
"lstrip": false,
|
| 46 |
+
"normalized": false,
|
| 47 |
+
"rstrip": false,
|
| 48 |
+
"single_word": false,
|
| 49 |
+
"special": true
|
| 50 |
+
},
|
| 51 |
+
"248046": {
|
| 52 |
+
"content": "<|im_end|>",
|
| 53 |
+
"lstrip": false,
|
| 54 |
+
"normalized": false,
|
| 55 |
+
"rstrip": false,
|
| 56 |
+
"single_word": false,
|
| 57 |
+
"special": true
|
| 58 |
+
},
|
| 59 |
+
"248047": {
|
| 60 |
+
"content": "<|object_ref_start|>",
|
| 61 |
+
"lstrip": false,
|
| 62 |
+
"normalized": false,
|
| 63 |
+
"rstrip": false,
|
| 64 |
+
"single_word": false,
|
| 65 |
+
"special": true
|
| 66 |
+
},
|
| 67 |
+
"248048": {
|
| 68 |
+
"content": "<|object_ref_end|>",
|
| 69 |
+
"lstrip": false,
|
| 70 |
+
"normalized": false,
|
| 71 |
+
"rstrip": false,
|
| 72 |
+
"single_word": false,
|
| 73 |
+
"special": true
|
| 74 |
+
},
|
| 75 |
+
"248049": {
|
| 76 |
+
"content": "<|box_start|>",
|
| 77 |
+
"lstrip": false,
|
| 78 |
+
"normalized": false,
|
| 79 |
+
"rstrip": false,
|
| 80 |
+
"single_word": false,
|
| 81 |
+
"special": true
|
| 82 |
+
},
|
| 83 |
+
"248050": {
|
| 84 |
+
"content": "<|box_end|>",
|
| 85 |
+
"lstrip": false,
|
| 86 |
+
"normalized": false,
|
| 87 |
+
"rstrip": false,
|
| 88 |
+
"single_word": false,
|
| 89 |
+
"special": true
|
| 90 |
+
},
|
| 91 |
+
"248051": {
|
| 92 |
+
"content": "<|quad_start|>",
|
| 93 |
+
"lstrip": false,
|
| 94 |
+
"normalized": false,
|
| 95 |
+
"rstrip": false,
|
| 96 |
+
"single_word": false,
|
| 97 |
+
"special": true
|
| 98 |
+
},
|
| 99 |
+
"248052": {
|
| 100 |
+
"content": "<|quad_end|>",
|
| 101 |
+
"lstrip": false,
|
| 102 |
+
"normalized": false,
|
| 103 |
+
"rstrip": false,
|
| 104 |
+
"single_word": false,
|
| 105 |
+
"special": true
|
| 106 |
+
},
|
| 107 |
+
"248053": {
|
| 108 |
+
"content": "<|vision_start|>",
|
| 109 |
+
"lstrip": false,
|
| 110 |
+
"normalized": false,
|
| 111 |
+
"rstrip": false,
|
| 112 |
+
"single_word": false,
|
| 113 |
+
"special": true
|
| 114 |
+
},
|
| 115 |
+
"248054": {
|
| 116 |
+
"content": "<|vision_end|>",
|
| 117 |
+
"lstrip": false,
|
| 118 |
+
"normalized": false,
|
| 119 |
+
"rstrip": false,
|
| 120 |
+
"single_word": false,
|
| 121 |
+
"special": true
|
| 122 |
+
},
|
| 123 |
+
"248055": {
|
| 124 |
+
"content": "<|vision_pad|>",
|
| 125 |
+
"lstrip": false,
|
| 126 |
+
"normalized": false,
|
| 127 |
+
"rstrip": false,
|
| 128 |
+
"single_word": false,
|
| 129 |
+
"special": true
|
| 130 |
+
},
|
| 131 |
+
"248056": {
|
| 132 |
+
"content": "<|image_pad|>",
|
| 133 |
+
"lstrip": false,
|
| 134 |
+
"normalized": false,
|
| 135 |
+
"rstrip": false,
|
| 136 |
+
"single_word": false,
|
| 137 |
+
"special": true
|
| 138 |
+
},
|
| 139 |
+
"248057": {
|
| 140 |
+
"content": "<|video_pad|>",
|
| 141 |
+
"lstrip": false,
|
| 142 |
+
"normalized": false,
|
| 143 |
+
"rstrip": false,
|
| 144 |
+
"single_word": false,
|
| 145 |
+
"special": true
|
| 146 |
+
},
|
| 147 |
+
"248058": {
|
| 148 |
+
"content": "<tool_call>",
|
| 149 |
+
"lstrip": false,
|
| 150 |
+
"normalized": false,
|
| 151 |
+
"rstrip": false,
|
| 152 |
+
"single_word": false,
|
| 153 |
+
"special": false
|
| 154 |
+
},
|
| 155 |
+
"248059": {
|
| 156 |
+
"content": "</tool_call>",
|
| 157 |
+
"lstrip": false,
|
| 158 |
+
"normalized": false,
|
| 159 |
+
"rstrip": false,
|
| 160 |
+
"single_word": false,
|
| 161 |
+
"special": false
|
| 162 |
+
},
|
| 163 |
+
"248060": {
|
| 164 |
+
"content": "<|fim_prefix|>",
|
| 165 |
+
"lstrip": false,
|
| 166 |
+
"normalized": false,
|
| 167 |
+
"rstrip": false,
|
| 168 |
+
"single_word": false,
|
| 169 |
+
"special": false
|
| 170 |
+
},
|
| 171 |
+
"248061": {
|
| 172 |
+
"content": "<|fim_middle|>",
|
| 173 |
+
"lstrip": false,
|
| 174 |
+
"normalized": false,
|
| 175 |
+
"rstrip": false,
|
| 176 |
+
"single_word": false,
|
| 177 |
+
"special": false
|
| 178 |
+
},
|
| 179 |
+
"248062": {
|
| 180 |
+
"content": "<|fim_suffix|>",
|
| 181 |
+
"lstrip": false,
|
| 182 |
+
"normalized": false,
|
| 183 |
+
"rstrip": false,
|
| 184 |
+
"single_word": false,
|
| 185 |
+
"special": false
|
| 186 |
+
},
|
| 187 |
+
"248063": {
|
| 188 |
+
"content": "<|fim_pad|>",
|
| 189 |
+
"lstrip": false,
|
| 190 |
+
"normalized": false,
|
| 191 |
+
"rstrip": false,
|
| 192 |
+
"single_word": false,
|
| 193 |
+
"special": false
|
| 194 |
+
},
|
| 195 |
+
"248064": {
|
| 196 |
+
"content": "<|repo_name|>",
|
| 197 |
+
"lstrip": false,
|
| 198 |
+
"normalized": false,
|
| 199 |
+
"rstrip": false,
|
| 200 |
+
"single_word": false,
|
| 201 |
+
"special": false
|
| 202 |
+
},
|
| 203 |
+
"248065": {
|
| 204 |
+
"content": "<|file_sep|>",
|
| 205 |
+
"lstrip": false,
|
| 206 |
+
"normalized": false,
|
| 207 |
+
"rstrip": false,
|
| 208 |
+
"single_word": false,
|
| 209 |
+
"special": false
|
| 210 |
+
},
|
| 211 |
+
"248066": {
|
| 212 |
+
"content": "<tool_response>",
|
| 213 |
+
"lstrip": false,
|
| 214 |
+
"normalized": false,
|
| 215 |
+
"rstrip": false,
|
| 216 |
+
"single_word": false,
|
| 217 |
+
"special": false
|
| 218 |
+
},
|
| 219 |
+
"248067": {
|
| 220 |
+
"content": "</tool_response>",
|
| 221 |
+
"lstrip": false,
|
| 222 |
+
"normalized": false,
|
| 223 |
+
"rstrip": false,
|
| 224 |
+
"single_word": false,
|
| 225 |
+
"special": false
|
| 226 |
+
},
|
| 227 |
+
"248068": {
|
| 228 |
+
"content": "<think>",
|
| 229 |
+
"lstrip": false,
|
| 230 |
+
"normalized": false,
|
| 231 |
+
"rstrip": false,
|
| 232 |
+
"single_word": false,
|
| 233 |
+
"special": false
|
| 234 |
+
},
|
| 235 |
+
"248069": {
|
| 236 |
+
"content": "</think>",
|
| 237 |
+
"lstrip": false,
|
| 238 |
+
"normalized": false,
|
| 239 |
+
"rstrip": false,
|
| 240 |
+
"single_word": false,
|
| 241 |
+
"special": false
|
| 242 |
+
},
|
| 243 |
+
"248070": {
|
| 244 |
+
"content": "<|audio_start|>",
|
| 245 |
+
"lstrip": false,
|
| 246 |
+
"normalized": false,
|
| 247 |
+
"rstrip": false,
|
| 248 |
+
"single_word": false,
|
| 249 |
+
"special": true
|
| 250 |
+
},
|
| 251 |
+
"248071": {
|
| 252 |
+
"content": "<|audio_end|>",
|
| 253 |
+
"lstrip": false,
|
| 254 |
+
"normalized": false,
|
| 255 |
+
"rstrip": false,
|
| 256 |
+
"single_word": false,
|
| 257 |
+
"special": true
|
| 258 |
+
},
|
| 259 |
+
"248072": {
|
| 260 |
+
"content": "<tts_pad>",
|
| 261 |
+
"lstrip": false,
|
| 262 |
+
"normalized": false,
|
| 263 |
+
"rstrip": false,
|
| 264 |
+
"single_word": false,
|
| 265 |
+
"special": true
|
| 266 |
+
},
|
| 267 |
+
"248073": {
|
| 268 |
+
"content": "<tts_text_bos>",
|
| 269 |
+
"lstrip": false,
|
| 270 |
+
"normalized": false,
|
| 271 |
+
"rstrip": false,
|
| 272 |
+
"single_word": false,
|
| 273 |
+
"special": true
|
| 274 |
+
},
|
| 275 |
+
"248074": {
|
| 276 |
+
"content": "<tts_text_eod>",
|
| 277 |
+
"lstrip": false,
|
| 278 |
+
"normalized": false,
|
| 279 |
+
"rstrip": false,
|
| 280 |
+
"single_word": false,
|
| 281 |
+
"special": true
|
| 282 |
+
},
|
| 283 |
+
"248075": {
|
| 284 |
+
"content": "<tts_text_bos_single>",
|
| 285 |
+
"lstrip": false,
|
| 286 |
+
"normalized": false,
|
| 287 |
+
"rstrip": false,
|
| 288 |
+
"single_word": false,
|
| 289 |
+
"special": true
|
| 290 |
+
},
|
| 291 |
+
"248076": {
|
| 292 |
+
"content": "<|audio_pad|>",
|
| 293 |
+
"lstrip": false,
|
| 294 |
+
"normalized": false,
|
| 295 |
+
"rstrip": false,
|
| 296 |
+
"single_word": false,
|
| 297 |
+
"special": true
|
| 298 |
+
}
|
| 299 |
+
}
|
| 300 |
+
}
|