Qwen3.8-Flash-Next Uncensored, IQR-V4 โ€” radiance container

orcarouter/Qwen3.8-Flash-Next-Uncensored โ€” Qwen3.8-Flash-Next with its refusal direction abliterated โ€” as a single .rad container for the radiance inference engine (AMD RDNA4, ROCm), with the model's MTP head for speculative decoding and its vision tower. It is the format of qwen3.8-next-flash-iqr-v4 and replaces qwen3.8-next-flash-uncensored-fp8-iq4r-moe, which radiance 2.x does not serve.

This model's safety alignment has been removed: it answers requests the original model refuses. The source model's card describes how it was made, how it was evaluated and what it is for.

Engine version. radiance 2.0.0 or newer.

File qwen3.8-next-flash-uncensored-iqr-v4.rad โ€” 113.3 GiB
Routed experts, gate_up 4-bit trellis codes after a 128-point Walshโ€“Hadamard rotation, chosen by GPTQ on Hessians recorded from the bf16 model
Routed experts, down 4-bit codes into a 16-level non-uniform codebook after the same rotation, one E4M3 scale per 64 weights, chosen by GPTQ and refined by three sweeps of coordinate descent
Protected experts ten experts that carry most of their layer's down-projection energy, kept bf16
Trunk the attention's query-gate and the delta net's input projection as 5-bit trellis codes (GPTQ); the other attention, delta-net and shared-expert linears and the n-gram layer's projections rotated int8, a scale per 128 columns searched for least error; the lm_head int8; hyper-connection mixing matrices E4M3
Speculator the model's MTP head (depth 3), its block as 3- and 4-bit trellis codes beside codebook experts
Vision the vision tower, bf16: images and video in chat requests
Context 262,144 tokens trained; served at 200K

Quality

This file was not measured against its own bf16 weights. The uncensored model differs from the original in the weights that write into the residual stream only, by one direction projected out of them, so the recipe's error on the original model is the figure to go by: the KL table of qwen3.8-next-flash-iqr-v4 (mean KL 0.0224 on model-written positions against 0.0250 for the earlier format).

Checked on this file: it loads and serves on radiance 2.1.0 (one card, tp: 1); on a handful of plain questions its answers match those of the original model's container, and it answers requests that container refuses.

Serve

The routed experts do not fit two 32 GB cards; the engine keeps the hottest in VRAM and streams the rest from a pinned host pool, so the host needs about 32 GB of free RAM. The n-gram table is read from the container on disk as tokens need it, so the container belongs on an NVMe drive.

# radiance.yaml -- run with: radiance --config radiance.yaml
server:
  host: 0.0.0.0
  port: 8000
memory:
  gpu-headroom-mib: 96
models:
  flashnext-uncensored:
    model: qwen3.8-next-flash-uncensored-iqr-v4.rad
    tp: 2
    tp-wire: wht6
    max-num-seqs: 8
    max-model-len: 200000
    placement: expert_tiered
    host-pool-mib: 12288
    kv-cache-dtype: i8
    prefix-cache-mib: 4096
    num-speculative-tokens: 3
    max-num-batched-tokens: 2048

It runs the kernels of the original model's container; its speed was not measured separately. The server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls, structured output, and image_url / video parts in chat messages. A deployment moving from the 1.x container replaces kv-cache-dtype: fp8 by i8.

How this file was made

It was not converted from the source checkpoint in one run. The uncensored model and the original differ in six groups of weights, all of which write into the residual stream: the routed and the shared experts' down projections, the delta net's and the attention's output projections, the token embedding and the n-gram layer's value projection. Every other weight is the original model's, bit for bit, and is copied from qwen3.8-next-flash-iqr-v4. The six groups come from three places:

  • The routed experts' down projections are the planes of qwen3.8-next-flash-uncensored-fp8-iq4r-moe, byte for byte. Both formats hold these weights the same way (the recipe line is the same, and the two containers of the original model hold identical planes, compared on 400 experts), and that container was converted from the uncensored checkpoint.
  • The token embedding and the n-gram value projection are the bf16 tensors that container holds, which are the checkpoint's.
  • The 98 output and shared-expert down projections of the trunk were rebuilt: the original model's bf16 matrix with the abliterated direction projected out of its output side, W' = W - r (r^T W). The direction r is read off the difference of the two models' embeddings, where it carries 99.3% of the energy. Rebuilt this way, each matrix differs from the int8 matrix the earlier uncensored container holds by 1.02 times what int8 rounding alone costs on the original model's matrix.

With those 100 tensors in place of the original checkpoint's, the conversion is the original container's, every weight that reads none of them copied:

CALIB=calib/w4nl-calib rad-convert orcarouter/Qwen3.8-Flash-Next-Uncensored \
    --recipe qwen4exp-iqr-v4.recipe \
    --reuse qwen3.8-next-flash-iqr-v4.rad --changed changed.txt \
    -o qwen3.8-next-flash-uncensored-iqr-v4.rad

changed.txt names the 100 tensors; the routed down planes were then written over the copied ones. The same command over the original checkpoint reproduces qwen3.8-next-flash-iqr-v4.rad's weights byte for byte. The recipe (qwen4exp-iqr-v4.recipe in this repository) is that container's; $CALIB and calib/qc-fn-bf16 are calibration data recorded on the original model and are not published:

blk.*.ple_key.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.ple_value.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.48.attn_qg.weight gptq codes=u3 tcq=16 tcq_gain=1.08 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 damp=0.01 tcq_order=1
blk.48.attn_k.weight gptq codes=u3 tcq=16 tcq_gain=1.08 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 damp=0.01 tcq_order=1
blk.48.attn_v.weight gptq codes=u3 tcq=16 tcq_gain=1.08 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 damp=0.01 tcq_order=1
blk.48.attn_output.weight gptq codes=u4 tcq=16 tcq_code=mad tcq_rows=256 group=128 scale=bf16 dyn=1 transform=fwht128 damp=0.01 tcq_gain=1.09
blk.48.ffn_gate_up_shexp.weight gptq codes=u3 tcq=16 tcq_gain=1.08 tcq_code=mad tcq_rows=128 block=1x* scale=f32 transform=fwht128 damp=0.01 tcq_order=1
blk.48.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
blk.*.attn_qg.weight gptq codes=u5 tcq=16 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 tcq_gain=1.16 damp=0.01 calib=calib/qc-fn-bf16 tcq_order=1
blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.ssm_inz.weight gptq codes=u5 tcq=16 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 tcq_gain=1.16 damp=0.01 calib=calib/qc-fn-bf16 tcq_order=1
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.34.ffn_*_exps.407.weight cast dtype=bf16
blk.34.ffn_*_exps.496.weight cast dtype=bf16
blk.44.ffn_*_exps.292.weight cast dtype=bf16
blk.44.ffn_*_exps.350.weight cast dtype=bf16
blk.46.ffn_*_exps.290.weight cast dtype=bf16
blk.46.ffn_*_exps.392.weight cast dtype=bf16
blk.47.ffn_*_exps.122.weight cast dtype=bf16
blk.47.ffn_*_exps.143.weight cast dtype=bf16
blk.47.ffn_*_exps.399.weight cast dtype=bf16
blk.47.ffn_*_exps.445.weight cast dtype=bf16
blk.*.ffn_gate_up_exps.*.weight gptq codes=u4 tcq=16 tcq_code=mad tcq_rows=128 block=1x* scale=f32 transform=fwht128 tcq_gain=1.12 damp=0.01 calib=calib/qc-fn-bf16 tcq_order=1
blk.*.ffn_down_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32
*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16

License

The Qwen community license of the base model; see LICENSE.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for StillDeadcode/qwen3.8-next-flash-uncensored-iqr-v4

Finetuned
(7)
this model