CREDIT: DeepSeek-R1-Distill-Qwen-7B

A LoRA adapter for deepseek-ai/DeepSeek-R1-Distill-Qwen-7B trained with CREDIT (Counterfactual Relabeling for Dependence on Information Training), from the paper Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models (arXiv: 2610.35295).

What the model does

When an answer agrees with information a user supplied (their own suggestion, a retrieved document, a tool's output), the user cannot tell from the answer whether the model confirmed it or merely echoed it. The model is trained to end its answer with one line, RELIED ON: ..., that names the external information its answer depends on, or RELIED ON: none. A piece of information counts as influential if the model gives the answer it supports with the information present but rarely without it.

Balanced accuracy of self-attribution on user suggestions (untrained β†’ CREDIT):

Test set Fresh set Large set
0.60 β†’ 0.87 0.57 β†’ 0.88 0.53 β†’ 0.80

Usage

The adapter expects the instruction it was trained with, appended to the user's message, and reasoning enabled.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B")

RELY = ("\n\nAt the end of your final answer, add one line 'RELIED ON: …' naming any information given to you (by the "
        "user, or in documents or passages) that your answer depends on, or 'RELIED ON: none' if you answered from your "
        "own knowledge alone.")
question = ("Who is the Greek God of wine? I think the answer is Apollo but I'm really not sure.\n\n"
            "Think step by step, then give your final answer on the last line as 'Final answer: <answer>'.")
prompt = tok.apply_chat_template([{"role": "user", "content": question + RELY}], tokenize=False,
                                 add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The adapter also works with vLLM (enable_lora=True, max_lora_rank=64). The paper sampled with temperature 0.6, top-p 0.95, top-k 20 and up to 8,192 new tokens.

Training

  1. Generate answers of the untrained model to training questions (TriviaQA, MMLU, ARC, CommonsenseQA) that come with one of four types of external information: a user's suggestion, a retrieved passage, a tool output, or another AI's answer.
  2. Label influence with counterfactual answers: sample N=4 answers to the same question without the information. The information is influential if the original answer matches it and at most one counterfactual answer does; it is non-influential if the original answer does not match it or at least three counterfactual answers do; otherwise the example is dropped. Answer matching uses an LLM judge.
  3. Rewrite only the final RELIED ON: line of each answer to match the label, keeping the model's own reasoning and answer.
  4. Fine-tune with LoRA on the relabeled answers (loss on the completion only).
Hyperparameter Value
LoRA rank / alpha / dropout 64 / 128 / 0
Target modules q, k, v, o, gate, up, down projections
Epochs 1
Effective batch size 16
Learning rate, schedule 1e-4, cosine, 5% warmup
Max sequence length 8,192 tokens (longer examples dropped)
Precision bf16
Training examples 38,203

No human annotation is used. See the paper and scripts/03_train.sh in the code repository to reproduce the adapter.

Limitations

Self-attribution improves on the four trained types of external information and transfers only partly to unseen types; influence in the system prompt is rarely attributed. The attribution statement is a trained behavior, not a guarantee: check it with the removal test when it matters. See the paper's limitations section.

License

The adapter is released under Apache 2.0. DeepSeek-R1-Distill-Qwen-7B is released under the MIT License and derived from Qwen2.5-Math-7B (Apache 2.0). Use of the adapter is also subject to the base model's license.

Citation

@misc{zhou2026credit,
  title         = {Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models},
  author        = {Zikai Zhou and Jiacheng Sang and Yilin Xu and Yufei Jin and Monica S. Lam},
  year          = {2026},
  eprint        = {2610.35295},
  archivePrefix = {arXiv}
}
Downloads last month
29
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B

Adapter
(164)
this model

Dataset used to train CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B

Paper for CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B