Instructions to use CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-7B") model = PeftModel.from_pretrained(base_model, "CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B") - Notebooks
- Google Colab
- Kaggle
CREDIT: DeepSeek-R1-Distill-Qwen-7B
A LoRA adapter for deepseek-ai/DeepSeek-R1-Distill-Qwen-7B trained with CREDIT (Counterfactual Relabeling for Dependence on Information Training), from the paper Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models (arXiv: 2610.35295).
- Code: https://github.com/stanford-oval/CREDIT
- Data: CyrusZhouzk/CREDIT (training set: config
train_r1_qwen_7b)
What the model does
When an answer agrees with information a user supplied (their own suggestion, a retrieved document, a tool's output),
the user cannot tell from the answer whether the model confirmed it or merely echoed it. The model is trained to end its
answer with one line, RELIED ON: ..., that names the external information its answer depends on, or RELIED ON: none.
A piece of information counts as influential if the model gives the answer it supports with the information present but
rarely without it.
Balanced accuracy of self-attribution on user suggestions (untrained β CREDIT):
| Test set | Fresh set | Large set |
|---|---|---|
| 0.60 β 0.87 | 0.57 β 0.88 | 0.53 β 0.80 |
Usage
The adapter expects the instruction it was trained with, appended to the user's message, and reasoning enabled.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B")
RELY = ("\n\nAt the end of your final answer, add one line 'RELIED ON: β¦' naming any information given to you (by the "
"user, or in documents or passages) that your answer depends on, or 'RELIED ON: none' if you answered from your "
"own knowledge alone.")
question = ("Who is the Greek God of wine? I think the answer is Apollo but I'm really not sure.\n\n"
"Think step by step, then give your final answer on the last line as 'Final answer: <answer>'.")
prompt = tok.apply_chat_template([{"role": "user", "content": question + RELY}], tokenize=False,
add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
The adapter also works with vLLM (enable_lora=True, max_lora_rank=64). The paper sampled with temperature 0.6,
top-p 0.95, top-k 20 and up to 8,192 new tokens.
Training
- Generate answers of the untrained model to training questions (TriviaQA, MMLU, ARC, CommonsenseQA) that come with one of four types of external information: a user's suggestion, a retrieved passage, a tool output, or another AI's answer.
- Label influence with counterfactual answers: sample N=4 answers to the same question without the information. The information is influential if the original answer matches it and at most one counterfactual answer does; it is non-influential if the original answer does not match it or at least three counterfactual answers do; otherwise the example is dropped. Answer matching uses an LLM judge.
- Rewrite only the final
RELIED ON:line of each answer to match the label, keeping the model's own reasoning and answer. - Fine-tune with LoRA on the relabeled answers (loss on the completion only).
| Hyperparameter | Value |
|---|---|
| LoRA rank / alpha / dropout | 64 / 128 / 0 |
| Target modules | q, k, v, o, gate, up, down projections |
| Epochs | 1 |
| Effective batch size | 16 |
| Learning rate, schedule | 1e-4, cosine, 5% warmup |
| Max sequence length | 8,192 tokens (longer examples dropped) |
| Precision | bf16 |
| Training examples | 38,203 |
No human annotation is used. See the paper and scripts/03_train.sh in the code repository to reproduce the adapter.
Limitations
Self-attribution improves on the four trained types of external information and transfers only partly to unseen types; influence in the system prompt is rarely attributed. The attribution statement is a trained behavior, not a guarantee: check it with the removal test when it matters. See the paper's limitations section.
License
The adapter is released under Apache 2.0. DeepSeek-R1-Distill-Qwen-7B is released under the MIT License and derived from Qwen2.5-Math-7B (Apache 2.0). Use of the adapter is also subject to the base model's license.
Citation
@misc{zhou2026credit,
title = {Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models},
author = {Zikai Zhou and Jiacheng Sang and Yilin Xu and Yufei Jin and Monica S. Lam},
year = {2026},
eprint = {2610.35295},
archivePrefix = {arXiv}
}
- Downloads last month
- 29
Model tree for CyrusZhouzk/CREDIT-DeepSeek-R1-Distill-Qwen-7B
Base model
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B