Hooshaai/distilbert-mnli

DistilBERT fine-tuned on MNLI

This checkpoint was fully fine-tuned (backbone + head) on the full MNLI training set as part of the A3 SVD compression project.

Results

Metric	Value
Accuracy	0.8211
F1 (macro)	0.8202

Training Details

• Base model: distilbert-base-uncased
• Epochs: 3
• Learning rate: 4e-05
• Batch size: 128
• Train samples: 392,702
• Val samples: 9,815

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("TahaaaaM/distilbert-mnli-a3")
tokenizer = AutoTokenizer.from_pretrained("TahaaaaM/distilbert-mnli-a3")

Citation

Part of the A3 SVD transformer compression research framework.