RoBERTa-Base Quantized Model Model for Topic Classification of News Articles

This repository contains a fine-tuned RoBERTa-Base model for topic classification on the AG News dataset. The model classifies news articles into one of four categories: World, Sports, Business, and Sci/Tech.

Model Details

Model Architecture: RoBERTa Base
Task: Topic Classification
Dataset: AG News
Use Case: Classifying news articles into broad topical categories
Fine-tuning Framework: Hugging Face Transformers

Usage

Installation

pip install transformers torch

Loading the Model

from transformers import RobertaTokenizerFast, RobertaForSequenceClassification
import torch

# Load tokenizer and model
tokenizer = RobertaTokenizerFast.from_pretrained("roberta-base")
model = RobertaForSequenceClassification.from_pretrained("path_to_your_fine_tuned_model")

# Sample input
text = "NASA launches a new satellite to study climate change."

# Tokenize and prepare input
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)

# Predict
with torch.no_grad():
    outputs = model(**inputs)
    predicted_class = torch.argmax(outputs.logits, dim=1).item()

# Map predicted class index to label
label_map = {0: "World", 1: "Sports", 2: "Business", 3: "Sci/Tech"}
print(f"Predicted Topic: {label_map[predicted_class]}")

Performance Metrics

Accuracy: 0.949737
Precision: 0.949787
Recall: 0.949737
F1 Score: 0.949729

Fine-Tuning Details

Dataset

The AG News dataset is sourced from Hugging Face and contains news headlines and descriptions categorized into four classes.

Training Configuration

Number of epochs: 3
Batch size: 16
Evaluation strategy: epoch
Learning rate: 2e-5

Quantization

Post-training quantization was applied using PyTorch's built-in quantization framework to reduce the model size and improve inference efficiency.

Repository Structure

.
├── config.json
├── tokenizer_config.json    
├── special_tokens_map.json 
├── tokenizer.json        
├── model.safetensors       # Fine-tuned RoBERTa model
├── README.md               # Documentation

Limitations

The model may not generalize well to domains outside the fine-tuning dataset.
Quantization may result in minor accuracy degradation compared to full-precision models.

Contributing

Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.