AventIQ-AI/gpt2-next-word-prediction

Next Word Prediction With GPT2

📌 Overview

This repository hosts the quantized version of the GPT2 model fine-tuned for next word prediction tasks. The model has been trained on the bookcorpus dataset from Hugging Face. The model is quantized to Float16 (FP16) to optimize inference speed and efficiency while maintaining high performance.

🏗 Model Details

• Model Architecture: GPT2
• Task: Next Word Prediction
• Dataset: Hugging Face's bookcorpus
• Quantization: Float16 (FP16) for optimized inference
• Fine-tuning Framework: Hugging Face Transformers

🚀 Usage

Installation

pip install transformers torch

Loading the Model

from transformers import GPT2LMHeadModel, GPT2Tokenizer
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

model_name = "AventIQ-AI/gpt2-next-word-prediction"
model = GPT2LMHeadModel.from_pretrained(model_name).to(device)
tokenizer = GPT2Tokenizer.from_pretrained(model_name)

Question Answer Example

# Input text
text = "Hi! How are"

# Tokenize input text
input_ids = tokenizer.encode(text, return_tensors="pt").to(device)

# Generate next word (max_length ensures we get only the next token)
output = model.generate(input_ids, max_length=input_ids.shape[1] + 1, do_sample=False)

# Decode output
generated_text = tokenizer.decode(output[0])

print("Generated Sentence:", generated_text)

⚡ Quantization Details

Post-training quantization was applied using PyTorch's built-in quantization framework. The model was quantized to Float16 (FP16) to reduce model size and improve inference efficiency while balancing accuracy.

Evaluation Metrics

A well-trained language model should have a perplexity closer to 10–50, depending on the dataset and domain and our model's perplexity score is 32.4.

🔧 Fine-Tuning Details

Dataset

The Bookcorpus dataset was used for training and evaluation. The dataset consists of texts.

Training Configuration

• Number of epochs: 3
• Batch size: 8
• Learning rate: 5e-5
• Evaluation strategy: steps

📂 Repository Structure

.
├── model/               # Contains the quantized model files
├── tokenizer_config/    # Tokenizer configuration and vocabulary files
├── model.safetensors/   # Quantized Model
├── README.md            # Model documentation

⚠️ Limitations

• The model may struggle for out of scope tasks.
• Quantization may lead to slight degradation in accuracy compared to full-precision models.
• Performance may vary across different writing styles and sentence structures.

🤝 Contributing

Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.