File size: 3,295 Bytes
28c40a3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
# 📝 Next Sentence Prediction Model
This repository hosts a fine-tuned **Flan-T5-Base** model optimized for **next-line prediction**. The model is trained to predict the next sentence in a given text, making it useful for applications like text completion, conversation modeling, and document coherence assessment.
## 📌 Model Details
- **Model Architecture**: Flan-T5-Base
- **Task**: Next Sentence Prediction
- **Dataset**: OpenWebText-10k (Preprocessed)
- **Fine-tuning Framework**: Hugging Face Transformers
- **Quantization**: FP16 for efficiency
## 🚀 Usage
### Installation
```bash
pip install transformers torch datasets
```
### Loading the Model
```python
from transformers import T5ForConditionalGeneration, T5Tokenizer
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
model_name = "AventIQ-AI/flan-t5-base-next-line-prediction"
model = T5ForConditionalGeneration.from_pretrained(model_name).to(device)
tokenizer = T5Tokenizer.from_pretrained(model_name)
```
### Perform Next Sentence Prediction
```python
def predict_next_sentence(model, tokenizer, input_sentence):
device = "cuda" if torch.cuda.is_available() else "cpu"
formatted_text = f"predict next line: {input_sentence}"
input_ids = tokenizer(formatted_text, return_tensors="pt").input_ids.to(device)
with torch.no_grad():
output_ids = model.generate(input_ids, max_length=50)
return tokenizer.decode(output_ids[0], skip_special_tokens=True)
# 🔹 **Test Prediction**
input_sentence = "The sun was setting behind the mountains."
predicted_sentence = predict_next_sentence(model, tokenizer, input_sentence)
print(f"Input Sentence: {input_sentence}")
print(f"Predicted Next Sentence: {predicted_sentence}")
```
## 📊 Evaluation Results
After fine-tuning, the model was evaluated on a test set, achieving the following performance:
| Metric | Score | Meaning |
| ------------------- | ------ | ----------------------------------- |
| **Perplexity** | 23 | Measures model confidence |
| **Inference Speed** | Fast | Optimized for real-time completion |
## 🔧 Fine-Tuning Details
### Dataset
The model was trained using the **OpenWebText-10k dataset**, containing **10,000 documents**. The dataset was preprocessed by splitting texts into sentence pairs, where the model learns to predict the next logical sentence.
### Training Configuration
- **Number of epochs**: 3
- **Batch size**: 8
- **Optimizer**: AdamW
- **Learning rate**: 2e-5
- **Evaluation strategy**: Epoch-based
### Quantization
The model was quantized using **FP16**, reducing memory usage while maintaining performance.
## 📂 Repository Structure
```bash
.
├── model/ # Fine-tuned model files
├── tokenizer_config/ # Tokenizer configuration
├── quantized_model/ # FP16 quantized model
├── README.md # Model documentation
```
## ⚠️ Limitations
- The model works best with **well-structured sentences**.
- May struggle with **long-range dependencies** in texts.
- **Contextual consistency** is limited to sentence pairs.
## 📬 Contact & Contributions
For improvements or questions, feel free to open an issue or contribute to this repository!
|