README.md

---
title: Gpt2 Tinystories Generator
emoji: 🏢
colorFrom: yellow
colorTo: red
sdk: gradio
sdk_version: 5.13.2
app_file: app.py
pinned: false
license: apache-2.0
short_description: Gradio
---
# GPT-2 TinyStories Generator 📖 FableWeaver AI

## Overview
This project fine-tunes a **GPT-2 model** on the **TinyStories dataset** to generate structured, coherent, and engaging short narratives. The model is hosted on **Hugging Face Spaces** and provides a user-friendly interface for story generation.

## Features
- ✅ **Story Generation:** Produces structured, child-friendly short stories.
- ✅ **Bias Monitoring:** Ensures balanced gender and cultural representation.
- ✅ **Efficient Training:** Fine-tuned on **200,000 training samples** and **20,000 test samples**.
- ✅ **Grammar & Readability Enhancements:** Integrated grammar-checking tools and text refinement.
- ✅ **Optimized Performance:** Uses loss tracking, sampling techniques, and bias mitigation strategies.

## 🚀 System Architecture
The model is designed for easy interaction via Hugging Face Spaces and follows this workflow:
1. **Data Preprocessing & Cleaning**
   - Tokenization, formatting, and encoding normalization.
   - Bias mitigation and balanced data preparation.
2. **Model Fine-Tuning**
   - Fine-tuned **GPT-2 (124M parameters)** using **Hugging Face Transformers**.
   - Hyperparameter optimization (batch size, learning rate, weight decay).
3. **Story Generation Pipeline**
   - Uses **top-k filtering (k=50)**, **top-p nucleus sampling (p=0.9)**, and **temperature adjustments**.
4. **Post-Processing & Bias Mitigation**
   - Contextual reinforcement and diversity-aware storytelling.
   - No-repeat n-gram settings and logical scene transitions.
5. **Evaluation & Performance Monitoring**
   - Tracked using **Weights & Biases (W&B)** and **TensorBoard**.
   - Analyzed validation loss and coherence checks.

## 🔹 Getting Started
### Accessing the Model
The model is available on **Hugging Face Spaces**: [GPT-2 TinyStories Generator](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator)

### Usage Instructions
1. Visit the **Hugging Face Space** linked above.
2. Enter a prompt (e.g., *"Once upon a time..."*) in the input field.
3. Click **Generate** to receive an AI-generated short story.
4. Modify the prompt and settings (temperature, top-k, top-p) for different results.

## 📊 Training Details
- **Model:** GPT-2 (124M)
- **Dataset:** RonenEldan/TinyStories
- **Training:** 3 epochs on Google Colab GPU (T4)
- **Loss Metrics:**
  - Training Loss: **3.08 → 2.86**
  - Validation Loss: **1.46 → 1.40**

## 📝 Evaluation & Observations
- **Perplexity improved from 8.12 → 2.09**, indicating better text fluency.
- **Validation loss decreased consistently**, suggesting effective generalization.
- **Human evaluation highlighted minor inconsistencies**, such as abrupt scene shifts and simplistic narratives.

## ⚖️ Ethical Considerations
- **Bias Monitoring:** Pronoun analysis and diversity checks to ensure fairness.
- **Harmful Content Mitigation:** Manually reviewed outputs for stereotypes.
- **Text Processing Issues:** UTF-8 encoding applied to prevent character errors.

## 🔮 Future Improvements
- **Enhancing Creativity:** Fine-tune temperature and randomness settings.
- **Genre-Specific Training:** Introduce theme-based datasets.
- **Larger Model Training:** Experiment with **GPT-2 (355M)** for richer storytelling.

## 🤝 Contributors
- **Charla Pia Vella** (Project Developer)
- **Affiliation:** ARI3333 Generative AI

## 📜 License
This project is released under the **Apache-2.0 License**.

## 🎓 Acknowledgments
- **OpenAI** for GPT-2
- **Hugging Face** for the fine-tuning framework
- **Ronen Eldan** for the TinyStories dataset

📌 For more details, visit the [Hugging Face Space](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator).








Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference