Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
@@ -12,95 +12,6 @@ short_description: Gradio
|
|
12 |
---
|
13 |
|
14 |
|
15 |
-
---
|
16 |
-
title: GPT-2 TinyStories Generator
|
17 |
-
emoji: ๐
|
18 |
-
colorFrom: yellow
|
19 |
-
colorTo: red
|
20 |
-
sdk: gradio
|
21 |
-
sdk_version: 5.13.2
|
22 |
-
app_file: app.py
|
23 |
-
pinned: false
|
24 |
-
license: apache-2.0
|
25 |
-
short_description: A fine-tuned GPT-2 model for generating structured, child-friendly short stories.
|
26 |
-
---
|
27 |
-
|
28 |
-
# GPT-2 TinyStories Generator ๐ FableWeaver AI
|
29 |
-
|
30 |
-
## Overview
|
31 |
-
This project fine-tunes a **GPT-2 model** on the **TinyStories dataset** to generate structured, coherent, and engaging short narratives. The model is hosted on **Hugging Face Spaces** and provides a user-friendly interface for story generation.
|
32 |
-
|
33 |
-
## Features
|
34 |
-
- โ
**Story Generation:** Produces structured, child-friendly short stories.
|
35 |
-
- โ
**Bias Monitoring:** Ensures balanced gender and cultural representation.
|
36 |
-
- โ
**Efficient Training:** Fine-tuned on **200,000 training samples** and **20,000 test samples**.
|
37 |
-
- โ
**Grammar & Readability Enhancements:** Integrated grammar-checking tools and text refinement.
|
38 |
-
- โ
**Optimized Performance:** Uses loss tracking, sampling techniques, and bias mitigation strategies.
|
39 |
-
|
40 |
-
## ๐ System Architecture
|
41 |
-
The model is designed for easy interaction via Hugging Face Spaces and follows this workflow:
|
42 |
-
1. **Data Preprocessing & Cleaning**
|
43 |
-
- Tokenization, formatting, and encoding normalization.
|
44 |
-
- Bias mitigation and balanced data preparation.
|
45 |
-
2. **Model Fine-Tuning**
|
46 |
-
- Fine-tuned **GPT-2 (124M parameters)** using **Hugging Face Transformers**.
|
47 |
-
- Hyperparameter optimization (batch size, learning rate, weight decay).
|
48 |
-
3. **Story Generation Pipeline**
|
49 |
-
- Uses **top-k filtering (k=50)**, **top-p nucleus sampling (p=0.9)**, and **temperature adjustments**.
|
50 |
-
4. **Post-Processing & Bias Mitigation**
|
51 |
-
- Contextual reinforcement and diversity-aware storytelling.
|
52 |
-
- No-repeat n-gram settings and logical scene transitions.
|
53 |
-
5. **Evaluation & Performance Monitoring**
|
54 |
-
- Tracked using **Weights & Biases (W&B)** and **TensorBoard**.
|
55 |
-
- Analyzed validation loss and coherence checks.
|
56 |
-
|
57 |
-
## ๐น Getting Started
|
58 |
-
### Accessing the Model
|
59 |
-
The model is available on **Hugging Face Spaces**: [GPT-2 TinyStories Generator](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator)
|
60 |
-
|
61 |
-
### Usage Instructions
|
62 |
-
1. Visit the **Hugging Face Space** linked above.
|
63 |
-
2. Enter a prompt (e.g., *"Once upon a time..."*) in the input field.
|
64 |
-
3. Click **Generate** to receive an AI-generated short story.
|
65 |
-
4. Modify the prompt and settings (temperature, top-k, top-p) for different results.
|
66 |
-
|
67 |
-
## ๐ Training Details
|
68 |
-
- **Model:** GPT-2 (124M)
|
69 |
-
- **Dataset:** RonenEldan/TinyStories
|
70 |
-
- **Training:** 3 epochs on Google Colab GPU (T4)
|
71 |
-
- **Loss Metrics:**
|
72 |
-
- Training Loss: **3.08 โ 2.86**
|
73 |
-
- Validation Loss: **1.46 โ 1.40**
|
74 |
-
|
75 |
-
## ๐ Evaluation & Observations
|
76 |
-
- **Perplexity improved from 8.12 โ 2.09**, indicating better text fluency.
|
77 |
-
- **Validation loss decreased consistently**, suggesting effective generalization.
|
78 |
-
- **Human evaluation highlighted minor inconsistencies**, such as abrupt scene shifts and simplistic narratives.
|
79 |
-
|
80 |
-
## โ๏ธ Ethical Considerations
|
81 |
-
- **Bias Monitoring:** Pronoun analysis and diversity checks to ensure fairness.
|
82 |
-
- **Harmful Content Mitigation:** Manually reviewed outputs for stereotypes.
|
83 |
-
- **Text Processing Issues:** UTF-8 encoding applied to prevent character errors.
|
84 |
-
|
85 |
-
## ๐ฎ Future Improvements
|
86 |
-
- **Enhancing Creativity:** Fine-tune temperature and randomness settings.
|
87 |
-
- **Genre-Specific Training:** Introduce theme-based datasets.
|
88 |
-
- **Larger Model Training:** Experiment with **GPT-2 (355M)** for richer storytelling.
|
89 |
-
|
90 |
-
## ๐ค Contributors
|
91 |
-
- **Charla Pia Vella** (Project Developer)
|
92 |
-
- **Affiliation:** ARI3333 Generative AI
|
93 |
-
|
94 |
-
## ๐ License
|
95 |
-
This project is released under the **Apache-2.0 License**.
|
96 |
-
|
97 |
-
## ๐ Acknowledgments
|
98 |
-
- **OpenAI** for GPT-2
|
99 |
-
- **Hugging Face** for the fine-tuning framework
|
100 |
-
- **Ronen Eldan** for the TinyStories dataset
|
101 |
-
|
102 |
-
๐ For more details, visit the [Hugging Face Space](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator).
|
103 |
-
|
104 |
|
105 |
|
106 |
|
|
|
12 |
---
|
13 |
|
14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
|
16 |
|
17 |
|