Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
@@ -10,129 +10,97 @@ pinned: false
|
|
10 |
license: apache-2.0
|
11 |
short_description: Gradio
|
12 |
---
|
13 |
-
GPT-2 TinyStories Generator ๐ข - FableWeaver AI
|
14 |
|
15 |
-
Overview
|
16 |
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
22 |
-
|
23 |
-
|
24 |
-
|
25 |
-
|
|
|
|
|
|
|
26 |
|
27 |
-
|
28 |
|
29 |
-
|
|
|
30 |
|
31 |
-
|
|
|
|
|
|
|
|
|
|
|
32 |
|
|
|
33 |
The model is designed for easy interaction via Hugging Face Spaces and follows this workflow:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
34 |
|
35 |
-
Data Preprocessing & Cleaning
|
36 |
-
|
37 |
-
Tokenization, formatting, and encoding normalization.
|
38 |
-
|
39 |
-
Bias mitigation and balanced data preparation.
|
40 |
-
|
41 |
-
Model Fine-Tuning
|
42 |
-
|
43 |
-
Fine-tuned GPT-2 (124M parameters) using Hugging Face Transformers.
|
44 |
-
|
45 |
-
Hyperparameter optimization (batch size, learning rate, weight decay).
|
46 |
-
|
47 |
-
Story Generation Pipeline
|
48 |
-
|
49 |
-
Uses top-k filtering (k=50), top-p nucleus sampling (p=0.9), and temperature adjustments.
|
50 |
-
|
51 |
-
Post-Processing & Bias Mitigation
|
52 |
-
|
53 |
-
Contextual reinforcement and diversity-aware storytelling.
|
54 |
-
|
55 |
-
No-repeat n-gram settings and logical scene transitions.
|
56 |
-
|
57 |
-
Evaluation & Performance Monitoring
|
58 |
-
|
59 |
-
Tracked using Weights & Biases (W&B) and TensorBoard.
|
60 |
-
|
61 |
-
Analyzed validation loss and coherence checks.
|
62 |
-
|
63 |
-
๐น Getting Started
|
64 |
-
|
65 |
-
Accessing the Model
|
66 |
-
|
67 |
-
The model is available on Hugging Face Spaces: GPT-2 TinyStories Generator
|
68 |
-
|
69 |
-
Usage Instructions
|
70 |
-
|
71 |
-
Visit the Hugging Face Space linked above.
|
72 |
-
|
73 |
-
Enter a prompt (e.g., "Once upon a time...") in the input field.
|
74 |
-
|
75 |
-
Click Generate to receive an AI-generated short story.
|
76 |
-
|
77 |
-
Modify the prompt and settings (temperature, top-k, top-p) for different results.
|
78 |
-
|
79 |
-
๐ Training Details
|
80 |
-
|
81 |
-
Model: GPT-2 (124M)
|
82 |
-
|
83 |
-
Dataset: RonenEldan/TinyStories
|
84 |
-
|
85 |
-
Training: 3 epochs on Google Colab GPU (T4)
|
86 |
-
|
87 |
-
Loss Metrics:
|
88 |
-
|
89 |
-
Training Loss: 3.08 โ 2.86
|
90 |
-
|
91 |
-
Validation Loss: 1.46 โ 1.40
|
92 |
-
|
93 |
-
๐ Evaluation & Observations
|
94 |
-
|
95 |
-
Perplexity improved from 8.12 โ 2.09, indicating better text fluency.
|
96 |
-
|
97 |
-
Validation loss decreased consistently, suggesting effective generalization.
|
98 |
-
|
99 |
-
Human evaluation highlighted minor inconsistencies, such as abrupt scene shifts and simplistic narratives.
|
100 |
-
|
101 |
-
โ๏ธ Ethical Considerations
|
102 |
-
|
103 |
-
Bias Monitoring: Pronoun analysis and diversity checks to ensure fairness.
|
104 |
-
|
105 |
-
Harmful Content Mitigation: Manually reviewed outputs for stereotypes.
|
106 |
-
|
107 |
-
Text Processing Issues: UTF-8 encoding applied to prevent character errors.
|
108 |
-
|
109 |
-
๐ฎ Future Improvements
|
110 |
-
|
111 |
-
Enhancing Creativity: Fine-tune temperature and randomness settings.
|
112 |
-
|
113 |
-
Genre-Specific Training: Introduce theme-based datasets.
|
114 |
-
|
115 |
-
Larger Model Training: Experiment with GPT-2 (355M) for richer storytelling.
|
116 |
-
|
117 |
-
๐ค Contributors
|
118 |
-
|
119 |
-
Charla Pia Vella (Project Developer)
|
120 |
-
|
121 |
-
Affiliation: ARI3333 Generative AI
|
122 |
-
|
123 |
-
๐ License
|
124 |
-
|
125 |
-
This project is released under the Apache-2.0 License.
|
126 |
-
|
127 |
-
๐ Acknowledgments
|
128 |
-
|
129 |
-
OpenAI for GPT-2
|
130 |
-
|
131 |
-
Hugging Face for the fine-tuning framework
|
132 |
-
|
133 |
-
Ronen Eldan for the TinyStories dataset
|
134 |
-
|
135 |
-
๐ For more details, visit the Hugging Face Space.
|
136 |
|
137 |
|
138 |
|
|
|
10 |
license: apache-2.0
|
11 |
short_description: Gradio
|
12 |
---
|
|
|
13 |
|
|
|
14 |
|
15 |
+
---
|
16 |
+
title: GPT-2 TinyStories Generator
|
17 |
+
emoji: ๐
|
18 |
+
colorFrom: yellow
|
19 |
+
colorTo: red
|
20 |
+
sdk: gradio
|
21 |
+
sdk_version: 5.13.2
|
22 |
+
app_file: app.py
|
23 |
+
pinned: false
|
24 |
+
license: apache-2.0
|
25 |
+
short_description: A fine-tuned GPT-2 model for generating structured, child-friendly short stories.
|
26 |
+
---
|
27 |
|
28 |
+
# GPT-2 TinyStories Generator ๐ FableWeaver AI
|
29 |
|
30 |
+
## Overview
|
31 |
+
This project fine-tunes a **GPT-2 model** on the **TinyStories dataset** to generate structured, coherent, and engaging short narratives. The model is hosted on **Hugging Face Spaces** and provides a user-friendly interface for story generation.
|
32 |
|
33 |
+
## Features
|
34 |
+
- โ
**Story Generation:** Produces structured, child-friendly short stories.
|
35 |
+
- โ
**Bias Monitoring:** Ensures balanced gender and cultural representation.
|
36 |
+
- โ
**Efficient Training:** Fine-tuned on **200,000 training samples** and **20,000 test samples**.
|
37 |
+
- โ
**Grammar & Readability Enhancements:** Integrated grammar-checking tools and text refinement.
|
38 |
+
- โ
**Optimized Performance:** Uses loss tracking, sampling techniques, and bias mitigation strategies.
|
39 |
|
40 |
+
## ๐ System Architecture
|
41 |
The model is designed for easy interaction via Hugging Face Spaces and follows this workflow:
|
42 |
+
1. **Data Preprocessing & Cleaning**
|
43 |
+
- Tokenization, formatting, and encoding normalization.
|
44 |
+
- Bias mitigation and balanced data preparation.
|
45 |
+
2. **Model Fine-Tuning**
|
46 |
+
- Fine-tuned **GPT-2 (124M parameters)** using **Hugging Face Transformers**.
|
47 |
+
- Hyperparameter optimization (batch size, learning rate, weight decay).
|
48 |
+
3. **Story Generation Pipeline**
|
49 |
+
- Uses **top-k filtering (k=50)**, **top-p nucleus sampling (p=0.9)**, and **temperature adjustments**.
|
50 |
+
4. **Post-Processing & Bias Mitigation**
|
51 |
+
- Contextual reinforcement and diversity-aware storytelling.
|
52 |
+
- No-repeat n-gram settings and logical scene transitions.
|
53 |
+
5. **Evaluation & Performance Monitoring**
|
54 |
+
- Tracked using **Weights & Biases (W&B)** and **TensorBoard**.
|
55 |
+
- Analyzed validation loss and coherence checks.
|
56 |
+
|
57 |
+
## ๐น Getting Started
|
58 |
+
### Accessing the Model
|
59 |
+
The model is available on **Hugging Face Spaces**: [GPT-2 TinyStories Generator](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator)
|
60 |
+
|
61 |
+
### Usage Instructions
|
62 |
+
1. Visit the **Hugging Face Space** linked above.
|
63 |
+
2. Enter a prompt (e.g., *"Once upon a time..."*) in the input field.
|
64 |
+
3. Click **Generate** to receive an AI-generated short story.
|
65 |
+
4. Modify the prompt and settings (temperature, top-k, top-p) for different results.
|
66 |
+
|
67 |
+
## ๐ Training Details
|
68 |
+
- **Model:** GPT-2 (124M)
|
69 |
+
- **Dataset:** RonenEldan/TinyStories
|
70 |
+
- **Training:** 3 epochs on Google Colab GPU (T4)
|
71 |
+
- **Loss Metrics:**
|
72 |
+
- Training Loss: **3.08 โ 2.86**
|
73 |
+
- Validation Loss: **1.46 โ 1.40**
|
74 |
+
|
75 |
+
## ๐ Evaluation & Observations
|
76 |
+
- **Perplexity improved from 8.12 โ 2.09**, indicating better text fluency.
|
77 |
+
- **Validation loss decreased consistently**, suggesting effective generalization.
|
78 |
+
- **Human evaluation highlighted minor inconsistencies**, such as abrupt scene shifts and simplistic narratives.
|
79 |
+
|
80 |
+
## โ๏ธ Ethical Considerations
|
81 |
+
- **Bias Monitoring:** Pronoun analysis and diversity checks to ensure fairness.
|
82 |
+
- **Harmful Content Mitigation:** Manually reviewed outputs for stereotypes.
|
83 |
+
- **Text Processing Issues:** UTF-8 encoding applied to prevent character errors.
|
84 |
+
|
85 |
+
## ๐ฎ Future Improvements
|
86 |
+
- **Enhancing Creativity:** Fine-tune temperature and randomness settings.
|
87 |
+
- **Genre-Specific Training:** Introduce theme-based datasets.
|
88 |
+
- **Larger Model Training:** Experiment with **GPT-2 (355M)** for richer storytelling.
|
89 |
+
|
90 |
+
## ๐ค Contributors
|
91 |
+
- **Charla Pia Vella** (Project Developer)
|
92 |
+
- **Affiliation:** ARI3333 Generative AI
|
93 |
+
|
94 |
+
## ๐ License
|
95 |
+
This project is released under the **Apache-2.0 License**.
|
96 |
+
|
97 |
+
## ๐ Acknowledgments
|
98 |
+
- **OpenAI** for GPT-2
|
99 |
+
- **Hugging Face** for the fine-tuning framework
|
100 |
+
- **Ronen Eldan** for the TinyStories dataset
|
101 |
+
|
102 |
+
๐ For more details, visit the [Hugging Face Space](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator).
|
103 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
104 |
|
105 |
|
106 |
|