cpv2280 commited on
Commit
0bd8929
ยท
verified ยท
1 Parent(s): 763b6bb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +83 -115
README.md CHANGED
@@ -10,129 +10,97 @@ pinned: false
10
  license: apache-2.0
11
  short_description: Gradio
12
  ---
13
- GPT-2 TinyStories Generator ๐Ÿข - FableWeaver AI
14
 
15
- Overview
16
 
17
- This project fine-tunes a GPT-2 model on the TinyStories dataset to generate structured, coherent, and engaging short narratives. The model is hosted on Hugging Face Spaces and provides a user-friendly interface for story generation.
18
-
19
- Features
20
-
21
- โœ… Story Generation: Produces structured, child-friendly short stories.
22
-
23
- โœ… Bias Monitoring: Ensures balanced gender and cultural representation.
24
-
25
- โœ… Efficient Training: Fine-tuned on 200,000 training samples and 20,000 test samples.
 
 
 
26
 
27
- โœ… Grammar & Readability Enhancements: Integrated grammar-checking tools and text refinement.
28
 
29
- โœ… Optimized Performance: Uses loss tracking, sampling techniques, and bias mitigation strategies.
 
30
 
31
- ๐Ÿš€ System Architecture
 
 
 
 
 
32
 
 
33
  The model is designed for easy interaction via Hugging Face Spaces and follows this workflow:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
- Data Preprocessing & Cleaning
36
-
37
- Tokenization, formatting, and encoding normalization.
38
-
39
- Bias mitigation and balanced data preparation.
40
-
41
- Model Fine-Tuning
42
-
43
- Fine-tuned GPT-2 (124M parameters) using Hugging Face Transformers.
44
-
45
- Hyperparameter optimization (batch size, learning rate, weight decay).
46
-
47
- Story Generation Pipeline
48
-
49
- Uses top-k filtering (k=50), top-p nucleus sampling (p=0.9), and temperature adjustments.
50
-
51
- Post-Processing & Bias Mitigation
52
-
53
- Contextual reinforcement and diversity-aware storytelling.
54
-
55
- No-repeat n-gram settings and logical scene transitions.
56
-
57
- Evaluation & Performance Monitoring
58
-
59
- Tracked using Weights & Biases (W&B) and TensorBoard.
60
-
61
- Analyzed validation loss and coherence checks.
62
-
63
- ๐Ÿ”น Getting Started
64
-
65
- Accessing the Model
66
-
67
- The model is available on Hugging Face Spaces: GPT-2 TinyStories Generator
68
-
69
- Usage Instructions
70
-
71
- Visit the Hugging Face Space linked above.
72
-
73
- Enter a prompt (e.g., "Once upon a time...") in the input field.
74
-
75
- Click Generate to receive an AI-generated short story.
76
-
77
- Modify the prompt and settings (temperature, top-k, top-p) for different results.
78
-
79
- ๐Ÿ“Š Training Details
80
-
81
- Model: GPT-2 (124M)
82
-
83
- Dataset: RonenEldan/TinyStories
84
-
85
- Training: 3 epochs on Google Colab GPU (T4)
86
-
87
- Loss Metrics:
88
-
89
- Training Loss: 3.08 โ†’ 2.86
90
-
91
- Validation Loss: 1.46 โ†’ 1.40
92
-
93
- ๐Ÿ“ Evaluation & Observations
94
-
95
- Perplexity improved from 8.12 โ†’ 2.09, indicating better text fluency.
96
-
97
- Validation loss decreased consistently, suggesting effective generalization.
98
-
99
- Human evaluation highlighted minor inconsistencies, such as abrupt scene shifts and simplistic narratives.
100
-
101
- โš–๏ธ Ethical Considerations
102
-
103
- Bias Monitoring: Pronoun analysis and diversity checks to ensure fairness.
104
-
105
- Harmful Content Mitigation: Manually reviewed outputs for stereotypes.
106
-
107
- Text Processing Issues: UTF-8 encoding applied to prevent character errors.
108
-
109
- ๐Ÿ”ฎ Future Improvements
110
-
111
- Enhancing Creativity: Fine-tune temperature and randomness settings.
112
-
113
- Genre-Specific Training: Introduce theme-based datasets.
114
-
115
- Larger Model Training: Experiment with GPT-2 (355M) for richer storytelling.
116
-
117
- ๐Ÿค Contributors
118
-
119
- Charla Pia Vella (Project Developer)
120
-
121
- Affiliation: ARI3333 Generative AI
122
-
123
- ๐Ÿ“œ License
124
-
125
- This project is released under the Apache-2.0 License.
126
-
127
- ๐ŸŽ“ Acknowledgments
128
-
129
- OpenAI for GPT-2
130
-
131
- Hugging Face for the fine-tuning framework
132
-
133
- Ronen Eldan for the TinyStories dataset
134
-
135
- ๐Ÿ“Œ For more details, visit the Hugging Face Space.
136
 
137
 
138
 
 
10
  license: apache-2.0
11
  short_description: Gradio
12
  ---
 
13
 
 
14
 
15
+ ---
16
+ title: GPT-2 TinyStories Generator
17
+ emoji: ๐Ÿ“–
18
+ colorFrom: yellow
19
+ colorTo: red
20
+ sdk: gradio
21
+ sdk_version: 5.13.2
22
+ app_file: app.py
23
+ pinned: false
24
+ license: apache-2.0
25
+ short_description: A fine-tuned GPT-2 model for generating structured, child-friendly short stories.
26
+ ---
27
 
28
+ # GPT-2 TinyStories Generator ๐Ÿ“– FableWeaver AI
29
 
30
+ ## Overview
31
+ This project fine-tunes a **GPT-2 model** on the **TinyStories dataset** to generate structured, coherent, and engaging short narratives. The model is hosted on **Hugging Face Spaces** and provides a user-friendly interface for story generation.
32
 
33
+ ## Features
34
+ - โœ… **Story Generation:** Produces structured, child-friendly short stories.
35
+ - โœ… **Bias Monitoring:** Ensures balanced gender and cultural representation.
36
+ - โœ… **Efficient Training:** Fine-tuned on **200,000 training samples** and **20,000 test samples**.
37
+ - โœ… **Grammar & Readability Enhancements:** Integrated grammar-checking tools and text refinement.
38
+ - โœ… **Optimized Performance:** Uses loss tracking, sampling techniques, and bias mitigation strategies.
39
 
40
+ ## ๐Ÿš€ System Architecture
41
  The model is designed for easy interaction via Hugging Face Spaces and follows this workflow:
42
+ 1. **Data Preprocessing & Cleaning**
43
+ - Tokenization, formatting, and encoding normalization.
44
+ - Bias mitigation and balanced data preparation.
45
+ 2. **Model Fine-Tuning**
46
+ - Fine-tuned **GPT-2 (124M parameters)** using **Hugging Face Transformers**.
47
+ - Hyperparameter optimization (batch size, learning rate, weight decay).
48
+ 3. **Story Generation Pipeline**
49
+ - Uses **top-k filtering (k=50)**, **top-p nucleus sampling (p=0.9)**, and **temperature adjustments**.
50
+ 4. **Post-Processing & Bias Mitigation**
51
+ - Contextual reinforcement and diversity-aware storytelling.
52
+ - No-repeat n-gram settings and logical scene transitions.
53
+ 5. **Evaluation & Performance Monitoring**
54
+ - Tracked using **Weights & Biases (W&B)** and **TensorBoard**.
55
+ - Analyzed validation loss and coherence checks.
56
+
57
+ ## ๐Ÿ”น Getting Started
58
+ ### Accessing the Model
59
+ The model is available on **Hugging Face Spaces**: [GPT-2 TinyStories Generator](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator)
60
+
61
+ ### Usage Instructions
62
+ 1. Visit the **Hugging Face Space** linked above.
63
+ 2. Enter a prompt (e.g., *"Once upon a time..."*) in the input field.
64
+ 3. Click **Generate** to receive an AI-generated short story.
65
+ 4. Modify the prompt and settings (temperature, top-k, top-p) for different results.
66
+
67
+ ## ๐Ÿ“Š Training Details
68
+ - **Model:** GPT-2 (124M)
69
+ - **Dataset:** RonenEldan/TinyStories
70
+ - **Training:** 3 epochs on Google Colab GPU (T4)
71
+ - **Loss Metrics:**
72
+ - Training Loss: **3.08 โ†’ 2.86**
73
+ - Validation Loss: **1.46 โ†’ 1.40**
74
+
75
+ ## ๐Ÿ“ Evaluation & Observations
76
+ - **Perplexity improved from 8.12 โ†’ 2.09**, indicating better text fluency.
77
+ - **Validation loss decreased consistently**, suggesting effective generalization.
78
+ - **Human evaluation highlighted minor inconsistencies**, such as abrupt scene shifts and simplistic narratives.
79
+
80
+ ## โš–๏ธ Ethical Considerations
81
+ - **Bias Monitoring:** Pronoun analysis and diversity checks to ensure fairness.
82
+ - **Harmful Content Mitigation:** Manually reviewed outputs for stereotypes.
83
+ - **Text Processing Issues:** UTF-8 encoding applied to prevent character errors.
84
+
85
+ ## ๐Ÿ”ฎ Future Improvements
86
+ - **Enhancing Creativity:** Fine-tune temperature and randomness settings.
87
+ - **Genre-Specific Training:** Introduce theme-based datasets.
88
+ - **Larger Model Training:** Experiment with **GPT-2 (355M)** for richer storytelling.
89
+
90
+ ## ๐Ÿค Contributors
91
+ - **Charla Pia Vella** (Project Developer)
92
+ - **Affiliation:** ARI3333 Generative AI
93
+
94
+ ## ๐Ÿ“œ License
95
+ This project is released under the **Apache-2.0 License**.
96
+
97
+ ## ๐ŸŽ“ Acknowledgments
98
+ - **OpenAI** for GPT-2
99
+ - **Hugging Face** for the fine-tuning framework
100
+ - **Ronen Eldan** for the TinyStories dataset
101
+
102
+ ๐Ÿ“Œ For more details, visit the [Hugging Face Space](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator).
103
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
104
 
105
 
106