cpv2280 commited on
Commit
8ae6c4b
ยท
verified ยท
1 Parent(s): 0bd8929

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -89
README.md CHANGED
@@ -12,95 +12,6 @@ short_description: Gradio
12
  ---
13
 
14
 
15
- ---
16
- title: GPT-2 TinyStories Generator
17
- emoji: ๐Ÿ“–
18
- colorFrom: yellow
19
- colorTo: red
20
- sdk: gradio
21
- sdk_version: 5.13.2
22
- app_file: app.py
23
- pinned: false
24
- license: apache-2.0
25
- short_description: A fine-tuned GPT-2 model for generating structured, child-friendly short stories.
26
- ---
27
-
28
- # GPT-2 TinyStories Generator ๐Ÿ“– FableWeaver AI
29
-
30
- ## Overview
31
- This project fine-tunes a **GPT-2 model** on the **TinyStories dataset** to generate structured, coherent, and engaging short narratives. The model is hosted on **Hugging Face Spaces** and provides a user-friendly interface for story generation.
32
-
33
- ## Features
34
- - โœ… **Story Generation:** Produces structured, child-friendly short stories.
35
- - โœ… **Bias Monitoring:** Ensures balanced gender and cultural representation.
36
- - โœ… **Efficient Training:** Fine-tuned on **200,000 training samples** and **20,000 test samples**.
37
- - โœ… **Grammar & Readability Enhancements:** Integrated grammar-checking tools and text refinement.
38
- - โœ… **Optimized Performance:** Uses loss tracking, sampling techniques, and bias mitigation strategies.
39
-
40
- ## ๐Ÿš€ System Architecture
41
- The model is designed for easy interaction via Hugging Face Spaces and follows this workflow:
42
- 1. **Data Preprocessing & Cleaning**
43
- - Tokenization, formatting, and encoding normalization.
44
- - Bias mitigation and balanced data preparation.
45
- 2. **Model Fine-Tuning**
46
- - Fine-tuned **GPT-2 (124M parameters)** using **Hugging Face Transformers**.
47
- - Hyperparameter optimization (batch size, learning rate, weight decay).
48
- 3. **Story Generation Pipeline**
49
- - Uses **top-k filtering (k=50)**, **top-p nucleus sampling (p=0.9)**, and **temperature adjustments**.
50
- 4. **Post-Processing & Bias Mitigation**
51
- - Contextual reinforcement and diversity-aware storytelling.
52
- - No-repeat n-gram settings and logical scene transitions.
53
- 5. **Evaluation & Performance Monitoring**
54
- - Tracked using **Weights & Biases (W&B)** and **TensorBoard**.
55
- - Analyzed validation loss and coherence checks.
56
-
57
- ## ๐Ÿ”น Getting Started
58
- ### Accessing the Model
59
- The model is available on **Hugging Face Spaces**: [GPT-2 TinyStories Generator](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator)
60
-
61
- ### Usage Instructions
62
- 1. Visit the **Hugging Face Space** linked above.
63
- 2. Enter a prompt (e.g., *"Once upon a time..."*) in the input field.
64
- 3. Click **Generate** to receive an AI-generated short story.
65
- 4. Modify the prompt and settings (temperature, top-k, top-p) for different results.
66
-
67
- ## ๐Ÿ“Š Training Details
68
- - **Model:** GPT-2 (124M)
69
- - **Dataset:** RonenEldan/TinyStories
70
- - **Training:** 3 epochs on Google Colab GPU (T4)
71
- - **Loss Metrics:**
72
- - Training Loss: **3.08 โ†’ 2.86**
73
- - Validation Loss: **1.46 โ†’ 1.40**
74
-
75
- ## ๐Ÿ“ Evaluation & Observations
76
- - **Perplexity improved from 8.12 โ†’ 2.09**, indicating better text fluency.
77
- - **Validation loss decreased consistently**, suggesting effective generalization.
78
- - **Human evaluation highlighted minor inconsistencies**, such as abrupt scene shifts and simplistic narratives.
79
-
80
- ## โš–๏ธ Ethical Considerations
81
- - **Bias Monitoring:** Pronoun analysis and diversity checks to ensure fairness.
82
- - **Harmful Content Mitigation:** Manually reviewed outputs for stereotypes.
83
- - **Text Processing Issues:** UTF-8 encoding applied to prevent character errors.
84
-
85
- ## ๐Ÿ”ฎ Future Improvements
86
- - **Enhancing Creativity:** Fine-tune temperature and randomness settings.
87
- - **Genre-Specific Training:** Introduce theme-based datasets.
88
- - **Larger Model Training:** Experiment with **GPT-2 (355M)** for richer storytelling.
89
-
90
- ## ๐Ÿค Contributors
91
- - **Charla Pia Vella** (Project Developer)
92
- - **Affiliation:** ARI3333 Generative AI
93
-
94
- ## ๐Ÿ“œ License
95
- This project is released under the **Apache-2.0 License**.
96
-
97
- ## ๐ŸŽ“ Acknowledgments
98
- - **OpenAI** for GPT-2
99
- - **Hugging Face** for the fine-tuning framework
100
- - **Ronen Eldan** for the TinyStories dataset
101
-
102
- ๐Ÿ“Œ For more details, visit the [Hugging Face Space](https://huggingface.co/spaces/cpv2280/gpt2-tinystories-generator).
103
-
104
 
105
 
106
 
 
12
  ---
13
 
14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
 
17