prasoonmhwr commited on
Commit
6b22ff5
·
verified ·
1 Parent(s): c4d5c1d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +149 -3
README.md CHANGED
@@ -1,3 +1,149 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ datasets:
4
+ - ahmadreza13/human-vs-Ai-generated-dataset
5
+ ---
6
+ # AI-Generated Content Detection Model
7
+
8
+ ## Model Description
9
+
10
+ This model is designed to detect AI-generated content by analyzing text using a combination of RoBERTa embeddings, Word2Vec embeddings, and engineered linguistic features.
11
+
12
+ ## Model Architecture
13
+
14
+ The model utilizes a hybrid architecture that combines:
15
+
16
+ - **RoBERTa Base**: For contextual text embeddings
17
+ - **Word2Vec Embeddings**: For additional semantic information
18
+ - **Engineered Linguistic Features**: Including sentiment analysis metrics, readability scores, and lexical diversity
19
+
20
+ The model architecture consists of:
21
+ - A pre-trained RoBERTa base model with the first 6 layers frozen
22
+ - Gradient checkpointing enabled for memory efficiency
23
+ - A fully connected network that combines RoBERTa embeddings with Word2Vec and engineered features
24
+ - Three fully connected layers (512 → 128 → 1) with ReLU activations and dropout
25
+
26
+ ## Training Information
27
+
28
+ - **Dataset**: https://huggingface.co/datasets/ahmadreza13/human-vs-Ai-generated-dataset
29
+ - **Training Strategy**:
30
+ - Mixed precision training with gradient accumulation
31
+ - OneCycleLR learning rate scheduler
32
+ - Early stopping based on validation F1 score
33
+ - **Hyperparameters**:
34
+ - Learning rate: 3e-5
35
+ - Batch size: 32
36
+ - Gradient accumulation steps: 2
37
+ - Dropout rate: 0.3
38
+ - Training epochs: Up to 3 with early stopping
39
+
40
+ ## Performance Metrics
41
+
42
+ | Metric | Score |
43
+ |-----------|-------|
44
+ | Precision | {f1:0.9979} |
45
+ | Recall | {f1:0.9989} |
46
+ | F1 Score | {f1:0.997} |
47
+ | ROC AUC | {roc_auc:0.998} |
48
+
49
+ ## Limitations
50
+
51
+ - The model's performance may vary based on the type of AI-generated content, as different AI models produce text with different characteristics
52
+ - Performance may be reduced on highly technical or domain-specific content that wasn't well-represented in the training data
53
+ - The model may produce occasional false positives on human-written content that exhibits unusually high coherence or consistency
54
+
55
+ ## Ethics & Responsible Use
56
+
57
+ This model is intended to be used as a tool for:
58
+ - Research on AI-generated content characteristics
59
+ - Content moderation and filtration where transparency about content source is important
60
+ - Educational purposes to understand differences between human and AI-written content
61
+
62
+ This model should NOT be used to:
63
+ - Make high-stakes decisions without human oversight
64
+ - Discriminate against content creators
65
+ - Falsely attribute content to AI or humans with absolute certainty
66
+
67
+ ## Usage Examples
68
+
69
+ ```python
70
+ from transformers import AutoTokenizer, AutoModel
71
+ import torch
72
+ import numpy as np
73
+
74
+ # Load model and tokenizer
75
+ from transformers import RobertaTokenizer, AutoModelForSequenceClassification
76
+ import torch
77
+
78
+ def predict_with_huggingface_model(text, repo_id="prasoonmhwr/ai_detection_model", device="cuda"):
79
+ """
80
+ Predicts using a model from the Hugging Face Model Hub.
81
+
82
+ Args:
83
+ text (str): The text to predict on.
84
+ repo_id (str): The repository ID of the model on Hugging Face Hub.
85
+ device (str): "cuda" if GPU is available, "cpu" otherwise
86
+
87
+ Returns:
88
+ float: The prediction probability (between 0 and 1).
89
+ """
90
+ # 1. Load the tokenizer
91
+ tokenizer = RobertaTokenizer.from_pretrained(repo_id)
92
+
93
+ # 2. Load the model
94
+ model = AutoModelForSequenceClassification.from_pretrained(repo_id).to(device)
95
+ model.eval() # Set the model to evaluation mode
96
+
97
+ # 3. Tokenize the input text
98
+ inputs = tokenizer(text,
99
+ add_special_tokens=True,
100
+ max_length=128,
101
+ padding='max_length',
102
+ truncation=True,
103
+ return_tensors='pt').to(device) # Move inputs to device
104
+
105
+ # 4. Make the prediction (no gradient calculation needed)
106
+ with torch.no_grad():
107
+ outputs = model(**inputs)
108
+ logits = outputs.logits
109
+ probabilities = torch.sigmoid(logits).cpu().numpy().flatten() # Get probabilities, move to CPU
110
+
111
+ return probabilities[0] # Return the probability for the positive class
112
+
113
+
114
+ if __name__ == '__main__':
115
+ # Example usage:
116
+ text_to_predict = "This is a sample text to check if it was written by a human or AI"
117
+ # text_to_predict = "This text was generated by an AI model." # uncomment to test on an AI generated text
118
+
119
+ # Set the device
120
+ device = "cuda" if torch.cuda.is_available() else "cpu"
121
+
122
+ repo_id = "prasoonmhwr/ai_detection_model"
123
+
124
+ # Make the prediction
125
+ prediction = predict_with_huggingface_model(text_to_predict, repo_id, device)
126
+
127
+ # Print the result
128
+ print(f"Text: '{text_to_predict}'")
129
+ print(f"Prediction (Probability of being AI-generated): {prediction:.4f}")
130
+
131
+ if prediction > 0.5:
132
+ print("The model predicts this text is likely AI-generated.")
133
+ else:
134
+ print("The model predicts this text is likely human-generated.")
135
+ ```
136
+
137
+ ## Citation
138
+
139
+ If you use this model in your research, please cite:
140
+
141
+ ```
142
+ @misc{ai_detection_model,
143
+ author = {Prasoon Mahawar},
144
+ title = {AI-Generated Content Detection Model},
145
+ year = {2025},
146
+ publisher = {HuggingFace},
147
+ url = {https://huggingface.co/prasoonmhwr/ai_detection_model}
148
+ }
149
+ ```