Image Classification
Transformers
Safetensors
PyTorch
timm_wrapper
Not-For-All-Audiences
ccabrerafreepik commited on
Commit
a6f99f9
·
1 Parent(s): cda8ddf

Improve readme

Browse files
Files changed (1) hide show
  1. README.md +24 -18
README.md CHANGED
@@ -35,7 +35,7 @@ The model can be used as a **binary (true/false) classifier if desired, or you c
35
 
36
  ### Global Performance
37
 
38
- | Category | Freepik | Falconasai | Adamcodd |
39
  |----------|-------------|------------------|----------------|
40
  | High | 99.54% | 97.92% | 98.62% |
41
  | Medium | 97.02% | 78.54% | 91.65% |
@@ -43,10 +43,23 @@ The model can be used as a **binary (true/false) classifier if desired, or you c
43
  | Neutral | 99.87% | 99.27% | 98.37% |
44
 
45
 
46
- **Notes:**
47
- * Values represent the percentage of correct predictions. For Falconasai and AdamCodd, these are True/False predictions, while for the Freepik model, they represent detecting at least 'low' level content.
48
- * AdamCodd and Falconasai show lower accuracy on intermediate categories because they are designed for binary classification only.
49
- * Our model outperforms both models in accuracy and granularity.
 
 
 
 
 
 
 
 
 
 
 
 
 
50
 
51
  ### Accuracy by AI Content
52
 
@@ -56,25 +69,18 @@ The following tables show detection accuracy percentages across different NSFW c
56
 
57
  #### AI-Generated Content
58
 
59
- | Category | Freepik Model | Falconasai Model | Adamcodd Model |
60
  |----------|-------------|------------------|----------------|
61
  | High | 100.00% | 84.00% | 92.00% |
62
  | Medium | 96.15% | 69.23% | 96.00% |
63
  | Low | 100.00% | 35.71% | 92.86% |
64
  | Neutral | 100.00% | 100.00% | 66.67% |
65
 
66
- ### Non-AI-Generated Content
67
-
68
- | Category | Freepik Model | Falconasai Model | Adamcodd Model |
69
- |----------|-------------|------------------|----------------|
70
- | High | 100.00% | 100.00% | 100.00% |
71
- | Medium | 100.00% | 70.00% | 100.00% |
72
- | Low | 100.00% | 0.00% | 100.00% |
73
- | Neutral | 100.00% | 88.88% | 100.00% |
74
 
75
- **Notes:**
 
 
76
 
77
- * Due to the small dataset size, some categories show 100.00% accuracy in some cases, expect less accuracy in real world
78
 
79
  ## Usage
80
 
@@ -95,7 +101,7 @@ detector = NSFWDetector(dtype=torch.bfloat16, device="cuda")
95
  # Load and classify an image
96
  image = Image.open("your_image")
97
 
98
- # Check if the image contains NSFW content sentivity level medium
99
  is_nsfw = detector.is_nsfw(image, "medium")
100
 
101
  # Get probability scores for all categories
@@ -261,7 +267,7 @@ Example output:
261
 
262
  **Notes:**
263
  * The model has been trained in bf16 so it is **recommended to use it in bf16**.
264
- * **Using torch tensor**: The speed using torch tensor is not achieved using pipeline. Avoid it in production.
265
  * Measurements taken on **NVIDIA RTX 3090**, expect better metrics in more powerful servers.
266
  * Throughput increases with larger batch sizes due to better GPU utilization. Consider your use case when selecting batch size.
267
  * Optimizations listed are suggestions that could further improve performance.
 
35
 
36
  ### Global Performance
37
 
38
+ | Category | Freepik | Falconsai | Adamcodd |
39
  |----------|-------------|------------------|----------------|
40
  | High | 99.54% | 97.92% | 98.62% |
41
  | Medium | 97.02% | 78.54% | 91.65% |
 
43
  | Neutral | 99.87% | 99.27% | 98.37% |
44
 
45
 
46
+ In the table below, the results are obtained as follows:
47
+
48
+ * For the **Falconsai and AdamCodd** models:
49
+ * A prediction is considered correct if the image is labeled "low", "medium", or "high" and the model returns true.
50
+ * If the label is "neutral", the correct output should be false.
51
+
52
+ * For the **Freepik model**:
53
+ * If the image label is "low", "medium", or "high", the model should return at least "low".
54
+ * If the label is "neutral", the correct output should be "neutral".
55
+
56
+
57
+ **Conclusions:**
58
+
59
+ * Our model **outperforms AdamCodd and Falconsai in accuracy**. It is entirely fair to compare them on the "high" and "neutral" labels.
60
+ * Our model **offers greater granularity**. It is not only suitable for detecting "high" and "neutral" content, but also performs excellently at identifying "low" and "medium" NSFW content.
61
+ * Falconsai may classify some "medium" and "low" images as not NSFW but mark others as safe for work(SFW), which could lead to unexpected results.
62
+ * AdamCodd classifies both "low" and "medium" categories as NSFW, which may not be desirable depending on your use case. Furthermore, a 10% of images in low and medium are considered SFW.
63
 
64
  ### Accuracy by AI Content
65
 
 
69
 
70
  #### AI-Generated Content
71
 
72
+ | Category | Freepik Model | Falconsai Model | Adamcodd Model |
73
  |----------|-------------|------------------|----------------|
74
  | High | 100.00% | 84.00% | 92.00% |
75
  | Medium | 96.15% | 69.23% | 96.00% |
76
  | Low | 100.00% | 35.71% | 92.86% |
77
  | Neutral | 100.00% | 100.00% | 66.67% |
78
 
 
 
 
 
 
 
 
 
79
 
80
+ **Conclusions:**
81
+ * **Avoid using Falconsai for AI-generated content** to prevent prediction errors.
82
+ * **Our model is the best option to detect NSFW content in AI-generated content**.
83
 
 
84
 
85
  ## Usage
86
 
 
101
  # Load and classify an image
102
  image = Image.open("your_image")
103
 
104
+ # Check if the image contains NSFW content sentivity level medium or higher
105
  is_nsfw = detector.is_nsfw(image, "medium")
106
 
107
  # Get probability scores for all categories
 
267
 
268
  **Notes:**
269
  * The model has been trained in bf16 so it is **recommended to use it in bf16**.
270
+ * **Using torch tensor**: The speed using torch tensor is not achieved using pipeline. Avoid pipeline use in production.
271
  * Measurements taken on **NVIDIA RTX 3090**, expect better metrics in more powerful servers.
272
  * Throughput increases with larger batch sizes due to better GPU utilization. Consider your use case when selecting batch size.
273
  * Optimizations listed are suggestions that could further improve performance.