Commit
·
a6f99f9
1
Parent(s):
cda8ddf
Improve readme
Browse files
README.md
CHANGED
@@ -35,7 +35,7 @@ The model can be used as a **binary (true/false) classifier if desired, or you c
|
|
35 |
|
36 |
### Global Performance
|
37 |
|
38 |
-
| Category | Freepik |
|
39 |
|----------|-------------|------------------|----------------|
|
40 |
| High | 99.54% | 97.92% | 98.62% |
|
41 |
| Medium | 97.02% | 78.54% | 91.65% |
|
@@ -43,10 +43,23 @@ The model can be used as a **binary (true/false) classifier if desired, or you c
|
|
43 |
| Neutral | 99.87% | 99.27% | 98.37% |
|
44 |
|
45 |
|
46 |
-
|
47 |
-
|
48 |
-
*
|
49 |
-
*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
50 |
|
51 |
### Accuracy by AI Content
|
52 |
|
@@ -56,25 +69,18 @@ The following tables show detection accuracy percentages across different NSFW c
|
|
56 |
|
57 |
#### AI-Generated Content
|
58 |
|
59 |
-
| Category | Freepik Model |
|
60 |
|----------|-------------|------------------|----------------|
|
61 |
| High | 100.00% | 84.00% | 92.00% |
|
62 |
| Medium | 96.15% | 69.23% | 96.00% |
|
63 |
| Low | 100.00% | 35.71% | 92.86% |
|
64 |
| Neutral | 100.00% | 100.00% | 66.67% |
|
65 |
|
66 |
-
### Non-AI-Generated Content
|
67 |
-
|
68 |
-
| Category | Freepik Model | Falconasai Model | Adamcodd Model |
|
69 |
-
|----------|-------------|------------------|----------------|
|
70 |
-
| High | 100.00% | 100.00% | 100.00% |
|
71 |
-
| Medium | 100.00% | 70.00% | 100.00% |
|
72 |
-
| Low | 100.00% | 0.00% | 100.00% |
|
73 |
-
| Neutral | 100.00% | 88.88% | 100.00% |
|
74 |
|
75 |
-
**
|
|
|
|
|
76 |
|
77 |
-
* Due to the small dataset size, some categories show 100.00% accuracy in some cases, expect less accuracy in real world
|
78 |
|
79 |
## Usage
|
80 |
|
@@ -95,7 +101,7 @@ detector = NSFWDetector(dtype=torch.bfloat16, device="cuda")
|
|
95 |
# Load and classify an image
|
96 |
image = Image.open("your_image")
|
97 |
|
98 |
-
# Check if the image contains NSFW content sentivity level medium
|
99 |
is_nsfw = detector.is_nsfw(image, "medium")
|
100 |
|
101 |
# Get probability scores for all categories
|
@@ -261,7 +267,7 @@ Example output:
|
|
261 |
|
262 |
**Notes:**
|
263 |
* The model has been trained in bf16 so it is **recommended to use it in bf16**.
|
264 |
-
* **Using torch tensor**: The speed using torch tensor is not achieved using pipeline. Avoid
|
265 |
* Measurements taken on **NVIDIA RTX 3090**, expect better metrics in more powerful servers.
|
266 |
* Throughput increases with larger batch sizes due to better GPU utilization. Consider your use case when selecting batch size.
|
267 |
* Optimizations listed are suggestions that could further improve performance.
|
|
|
35 |
|
36 |
### Global Performance
|
37 |
|
38 |
+
| Category | Freepik | Falconsai | Adamcodd |
|
39 |
|----------|-------------|------------------|----------------|
|
40 |
| High | 99.54% | 97.92% | 98.62% |
|
41 |
| Medium | 97.02% | 78.54% | 91.65% |
|
|
|
43 |
| Neutral | 99.87% | 99.27% | 98.37% |
|
44 |
|
45 |
|
46 |
+
In the table below, the results are obtained as follows:
|
47 |
+
|
48 |
+
* For the **Falconsai and AdamCodd** models:
|
49 |
+
* A prediction is considered correct if the image is labeled "low", "medium", or "high" and the model returns true.
|
50 |
+
* If the label is "neutral", the correct output should be false.
|
51 |
+
|
52 |
+
* For the **Freepik model**:
|
53 |
+
* If the image label is "low", "medium", or "high", the model should return at least "low".
|
54 |
+
* If the label is "neutral", the correct output should be "neutral".
|
55 |
+
|
56 |
+
|
57 |
+
**Conclusions:**
|
58 |
+
|
59 |
+
* Our model **outperforms AdamCodd and Falconsai in accuracy**. It is entirely fair to compare them on the "high" and "neutral" labels.
|
60 |
+
* Our model **offers greater granularity**. It is not only suitable for detecting "high" and "neutral" content, but also performs excellently at identifying "low" and "medium" NSFW content.
|
61 |
+
* Falconsai may classify some "medium" and "low" images as not NSFW but mark others as safe for work(SFW), which could lead to unexpected results.
|
62 |
+
* AdamCodd classifies both "low" and "medium" categories as NSFW, which may not be desirable depending on your use case. Furthermore, a 10% of images in low and medium are considered SFW.
|
63 |
|
64 |
### Accuracy by AI Content
|
65 |
|
|
|
69 |
|
70 |
#### AI-Generated Content
|
71 |
|
72 |
+
| Category | Freepik Model | Falconsai Model | Adamcodd Model |
|
73 |
|----------|-------------|------------------|----------------|
|
74 |
| High | 100.00% | 84.00% | 92.00% |
|
75 |
| Medium | 96.15% | 69.23% | 96.00% |
|
76 |
| Low | 100.00% | 35.71% | 92.86% |
|
77 |
| Neutral | 100.00% | 100.00% | 66.67% |
|
78 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
79 |
|
80 |
+
**Conclusions:**
|
81 |
+
* **Avoid using Falconsai for AI-generated content** to prevent prediction errors.
|
82 |
+
* **Our model is the best option to detect NSFW content in AI-generated content**.
|
83 |
|
|
|
84 |
|
85 |
## Usage
|
86 |
|
|
|
101 |
# Load and classify an image
|
102 |
image = Image.open("your_image")
|
103 |
|
104 |
+
# Check if the image contains NSFW content sentivity level medium or higher
|
105 |
is_nsfw = detector.is_nsfw(image, "medium")
|
106 |
|
107 |
# Get probability scores for all categories
|
|
|
267 |
|
268 |
**Notes:**
|
269 |
* The model has been trained in bf16 so it is **recommended to use it in bf16**.
|
270 |
+
* **Using torch tensor**: The speed using torch tensor is not achieved using pipeline. Avoid pipeline use in production.
|
271 |
* Measurements taken on **NVIDIA RTX 3090**, expect better metrics in more powerful servers.
|
272 |
* Throughput increases with larger batch sizes due to better GPU utilization. Consider your use case when selecting batch size.
|
273 |
* Optimizations listed are suggestions that could further improve performance.
|