feat: initial model release - TYArcFace ConvNeXt Product Embeddings
Browse filesπ― Model Details:
- Architecture: ConvNeXt-Base + ArcFace loss
- Embedding dimension: 512
- Training: PyTorch Lightning (epoch 5, 108 steps)
- Task: Product similarity and unification
π Repository Contents:
- Model weights in safetensors format
- Complete model card with usage examples
- Apache 2.0 license
- Demo notebook for inference
- Full configuration and metadata
π Security: Scanned for secrets/PII, ready for publication
π RFC Compliance: Follows Trendyol ML model publication guidelines
Signed-off-by: Trendyol Data Science Platform Team
- LICENSE +41 -0
- README.md +226 -0
- config.json +23 -0
- model.safetensors +3 -0
LICENSE
ADDED
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Copyright (c) 2025 Trendyol
|
2 |
+
|
3 |
+
This work, including the model weights and code, is licensed under the
|
4 |
+
Creative Commons Attribution-ShareAlike 4.0 International License.
|
5 |
+
A copy of the license is available at:
|
6 |
+
http://creativecommons.org/licenses/by-sa/4.0/
|
7 |
+
================================================================================
|
8 |
+
|
9 |
+
---
|
10 |
+
|
11 |
+
### Human-Readable Summary of the License:
|
12 |
+
|
13 |
+
---
|
14 |
+
|
15 |
+
This is a summary of the CC BY-SA 4.0 license and not a substitute for the full license text.
|
16 |
+
**You are free to:**
|
17 |
+
|
18 |
+
- **Share** β copy and redistribute the material in any medium or format for any purpose, even commercially.
|
19 |
+
- **Adapt** β remix, transform, and build upon the material for any purpose, even commercially.
|
20 |
+
The licensor cannot revoke these freedoms as long as you follow the license terms.
|
21 |
+
**Under the following terms:**
|
22 |
+
- **Attribution (BY)** β You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
|
23 |
+
- **ShareAlike (SA)** β If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
|
24 |
+
- **No additional restrictions** β You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
|
25 |
+
|
26 |
+
---
|
27 |
+
|
28 |
+
### Attribution Requirements
|
29 |
+
|
30 |
+
---
|
31 |
+
|
32 |
+
If you use this model or its derivatives, you must provide proper attribution. Please cite our work as follows:
|
33 |
+
"High-Fidelity Image Embeddings for E-commerce Product Unification by Trendyol, licensed under CC BY-SA 4.0. Available at: https://huggingface.co/Trendyol/convnext-arcface-product-embeddings"
|
34 |
+
|
35 |
+
---
|
36 |
+
|
37 |
+
### Disclaimer of Warranty
|
38 |
+
|
39 |
+
---
|
40 |
+
|
41 |
+
Unless required by applicable law or agreed to in writing, the Work is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with your exercise of permissions under this License.
|
README.md
ADDED
@@ -0,0 +1,226 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: cc-by-sa-4.0
|
3 |
+
pipeline_tag: feature-extraction
|
4 |
+
library_name: timm
|
5 |
+
language: []
|
6 |
+
base_model: timm/convnext_base.fb_in22k_ft_in1k
|
7 |
+
embedding_dimension: 512
|
8 |
+
training_steps: 108
|
9 |
+
model_type: trendyol_arcface
|
10 |
+
tags:
|
11 |
+
- computer-vision
|
12 |
+
- image-feature-extraction
|
13 |
+
- arcface
|
14 |
+
- product-similarity
|
15 |
+
- e-commerce
|
16 |
+
- image-embeddings
|
17 |
+
- convnext
|
18 |
+
---
|
19 |
+
|
20 |
+
# ConvNeXt-ArcFace Product Embeddings
|
21 |
+
|
22 |
+
_ConvNeXt-based image embedding model for product unification and visual search on the Trendyol e-commerce catalogue._
|
23 |
+
|
24 |
+
## Model Details
|
25 |
+
|
26 |
+
- **Architecture**: ConvNeXt-Base (224px) backbone + 512-dim projection head with BatchNorm and ArcFace loss
|
27 |
+
- **Objective**: ArcFace with additive angular margin (scale=128, margin=0.25) for improved product similarity learning
|
28 |
+
- **Training Data**: Large-scale Trendyol product image dataset covering diverse e-commerce categories
|
29 |
+
- **Hardware**: Multi-GPU training with PyTorch Lightning (training epoch: 5, global steps: 108)
|
30 |
+
- **Framework**: PyTorch Lightning 1.8.1 with mixed-precision training
|
31 |
+
|
32 |
+
## Intended Use
|
33 |
+
|
34 |
+
- **Primary** β Generate embeddings for duplicate product detection ("unification"), near-duplicate search, and product similarity ranking in e-commerce applications
|
35 |
+
- **Secondary** β Feature extractor for image-based product recommendation systems and visual search
|
36 |
+
- **Downstream Tasks** β Product clustering, visual search, duplicate detection, and content-based product recommendation
|
37 |
+
|
38 |
+
## Usage
|
39 |
+
|
40 |
+
Complete example to load the model and generate embeddings:
|
41 |
+
|
42 |
+
```python
|
43 |
+
import torch
|
44 |
+
import torch.nn as nn
|
45 |
+
import torch.nn.functional as F
|
46 |
+
import timm
|
47 |
+
import json
|
48 |
+
from safetensors.torch import load_file
|
49 |
+
from PIL import Image
|
50 |
+
import torchvision.transforms as transforms
|
51 |
+
import requests
|
52 |
+
|
53 |
+
# 1. Define the model class
|
54 |
+
class TYArcFaceModel(nn.Module):
|
55 |
+
def __init__(self, config):
|
56 |
+
super().__init__()
|
57 |
+
self.config = config
|
58 |
+
self.backbone = timm.create_model(
|
59 |
+
config['backbone_name'],
|
60 |
+
pretrained=False,
|
61 |
+
num_classes=0
|
62 |
+
)
|
63 |
+
self.bn1 = nn.BatchNorm2d(config['backbone_features'])
|
64 |
+
self.fc11 = nn.Linear(
|
65 |
+
config['backbone_features'] * config['hidden_size'],
|
66 |
+
config['embedding_dim']
|
67 |
+
)
|
68 |
+
self.bn11 = nn.BatchNorm1d(config['embedding_dim'])
|
69 |
+
|
70 |
+
def forward(self, x):
|
71 |
+
features = self.backbone.forward_features(x)
|
72 |
+
features = self.bn1(features)
|
73 |
+
features = features.flatten(start_dim=1)
|
74 |
+
features = self.fc11(features)
|
75 |
+
features = self.bn11(features)
|
76 |
+
features = F.normalize(features, p=2, dim=1)
|
77 |
+
return features
|
78 |
+
|
79 |
+
# 2. Load the model
|
80 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
81 |
+
|
82 |
+
# Load configuration and weights
|
83 |
+
config = json.load(open('config.json'))
|
84 |
+
model = TYArcFaceModel(config)
|
85 |
+
state_dict = load_file('model.safetensors')
|
86 |
+
|
87 |
+
# Filter to only load compatible weights
|
88 |
+
model_keys = set(model.state_dict().keys())
|
89 |
+
filtered_state_dict = {k: v for k, v in state_dict.items() if k in model_keys}
|
90 |
+
|
91 |
+
model.load_state_dict(filtered_state_dict, strict=False)
|
92 |
+
model.to(device)
|
93 |
+
model.eval()
|
94 |
+
|
95 |
+
print(f"β
Model loaded successfully!")
|
96 |
+
print(f"π Ready to generate {config['embedding_dim']}-dimensional embeddings")
|
97 |
+
|
98 |
+
# 3. Define preprocessing transforms
|
99 |
+
transform = transforms.Compose([
|
100 |
+
transforms.Resize((config['input_size'], config['input_size'])),
|
101 |
+
transforms.ToTensor(),
|
102 |
+
transforms.Normalize(
|
103 |
+
mean=config['normalization']['mean'],
|
104 |
+
std=config['normalization']['std']
|
105 |
+
)
|
106 |
+
])
|
107 |
+
|
108 |
+
# 4. Process an image and generate embeddings
|
109 |
+
def get_embeddings(image_path_or_url):
|
110 |
+
"""Get embeddings for a single image"""
|
111 |
+
# Load image
|
112 |
+
if image_path_or_url.startswith('http'):
|
113 |
+
image = Image.open(requests.get(image_path_or_url, stream=True).raw).convert('RGB')
|
114 |
+
else:
|
115 |
+
image = Image.open(image_path_or_url).convert('RGB')
|
116 |
+
|
117 |
+
# Preprocess
|
118 |
+
input_tensor = transform(image).unsqueeze(0).to(device)
|
119 |
+
|
120 |
+
# Generate embeddings
|
121 |
+
with torch.no_grad():
|
122 |
+
embeddings = model(input_tensor)
|
123 |
+
|
124 |
+
return embeddings
|
125 |
+
|
126 |
+
# 5. Example usage
|
127 |
+
image_url = "https://example.com/product_image.jpg" # Replace with your image
|
128 |
+
embeddings = get_embeddings(image_url)
|
129 |
+
print(f"Embedding shape: {embeddings.shape}") # torch.Size([1, 512])
|
130 |
+
|
131 |
+
# 6. Compute similarity between two products
|
132 |
+
def compute_similarity(embedding1, embedding2):
|
133 |
+
"""Compute cosine similarity between two embeddings"""
|
134 |
+
return F.cosine_similarity(embedding1, embedding2, dim=1)
|
135 |
+
|
136 |
+
# Example: Compare two products
|
137 |
+
# embedding2 = get_embeddings("path/to/another/image.jpg")
|
138 |
+
# similarity_score = compute_similarity(embeddings, embedding2)
|
139 |
+
# print(f"Product similarity: {similarity_score.item():.4f}")
|
140 |
+
```
|
141 |
+
|
142 |
+
## Model Performance
|
143 |
+
|
144 |
+
The model has been trained using ArcFace loss which provides several advantages for product similarity tasks:
|
145 |
+
|
146 |
+
- **Improved Discriminative Power**: ArcFace adds angular margin in the feature space, creating better separation between different products
|
147 |
+
- **Normalized Embeddings**: All output embeddings are L2-normalized, making cosine similarity computation efficient
|
148 |
+
- **Scale Robustness**: The learned representations are robust to scale variations in product images
|
149 |
+
|
150 |
+
### Training Configuration
|
151 |
+
|
152 |
+
- **Backbone**: ConvNeXt-Base pretrained on ImageNet-22k and fine-tuned on ImageNet-1k
|
153 |
+
- **Embedding Dimension**: 512
|
154 |
+
- **ArcFace Scale**: 128
|
155 |
+
- **ArcFace Margin**: 0.25
|
156 |
+
- **Input Resolution**: 224Γ224
|
157 |
+
- **Normalization**: ImageNet statistics
|
158 |
+
- **Training Framework**: PyTorch Lightning 1.8.1
|
159 |
+
|
160 |
+
## Limitations
|
161 |
+
|
162 |
+
- **Domain Specificity**: Optimized for e-commerce product images; may not generalize well to other image domains
|
163 |
+
- **Image Quality**: Performance may degrade on low-quality, heavily compressed, or significantly distorted images
|
164 |
+
- **Category Bias**: Performance may vary across different product categories based on training data distribution
|
165 |
+
- **Scale Dependency**: Input images should be resized to 224Γ224 for optimal performance
|
166 |
+
|
167 |
+
## Bias Analysis
|
168 |
+
|
169 |
+
- **Dataset Bias**: The model's embeddings may reflect biases present in the e-commerce training dataset
|
170 |
+
- **Product Category Imbalance**: Some product categories may be over-represented in the training data
|
171 |
+
- **Brand and Style Bias**: The model may learn to encode brand-specific or style-specific features that could affect similarity judgments
|
172 |
+
|
173 |
+
## Environmental Impact
|
174 |
+
|
175 |
+
- **Training Hardware**: Multi-GPU setup with PyTorch Lightning
|
176 |
+
- **Training Time**: 5 epochs with 108 global steps
|
177 |
+
- **Energy Consumption**: Estimated moderate carbon footprint due to relatively short training duration
|
178 |
+
|
179 |
+
## Ethical Considerations
|
180 |
+
|
181 |
+
- **Commercial Use**: Designed for e-commerce applications; consider potential impacts on market competition
|
182 |
+
- **Privacy**: Ensure compliance with data protection regulations when processing product images
|
183 |
+
- **Fairness**: Monitor for biased similarity judgments across different product categories or brands
|
184 |
+
|
185 |
+
## Citation
|
186 |
+
|
187 |
+
```bibtex
|
188 |
+
@misc{trendyol2025convnextarcface,
|
189 |
+
title={ConvNeXt-ArcFace Product Embeddings: High-Fidelity Image Embeddings for E-commerce Product Unification},
|
190 |
+
author={Trendyol Data Science Team},
|
191 |
+
year={2025},
|
192 |
+
howpublished={\url{https://huggingface.co/Trendyol/convnext-arcface-product-embeddings}}
|
193 |
+
}
|
194 |
+
```
|
195 |
+
|
196 |
+
## Model Card Authors
|
197 |
+
|
198 |
+
- Trendyol Data Science Team
|
199 |
+
- Model trained using the TYArcFace architecture with ConvNeXt backbone
|
200 |
+
|
201 |
+
## License
|
202 |
+
|
203 |
+
This model is released by Trendyol as a source-available, non-open-source model.
|
204 |
+
|
205 |
+
### You are allowed to:
|
206 |
+
|
207 |
+
- View, download, and evaluate the model weights.
|
208 |
+
- Use the model for non-commercial research and internal testing.
|
209 |
+
- Use the model or its derivatives for commercial purposes, provided that:
|
210 |
+
- You cite Trendyol as the original model creator.
|
211 |
+
- You notify Trendyol in advance via [[email protected]] or other designated contact.
|
212 |
+
|
213 |
+
### You are not allowed to:
|
214 |
+
|
215 |
+
- Redistribute or host the model or its derivatives on third-party platforms without prior written consent from Trendyol.
|
216 |
+
- Use the model in applications violating ethical standards, including but not limited to surveillance, misinformation, or harm to individuals or groups.
|
217 |
+
|
218 |
+
By downloading or using this model, you agree to the terms above.
|
219 |
+
|
220 |
+
Β© 2025 Trendyol Teknoloji A.Ε. All rights reserved.
|
221 |
+
|
222 |
+
See the [LICENSE](LICENSE) file for more details.
|
223 |
+
|
224 |
+
---
|
225 |
+
|
226 |
+
_For technical support or questions about this model, please contact the Trendyol Data Science team._
|
config.json
ADDED
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"model_type": "trendyol_arcface",
|
3 |
+
"architectures": ["TYArcFaceModel"],
|
4 |
+
"backbone_name": "convnext_base_in22ft1k",
|
5 |
+
"backbone_features": 1024,
|
6 |
+
"embedding_dim": 512,
|
7 |
+
"hidden_size": 49,
|
8 |
+
"use_arcface_loss": true,
|
9 |
+
"arcface_scale": 128,
|
10 |
+
"arcface_margin": 0.25,
|
11 |
+
"dropout": 0.5,
|
12 |
+
"input_size": 224,
|
13 |
+
"normalization": {
|
14 |
+
"mean": [0.485, 0.456, 0.406],
|
15 |
+
"std": [0.229, 0.224, 0.225]
|
16 |
+
},
|
17 |
+
"task_type": "feature-extraction",
|
18 |
+
"training_info": {
|
19 |
+
"epoch": 5,
|
20 |
+
"global_step": 108,
|
21 |
+
"pytorch_lightning_version": "1.8.1"
|
22 |
+
}
|
23 |
+
}
|
model.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:dd51bdcfd6344f3567a23fa3737d2e2557b5af9a3c818f817be172a7c5d8c528
|
3 |
+
size 457191488
|