File size: 6,425 Bytes
c474a69 8ea0aa8 1e53214 359de93 8ea0aa8 1e53214 8ea0aa8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 |
---
license: apache-2.0
datasets:
- VinayHajare/Marathi-Sign-Language
language:
- en
base_model:
- google/siglip2-base-patch16-224
pipeline_tag: image-classification
library_name: transformers
tags:
- Marathi-Sign-Language-Detection
- SigLIP2
- 93M
---
# Marathi-Sign-Language-Detection
> Marathi-Sign-Language-Detection is a vision-language model fine-tuned from google/siglip2-base-patch16-224 for multi-class image classification. It is trained to recognize Marathi sign language hand gestures and map them to corresponding Devanagari characters using the SiglipForImageClassification architecture.
```py
Classification Report:
precision recall f1-score support
अ 0.9881 0.9911 0.9896 1009
आ 0.9926 0.9237 0.9569 1022
इ 0.8132 0.9609 0.8809 1101
ई 0.9424 0.8894 0.9151 1103
उ 0.9477 0.9073 0.9271 1198
ऊ 0.9436 1.0000 0.9710 1071
ए 0.9153 0.9378 0.9264 1141
ऐ 0.7790 0.8871 0.8295 1089
ओ 0.9188 0.9581 0.9381 1075
औ 1.0000 0.9226 0.9598 1021
क 0.9566 0.9160 0.9358 1083
क्ष 0.9287 0.9667 0.9473 1200
ख 0.9913 1.0000 0.9956 1140
ग 0.9753 0.9982 0.9866 1109
घ 0.8398 0.7908 0.8146 1200
च 0.9388 0.9016 0.9198 1158
छ 0.9764 0.8127 0.8870 1169
ज 0.9599 0.9967 0.9779 1200
ज्ञ 0.9878 0.9483 0.9677 1200
झ 0.9939 0.9567 0.9749 1200
ट 0.8917 0.8992 0.8954 1200
ठ 0.9075 0.8425 0.8738 1200
ड 0.9354 0.9900 0.9619 1200
ढ 0.8616 0.9025 0.8816 1200
ण 0.9114 0.9425 0.9267 1200
त 0.9280 0.9025 0.9151 1200
थ 0.9388 0.9717 0.9550 1200
द 0.8648 0.9275 0.8951 1200
ध 0.9876 0.9917 0.9896 1200
न 0.7256 0.8967 0.8021 1200
प 0.9991 0.9683 0.9835 1200
फ 0.8909 0.8575 0.8739 1200
ब 0.9814 0.7917 0.8764 1200
भ 0.9758 0.8383 0.9018 1200
म 0.8121 0.8142 0.8132 1200
य 0.5726 0.9133 0.7039 1200
र 0.7635 0.7339 0.7484 1210
ल 0.9239 0.8800 0.9014 1200
ळ 0.8950 0.7533 0.8181 1200
व 0.9597 0.7542 0.8446 1200
श 0.8829 0.8667 0.8747 1200
स 0.8449 0.8758 0.8601 1200
ह 0.9604 0.8883 0.9229 1200
accuracy 0.9027 50099
macro avg 0.9117 0.9039 0.9051 50099
weighted avg 0.9107 0.9027 0.9040 50099
```
---
## Label Space: 43 Classes
The model classifies a hand sign into one of the following 43 Marathi characters:
```json
"id2label": {
"0": "अ", "1": "आ", "2": "इ", "3": "ई", "4": "उ", "5": "ऊ",
"6": "ए", "7": "ऐ", "8": "ओ", "9": "औ", "10": "क", "11": "क्ष",
"12": "ख", "13": "ग", "14": "घ", "15": "च", "16": "छ", "17": "ज",
"18": "ज्ञ", "19": "झ", "20": "ट", "21": "ठ", "22": "ड", "23": "ढ",
"24": "ण", "25": "त", "26": "थ", "27": "द", "28": "ध", "29": "न",
"30": "प", "31": "फ", "32": "ब", "33": "भ", "34": "म", "35": "य",
"36": "र", "37": "ल", "38": "ळ", "39": "व", "40": "श", "41": "स", "42": "ह"
}
```
---
## Install Dependencies
```bash
pip install -q transformers torch pillow gradio
```
---
## Inference Code
```python
import gradio as gr
from transformers import AutoImageProcessor, SiglipForImageClassification
from PIL import Image
import torch
# Load model and processor
model_name = "prithivMLmods/Marathi-Sign-Language-Detection" # Replace with actual path
model = SiglipForImageClassification.from_pretrained(model_name)
processor = AutoImageProcessor.from_pretrained(model_name)
# Marathi label mapping
id2label = {
"0": "अ", "1": "आ", "2": "इ", "3": "ई", "4": "उ", "5": "ऊ",
"6": "ए", "7": "ऐ", "8": "ओ", "9": "औ", "10": "क", "11": "क्ष",
"12": "ख", "13": "ग", "14": "घ", "15": "च", "16": "छ", "17": "ज",
"18": "ज्ञ", "19": "झ", "20": "ट", "21": "ठ", "22": "ड", "23": "ढ",
"24": "ण", "25": "त", "26": "थ", "27": "द", "28": "ध", "29": "न",
"30": "प", "31": "फ", "32": "ब", "33": "भ", "34": "म", "35": "य",
"36": "र", "37": "ल", "38": "ळ", "39": "व", "40": "श", "41": "स", "42": "ह"
}
def classify_marathi_sign(image):
image = Image.fromarray(image).convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
logits = outputs.logits
probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist()
prediction = {
id2label[str(i)]: round(probs[i], 3) for i in range(len(probs))
}
return prediction
# Gradio Interface
iface = gr.Interface(
fn=classify_marathi_sign,
inputs=gr.Image(type="numpy"),
outputs=gr.Label(num_top_classes=5, label="Marathi Sign Classification"),
title="Marathi-Sign-Language-Detection",
description="Upload an image of a Marathi sign language hand gesture to identify the corresponding character."
)
if __name__ == "__main__":
iface.launch()
```
---
## Intended Use
Marathi-Sign-Language-Detection can be applied in:
* Educational platforms for learning regional sign language.
* Assistive communication tools for Marathi-speaking users with hearing impairments.
* Interactive applications that translate signs into text.
* Research and data collection for sign language development and recognition. |