First README file.
Browse files
README.md
CHANGED
@@ -1,3 +1,92 @@
|
|
1 |
-
---
|
2 |
-
license: mit
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
base_model:
|
6 |
+
- microsoft/deberta-v3-base
|
7 |
+
pipeline_tag: zero-shot-classification
|
8 |
+
tags:
|
9 |
+
- smart
|
10 |
+
- city
|
11 |
+
- classifier
|
12 |
+
- genai
|
13 |
+
---
|
14 |
+
# GenAI Smart City Classifier (DeBERTa v3 Base Fine-Tune)
|
15 |
+
|
16 |
+
Binary transformer classifier detecting whether a text describes a Generative AI (GenAI) application in a smart city context.
|
17 |
+
|
18 |
+
## Labels
|
19 |
+
- 0: GenAI used for smart city application
|
20 |
+
- 1: Not related
|
21 |
+
|
22 |
+
`id2label = {0: "GenAI used for smart city application", 1: "Not related"}`
|
23 |
+
|
24 |
+
## Model Card Summary
|
25 |
+
- Base: microsoft/deberta-v3-base
|
26 |
+
- Tokenizer: DebertaV2Tokenizer (same as base)
|
27 |
+
- Max length used in training batches: 512 (inference examples use 256)
|
28 |
+
- Loss: Custom focal loss (γ=2) + label smoothing (0.1)
|
29 |
+
- Scheduler: Cosine, warmup 10%
|
30 |
+
- Epochs: 8, batch size 8 (train) / 16 (eval)
|
31 |
+
- Calibration: Temperature scaling (optimal ≈ 0.602)
|
32 |
+
|
33 |
+
## Quick Start
|
34 |
+
```python
|
35 |
+
import torch
|
36 |
+
from transformers import DebertaV2Tokenizer, AutoModelForSequenceClassification
|
37 |
+
|
38 |
+
MODEL_ID = "joaocarlosnb/genai-smartcity-classifier" # replace with actual repo id
|
39 |
+
TEMP = 0.602
|
40 |
+
id2label = {0: "GenAI used for smart city application", 1: "Not related"}
|
41 |
+
|
42 |
+
tokenizer = DebertaV2Tokenizer.from_pretrained(MODEL_ID)
|
43 |
+
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
|
44 |
+
model.eval()
|
45 |
+
|
46 |
+
def predict(text, max_length=256, apply_temp=True):
|
47 |
+
inputs = tokenizer(text, truncation=True, padding="max_length",
|
48 |
+
max_length=max_length, return_tensors="pt")
|
49 |
+
with torch.no_grad():
|
50 |
+
logits = model(**inputs).logits
|
51 |
+
if apply_temp:
|
52 |
+
logits = logits / TEMP
|
53 |
+
probs = torch.softmax(logits, dim=-1)[0]
|
54 |
+
top = int(probs.argmax())
|
55 |
+
return {
|
56 |
+
"label": id2label[top],
|
57 |
+
"probabilities": {id2label[i]: float(p) for i, p in enumerate(probs)}
|
58 |
+
}
|
59 |
+
|
60 |
+
print(predict("We apply a diffusion model to simulate traffic for urban planning."))
|
61 |
+
```
|
62 |
+
|
63 |
+
## Installation
|
64 |
+
```bash
|
65 |
+
pip install transformers torch
|
66 |
+
```
|
67 |
+
|
68 |
+
## Input Guidance
|
69 |
+
Short technical sentences or abstract fragments (English). Truncate >512 tokens automatically.
|
70 |
+
|
71 |
+
## Limitations
|
72 |
+
- Binary only (no “mentioned, not used” middle class)
|
73 |
+
- English academic / technical domain bias
|
74 |
+
- Not evaluated for adversarial or multilingual robustness
|
75 |
+
|
76 |
+
## Intended Use
|
77 |
+
Research, corpus analysis, and exploratory filtering. Human review is recommended before operational deployment.
|
78 |
+
|
79 |
+
## Dataset
|
80 |
+
Training data hosted separately (same namespace). Contains augmented, adaptive, contrastive, and diagnostic subsets.
|
81 |
+
|
82 |
+
## Reproducibility Notes
|
83 |
+
Set `seed=42`. Use DebertaV2Tokenizer with max_length=512 for full retraining.
|
84 |
+
|
85 |
+
## Citation (Placeholder)
|
86 |
+
> Bittencourt, J. C. N., Flores, T. K. S., Jesus, T. C., & Costa, D. G. (2025). On the Role of AI in Building Generative Urban Intelligence. In Review. https://doi.org/10.21203/rs.3.rs-7131966/v1
|
87 |
+
>
|
88 |
+
## License
|
89 |
+
See repository LICENSE (ensure compatibility with upstream model license).
|
90 |
+
|
91 |
+
## Security
|
92 |
+
Do not hard-code Hugging Face tokens. Use environment variable: `export
|