File size: 2,997 Bytes
2f1e4a2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 |
# DistilBERT-Based Intent Detection Model for Banking Customer Queries
This repository contains a fine-tuned **DistilBERT** model for **intent detection** in banking customer support scenarios. It is trained on the **BANKING77 dataset** and designed to accurately classify user queries into 77 distinct banking-related intents.
## Model Details
- **Model Architecture:** DistilBERT Base Uncased
- **Task:** Intent Detection for Banking Queries
- **Dataset:** [BANKING77](https://huggingface.co/datasets/banking77)
- **Fine-tuning Framework:** Hugging Face Transformers
- **Language:** English
- **Number of Labels:** 77
- **Quantization:** *Not applied (full precision)*
## Usage
### Installation
```bash
pip install transformers torch datasets
```
### Loading the Model
```python
from transformers import DistilBertTokenizer, DistilBertForSequenceClassification
import torch
# Load fine-tuned model
model_path = "./banking77-distilbert" # Adjust path if different
model = DistilBertForSequenceClassification.from_pretrained(model_path)
tokenizer = DistilBertTokenizer.from_pretrained(model_path)
model.eval()
# Sample query
text = "I need to reset my online banking password."
# Tokenize and predict
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
outputs = model(**inputs)
predicted_class = torch.argmax(outputs.logits, dim=1).item()
# Example label map (replace with the full BANKING77 map)
label_map = {0: "activate_my_card", 1: "balance", 2: "card_arrival", ..., 76: "why_was_my_card_declined"}
print(f"Predicted Intent: {label_map[predicted_class]}")
```
## Performance Metrics
- **Accuracy:** ~95% (on the BANKING77 test split)
- **Loss:** ~0.13 (after fine-tuning for 4 epochs)
## Fine-Tuning Details
### Dataset
- **Name:** BANKING77
- **Size:** ~13,000 customer support queries
- **Intents:** 77 unique labeled banking intents
### Training
- **Epochs:** 4
- **Batch Size:** 16
- **Learning Rate:** 2e-5
- **Optimizer:** AdamW
- **Evaluation Strategy:** per epoch
- **Loss Function:** CrossEntropyLoss
### Hardware
- **GPU Used:** NVIDIA Tesla T4 (via Google Colab)
- **Training Time:** ~15 minutes
## Repository Structure
```
.
├── banking77-distilbert/ # Fine-tuned model directory (saved via trainer.save_model)
│ ├── config.json
│ ├── pytorch_model.bin
│ ├── tokenizer_config.json
│ ├── vocab.txt
├── intent_predictor.py # Script for predicting intents (with preprocessing)
├── README.md # Model documentation
```
## Limitations
- The model is trained only on banking-related intents; it may misclassify out-of-domain queries.
- Multilingual support is not included — limited to English.
- Model does not handle multiple intents per query.
## Contributing
Contributions and suggestions are welcome. Please open an issue or pull request for improvements or additional features.
|