DeepakKumarMSL commited on
Commit
adc3cce
Β·
verified Β·
1 Parent(s): 2275eaa

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -40
README.md CHANGED
@@ -1,75 +1,75 @@
1
- # 🎯 Tone Detection using `facebook/bart-large-mnli` (Zero-Shot Classification)
2
 
3
- This project demonstrates how to perform **Tone Detection** using the [`facebook/bart-large-mnli`](https://huggingface.co/facebook/bart-large-mnli) model through **zero-shot classification** based on Natural Language Inference (NLI).
4
 
5
- This approach enables you to classify emotional tone (e.g., joy, anger, sadness) **without training**, by framing it as a textual entailment task.
 
 
 
 
6
 
7
  ---
8
 
9
- ## πŸ“Œ Model Details
10
 
11
  - **Model:** `facebook/bart-large-mnli`
12
- - **Task:** Zero-shot classification via NLI
13
- - **Approach:** Checks if the input sentence entails a hypothesis (e.g., "This text expresses anger.")
14
- - **Strength:** No labeled training data required
15
 
16
  ---
17
 
18
- ## πŸ“‚ Dataset Used
19
 
20
- For benchmarking and scoring, we use the [`go_emotions`](https://huggingface.co/datasets/go_emotions) dataset:
21
 
22
  ```python
23
  from datasets import load_dataset
24
 
25
- dataset = load_dataset("go_emotions")
26
  ```
27
 
28
- # 🧠 Tone Detection (Inference)
29
- ```Python
 
 
 
 
 
 
 
30
  from transformers import pipeline
31
 
32
  classifier = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
33
 
34
- labels = ["joy", "anger", "sadness", "fear", "surprise", "neutral"]
35
-
36
- text = "I can't believe this is happening again. So frustrating."
37
 
38
- result = classifier(text, candidate_labels=labels, hypothesis_template="This text expresses {}.")
39
  print(result)
40
  ```
41
 
42
- # πŸ§ͺ Evaluation with Scoring
 
43
 
44
  ```python
45
  from sklearn.metrics import accuracy_score
46
 
47
- # Mapping GoEmotions label indices to names
48
- id2label = dataset["train"].features["labels"].feature.names
49
-
50
- # Evaluate on a small sample
51
- def evaluate(dataset, candidate_labels):
52
  correct = 0
53
  total = 0
54
- for row in dataset.select(range(100)): # Use more samples as needed
55
- text = row["text"]
56
- true_labels = [id2label[i] for i in row["labels"]]
57
- result = classifier(text, candidate_labels=candidate_labels, hypothesis_template="This text expresses {}.")
58
  predicted = result["labels"][0]
59
- if predicted in true_labels:
60
- correct += 1
61
  total += 1
62
- return correct/total
63
-
64
- accuracy = evaluate(dataset["test"], candidate_labels=labels)
65
- print(f"Zero-shot Accuracy: {accuracy:.2%}")
66
- ```
67
-
68
- # βš™οΈ Use Cases
69
- Customer support tone analysis
70
-
71
- Chat moderation for emotional tone
72
 
73
- Feedback sentiment detection
 
 
74
 
75
- Real-time conversation emotion tagging
 
 
 
1
+ # Zero-Shot Text Classification using `facebook/bart-large-mnli`
2
 
3
+ This repository demonstrates how to use the [`facebook/bart-large-mnli`](https://huggingface.co/facebook/bart-large-mnli) model for **zero-shot text classification** based on **natural language inference (NLI)**.
4
 
5
+ We extend the base usage by:
6
+ - Using a labeled dataset for benchmarking
7
+ - Performing optional fine-tuning
8
+ - Quantizing the model to FP16
9
+ - Scoring model performance
10
 
11
  ---
12
 
13
+ ## πŸ“Œ Model Description
14
 
15
  - **Model:** `facebook/bart-large-mnli`
16
+ - **Type:** NLI-based zero-shot classifier
17
+ - **Architecture:** BART (Bidirectional and Auto-Regressive Transformers)
18
+ - **Usage:** Classifies text by scoring label hypotheses as NLI entailment
19
 
20
  ---
21
 
22
+ ## πŸ“‚ Dataset
23
 
24
+ We use the [`yahoo_answers_topics`](https://huggingface.co/datasets/yahoo_answers_topics) dataset from Hugging Face for evaluation. It contains questions categorized into 10 topics.
25
 
26
  ```python
27
  from datasets import load_dataset
28
 
29
+ dataset = load_dataset("yahoo_answers_topics")
30
  ```
31
 
32
+ # 🧠 Zero-Shot Classification Logic
33
+ The model checks whether a text entails a hypothesis like:
34
+
35
+ "This text is about sports."
36
+
37
+ For each candidate label (e.g., "sports", "education", "health"), we convert them into such hypotheses and use the model to score them.
38
+
39
+ # βœ… Example: Inference with Zero-Shot Pipeline
40
+ ```python
41
  from transformers import pipeline
42
 
43
  classifier = pipeline("zero-shot-classification", model="facebook/bart-large-mnli")
44
 
45
+ sequence = "The team played well and won the championship."
46
+ labels = ["sports", "politics", "education", "technology"]
 
47
 
48
+ result = classifier(sequence, candidate_labels=labels)
49
  print(result)
50
  ```
51
 
52
+ # πŸ“Š Scoring / Evaluation
53
+ Evaluate zero-shot classification using accuracy or top-k accuracy:
54
 
55
  ```python
56
  from sklearn.metrics import accuracy_score
57
 
58
+ def evaluate_zero_shot(dataset, labels):
 
 
 
 
59
  correct = 0
60
  total = 0
61
+ for example in dataset:
62
+ result = classifier(example["question_content"], candidate_labels=labels)
 
 
63
  predicted = result["labels"][0]
64
+ true = labels[example["topic"]]
65
+ correct += int(predicted == true)
66
  total += 1
67
+ return correct / total
 
 
 
 
 
 
 
 
 
68
 
69
+ labels = ["Society & Culture", "Science & Mathematics", "Health", "Education",
70
+ "Computers & Internet", "Sports", "Business & Finance", "Entertainment & Music",
71
+ "Family & Relationships", "Politics & Government"]
72
 
73
+ acc = evaluate_zero_shot(dataset["test"].select(range(100)), labels)
74
+ print(f"Accuracy: {acc:.2%}")
75
+ ```