Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1 |
---
|
2 |
title: H4rmonyEval
|
3 |
-
emoji:
|
4 |
colorFrom: purple
|
5 |
colorTo: gray
|
6 |
sdk: gradio
|
@@ -11,4 +11,94 @@ license: mit
|
|
11 |
short_description: Model Judge For Ecolinguistic Evaluation
|
12 |
---
|
13 |
|
14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
title: H4rmonyEval
|
3 |
+
emoji: π
|
4 |
colorFrom: purple
|
5 |
colorTo: gray
|
6 |
sdk: gradio
|
|
|
11 |
short_description: Model Judge For Ecolinguistic Evaluation
|
12 |
---
|
13 |
|
14 |
+
# π¦ββ¬ H4rmonyEval: Model Judge for Ecolinguistic Evaluation
|
15 |
+
|
16 |
+
**H4rmonyEval** is a specialised evaluation tool designed to assess language model outputs through an **ecolinguistic** perspective. It enables both single prompt evaluation and batch assessments via CSV, focusing on how well model responses align with ecological and sustainable narratives.
|
17 |
+
|
18 |
+
Hosted on **Hugging Face Spaces**, powered by **Gradio**.
|
19 |
+
|
20 |
+
---
|
21 |
+
|
22 |
+
## πΏ Features
|
23 |
+
|
24 |
+
- **Single Q&A Evaluation**
|
25 |
+
Quickly score individual prompt/answer pairs on a scale of **0β5**.
|
26 |
+
|
27 |
+
- **Batch CSV Evaluation**
|
28 |
+
Upload a CSV with questions (and optional answers) for automated scoring. Download detailed results.
|
29 |
+
|
30 |
+
- **Ecolinguistic-Aware Judging**
|
31 |
+
Leveraging a Cohere language model with custom system instructions based on ecolinguistic principles.
|
32 |
+
|
33 |
+
---
|
34 |
+
|
35 |
+
## π CSV Format
|
36 |
+
|
37 |
+
- Required: `question` column.
|
38 |
+
- Optional: `answer` column (if empty, the system generates answers using a selected model).
|
39 |
+
|
40 |
+
Example:
|
41 |
+
|
42 |
+
| question | answer |
|
43 |
+
|-------------------------|----------------------|
|
44 |
+
| What is green AI? | AI focused on... |
|
45 |
+
| Define ecological ethics| |
|
46 |
+
|
47 |
+
---
|
48 |
+
|
49 |
+
## π― Scoring System
|
50 |
+
|
51 |
+
- **5** β Perfect, accurate, and ecologically aligned.
|
52 |
+
- **3** β Partially correct, limited ecological framing.
|
53 |
+
- **0** β Incorrect or ecologically harmful/misaligned.
|
54 |
+
|
55 |
+
Final scores are presented as percentages for batch evaluations.
|
56 |
+
|
57 |
+
---
|
58 |
+
|
59 |
+
## βοΈ How It Works
|
60 |
+
|
61 |
+
- **Judge:** Cohere `command-r-plus` model guided by custom instructions.
|
62 |
+
- **Answer Generation:** Hugging Face Inference API (if no answer is provided).
|
63 |
+
- **Frontend:** Gradio `5.27.0`.
|
64 |
+
|
65 |
+
---
|
66 |
+
|
67 |
+
## π Quick Start
|
68 |
+
|
69 |
+
1. Upload your CSV **or** enter a single Q&A.
|
70 |
+
2. Click the appropriate evaluation button.
|
71 |
+
3. View your score and download results.
|
72 |
+
|
73 |
+
---
|
74 |
+
|
75 |
+
## π Environment Variables
|
76 |
+
|
77 |
+
- `COHERE_API_KEY` β Required.
|
78 |
+
- `HF_TOKEN` β *(Optional)* For Hugging Face Inference.
|
79 |
+
|
80 |
+
---
|
81 |
+
|
82 |
+
## π About H4rmony
|
83 |
+
|
84 |
+
Part of the **H4rmony Project**, promoting AI aligned with ecological justice and sustainability through language.
|
85 |
+
|
86 |
+
Learn more:
|
87 |
+
π https://TheH4rmonyProject.org
|
88 |
+
---
|
89 |
+
|
90 |
+
## π€ Acknowledgements
|
91 |
+
|
92 |
+
- **Cohere** for LLM infrastructure.
|
93 |
+
- **Hugging Face** for Spaces and APIs.
|
94 |
+
- Inspired by the work of the **The H4rmony Project, University of Gloucestershire & International Ecolinguistics Association**.
|
95 |
+
|
96 |
+
---
|
97 |
+
|
98 |
+
## π License
|
99 |
+
|
100 |
+
MIT License.
|
101 |
+
|
102 |
+
---
|
103 |
+
|
104 |
+
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|