Spaces:
Running
Running
| title: H4rmonyEval | |
| emoji: 🌍 | |
| colorFrom: purple | |
| colorTo: gray | |
| sdk: gradio | |
| sdk_version: 5.27.0 | |
| app_file: app.py | |
| pinned: false | |
| license: mit | |
| short_description: Model Judge For Ecolinguistic Evaluation | |
| # 🐦⬛ H4rmonyEval: Model Judge for Ecolinguistic Evaluation | |
| **H4rmonyEval** is a specialised evaluation tool designed to assess language model outputs through an **ecolinguistic** perspective. It enables both single prompt evaluation and batch assessments via CSV, focusing on how well model responses align with ecological and sustainable narratives. | |
| Hosted on **Hugging Face Spaces**, powered by **Gradio**. | |
| --- | |
| ## 🌿 Features | |
| - **Single Q&A Evaluation** | |
| Quickly score individual prompt/answer pairs on a scale of **0–5**. | |
| - **Batch CSV Evaluation** | |
| Upload a CSV with questions (and optional answers) for automated scoring. Download detailed results. | |
| - **Ecolinguistic-Aware Judging** | |
| Leveraging a Cohere language model with custom system instructions based on ecolinguistic principles. | |
| --- | |
| ## 📄 CSV Format | |
| - Required: `question` column. | |
| - Optional: `answer` column (if empty, the system generates answers using a selected model). | |
| Example: | |
| | question | answer | | |
| |-------------------------|----------------------| | |
| | What is green AI? | AI focused on... | | |
| | Define ecological ethics| | | |
| --- | |
| ## 🎯 Scoring System | |
| - **5** – Perfect, accurate, and ecologically aligned. | |
| - **3** – Partially correct, limited ecological framing. | |
| - **0** – Incorrect or ecologically harmful/misaligned. | |
| Final scores are presented as percentages for batch evaluations. | |
| --- | |
| ## ⚙️ How It Works | |
| - **Judge:** Cohere `command-r-plus` model guided by custom instructions. | |
| - **Answer Generation:** Hugging Face Inference API (if no answer is provided). | |
| - **Frontend:** Gradio `5.27.0`. | |
| --- | |
| ## 🚀 Quick Start | |
| 1. Upload your CSV **or** enter a single Q&A. | |
| 2. Click the appropriate evaluation button. | |
| 3. View your score and download results. | |
| --- | |
| ## 🔐 Environment Variables | |
| - `COHERE_API_KEY` – Required. | |
| - `HF_TOKEN` – *(Optional)* For Hugging Face Inference. | |
| --- | |
| ## 🌍 About H4rmony | |
| Part of the **H4rmony Project**, promoting AI aligned with ecological justice and sustainability through language. | |
| Learn more: | |
| 🔗 https://TheH4rmonyProject.org | |
| --- | |
| ## 🤝 Acknowledgements | |
| - **Cohere** for LLM infrastructure. | |
| - **Hugging Face** for Spaces and APIs. | |
| - Inspired by the work of the **The H4rmony Project, University of Gloucestershire & International Ecolinguistics Association**. | |
| --- | |
| ## 📜 License | |
| MIT License. | |
| --- | |
| Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference |