README.md

metadata

title: Email Triage OpenEnv
emoji: 📧
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false
short_description: OpenEnv email triage benchmark — GRPO fine-tuned agent
tags:
  - openenv
  - agents
  - evaluation
  - email-triage

📧 Multi-Agent Email Triage Environment

A real-world RL environment that trains LLMs to intelligently triage a corporate inbox — classifying, routing, prioritizing, and drafting replies for incoming emails using GRPO reinforcement learning.

---

🔗 Deliverables

Item	Link
🤗 HF Space	(this repo)
📓 Training Notebook	Open in Kaggle
📝 Blog / Writeup	BLOG.md in this repo
📊 wandb Training Run	Live metrics

---

🧠 What Was Trained

A Qwen/Qwen2.5-3B-Instruct model fine-tuned with GRPO (Group Relative Policy Optimization) via HuggingFace TRL.

• Base model: Qwen/Qwen2.5-3B-Instruct
• Method: GRPO with LoRA (r=4, alpha=8, target: q_proj + v_proj)
• Training steps: 200
• Reward signal: Environment reward (priority + category + routing + reply quality + SLA compliance)

---

📈 Training Results

Reward Curve

Red line = 10-step rolling average. Agent improves from 3.47 → 4.10+ over 200 GRPO steps.

Before vs After

Key Numbers

Metric	Baseline (untrained)	After GRPO (200 steps)
Avg reward	3.47	4.10+
Best episode reward	3.65	4.41
Worst episode reward	2.78	3.60
Final evaluation score	—	4.189 / 5.0
Correct routing	~40%	100% (5/5)
Null reply drafts	frequent	eliminated
Invalid categories	occasional	eliminated
Escalation budget abuse	frequent	eliminated

Before vs After Routing (Final Evaluation)

Email	Untrained	Trained	Per-email Reward
URGENT: My account has been hacked!	general_inquiry → support_tier1	customer_complaint → support_tier2 ✅	0.705
Congratulations! You've WON $1,000,000!	urgent → support_tier1	spam_phishing → trash ✅	0.840
Team lunch this Friday	low → trash	internal_hr → hr ✅	0.988
Invoice #INV-2024-0042 - Payment Overdue	medium → support_tier1	billing_inquiry → billing ✅	0.798
Interested in your Enterprise plan	general_inquiry → archive	sales_lead → sales ✅	0.858
Total			4.189 / ~5.0

---

🎯 Problem Statement

Every company receives hundreds of emails daily. Manually reading, classifying, and routing each one is time-consuming and error-prone. This environment trains an LLM agent to handle this automatically.

This falls under Theme #3.2 — Personalized Tasks (World Modeling).

---

🌍 Environment Interface

The environment follows the OpenEnv-style interface:

• reset() — starts a fresh episode
• step(action) — applies action, returns next observation + reward
• state() — returns current internal state

Observation Space

Field	Description
current_email	Email to triage (header + body)
inbox	Full list of emails in episode
processed	IDs of already handled emails
escalation_budget_remaining	Remaining flag_review capacity
team_queue_remaining	Capacity per routing destination
active_sla_warnings	Emails close to SLA deadline
sla_breaches_so_far	Number of SLA breaches triggered

Action Space

Field	Valid Values
email_id	Copy exactly from current_email
priority	urgent | high | medium | low | spam
category	customer_complaint | billing_inquiry | technical_support | sales_lead | internal_hr | legal_compliance | spam_phishing | general_inquiry
route_to	support_tier1 | support_tier2 | billing | sales | legal | hr | management | trash | archive
summary	≤280 chars
flag_review	true or false (uses escalation budget)
reply_draft	Professional reply to sender

Tasks

Task	Emails	Difficulty	Expected Score
easy	5	Easy	0.75–0.99
medium	8	Medium	0.55–0.80
hard	12	Hard	0.35–0.65

Reward Design

Per-email scoring:

• ✅ Correct priority → ~0.2 pts
• ✅ Correct category → ~0.2 pts
• ✅ Correct routing → ~0.3 pts
• ✅ Reply draft quality → ~0.2 pts (20% of score)
• ✅ Appropriate escalation
• ❌ Penalty for SLA breaches
• ❌ Penalty for invalid field values

---

🏋️ Training

MODEL_NAME      = "Qwen/Qwen2.5-3B-Instruct"
TRAINING_STEPS  = 200
BATCH_SIZE      = 1
NUM_GENERATIONS = 4
learning_rate   = 2e-6
lora_r          = 4

To run training:

pip install trl transformers datasets peft accelerate bitsandbytes wandb
# Then open the Colab notebook linked above

---

🔍 Inference

import requests

BASE_URL = "https://Vansh04092003-multi-agent-email-env2.hf.space"

# Start episode
obs = requests.post(f"{BASE_URL}/reset").json()["observation"]

# Take action
action = {
    "email_id":    "e001",
    "priority":    "urgent",
    "category":    "customer_complaint",
    "route_to":    "support_tier2",
    "summary":     "Account hacked, needs immediate lock.",
    "flag_review": True,
    "reply_draft": "We are securing your account immediately."
}
result = requests.post(f"{BASE_URL}/step", json=action).json()
print(f"Reward: {result['reward']}")

---

📁 Repository Structure

├── README.md
├── BLOG.md
├── openenv.yaml
├── training/
│   ├── notebook_training.ipynb
│   ├── train_grpo.py
│   └── plots/
│       ├── reward_curve.png
│       └── before_after.png
└── src/
    └── openenv_email_triage/
        ├── environment.py
        └── models.py

---

🐳 Local Run

export HF_TOKEN=your_token_here
uv run server

Docker

docker build -t email-triage-env .
docker run -p 7860:7860 -e HF_TOKEN=$HF_TOKEN email-triage-env

Validate

openenv validate