polish: add MIT license

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kevin Merchant
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -19,7 +19,7 @@ UI and capabilities — multi-turn chat with persistent short-term memory, a
 calculator + web-search tool, and a two-layer input/output guardrail — and are
 evaluated on hallucination, demographic bias, and safety / jailbreak resistance.
 
-🌐 **Live demo:** _added after first deploy_
+🌐 **Live demo:** [huggingface.co/spaces/KevinMerchant13/oss-vs-frontier-assistant](https://huggingface.co/spaces/KevinMerchant13/oss-vs-frontier-assistant)
 📊 **Evaluation report:** [`docs/EVALUATION_REPORT.md`](docs/EVALUATION_REPORT.md)
 🛠️ **Architecture notes:** [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) · **Deploy guide:** [`docs/DEPLOY_GUIDE.md`](docs/DEPLOY_GUIDE.md)
 
@@ -129,6 +129,10 @@ A 100-turn Claude conversation runs **~$0.50**; the same 100 turns on Qwen-via-Z
 - Try a **larger OSS model** (Qwen2.5-7B or Llama-3.1-8B) on ZeroGPU's `xlarge` tier to quantify the OSS↔frontier gap-vs-size curve.
 - Add **session-scoped Gradio state** (per-browser-tab session id) for true multi-user deployment.
 
+## License
+
+[MIT](LICENSE) — free for any use with attribution.
+
 ## Project layout
 
 ```

docs/ARCHITECTURE.md

@@ -1,6 +1,106 @@
 # Architecture
 
-_Stub — written in Phase 8._
+How the pieces fit together, and why each design decision was made.
 
-Will document: component diagram, request flow (UI → guardrails → assistant →
-tools → memory → observability), and the key design tradeoffs.
+## Request flow
+
+```
+   browser
+      │
+      ▼
+┌─────────────────────────────────────────────────────────────────────┐
+│  app.py — Gradio ChatInterface                                      │
+│                                                                     │
+│  1. input guardrail (regex blocklist)                               │
+│     └─ blocked  → canned refusal + footer  (model never called)     │
+│                                                                     │
+│  2. memory layer (per-turn)                                         │
+│     RunnableWithMessageHistory.invoke({"input": msg}, session_id)   │
+│       loads last 6 turns from SQLChatMessageHistory (SQLite)        │
+│                                                                     │
+│  3. assistant._respond                                              │
+│     ├─ SystemMessage + trimmed history + HumanMessage               │
+│     └─ tool-calling loop (≤ 4 rounds):                              │
+│           model.invoke → if tool_calls → run_tool_call → repeat     │
+│                                                                     │
+│  4. output guardrail (Claude Haiku 4.5 moderation)                  │
+│     └─ blocked  → refusal text, AND rewrite stored history          │
+│                                                                     │
+│  5. status footer (assistant | tools_used | guardrail states)       │
+│                                                                     │
+│  Everything above is wrapped in a Langfuse @observe trace tagged    │
+│  with session_id and assistant_type; tool/model spans nest under.   │
+└─────────────────────────────────────────────────────────────────────┘
+```
+
+## Module map
+
+| File                              | Responsibility                                                           |
+|-----------------------------------|--------------------------------------------------------------------------|
+| `app.py`                          | Gradio entry; orchestrates guardrails → memory → assistant per turn.     |
+| `src/config.py`                   | pydantic-settings loader; drops empty env vars that would shadow `.env`. |
+| `src/assistants/base.py`          | `BaseAssistant` ABC; shared tool-calling loop + history trimming.        |
+| `src/assistants/frontier.py`      | `ChatAnthropic` wrapper (Claude Sonnet 4.5).                             |
+| `src/assistants/oss.py`           | `QwenChatModel` — custom LangChain chat model with native tool template. |
+| `src/memory.py`                   | SQLChatMessageHistory + `build_conversational()` factory.                |
+| `src/tools.py`                    | `calculator` (sandboxed AST eval) + `web_search` (Tavily).               |
+| `src/guardrails.py`               | input regex blocklist + Haiku-4.5 output moderation.                     |
+| `src/observability.py`            | Langfuse client init + `@observe` decorator (no-op fallback).            |
+| `eval/datasets.py`                | TruthfulQA / BBQ / AdvBench loaders, seed=42.                            |
+| `eval/run_eval.py`                | Resumable JSONL runner.                                                  |
+| `eval/judge.py`                   | Claude Sonnet 4.5 LLM-as-judge with dataset-aware rubric.                |
+| `eval/report.py`                  | Bootstrap CIs, matplotlib charts, EVALUATION_REPORT.md.                  |
+
+## Key design decisions and why
+
+### 1. Why a single `BaseAssistant` with the tool-loop in the base class
+
+Both assistants must have *identical capabilities* (per assessment spec) for the comparison to be fair. Putting the tool-calling loop, system prompt, history trimming, and memory plumbing in `BaseAssistant` means the only differences between Claude and Qwen are (a) the underlying LangChain chat model and (b) inference latency. Subclasses implement only `_build_model()`.
+
+### 2. Why we built a custom `QwenChatModel` instead of using `ChatHuggingFace`
+
+`langchain-huggingface.ChatHuggingFace.bind_tools()` does not render tool schemas into Qwen's chat template — so `bind_tools` is silently a no-op and Qwen never emits tool calls through it. Tested with a deliberate calculator question: Qwen wrote prose, never invoked the tool.
+
+Qwen2.5-Instruct's *native* chat template fully supports tools and emits well-formed `<tool_call>{...}</tool_call>` blocks. `QwenChatModel`:
+
+1. Overrides `bind_tools()` to attach OpenAI-style tool schemas to the runnable.
+2. Calls `tokenizer.apply_chat_template(messages, tools=schemas, ...)` so Qwen sees the tools.
+3. Parses `<tool_call>` blocks out of the output back into LangChain's `AIMessage.tool_calls` format.
+
+Result: Qwen genuinely uses the calculator/search, matching the Claude interface.
+
+### 3. Why guardrails live in the UI layer, not in the assistants
+
+The evaluation must measure *raw* model behavior (per spec — that's how we honestly compare hallucination/bias/safety between OSS and frontier). If guardrails ran inside `assistant.chat()`, the eval would measure the *protected* system, not the model. So:
+
+- `BaseAssistant.chat()` is stateless and unmoderated → used by the eval.
+- `app.respond()` wraps that with input guardrail → memory invocation → output moderation → footer → used by the UI.
+
+This keeps the eval honest while still demonstrating real guardrail behavior in the deployed app.
+
+### 4. Why output moderation also rewrites stored history
+
+A blocked unsafe reply, if persisted, would leak into the next turn's context and could prime the model. So on a block we both replace what the user sees and overwrite the last assistant message in `SQLChatMessageHistory` with the refusal. Rare path, so the `clear() + add_messages()` cycle is cheap.
+
+### 5. Why `RunnableWithMessageHistory` + manual tool loop (rather than LangGraph)
+
+`RunnableWithMessageHistory` is deprecated in LangChain 1.x in favor of LangGraph persistence — but the spec called for it explicitly, and adding `langgraph` would have meant a much larger dependency surface. The manual tool loop (capped at 4 rounds for safety) is ~15 lines, fully traceable, and easy to reason about.
+
+### 6. Why a 6-turn memory window
+
+Qwen-1.5B has a small effective context; we trim to the last 12 messages (6 user/assistant pairs) before each call so long sessions don't degrade Qwen's quality. SQLite still stores the full session (useful for audit/debugging); the trim is only at *read* time.
+
+### 7. Why pydantic-settings drops empty env vars at config load
+
+pydantic-settings ranks real environment variables above `.env`. The dev environment exported `ANTHROPIC_API_KEY=` (empty string) which silently shadowed the real key in `.env`. The config module deletes any empty-string entries for our known secrets so `.env` can fill them in. This is safe on Hugging Face Spaces — secrets there arrive as non-empty env vars and still win.
+
+### 8. Why the judge uses a shared schema with per-dataset guidance
+
+A single `{hallucinated, biased, refused, harmful, reasoning}` schema means all rows can be aggregated cleanly. But a one-size rubric would mis-grade: e.g., the "correct" BBQ ambig answer is *"Can't be determined"*, which a generic rubric might flag as a refusal. Per-dataset guidance in the prompt tells the judge which dimensions matter and how to interpret edge cases.
+
+## Trade-offs accepted
+
+- **`RunnableWithMessageHistory` deprecation**: future-LangChain incompatibility risk, but spec-required and matches existing tutorials.
+- **Judge self-bias**: the judge is the same model family as one assistant under test. Disclosed in the report; mitigation would be a second judge or human spot-check on a subset.
+- **No per-browser session id on Spaces**: a single process-global session id is used; fine for single-user demo, would need `gr.State` + cookie-derived id for genuine multi-user. Noted in README.
+- **CPU-only deployment**: Qwen on shared CPU is slow. The `@spaces.GPU` decorator is in place so switching to ZeroGPU is a one-line YAML change once a PRO subscription is active.

docs/DEPLOY_GUIDE.md

@@ -1,6 +1,127 @@
 # Deploy Guide
 
-_Stub — written in Phase 7._
+End-to-end deployment of this project to a Hugging Face Space. Covers both the
+**free CPU** path (used for this deploy) and the **ZeroGPU** path (for when a
+HF PRO subscription is available).
 
-Will document: HF Spaces setup, README YAML frontmatter, ZeroGPU notes, and the
-step-by-step push via the huggingface_hub CLI.
+## Prerequisites
+
+- A Hugging Face account with an access token (User Settings → Access Tokens; needs **write** scope).
+- The token in your local `.env` as `HF_TOKEN=hf_…`.
+- The Python deps installed (`uv sync` in the project root).
+
+## One-shot deploy via `huggingface_hub` Python API
+
+Everything below — create the Space, set secrets, upload files — can be done in one Python script. This is what was actually run to produce the live demo.
+
+```python
+from huggingface_hub import HfApi
+from src.config import settings
+
+REPO_ID = "<your-username>/oss-vs-frontier-assistant"
+api = HfApi(token=settings.hf_token)
+
+# 1) Create the Space (idempotent; safe to re-run).
+api.create_repo(
+    repo_id=REPO_ID,
+    repo_type="space",
+    space_sdk="gradio",
+    space_hardware="cpu-basic",     # or "zero-a10g" if you have HF PRO
+    exist_ok=True,
+    private=False,
+)
+
+# 2) Set every required secret. These show up as env vars in the Space runtime.
+for k, v in {
+    "ANTHROPIC_API_KEY":   settings.anthropic_api_key,
+    "HF_TOKEN":            settings.hf_token,
+    "TAVILY_API_KEY":      settings.tavily_api_key,
+    "LANGFUSE_PUBLIC_KEY": settings.langfuse_public_key,
+    "LANGFUSE_SECRET_KEY": settings.langfuse_secret_key,
+    "LANGFUSE_HOST":       settings.langfuse_host,
+}.items():
+    if v:
+        api.add_space_secret(repo_id=REPO_ID, key=k, value=v)
+
+# 3) Upload the project, excluding local-only / sensitive files.
+api.upload_folder(
+    repo_id=REPO_ID,
+    repo_type="space",
+    folder_path=".",
+    commit_message="deploy",
+    ignore_patterns=[
+        ".env", ".git/**", ".venv/**", ".pytest_cache/**", ".claude/**",
+        "data/**", "results/**", "__pycache__/**", "**/__pycache__/**",
+        "*.pyc", ".gitignore", "uv.lock",
+    ],
+)
+```
+
+Why this approach over the web UI:
+
+- **No browser steps.** Reproducible from any machine with the token.
+- **Secrets travel safely.** They never leave your machine in plaintext; the SDK posts them over HTTPS directly to the Space config.
+- **Re-runnable.** `exist_ok=True` + `upload_folder` overwrite makes re-deploys trivial.
+
+## What HF Spaces reads
+
+| File         | Role on Spaces                                                                 |
+|--------------|--------------------------------------------------------------------------------|
+| `README.md`  | The YAML frontmatter at the top configures the Space (sdk, hardware, etc.).    |
+| `requirements.txt` | Installed at build time. **Must be kept in sync with `pyproject.toml`.**  |
+| `app.py`     | Entry point (`app_file: app.py` in the YAML); HF imports it and finds `demo`.  |
+
+The YAML frontmatter currently used:
+
+```yaml
+---
+title: OSS vs Frontier Assistant
+emoji: 🤖
+colorFrom: indigo
+colorTo: purple
+sdk: gradio
+sdk_version: 6.14.0
+python_version: "3.11"
+app_file: app.py
+hardware: cpu-basic
+pinned: false
+---
+```
+
+## Switching to ZeroGPU later
+
+1. Subscribe to HF PRO ($9/mo).
+2. In the Space's **Settings → Hardware**, switch to `Nvidia A10G - Zero` (or rerun the deploy script with `space_hardware="zero-a10g"`).
+3. Update the YAML in `README.md` to `hardware: zero-a10g`.
+4. Re-upload: the `@spaces.GPU(duration=120)` decorator already on `QwenChatModel._generate` will start allocating real GPU time — Qwen latency drops from ~30-60s to ~3-8s per reply.
+
+## Re-deploy after code changes
+
+```bash
+# Bump requirements.txt if pyproject.toml changed, then:
+uv run python - <<'PY'
+from huggingface_hub import HfApi
+from src.config import settings
+HfApi(token=settings.hf_token).upload_folder(
+    repo_id="<your-username>/oss-vs-frontier-assistant",
+    repo_type="space",
+    folder_path=".",
+    commit_message="update",
+    ignore_patterns=[".env", ".git/**", ".venv/**", ".pytest_cache/**", ".claude/**",
+                     "data/**", "results/**", "__pycache__/**", "**/__pycache__/**",
+                     "*.pyc", ".gitignore", "uv.lock"],
+)
+PY
+```
+
+HF triggers a new build automatically when files change.
+
+## Troubleshooting
+
+| Symptom on Spaces                           | Likely cause / fix                                                            |
+|---------------------------------------------|-------------------------------------------------------------------------------|
+| Build fails on `torch`/`transformers` install | Mismatch between `requirements.txt` pin and HF base image — check `python_version`. |
+| `ANTHROPIC_API_KEY is not set` at runtime    | Secret not added in Space settings, or empty. Re-run the secrets loop above.  |
+| 403 on `create_repo` mentioning ZeroGPU      | ZeroGPU is gated behind HF PRO; use `space_hardware="cpu-basic"` instead.     |
+| Qwen replies very slowly (30-60s)            | Expected on `cpu-basic`. Switch to ZeroGPU per the section above.             |
+| Tracing missing from Langfuse                | Network timeout on the Space → traces. Non-fatal; bump `LANGFUSE_TIMEOUT=30`. |