File size: 2,316 Bytes
6df9665 dd850a7 6df9665 8473a16 6df9665 dd850a7 6df9665 dd850a7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 |
---
title: Token Attention Visualizer
emoji: 🔍
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.42.0
app_file: app.py
pinned: false
license: apache-2.0
---
# Token Attention Visualizer
An interactive tool for visualizing attention patterns in Large Language Models during text generation.
## Features
- 🚀 **Real-time Generation**: Generate text with any Hugging Face model
- 🔍 **Attention Visualization**: Explore attention patterns with clear visual representations
- 📊 **Dual Normalization**: Choose between separate or joint attention normalization
- ⚡ **Smart Caching**: Fast response with intelligent result caching
- 🎯 **Token Selection**: Use dropdown menus to select and filter token connections
- 📈 **Step Navigation**: Navigate through generation steps
- 🎨 **Customizable Threshold**: Filter weak attention connections
## How It Works
The visualizer shows how tokens attend to each other during text generation:
- **Blue lines**: Attention from input tokens to output tokens
- **Orange curves**: Attention between output tokens
- **Line thickness**: Represents attention weight strength
## Usage
1. **Load a Model**: Enter a Hugging Face model name (default: HuggingFaceTB/SmolLM-135M-Instruct)
2. **Enter Prompt**: Type your input text
3. **Configure Settings**: Adjust max tokens, temperature, and normalization
4. **Generate**: Click to generate text and visualize attention
5. **Explore**: Use dropdown menus to select tokens and view their attention patterns
## Technical Details
- Built with Gradio for the interface
- Visualization system with dropdown-based token selection
- Supports any Hugging Face causal language model
- Optimized for smaller models like SmolLM for efficient deployment
- Implements efficient attention processing and caching
## Local Development
```bash
# Clone the repository
git clone <repo-url>
cd token-attention-viz
# Install dependencies
pip install -r requirements.txt
# Run the app
python app.py
```
## Deployment
This app is designed for easy deployment on Hugging Face Spaces. Simply:
1. Create a new Space
2. Upload the project files
3. The app will automatically start
## Requirements
- Python 3.8+
- 4GB+ RAM (SmolLM models are lightweight)
- GPU acceleration optional (works well on CPU)
## License
Apache 2.0 |