README.md

---
library_name: transformers
datasets:
- DataSeer/si-summarization-votes-r1-081725
base_model: Qwen/Qwen3-32B
tags:
- lora
- supervised-fine-tuning
- summarization
- qwen3
---

# Qwen3-32B Summarization LoRA Adapter

A LoRA (Low-Rank Adaptation) fine-tuned adapter for the Qwen3-32B model, specifically trained for summarizing supplemental information for articles. We used multi-turn reinforcement learning based on the rollouts in the DataSeer summarization votes dataset (human preference data).

## Model Details

### Model Description

This adapter fine-tunes the Qwen3-32B base model for improved summarization capabilities using LoRA technique.

- **Developed by:** DataSeer
- **Model type:** Causal Language Model (LoRA Adapter)
- **Language:** English
- **Base model:** Qwen/Qwen3-32B
- **Training approach:** Multi-turn RL with LoRA
- **Dataset:** DataSeer/si-summarization-votes-r1-081725

### Model Architecture

- **Base Model:** Qwen3-32B (32.8B parameters)
- **LoRA Configuration:**
  - Rank (r): 8
  - Alpha: 32
  - Target modules: q_proj, v_proj, k_proj, o_proj, gate_proj, up_proj, down_proj
  - Dropout: 0
- **Precision:** bfloat16

## Training Details

### Training Data

The model was trained on the `DataSeer/si-summarization-votes-r1-081725` dataset, which contains summarization rollouts with annotator votes. The dataset was filtered to include only positively-voted examples (label=True).

### Training Configuration

- **Training epochs:** 2
- **Learning rate:** 1e-3 (0.001)
- **Batch size:** 1 per device
- **Gradient accumulation steps:** 8
- **Effective batch size:** 8
- **Learning rate scheduler:** Cosine
- **Optimizer:** AdamW (torch fused)
- **Precision:** bfloat16
- **Gradient checkpointing:** Enabled
- **Max sequence length:** 18,893 tokens

### Training Results

- **Final training loss:** 0.3414
- **Mean token accuracy:** 88.13%
- **Total training steps:** 62
- **Training runtime:** 37.9 minutes (2,273 seconds)
- **Training samples per second:** 0.216
- **Final learning rate:** 5.77e-6

### Hardware & Performance

- **Hardware:** 8x NVIDIA H100 80GB HBM3
- **Training time:** ~38 minutes
- **Memory optimization:** Gradient checkpointing, bfloat16 precision

## Usage

### Loading the Model

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-32B",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-32B")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "path/to/adapter")
```


### Environmental Impact
Training was conducted on high-performance H100 GPUs for approximately 38 minutes, representing a relatively efficient fine-tuning process thanks to the LoRA approach which only trains ~0.1% of the total model parameters.