Instructions to use buddhist-nlp/mitra-qwen35-2b-embedder-sentence-alignment with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use buddhist-nlp/mitra-qwen35-2b-embedder-sentence-alignment with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="buddhist-nlp/mitra-qwen35-2b-embedder-sentence-alignment")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("buddhist-nlp/mitra-qwen35-2b-embedder-sentence-alignment") model = AutoModel.from_pretrained("buddhist-nlp/mitra-qwen35-2b-embedder-sentence-alignment", device_map="auto") - Notebooks
- Google Colab
- Kaggle
mitra-qwen35-2b-embedder-sentence-alignment
A variant of mitra-qwen35-2b-embedder
specialised for sentence alignment of Buddhist texts and their translations
(e.g. as the scoring model in a vecalign-style aligner). It is trained to give the
highest similarity to the exact translation of a sentence, and lower similarity to
spans that contain the translation plus neighbouring material.
Use the base embedder for general retrieval. Use this model for alignment.
Training
Both stages start from the same Qwen3.5-2B base as the general embedder: Buddhist-domain continued pretraining followed by stage-2 SFT. Both stages train the same LoRA adapter (r=32, alpha 64, all attention and MLP projections), which is merged into this checkpoint.
- Parallel-sentence contrastive stage. 8.8M strictly parallel sentence pairs over 27 language pairs (Sanskrit, Tibetan, Chinese and Pāli with each other and with modern languages), decontaminated against the evaluation sets. Symmetric InfoNCE with in-batch negatives only (512 pairs per step), temperature 0.02, lr 5e-5, 4000 steps.
- Span-precision stage. Continues the stage-1 adapter for 1500 steps on runs of consecutive sentence pairs drawn from the same data. Each query's positive is its exact translation. Hard negatives are the same translation padded with the preceding and/or following sentence, on both the target side and the query side. 20% of the units are two-sentence pairs whose single-sentence halves serve as negatives, so the model does not simply prefer shorter spans. Batches hold 256 units, plus all of their hard negatives. Temperature 0.02, lr 5e-5.
Difference from the general embedder. The released mitra-qwen35-2b-embedder (v2)
is trained on a mix of retrieval data with mined hard negatives. This model sees only parallel sentence pairs and
span-level negatives. It is much better at choosing the exact sentence span. It is
weaker on loose, long-range matches such as commentary retrieval and question answering.
Evaluation (vs. mitra-qwen35-2b-embedder v2)
| this model | v2 | |
|---|---|---|
| Span precision: exact translation beats padded variants (5,100 items, 34 directions) | 0.950 | 0.766 |
| Complete translation beats partial or over-long spans | 0.908 | 0.735 |
| Document alignment, link F1 (vecalign, 18 language pairs, 6 noise variants) | 0.942 | 0.913 |
| Document alignment, hardest variant (merges + distractors + drift), link F1 | 0.851 | 0.753 |
| Internal quick suite R@1: multilingual parallels / English→classical | 0.857 / 0.954 | 0.838 / 0.954 |
| Internal quick suite R@1: verse→commentary / question→passage | 0.450 / 0.558 | 0.639 / 0.657 |
In the document alignment evaluation, the deletion penalty is chosen per model by cross-validation. With this model, a fixed deletion penalty of about 0.26–0.30 (in vecalign's normalized cost units) works much better than vecalign's percentile-based default. Use 0.26 for noisy documents.
Usage
Same prompting and pooling as the general 2B embedder. Source sentences get the instruction, and target sentences are raw text. Tibetan in native script, Sanskrit and Pāli in IAST.
import torch
from transformers import AutoModel, AutoTokenizer
name = "buddhist-nlp/mitra-qwen35-2b-embedder-sentence-alignment"
tok = AutoTokenizer.from_pretrained(name)
tok.padding_side = "left"
model = AutoModel.from_pretrained(name, dtype=torch.bfloat16, device_map="cuda").eval()
def embed(texts, target_lang=None, batch_size=32):
if target_lang: # source side
texts = [f"<instruct>Please find the semantically most similar text in {target_lang}.\n<query>{t}"
for t in texts]
out = []
with torch.no_grad():
for i in range(0, len(texts), batch_size):
enc = tok(texts[i:i+batch_size], padding=True, truncation=True,
max_length=512, return_tensors="pt").to(model.device)
h = model(**enc).last_hidden_state[:, -1]
out.append(torch.nn.functional.normalize(h.float(), dim=-1).cpu())
return torch.cat(out)
src = embed(["...Tibetan sentences..."], target_lang="English")
tgt = embed(["...English sentences..."])
EOS is appended by the bundled tokenizer. Pooling is last-token, L2-normalized, dimension 2048.
Citation
If you use this model, please cite the Dharmamitra project (https://dharmamitra.org). A technical report is in preparation.
- Downloads last month
- 90