Instructions to use amirzandi/Chatterbox-Persian-TTS with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Chatterbox
How to use amirzandi/Chatterbox-Persian-TTS with Chatterbox:
# pip install chatterbox-tts import torchaudio as ta from chatterbox.tts import ChatterboxTTS model = ChatterboxTTS.from_pretrained(device="cuda") text = "Ezreal and Jinx teamed up with Ahri, Yasuo, and Teemo to take down the enemy's Nexus in an epic late-game pentakill." wav = model.generate(text) ta.save("test-1.wav", wav, model.sr) # If you want to synthesize with a different voice, specify the audio prompt AUDIO_PROMPT_PATH="YOUR_FILE.wav" wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH) ta.save("test-2.wav", wav, model.sr) - Notebooks
- Google Colab
- Kaggle
Chatterbox TTS — Persian
A full fine-tune (FFT) of the Chatterbox Multilingual T3 backbone on Thomcles/Persian-Farsi-Speech (~109k clips / 417 hours / 16kHz).
Feed it raw Persian text — no G2P, no phonemes. Honest model card: weaknesses listed below.
🔊 Demo
| Sample | Text |
|---|---|
| «صبح زود، وقتی از خانه بیرون آمدم، متوجه شدم هوای شهر برخلاف روزهای گذشته آرام و دلپذیر است. در مسیر، یک صدای غیرعادی از خیابان کناری شنیدم و برای چند لحظه مکث کردم، اما ناگهانی اتفاقی افتاد که باعث شد همهی رهگذران به سمت دیگری نگاه کنند. من بدون عجله به راهم ادامه دادم و سعی کردم لحن صحبت کردنم را طبیعی و آرام نگه دارم، چون حتی یک تغییر کوچک در تلفظ یا مکث میتواند کیفیت گفتار را تحت تأثیر قرار دهد.» | |
| «این یک پاراگراف بلندتره تا سیستم چانکینگ رو تست کنه و کلمهها رو درست سر هم بذاره.» |
Generate anything yourself in the ready Colab notebook — zero config, free GPU.
🚀 Usage
pip install git+https://github.com/resemble-ai/chatterbox.git
pip install transformers==4.46.3 safetensors soundfile
import torch
from chatterbox.mtl_tts import ChatterboxMultilingualTTS, punc_norm
from safetensors.torch import load_file
model = ChatterboxMultilingualTTS.from_pretrained(device="cuda")
sd = load_file("t3_persian_v1.safetensors")
model.t3.load_state_dict(
{k[3:] if k.startswith("t3.") else k: v for k, v in sd.items()}, strict=False)
model.t3.to("cuda").eval()
wav = model.generate(
punc_norm("سلام، امروز هوا خوب است."),
language_id=None, # raw Persian text, no language tag
audio_prompt_path="ref.wav", # any clean 5–10s Persian voice clip
temperature=0.6, repetition_penalty=1.5,
)
import torchaudio
torchaudio.save("out.wav", wav.squeeze(0).cpu(), model.sr)
The ve and s3gen modules and the tokenizer load automatically from the official base repo — only the file above comes from here.
Tips: use temperature=0.6 + repetition_penalty=1.5 (better than library defaults), and generate 2–3 takes per sentence, keeping the best. Occasional hallucination is inherent to the base architecture; a retry usually fixes it.
📊 Evaluation (Whisper large-v3-turbo CER, best-of-8)
| Text type | CER |
|---|---|
| Natural Persian (incl. long paragraphs) | 0.01–0.09 |
| 22-sentence stress set (numbers, clock times, foreign words, tongue twisters) | 0.038–0.062 |
| Artificial tongue-twisters | ≈ 0.3 (weakest spot) |
⚠️ Limitations
- 16kHz output; cloning quality depends entirely on your reference clip (clean, mono, 5–10s)
- Numbers and foreign words mostly work, but are not guaranteed
- Trained on 417h of general speech: it improves Persian voice cloning — it does not reproduce one specific speaker
📄 License & credits
Fine-tuned weights: MIT. Base model: ResembleAI/chatterbox (MIT). Training data: Thomcles/Persian-Farsi-Speech (CC BY 4.0) — special thanks to Thomcles and Resemble AI.
- Downloads last month
- -
Model tree for amirzandi/Chatterbox-Persian-TTS
Base model
ResembleAI/chatterbox