Instructions to use MERaLiON/MERaLiON-2-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MERaLiON/MERaLiON-2-3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="MERaLiON/MERaLiON-2-3B", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForSpeechSeq2Seq model = AutoModelForSpeechSeq2Seq.from_pretrained("MERaLiON/MERaLiON-2-3B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Remove flash-attn from requirements and GPU inference example
#2
by YingxuHe - opened
README.md
CHANGED
|
@@ -537,16 +537,6 @@ pip install transformers==4.50.1
|
|
| 537 |
pip install librosa
|
| 538 |
```
|
| 539 |
|
| 540 |
-
To run in GPU, MERaLiON-2 requires `flash-attn`.
|
| 541 |
-
|
| 542 |
-
```
|
| 543 |
-
pip install flash-attn --no-build-isolation
|
| 544 |
-
```
|
| 545 |
-
|
| 546 |
-
> [!TIP]
|
| 547 |
-
> Should you face any difficulties installing the above packages, you can refer to this docker:
|
| 548 |
-
> `pytorch/pytorch:2.5.1-cuda12.1-cudnn9-devel` as an example for a pre-built torch & cuda environment.
|
| 549 |
-
|
| 550 |
### Audio Input
|
| 551 |
|
| 552 |
- For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
|
|
@@ -666,7 +656,6 @@ model = AutoModelForSpeechSeq2Seq.from_pretrained(
|
|
| 666 |
repo_id,
|
| 667 |
use_safetensors=True,
|
| 668 |
trust_remote_code=True,
|
| 669 |
-
attn_implementation="flash_attention_2",
|
| 670 |
torch_dtype=torch.bfloat16
|
| 671 |
).to(device)
|
| 672 |
|
|
|
|
| 537 |
pip install librosa
|
| 538 |
```
|
| 539 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 540 |
### Audio Input
|
| 541 |
|
| 542 |
- For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
|
|
|
|
| 656 |
repo_id,
|
| 657 |
use_safetensors=True,
|
| 658 |
trust_remote_code=True,
|
|
|
|
| 659 |
torch_dtype=torch.bfloat16
|
| 660 |
).to(device)
|
| 661 |
|