Automatic Speech Recognition
Transformers
Safetensors
meralion2
meralion
meralion-2
custom_code

Remove flash-attn from requirements and GPU inference example

#2
Files changed (1) hide show
  1. README.md +0 -11
README.md CHANGED
@@ -537,16 +537,6 @@ pip install transformers==4.50.1
537
  pip install librosa
538
  ```
539
 
540
- To run in GPU, MERaLiON-2 requires `flash-attn`.
541
-
542
- ```
543
- pip install flash-attn --no-build-isolation
544
- ```
545
-
546
- > [!TIP]
547
- > Should you face any difficulties installing the above packages, you can refer to this docker:
548
- > `pytorch/pytorch:2.5.1-cuda12.1-cudnn9-devel` as an example for a pre-built torch & cuda environment.
549
-
550
  ### Audio Input
551
 
552
  - For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
@@ -666,7 +656,6 @@ model = AutoModelForSpeechSeq2Seq.from_pretrained(
666
  repo_id,
667
  use_safetensors=True,
668
  trust_remote_code=True,
669
- attn_implementation="flash_attention_2",
670
  torch_dtype=torch.bfloat16
671
  ).to(device)
672
 
 
537
  pip install librosa
538
  ```
539
 
 
 
 
 
 
 
 
 
 
 
540
  ### Audio Input
541
 
542
  - For ASR tasks, the maximum audio length is suggested to be 30 seconds at 16,000 Hz.
 
656
  repo_id,
657
  use_safetensors=True,
658
  trust_remote_code=True,
 
659
  torch_dtype=torch.bfloat16
660
  ).to(device)
661