[ROCm] Failure to run chandra on AMD MI210
#4
by
XianHong1101
- opened
We are attempting to run the VLM model datalab-to/chandra on an AMD MI210 GPU using the official ROCm Docker images.
Does vLLM on the ROCm (AMD) platform currently support VLM models like datalab-to/chandra that require --trust-remote-code?
It seems the custom code for these models is CUDA-specific and/or the vLLM-ROCm loader does not support their architecture.
Thank you!
If you're able to install the latest vllm on your AMD GPU, you can run vllm serve datalab-to/chandra. You don't need to add the --trust-remote-code flag
tarun-menta
changed discussion status to
closed