Which vLLM version am I supposed to install to match latest SGLang? #11180
Unanswered
Ouna-the-Dataweaver
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I tried running an AWQ quant of GLM-4.5-Air on an A100, but SGLang threw kernel-name mismatch errors. I patched a few things(like there were mismatches related to tenson names and kernel names), and eventually it finished at mismatch between vLLM cuda version and SGLang. Which vLLM version (both the library and CUDA wheel) should I install alongside SGLang to avoid these mismatches?” I found that AWQ sort of expect 0.8.4, but it's a really old vLLM, and it requires like 2 versions older torch (with different cuda at that) compared to newest SGLang...
Beta Was this translation helpful? Give feedback.
All reactions