Does sglang support offline benchmark tools? #12260
Unanswered
NiuMa-1234
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I'm using sglang's offline inference for RL, and I found that there seems to be only online benchmark tools. Is there any offline tool (and how to use it) or would this feature be supported in the future?
My offline call is like this:
An online benchmark example:
Beta Was this translation helpful? Give feedback.
All reactions