How can I use EP MoE without setting up TP? #4475
-
|
I have two GPUs on my server, but when I set ep-size=2, the log shows ep-size=1, and the inference is still performed on only one GPU. I have read the relevant source code of |
Beta Was this translation helpful? Give feedback.
Answered by
Fridge003
Mar 17, 2025
Replies: 1 comment 1 reply
-
|
Currently ep-size will be automatically set to tp-size. You can have a look at this doc . |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thanks. |
Beta Was this translation helpful? Give feedback.
Answer selected by
Fridge003
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently ep-size will be automatically set to tp-size. You can have a look at this doc .