vulkan: Remove transfer_ctx, do everything in compute_ctx. #18945
Conversation
github-actions
bot
added
Vulkan
|
llama-embedding test is failing in CI, I assume this means the output is corrupt somehow. I'll look at it closer tomorrow. |
We had a bug where a set_tensor_async (using transfer_ctx) didn't get submitted before the graph_compute (using compute_ctx) that came after it. To avoid this sort of issue, just do everything in compute_ctx. Remove transfer_cmd_pool, which was already unused.
jeffbolznv
force-pushed
the
issue_18786
branch
from
8972611 to
e93b5b4
Compare
|
Isn't there an advantage on certain hardware to using a transfer-specific queue? Maybe there's a way to solve this with synchronization between the queues. |
|
If the transfer queue can run in parallel with the compute work then that's potentially an advantage, but if we'd need to sync them against each other with semaphores to preserve the ordering that ggml expects, then we wouldn't be able to get that. And these operations were already on the compute queue (being created from compute_cmd_pool) so that hasn't changed. The commands in the ggml_backend_buffer_i interface still use the transfer queue. |
|
Hmm, I'm seeing a crash somewhere, set to draft for now. |
|
The crash was with the perf_logger enabled, fixed now. |
2 participants
We had a bug where a set_tensor_async (using transfer_ctx) didn't get submitted before the graph_compute (using compute_ctx) that came after it. To avoid this sort of issue, just do everything in compute_ctx.
Remove transfer_cmd_pool, which was already unused.
Fixes #18786.