Search results

'cudaMemcpy(ith_context_req_token_index.ptr, ith_context_req_token_index_cpu, sizeof(int32_t) * (batch_size+1), cudaMemcpyHostToDevice)'

Status: Open.

#66 In LLMServe/DistServe;

· Sabiha1225 opened on Jun 27, 2025

关于论文引用的小小问题

Status: Open.

#65 In LLMServe/DistServe;

· Qiu-Jianrong opened on Apr 28, 2025

初始化的时候， 报错

Status: Open.

#64 In LLMServe/DistServe;

· Haoyanlong opened on Apr 23, 2025

the inference result sentence is null

Status: Open.

#63 In LLMServe/DistServe;

· Dreamer-HIT opened on Apr 8, 2025

How to guarantee that kv cache transmission finished

Status: Open.

#61 In LLMServe/DistServe;

· HarryWu99 opened on Mar 17, 2025

How to handle preemption cases

Status: Open.

#59 In LLMServe/DistServe;

· Mrxiangli opened on Feb 26, 2025

How to use DistServe with ray?

Status: Open.

#58 In LLMServe/DistServe;

· Liaukx opened on Feb 25, 2025

Error: Peer-to-peer access is unsupported on this platform.

Status: Open.

#57 In LLMServe/DistServe;

· hh001-cyber opened on Feb 21, 2025

When using the cmake - B build command, the following message appears in a continuous loop

Status: Open.

#56 In LLMServe/DistServe;

· 67lc opened on Feb 7, 2025

两个节点通过ray部署，每个节点一个GPU，运行不起来。

Status: Open.

#55 In LLMServe/DistServe;

· Liufeiran123 opened on Jan 17, 2025

How to independently measure the performance of the Prefill phase and the Decode phase？

Status: Open.

#50 In LLMServe/DistServe;

· J1nLo opened on Nov 19, 2024

offline/online serving is stuck at fetching files in paraworkers

Status: Open.

#48 In LLMServe/DistServe;

· hyuenmin-choi opened on Oct 16, 2024