Consider add Mercury Coder as prediction provider or make Zeta a dLLM

[LaneSun]

Today, I happened to come across this blog post mentioning that Continue uses Mercury Coder as their Prediction model, and I thought this might be something worth considering for Zed.

First, I want to state that the current Edit Prediction feature works quite well, and using it occasionally does save some time.

That said, the current Predictions are still somewhat limited in intelligence, making it feel more like an extension of LSP functionality rather than a true AI feature.

In the past, I would have said, well, there's no way around it—we can’t request a heavyweight model for every keystroke.

But now, the combination of Mercury Coder and Continue has broken that barrier. Perhaps behind the scenes, Mercury Coder isn’t a compact model, but it is undoubtedly very fast. Also, like their viewpoint, I believe that dLLM is more suitable for Edit Prediction tasks than general LLMs.

We have an opportunity to obtain a powerful yet fast method for Edit Prediction.

Therefore, I’d like to propose two different suggestions:

1. Like Continue, add Mercury Coder as a provider: Personally, I’m not in favor of this approach. I don’t want the core functionality of the editor to be out of the team’s control. However, this would likely be the easiest way for the team and the fastest method for us to gain a more powerful experience.

2. Consider changing the base model of Zeta to something like Dream-Coder 7B: Given the masked generation characteristics of dLLMs, we could skip the prediction process for contextually known code or templates while the dLLM is completing, and simultaneously leverage the dLLM’s adjustable inference effort—using high effort at the beginning and low effort toward the end—to further improve prediction speed.

Additionally, considering the content mentioned in the blog post (such as automated refactoring of entire codebases), I suspect that Continue uses RAG to power their Edit Prediction feature behind the scenes (though I haven’t used it and have no evidence). Therefore, I think it might also be worth considering implementing a RAG system based on code semantics as part of the AI functionality infrastructure. Given that we have a fully consistent LSP interface, this should be feasible. If implemented, this could completely transform how we search and refactor code. Moreover, for dynamic languages like JS and Python, this might be the only way to achieve refactoring capabilities close to those of statically analyzed languages.