Configure a local AI autocomplete model for Zed

Zed's AI Autocomplete feature is powered by Zeta, Zed’s open-weight edit prediction model. By default, Zeta hosted by Zed. This is a great default. However, if you’d like to use Zed at work, you may have to disable this handy feature as many employers don’t permit using models hosted by third party providers.

Fortunately, this is easy to resolve. Zed gives its users the option to use an edit prediction model that’s running locally

https://zed.dev/blog/edit-prediction-providers
https://zed.dev/docs/ai/edit-prediction#local-and-self-hosted-models

Since Zeta itself is an open-weight model, you can use Zeta. To use Zeta, follow the steps below.

Install ollama (unless you already have it installed)

brew install ollama

Start ollama server

ollama server

Download Zeta 2.1, the current latest version of Zeta

ollama run hf.co/mradermacher/zeta-2.1-GGUF:Q4_K_M

After ollama run downloads Zeta, type /bye to exit ollama run

❯ ollama run hf.co/mradermacher/zeta-2.1-GGUF:Q4_K_M
pulling manifest
pulling f031e71d3a46: 100% ▕███████████████████████████▏ 5.1 GB
pulling a4da62026d0c: 100% ▕███████████████████████████▏  404 B
verifying sha256 digest
writing manifest
success
>>> /bye

Lastly, open Zed’s settings file

vim ~/.config/zed/config.json

and configure Zed to use ollama’s server

{
  // ...
  "edit_predictions": {
    "provider": "ollama",
    "ollama": {
      "api_url": "http://localhost:11434",
      "model": "hf.co/mradermacher/zeta-2.1-GGUF:Q4_K_M",
      "prompt_format": "zeta2_1",
      "max_output_tokens": 512,
    }
  }
  // ...
}

That's it! Start typing in any file and predictions should appear

If prediction latency is too high, try running a smaller model (qwen2.5-coder:7b-base, starcoder2:3b or deepseek-coder:6.7b-base) with the an appropriate prompt format (qwen, star_coder and deepseek_coder respectively) https://zed.dev/docs/ai/edit-prediction#local-and-self-hosted-models

Configure a local AI autocomplete model for Zed

Resources