Choosing who answers
"Who answers" is the language model behind the assistant. It can be a cloud model (OpenAI, Anthropic, Google Gemini, or another service that speaks the OpenAI chat interface: you need a key, and the provider bills you) or a model on your own computer or another one in your home (Ollama, LM Studio, a llama.cpp server, vLLM, or any server that speaks the OpenAI-compatible API: free, private, but needs a good graphics card). Lessons need none of them.
How to choose
- Open Settings ▸ Assistant: the gear ▸ Assistant, Assistant ▸ Assistant settings…, or Choose… (or Choose an assistant…) on the welcome card.
- Pick a card under Provider.
- Fill in what that kind needs (below), pick the Model, press Test connection.
- Save settings.
Later you can switch between the ones you set up with the chooser on the toolbar or Assistant ▸ Choose the assistant; the choice is saved at once.
What each kind needs
| Kind | Needs | Page |
|---|---|---|
| OpenAI | a key, a model name | OpenAI |
| Anthropic | a key, a model name | Anthropic |
| Google Gemini | a key, a model name | Google Gemini |
| OpenAI-compatible | the service address with https, its own key, a model | OpenAI-compatible services |
| Your own server: Ollama | the address (http://127.0.0.1:11434/v1), a model | Ollama |
| Your own server: LM Studio | the address (http://127.0.0.1:1234/v1), a model | LM Studio |
| Your own server: llama.cpp | the address of llama-server (often http://127.0.0.1:8080/v1), a model | llama.cpp server |
| Your own server: vLLM or another compatible server | its address with /v1, a model | vLLM and other compatible servers |
How to pick a model
- It must call tools. The assistant reads your catalog and builds through tools. The model list marks each model tools or no tool calls; a model without tools gets "This model cannot call tools; the assistant needs them to read the catalog and build."
- Bigger is better at building. A small local model (a few billion parameters) answers simple questions but often loses its way in a build.
- The context window matters for a local model: 16 384 tokens or more. See The context window.
- Which model to start with, what it costs and how much memory it needs: Which model to take.
Cloud or local?
| Cloud | Your own server | |
|---|---|---|
| Cost | the provider bills per use | free, your electricity |
| Privacy | your messages and graph go to the provider | stay on your computers |
| Quality for building | high | depends on the model and your hardware |
| Setup | paste a key | install and start a server |
What goes where: What is sent to the model. If your network needs a proxy for the cloud, set it under Settings ▸ Assistant ▸ Network; your own server is always reached directly.