Configuring Models
Anyy uses a selected provider/model for ordinary conversations. You can configure ordered fallbacks and separate image, video, and speech routes. This guide describes v0.1.107.
Choose a Configuration Surface
| Goal | Surface |
|---|---|
| Configure a provider and main model | anyy setup model |
| View and edit model routes | Dashboard → Models |
| Configure fallback routing | anyy setup model fallback or Dashboard → Models |
| Inspect paths and credential-reference status | anyy config |
| Switch the current conversation | Chat model selector or TUI /model |
Start with:
anyy setup model
A route needs a provider, a supported endpoint/protocol, authentication, and the exact model ID accepted by that endpoint. The wizard discovers models where possible, permits manual entry, and shows a review before writing. Dashboard’s Test connection checks catalog connectivity and whether the selected model is listed, using the pending or saved connection settings. It sends no conversation-generation request.
After saving, open a new Chat session and check that its selected provider/model is the intended target. Send a short prompt such as “Reply with Ready,” and confirm that an actual reply returns. For this check, temporarily disable configured fallback routes and restore their previous settings afterward. If you keep fallback enabled, confirm that the actual reply was produced by the tested provider/model without fallback before treating that target as working. The session’s selected model and a successful reply alone do not establish which target generated it; catalog discovery, login, or configuration status does not either.
Supported Protocols
The current five conversation protocol families are openai-compatible, openai-responses, codex-responses, anthropic-messages, and google-gemini. They follow Anyy's fixed Pi v1.2.0 reference for shared behavior.
Azure OpenAI Responses, Bedrock Converse, Google Vertex, Mistral Conversations, and pi-messages do not have independent native implementations in this release. A catalog entry, provider brand, or compatible Base URL is not evidence that a distinct native API is supported. See Provider Protocols and Model Interfaces for the full boundary, model types, and retry behavior.
Local Providers
Start Ollama or LM Studio before setup and install at least one model. Ollama uses its OpenAI endpoint with openai-compatible and default Base URL http://127.0.0.1:11434/v1; /api/tags is catalog discovery, not a separate inference protocol. Ollama Cloud uses https://ollama.com/v1. Local Ollama defaults to no authentication; a protected endpoint can use its supported API-key option.
Old api_mode: ollama, ollama.local.native, and ollama.cloud.native inference settings are rejected. Rerun setup and select the OpenAI endpoint, or explicitly migrate to openai-compatible, the service's /v1 Base URL, and ollama.local.openai or ollama.cloud.openai. Anyy does not silently rewrite these values.
Compatible Endpoints
Use OpenAI-compatible or Anthropic-compatible only when the service implements that protocol. Enter its required API root and exact model ID. Capability metadata must match the model; manual entry does not make an unknown model support images, tools, or thinking.
The provider can receive prompts, context, attachments, and tool results needed for the request. Review its privacy, retention, account, and billing terms.
Main Model and Credentials
After setup, inspect the result:
anyy config
anyy status
anyy config shows non-secret configuration, resolved paths, and credential-reference health. Anyy-managed keys are stored under <ProfileHome>/secrets/ with file mode 0600; configuration stores references. An environment reference such as env:MY_MODEL_KEY works only if the resident Gateway inherits that variable.
Dashboard → Models supports entering, replacing, and clearing keys, plus the subscription actions exposed by the selected endpoint. Codex uses its ChatGPT subscription device login. The Anthropic Claude Code subscription option accepts an externally obtained setup token; it is not a generic login for every OAuth provider. Blocked catalog choices remain unavailable. Successful login or catalog discovery still does not prove that every model or remote account works.
Use the same Models/setup surface to reconnect or disconnect a supported credential. Avoid secrets in prompts, shell history, screenshots, and support reports.
Fallback Routing
After the main route works:
anyy setup model fallback
Add independent provider/model targets in preference order. Dashboard can enable or disable each entry and set allow_reasoning_drop, allow_context_drop, and max_attempts. To check a fallback’s generation access, explicitly select that provider/model in a new conversation and send a short prompt; apply the same disabled-fallback or confirmed-no-fallback check above. Its catalog check alone is insufficient.
Fallback is conditional availability routing. It checks the classified error and target compatibility, including tool/media requirements, reasoning history, and context size; it does not run every model or merge answers. Allowing a smaller context does not make the request fit automatically. Use independent endpoints and credentials when resilience matters.
Fallback attempt limits and Session auto-retry are separate controls. See Retry, fallback, and virtual routing.
Change the Current Session
Saving the main model sets defaults for new sessions. Existing sessions can retain their selected target and thinking state. Change that session explicitly in Dashboard Chat or use TUI commands:
/model
/provider
/effort
Use /new when you want a fresh conversation. Changing model or thinking can reduce prompt-cache reuse and increase cost.
Thinking Defaults
The named levels are off, minimal, low, medium, high, xhigh, and max. Controls show the selected model's supported choices, and requests are clamped to that model's mapping. Off can resolve to another supported level when a model cannot disable thinking. Non-reasoning models use off and have no effort picker.
| Action | Preference order before model clamping |
|---|---|
| New session | Per-model setting → global setting → medium |
| Switch model | Target per-model setting → global setting → current session thinking → medium |
| Resume a branch with messages | Saved branch thinking; if absent, global → medium |
| Resume a branch without messages | Initial model defaults and per-model → global → medium |
Saved model/thinking wins over later settings when resuming messages. Busy sessions are left unchanged. Restoring selection sends no model request. With no selected model, initial thinking is off.
inference.level retains the requested preference; read-only inference.effective_level shows its clamped result. The global preview uses the main model without per-model overrides, so it can differ from a new session. Saving an untouched Dashboard effort control does not write its derived value as a preference.
In TUI /effort, Enter applies the session choice and saves the requested global preference while preserving the Profile's saved provider/model; Alt+S changes only this session. Neither writes a per-model setting. See Thinking configuration and migration for YAML keys and retired settings.
Thinking History
OpenAI-compatible Chat retains nonempty native reasoning_content, reasoning, or reasoning_text, including when the catalog did not mark the model as reasoning-capable. Structured reasoning_details stays opaque history. Same-target continuations restore the matching native field; changing targets follows the protocol's history conversion rules. Do not copy hidden thinking into answer text or invent extra capability switches to enable replay.
Auxiliary Routes
Dashboard → Models groups image understanding, video understanding, image generation, TTS, and STT. Native image/video input requires a model that declares the capability. summary_first understanding requires a dedicated route. Speech endpoints have their own protocols and credentials; a working text model does not enable speech automatically.
The product image-generation tool uses its supported OpenAI Responses route. Codemode's models.generateImages() is a separate typed model operation; configuring one does not configure the other. See Image Generation and Voice & Speech.
Troubleshooting
| Symptom | Check |
|---|---|
| Authentication error | Account, key/login, endpoint credential realm, and resident process environment |
| Model not found | Refresh discovery or enter the endpoint's exact model ID |
| Local connection refused | Start the local server and verify its API root |
| Fallback never helps | Test routes separately and check enablement, order, and compatibility |
| Old chat keeps its model | Switch that session or start /new |
| Unexpected cost or latency | Check model, thinking, retry limits, fallback attempts, and media routes |
For visual controls, see Dashboard Models.