Media Integrations

Media integrations wire Anyy to implemented media services. They do not replace the main model; they provide specific capability routes for image generation, text to speech, and audio transcription.

Image Generation

Image generation uses the image_generate tool from the optional media_generation toolset.

Minimal config:

auxiliary_models:
  image_generation:
    - provider_id: image-provider
      model: image-model

media_generation:
  image:
    enabled: true
    default_aspect_ratio: square
    timeout_ms: 120000
    max_bytes: 5242880

Use Image Generation for runtime behavior, artifact metadata, routing order, and failure modes.

Text To Speech

TTS uses the text_to_speech tool from the tts toolset.

Minimal OpenAI-compatible config:

tts:
  enabled: true
  provider: openai
  timeout_ms: 60000
  max_chars: 4096
  openai:
    model: tts-1
    voice: alloy
    base_url: https://api.openai.com/v1
    api_key_ref: env:OPENAI_API_KEY
    response_format: mp3

voice:
  auto_tts: false

Use Text To Speech for provider types, artifacts, preview behavior, and delivery notes.

Audio Transcription

Audio transcription is part of media understanding. It runs before the assistant turn when inbound attachments are marked as audio or voice.

media_understanding:
  enabled: true
  audio:
    enabled: true
    provider: openai
    mode: auto
    on_failure: annotate
    timeout_ms: 60000
    max_bytes: 25000000
    max_chars: 12000
    providers:
      openai:
        type: openai
        model: gpt-4o-mini-transcribe
        base_url: https://api.openai.com/v1
        api_key_ref: env:OPENAI_API_KEY

Use Transcription for attachment rules, failure policy, and provider configuration.

Toolsets And Visibility

Media features are not all visible by default:

Capability	Runtime path	Toolset
Image generation	model-visible tool	media_generation
Text to speech	model-visible tool	tts
Audio transcription	inbound preprocessing	no chat tool

Enable only the toolsets you want the assistant to use in a session or default toolset profile.

Troubleshooting

• Image tool is missing: check media_generation.image.enabled, image-capable model selection, and the media_generation toolset.
• TTS tool is missing: check tts.provider, provider credentials, and the tts toolset.
• Audio does not transcribe: check media_understanding.audio.enabled, audio.provider, attachment kind, and size limits.
• Provider auth fails: use credential refs such as env:NAME, file:/path, or secret:name; do not put raw secrets in YAML.