Documentation
Model Compatibility

LLM Providers that work with Monty

OpenAI

Get an API key (opens in a new tab)

Available models:

  • gpt-4-turbo-preview
  • gpt-3.5-turbo

Groq

Get an API key (opens in a new tab)

Available models:

  • mixtral-8x7b-32768
  • llama2-70b-4096
  • gemma-7b-it

Custom

Monty works with OpenAI API-compatible LLMs. More specifically, models that support the Chat Completions API (opens in a new tab).

Providers we've tested

OpenAI (opens in a new tab)

Groq (opens in a new tab)

Together AI (opens in a new tab)

  • Base url: https://api.together.xyz/v1
  • API keys (opens in a new tab)
  • Recommended models: mistralai/Mixtral-8x22B-Instruct-v0.1, and mistralai/Mixtral-8x7B-Instruct-v0.1

Anyscale (opens in a new tab)

If you discover other providers that work with Monty please send us a message in our Discord (opens in a new tab).

Local models with Ollama (opens in a new tab)

Ollama includes a local server that is OpenAI API-compatible (opens in a new tab). We've tested the following:

  • Base url: http://localhost:11434/v1
  • API key can be anything, e.g foo
  • Recommended model: phi3

Azure OpenAI

Use your own private deployment. Learn more (opens in a new tab)

Anthropic

Get an API key (opens in a new tab)

Available models:

  • claude-3-opus-20240229
  • claude-3-sonnet-20240229
  • claude-3-haiku-20240307

Google AI Studio

Get an API key (opens in a new tab)

Available models:

  • gemini-1.5-pro-latest

Unsupported providers (for now)

  • AWS Bedrock
  • Google Vertex AI