LLM Providers that work with Monty

OpenAI

Get an API key (opens in a new tab)

Available models:

• gpt-4-turbo-preview
• gpt-3.5-turbo

Groq

Get an API key (opens in a new tab)

Available models:

• mixtral-8x7b-32768
• llama2-70b-4096
• gemma-7b-it

Custom

Monty works with OpenAI API-compatible LLMs. More specifically, models that support the Chat Completions API (opens in a new tab).

Providers we've tested

OpenAI (opens in a new tab)

• Base url: https://api.openai.com/v1
• API keys (opens in a new tab)
• Recommended models: gpt-4-turbo, and gpt-3.5-turbo

Groq (opens in a new tab)

• Base url: https://api.groq.com/openai/v1
• API keys (opens in a new tab)
• Recommended model: mixtral-8x7b-32768

Together AI (opens in a new tab)

• Base url: https://api.together.xyz/v1
• API keys (opens in a new tab)
• Recommended models: mistralai/Mixtral-8x22B-Instruct-v0.1, and mistralai/Mixtral-8x7B-Instruct-v0.1

Anyscale (opens in a new tab)

• Base url: https://api.endpoints.anyscale.com/v1
• API keys (opens in a new tab)
• Recommended model: mistralai/Mixtral-8x7B-Instruct-v0.1

If you discover other providers that work with Monty please send us a message in our Discord (opens in a new tab).

Local models with Ollama (opens in a new tab)

Ollama includes a local server that is OpenAI API-compatible (opens in a new tab). We've tested the following:

• Base url: http://localhost:11434/v1
• API key can be anything, e.g foo
• Recommended model: phi3

Azure OpenAI

Use your own private deployment. Learn more (opens in a new tab)

Anthropic

Get an API key (opens in a new tab)

Available models:

• claude-3-opus-20240229
• claude-3-sonnet-20240229
• claude-3-haiku-20240307

Google AI Studio

Get an API key (opens in a new tab)

Available models:

• gemini-1.5-pro-latest

Unsupported providers (for now)

• AWS Bedrock
• Google Vertex AI