LLM Providers that work with Monty
OpenAI
Get an API key (opens in a new tab)
Available models:
gpt-4-turbo-preview
gpt-3.5-turbo
Groq
Get an API key (opens in a new tab)
Available models:
mixtral-8x7b-32768
llama2-70b-4096
gemma-7b-it
Custom
Monty works with OpenAI API-compatible LLMs. More specifically, models that support the Chat Completions API (opens in a new tab).
Providers we've tested
- Base url:
https://api.openai.com/v1
- API keys (opens in a new tab)
- Recommended models:
gpt-4-turbo
, andgpt-3.5-turbo
- Base url:
https://api.groq.com/openai/v1
- API keys (opens in a new tab)
- Recommended model:
mixtral-8x7b-32768
Together AI (opens in a new tab)
- Base url:
https://api.together.xyz/v1
- API keys (opens in a new tab)
- Recommended models:
mistralai/Mixtral-8x22B-Instruct-v0.1
, andmistralai/Mixtral-8x7B-Instruct-v0.1
- Base url:
https://api.endpoints.anyscale.com/v1
- API keys (opens in a new tab)
- Recommended model:
mistralai/Mixtral-8x7B-Instruct-v0.1
If you discover other providers that work with Monty please send us a message in our Discord (opens in a new tab).
Local models with Ollama (opens in a new tab)
Ollama includes a local server that is OpenAI API-compatible (opens in a new tab). We've tested the following:
- Base url:
http://localhost:11434/v1
- API key can be anything, e.g
foo
- Recommended model:
phi3
Azure OpenAI
Use your own private deployment. Learn more (opens in a new tab)
Anthropic
Get an API key (opens in a new tab)
Available models:
claude-3-opus-20240229
claude-3-sonnet-20240229
claude-3-haiku-20240307
Google AI Studio
Get an API key (opens in a new tab)
Available models:
gemini-1.5-pro-latest
Unsupported providers (for now)
- AWS Bedrock
- Google Vertex AI