Supported LLMs
Glean supports a variety of LLMs that can be leveraged for the platform’s Generative AI features
To enable Glean’s Generative AI features, you need to select a LLM provider. We recommend using Glean’s Azure OpenAI key. We also provide the option to Bring Your Own LLM key (BYOK) with Azure OpenAI, Google Vertex AI (if your deployment is on GCP), and Amazon Bedrock (if your deployment is on AWS).
This can can be setup in Admin Console > Platform > Assistant > Setup
Glean currently supports the following LLMs:
| LLM | Glean Key | BYOK |
|---|---|---|
| GPT | ✅ | ✅ (Azure OpenAI & OpenAI) |
| Claude (Anthropic API) | ❌ | ✅ |
| Claude (Bedrock) | ❌ | ✅ |
| Claude (Vertex AI) | ✅* | ✅ |
| Gemini | ✅* | ✅ (Google Vertex AI) |
*Glean Key is only supported for these models when leveraging Glean SaaS.
Supported LLM Options
Capacity Requirements
Query Planning and Answer Generation
Capacity requirements for GPT-4o, GPT-4 Turbo, Claude 3.5 Sonnet, or Gemini 1.5 Pro:
| Users | RPM | TPM |
|---|---|---|
| 500 | 20 | 70,000 |
| 1,000 | 40 | 135,000 |
| 2,500 | 100 | 335,000 |
| 5,000 | 200 | 665,000 |
| 10,000 | 350 | 1,165,000 |
| 20,000 | 500 | 1,665,000 |
Tool Selection and Followup Question Generation
Capacity requirements for GPT-4o mini, GPT-3.5-Turbo, Claude 3 Haiku, or Gemini 1.5 Flash:
| Users | RPM | TPM |
|---|---|---|
| 500 | 10 | 15,000 |
| 1,000 | 20 | 25,000 |
| 2,500 | 50 | 60,000 |
| 5,000 | 100 | 115,000 |
| 10,000 | 175 | 205,000 |
| 20,000 | 250 | 290,000 |
Citation Generation
OpenAI Ada Embeddings or Google Embeddings for Text
| Users | RPM | TPM |
|---|---|---|
| 500 | 15 | 13,000 |
| 1,000 | 30 | 25,000 |
| 2,500 | 75 | 63,000 |
| 5,000 | 150 | 125,000 |
| 10,000 | 260 | 219,000 |
| 20,000 | 375 | 313,000 |
Amazon Titan Embeddings G1 - Text
| Users | RPM | TPM |
|---|---|---|
| 500 | 500 | 13,000 |
| 1,000 | 1,000 | 25,000 |
| 2,500 | 2,500 | 63,000 |
| 5,000 | 5,000 | 125,000 |
| 10,000 | 8,750 | 219,000 |
| 20,000 | 12,500 | 313,000 |
Was this page helpful?