Bring Your Own Key (BYOK)
You can opt to use an LLM key for your desired LLM provider as long as the provider is supported by Glean.
Overview
To enable Glean Generative AI features, you need to select an LLM provider and an associated account.
By default, generative AI features will use an underlying account key provided by Glean, however you can opt to use your own account key for the LLM provider of your choice (as long as the underlying provider is supported by Glean).
Glean currently supports the following LLM Providers:
- Anthropic
- OpenAI
- Azure OpenAI
- Google Vertex AI
- Amazon Bedrock
There are minimum throughput and capacity requirements that must be met when using your own key. Failure to meet these requirements may result in degraded performance for your users.
For more information, please see the Supported LLMs page.
Configuring BYOK
-
Navigate to Admin Console > Platform > Assistant > Setup, and expand the LLM Provider box.
-
By default Use Glean’s LLM key will be selected. Change this to Use your own LLM provider.
-
Follow the LLM provider-specific instructions from one of the tabs below.
You can switch back to using the Glean-provided key at any time by selecting Use Glean’s LLM key (Recommended) from the LLM Provider dropdown.
You will need an Anthropic API Key from the Anthropic Console to continue.
To use an Anthropic API key:
- Select Anthropic from the dropdown.
- Paste your Anthropic API Key into the Anthropic key field.
- Click Submit to validate and save your key.
Glean Assistant will now use your Anthropic key for all Generative AI features. Your changes may take up to an hour to take effect.
You will need an Anthropic API Key from the Anthropic Console to continue.
To use an Anthropic API key:
- Select Anthropic from the dropdown.
- Paste your Anthropic API Key into the Anthropic key field.
- Click Submit to validate and save your key.
Glean Assistant will now use your Anthropic key for all Generative AI features. Your changes may take up to an hour to take effect.
You will need an OpenAI API Key and your OpenAI Organization ID to continue.
To use an OpenAI API key:
- Select OpenAI from the dropdown.
- Select the OpenAI LLM model to use from the dropdown. We recommend selecting the most recent frontier model from the list (e.g. GPT-4o).
- Paste your OpenAI API Key into the OpenAI key field.
- Paste your OpenAI Organization ID into the Organization ID field.
- Click Submit to validate and save your key.
Glean Assistant will now use your OpenAI key for all Generative AI features. Your changes may take up to an hour to take effect.
You will need the following to continue:
- Azure OpenAI Key & Endpoint.
- Large GPT model deployment name (e.g. GPT-4o).
- Small GPT model deployment name (e.g. GPT-4o mini.
- Azure embeddings deployment name (e.g. Ada-Embedding, text-embedding-ada-004).
To use your Azure OpenAI key:
- Select Azure OpenAI from the dropdown in the Glean UI.
- Paste your Azure OpenAI key into the Key 1 field.
- Paste the endpoint URL into the Endpoint field (drop the
https://as it is not required). - For each of the models listed, paste the deployment name into the field.
- Click Submit to validate and save your settings.
Glean Assistant will now use your Azure OpenAI key for all Generative AI features. Your changes may take up to an hour to take effect.
Vertex AI is only supported when your Glean instance is hosted on GCP’s infrastructure.
- Select Vertex AI from the dropdown in the Glean UI.
- Select the Vertex AI model to use: Gemini or Claude.
- Follow the prompts to complete the setup process.
Glean Assistant will now use your Vertex AI key for all Generative AI features. Your changes may take up to an hour to take effect.
Amazon Bedrock is only supported when your Glean instance is hosted on AWS’s infrastructure.
- Select Amazon Bedrock from the dropdown in the Glean UI.
- Select the Claude LLM model to use.
- Follow the prompts to complete the setup process.
Glean Assistant will now use your Amazon Bedrock key for all Generative AI features. Your changes may take up to an hour to take effect.
Estimating LLM Usage costs
The following data should be considered an estimate only. It is based on average usage across multiple Glean customers, however your actual usage may vary.
We recommend using Glean’s LLM key if you have concerns around LLM usage costs.
The following costs per query can be used as a rough guide to help you estimate your LLM usage costs with Glean. All costs are in USD.
| LLM Provider | Models Used | Aggregate Avg Cost per Glean Query |
|---|---|---|
| OpenAI | GPT-4o GPT-4o mini Ada Embeddings | $0.07 |
| OpenAI | GPT-4-Turbo GPT-3.5-Turbo Ada Embeddings | $0.14 |
| Azure OpenAI | GPT-4o GPT-4o mini Ada Embeddings | $0.07 |
| Azure OpenAI | GPT-4-Turbo GPT-3.5-Turbo Ada Embeddings | $0.14 |
| Anthropic | Claude 3.5 Sonnet Claude 3 Haiku Voyage Embeddings | $0.05 |
| Vertex AI - Gemini | Gemini 1.5 Pro Gemini 1.5 Flash Gecko Embeddings | $0.02 |
| Vertex AI - Claude | Claude 3.5 Sonnet Claude 3 Haiku Gecko Embeddings | $0.04 |
| Amazon Bedrock - Claude | Claude 3.5 Sonnet Claude 3 Haiku Titan Text Embeddings G1 | $0.07 |
Was this page helpful?