Set up LLMs

The Glean model hub provides secure and easy access to a variety of large language models (LLMs), both commercial and open-source for use with Glean agents. Users can pick-and-choose models for their agents and each step in an agentic workflow without needing technical expertise or engineering resources.

​

What is the Model Hub?

• Easy Access: Choose from over 15 models from providers like OpenAI, Google, Anthropic, Amazon, and Meta.
• Growing Selection: New models from Amazon Bedrock, Google Vertex AI, and Azure OpenAI are regularly added.
• Flexible Key Options:
  • Glean Universal Model Key: Access pre-procured and managed models. Glean handles agreements to ensure your data is never retained or used for model training. This is available for Glean Key customers.
  • Bring Your Own Model Key: Use your existing model keys for LLM providers in Glean

​

Supported Models in Glean Agents

• OpenAI (via Azure or OpenAI):
  • GPT 4.1
  • GPT 4.1 mini
  • GPT-4o
  • GPT-4o-mini
  • o3
  • o3-mini
  • o4-mini
  • o1 (Customer key only)
• Google Gemini (via Google Vertex AI):
  • Gemini 2.5 Pro
  • Gemini 2.5 Flash
  • Gemini 2.0 Flash
• Anthropic (via Google Vertex AI or Amazon Bedrock):
  • Claude Sonnet 4
  • Claude Sonnet 3.7
  • Claude Sonnet 3.5 V2
  • Claude 3.5 Haiku
• Llama (via Google Vertex AI or Amazon Bedrock):
  • Llama 4 Maverick
  • Llama 4 Scout
• Amazon:
  • Nova Pro 1.0 (via Amazon Bedrock)
• DeepSeek:
  • R-1 (via Amazon Bedrock)

​

Enabling the model hub

​

For Glean universal model key (aka Glean key)

Models are preconfigured and managed by Glean. You simply enable or disable them. Deprecated models are automatically replaced with the latest version.

1. Go to the admin console.
2. Select “LLM” under the “Platform” tab.
3. Click “Add LLM.”
4. Select a hosting provider.
5. Enable all available models for that provider.
6. You can disable models from the same page at any time.

Important notes for universal model key:

• If GCP is your hosting provider, you cannot add models from AWS, and vice versa.
• The Glean universal model key for Agents is exclusively for Glean Key customers.

​

For customer key (Bring Your Own Keys):

1. Go to the Admin Console.
2. Select “LLM” under the “Platform” tab.
3. Click “Add LLM.”
4. Select a hosting provider and follow the specific configuration steps:
   • Azure OpenAI:
     1. Ensure that assistant is configured
     2. In your Azure Portal, go to “Keys and Endpoints”, determine the Key and Endpoint to be used with Glean and add them here
     3. Select models for Agents.
        • For each selected model, add the deployment name configured on Azure.
     4. Validate the connection.
     5. Click “Save.”
     6. To remove a model, uncheck it, validate, and save.
   • Amazon Bedrock (self-hosted on AWS only):
     1. Enter your preferred region.
     2. Select models for Agents.
     3. Validate the connection.
     4. Click “Save.”
     5. To remove a model, uncheck it, validate, and save.
   • Google Vertex AI (self-hosted on GCP only):
     1. Select models for Agents.
     2. Ensure selected models are enabled in Google Model Garden.
     3. Validate the connection.
     4. Click “Save.”
     5. To remove a model, uncheck it, validate, and save.
   • OpenAI:
     1. Enter your API key to be used with Glean from the OpenAI Portal.
     2. Select models for Agents.
     3. Validate the connection.
     4. Click “Save.”
     5. To remove a model, uncheck it, validate, and save.

​

How to change the model used by your agent

To set the default model that will be used in all steps of the agent, go to agent settings (gear icon in the header) and click on Select Model.

​

How to change the model used in an agent step

You also have the option of changing the model for only a single step.

Click on the step in the canvas and then click on the three dots menu in the drawer and select Advanced settings.

Then click on Select Model.