Overview
If you have received an email stating that "Endpoint XYZ is running into errors" with a distribution of HTTP status codes, this guide will help you understand what it means and how to address the issue.
What does this mean?
Glean calls the endpoint mentioned in the email to make requests to your LLM provider (Azure, OpenAI, Google Vertex AI, or Amazon Bedrock). You are receiving the alert because the endpoint has been returning errors. Glean does not have administrative access to your LLM provider, so we need your help to resolve these issues.
How to resolve the errors
Status code 401/403
These status codes indicate an authentication or access issue with the endpoint. Check that your account with your LLM provider is still active and that the settings at Admin console > Platform > LLMs are correct. For 403 errors, also verify that your provider's network or firewall settings allow traffic from Glean.
Status code 404
This status code indicates that the endpoint might be underprovisioned for your traffic. Please refer to Glean LLM Providers to check if you are appropriately provisioned. Contact a Glean representative if you believe you are provisioned correctly but are still receiving 429 errors.
Status code 429
This status code indicates that the endpoint is underprovisioned for your traffic. Refer to the supported LLMs page to check whether you're appropriately provisioned. Contact a Glean representative if you believe you're provisioned correctly but are still receiving 429 errors.
Status code 500
This status code indicates a server error on the LLM provider's side. Check whether there's an outage or service disruption from the provider:
If you use a custom gateway or proxy, check its logs for more specific error details.
Timeout errors
If the alert references timeouts or unresponsiveness, your LLM endpoint may be slow or unreachable. Verify the endpoint is responding and check for any network or gateway issues.
Once the errors are resolved, you'll stop receiving alert emails and no further action is required.
For a comprehensive troubleshooting guide covering all error codes and resolution steps, see Troubleshoot LLM provider errors.