GitHub Server

The GitHub Server connector lets Glean fetch and index content from an on-prem GitHub Server instance, so users can search the repositories, code, and activity they have access to. Glean authenticates by creating a GitHub App on your instance, and enforces all user access permissions at query time so users only see results they are allowed to see. All data is stored in the customer's cloud account and no data leaves the customer's environment.

Network reachability

These instructions work for on-prem instances the Glean crawler can access. Glean supports deployments on both GCP and AWS. Your GitHub Server instance must be network-accessible to the Glean crawler running in your cloud. Contact Glean Support for any network configuration required.

When you are ready to connect, follow the setup guide.

Key features

• Glean authenticates to your GitHub instance to fetch relevant information.
• Authentication is done by creating a GitHub App on your instance.
• Glean enforces all user access permissions at query time, so users cannot see results they do not have access to.
• All data is stored in the customer's cloud account; no data leaves the customer's environment.

Supported objects

For GitHub Server, Glean captures the following content:

• PR descriptions
• PR conversations and comments
• Issue threads
• Commit messages for the main branch
• Wikis
• Code: Code Search is supported for repositories connected via GitHub Server. It is enabled by default once connected; the Admin console toggle has been removed. See Code Search for more details.
• GitHub Pages: you must enter a comma-separated list of repository names to include Pages in your search index. Only repositories that use the legacy gh-pages branch based workflow are supported, and within that branch only HTML and Markdown files are indexed.

File types

Content indexing is limited to specific file types for Text and Markdown (.txt, .md) and Code and Configuration files (.cmd, .c, .h, .cs, .csx, .cpp, .c++, .cc, .h++, .hh, .hpp, .cmake, .cob, .cbl, .css, .clj, .cljs, .cls, .dockerfile, .go, .graphql, .gql, .graphqls, .hs, .ini, .cfg, .cnf, .java, .js, .json, .jsx, .jl, .kt, .kts, .lua, .ml, .php, .pas, .pl, .perl, .proto, .py, .py3, .r, .rd, .rb, .rs, .scss, .sql, .mysql, .scala, .sc, .sh, .bash, .swift, .tf, .ts, .tsx, .us, .vhdl, .vhd, .vlg, .verilog, .yaml, .yml).

note

Contact Glean Support to explore crawling additional content types beyond those listed above.

Files on non-default branches, tags, and commits

Glean indexes code files on each repository's default branch. When a user opens or references a file on another branch, a tag, or a specific commit, Glean can fetch that file directly from your GitHub Server instance at that moment using the user's own access. This makes files outside the default branch available even though they aren't indexed, and the user's GitHub permissions are still enforced.

Files on a branch whose name contains a forward slash (for example, feature/login) can't be fetched this way. To reach such a file on demand, reference it by its commit SHA instead of the branch name.

Crawl strategy and indexed content

This table outlines the purpose, frequency, and corresponding API endpoints for each crawl scope.

Crawl scope	Purpose	Frequency	API endpoint
Repositories	Discover all repositories in the organization.	Every 4 hours	GET /orgs/{org}/repos
GIT	Clone repositories and crawl files, commits, and READMEs.	Full: 28 days; Incremental: 1 hour
PRs	Crawl pull requests, comments, reviews, diffs, and changed files.	Full: 28 days; Incremental: 1 hour	GET /repos/{owner}/{repo}/pulls
Issues	Crawl issues (non-PR) and comments.	Full: 28 days; Incremental: 1 hour	GET /repos/{owner}/{repo}/issues
Pages	Crawl GitHub Pages content (HTML and Markdown from legacy gh-pages repositories).
Identity	Discover users, groups (repositories and teams), team members, and collaborators.	Every 10 minutes	GET /orgs/{org}/members (users); GET /repos/{owner}/{repo}/collaborators (collaborators)

note

In addition to the scheduled identity crawl, Glean supports identity webhooks for GitHub Server. Additive permission changes, such as adding users to a repository, adding teams to a repository, adding users to a team, or adding and removing users from the organization, are typically reflected in Glean within an hour.

Identity

• Users: Glean syncs GitHub users for permission resolution.
• Teams and groups: Glean syncs GitHub teams, team memberships, and repository-level access groups.
• Scoped identity crawling: GitHub identity sync uses separate crawl scopes for users and teams to improve permission freshness.
• Full identity crawls: Conducted periodically to ensure all identity data stays up to date.

Webhook events

Glean handles GitHub identity webhook events for membership and team changes. Events such as Member, Organization, and Team help refresh permissions faster.

Event type	Trigger	Behavior
pull_request	opened, edited, closed, reopened, etc.	Create dirty nodes for PR plus comments, reviews, diff, files
pull_request_review	submitted, edited	Create dirty node for reviews
pull_request_review_comment	created, edited, deleted	Create dirty node for review comments or publish deletion doc
issue_comment	created, edited, deleted	Create dirty node for issue comments or publish deletion doc
issues	opened, edited, closed, reopened, etc.	Create dirty node for issue plus comments
installation	created, deleted	Trigger admin re-auth
member	added, removed, edited	Update user identity (if enabled)
membership	added, removed	Update team memberships (if enabled)
organization	member_added, member_removed	Update user identity (if enabled)
team	created, deleted, edited	Update team metadata (if enabled)

GitHub authentication system

The system uses two separate flows to manage access: App authentication (for organizational data) and user token refresh (for individual sessions).

GitHub App authentication (installation token)

This flow manages the application's core access token (AUTH_ACCESS_TOKEN), which lets Glean read organizational data through the GitHub App installation.

Process stage	Purpose	Key artifacts	Expiry logic
Token request	Generate a JWT to request a new access token from the GitHub API.	JWT, installation ID (cached for 24h)
Active token	The current token used for all API calls.	Access token	1 hour expiry.
Pre-fetch	Store a pre-fetched token 30 minutes before the active token expires.	Next access token
Validation			If the active token is under 5 minutes from expiry, the pending token is immediately accepted.

User token refresh

This flow manages renewal of individual user sessions through the OAuth token refresh mechanism.

Process stage	Purpose	Condition or endpoint
Discovery	The admin crawl queries the user token store for sessions needing renewal.	Targets tokens expiring within a two-hour buffer (for 8-hour tokens).
Renewal	The system posts the refresh token to the GitHub OAuth endpoint to retrieve a new pair.	gitDomain/login/oauth/access_token
Update	The user token store is updated with the new access and refresh token pair and their updated expiry time.	Performed for each user requiring a refresh.