Data ingestion flow
Glean's data ingestion process is built around specialized connectors deployed within your tenant's dedicated cloud project. These connectors serve multiple purposes:
- Content retrieval: Fetches content from connected enterprise sources. See About connectors.
- Activity tracking: Monitors user interaction data.
- Permission mapping: Maps and maintains access controls.
Connection methods
Data retrieval occurs over HTTPS, with two primary connection patterns depending on the connector location.
- SaaS applications: For services like Google Drive, connections occur over the public internet using HTTPS.
- On-premises systems: For internal systems like on-premises Jira, secure private connections are established through a VPN or Shared VPC. See GCP on-premises connectivity.
Ingestion paths
Glean has three primary data ingestion paths.
Indexing
Glean's standard approach to ingesting unstructured and semi-structured data, such as documents, wikis, emails, and files. Supported sources include Google Drive and Confluence.
Tools
Integration of business logic and workflows through tools. Tools enable direct communication between Glean and third-party systems to trigger or retrieve information as needed. Typical use cases include ticket lookup, status update, quick access to app-level data, and dynamic retrieval from data warehouse platforms like Snowflake and Databricks.
Context provisioning (MCP, Model Context Provisioning) [upcoming]
The ability to programmatically provide context or signals to Glean without indexing content. This path highlights dynamic context injection, such as relevant metadata or recent activity, to improve search and responses.
Glean's approach to structured data
- Glean does not need to ingest or index structured data from systems like Snowflake, Databricks, or other data warehouses.
- Glean focuses on knowledge work, which is unstructured and conversational data, rather than analytics or raw data processing.
- Customers' structured data in these platforms remains untouched. Glean relies on metadata and context, not direct access.
Key takeaways
- Glean ingests only what is necessary, respecting security and privacy boundaries.
- Glean does not index structured datasets from data warehouse and business intelligence tools like Snowflake and Databricks.
- Data ingestion is efficient, secure, and focused, which empowers end users without unnecessary data movement.
