Crawl and index connectors

Glean continuously crawls connected data sources to keep search results fresh and accurate. This section covers how crawling works, how to configure it, and what constraints apply.

Strategy

Crawl frequency, full versus incremental crawls, deletion handling, and real-time update support by connector.

Limits

File size, item size, and processing limits that apply to all connectors.

Content exclusion

Greenlist and redlist restrictions for managing what Glean crawls and indexes from each connector.

FAQ

Answers to common questions about monitoring crawl progress, content restrictions, deletion, and freshness.