Migrate from a native connector to a custom connector

If you've built a custom connector that covers more data types or offers better control than a native connector, you can replace the native connector with your custom one. This guide walks through the migration process.

Before you begin

warning

Native and custom datasources occupy separate namespaces in Glean. The system rejects any attempt to push content into a built-in datasource name. Your custom datasource will always have its own distinct identity, document IDs, and datasource reference.

Because of this namespace separation, agents, saved searches, app filters, and pinned results scoped to the native datasource won't automatically recognize content in the custom datasource. You'll need to migrate these artifacts as part of the transition.

Migration steps

1. Create and populate the custom datasource

   Set up a new custom datasource with a distinct name and push all content on a regular schedule. Since custom datasources don't have an automated crawl like native connectors, your team manages the ingestion cadence by pushing content through the Indexing API:

   • Use /indexdocument for incremental updates.
   • Use /bulkindexdocuments for full refreshes.
   tip

   Run both the native and custom connectors in parallel during the transition period. This avoids gaps in search coverage while you migrate.

2. Migrate user-facing artifacts

   Users and admins need to update any artifacts that reference the native datasource to point at the new custom datasource. There's no automatic carry-over for these items:

   • Agents — update any agents that filter by or reference the native datasource.
   • Saved searches — re-create or update saved searches to scope to the new datasource name.
   • App filters — update any preconfigured app filters in the Admin console.
   • Pinned results — re-pin results from the new custom datasource if you've pinned native datasource results.

3. Deactivate the native connector

   Only deactivate the native connector after you've confirmed that the custom datasource is fully populated and all user-facing artifacts point to it.

   warning

   When you deactivate a native connector, its content is removed from the search index. Make sure the custom datasource is serving all the content your users need before you deactivate.

Verify the migration

After completing the migration steps, verify that everything works as expected:

1. Search in Glean for content that was previously only available through the native connector. Confirm it appears as a result from the custom datasource.
2. Test with users who have different permission levels to verify that document-level access controls are working correctly.
3. Confirm that agents referencing the new datasource return the expected results.
4. Check the custom datasource status using the debug status endpoint to verify that documents are indexed and processing is complete.

See also

Custom data sources

Learn how to create and configure custom datasources in Glean.

Troubleshooting

Debug endpoints and tools for investigating indexing API issues.

Indexing API overview

Full API documentation for the indexing endpoints.