Data Connectors allow you to ingest external data into your AI21 Maestro workspace. Once connected, this data can then be indexed and used to ground responses in external knowledge, enabling your AI applications to provide more accurate, relevant, and up-to-date answers.Through the AI21 Studio UI, users can easily connect data sources manage ingestion, and monitor indexing via the File Library.This guide walks you through the process of using Data Connectors via the AI21 Studio.
You have two options to connect the Index Website:Option A: Sitemap XMLPaste a sitemap link to ingest content automatically.
Only the first 100 links in the sitemap will be indexed.
Option B: Specific URLs
Paste up to 100 URLs.
When you past the urls, make sure to press enter.Advanced Settings
There are additional ingestion options, such as:
Block URL Types
Define URL Types
Specify Localization
Content Load Delay
Extract Files (e.g., PDFs, Docs)
These advanced features are available to Enterprise plan users. Contact Sales for access.
Once the data is connected you’ll see a checkmark (✓) next to it. Navigate to the File Library to see your connected data. It will appear in a table showing source, type, and ingestion status.