File Library

Overview

The library can hold an unlimited number of files, but a maximum of 1GB of files.

File metadata

Files support the following metadata:

Show properties

fileId

uuid

The unique identifier of the file, generated by AI21.

name

string

The name of the file specified by you.

path

string

An arbitrary file-path-like string you can assign to indicate the content of a file.

fileType

string

The type of the file.

sizeBytes

int

The size of the file in bytes.

labels

string[]

The labels associated with the file. You can apply arbitrary string labels to your files and limit queries to files with one or more labels. Similar to paths, but labels do not prefix match. Labels are case-sensitive.

publicUrl

string

The public URL of the file, specified by you. This URL is not validated by AI21 or used in any way. It is strictly a piece of metadata that you can optionally attach to a file.

createdBy

uuid

The identifier of the user who uploaded the file.

creationDate

date

The date when the file was uploaded.

lastUpdated

date

The last update date of the file in the library.

dataSource

string

Where was the file uploaded from.

status

string

The status of the file in the library.

Show supported values

Value	Description
`DB_RECORD_CREATED`	The file record has been created in the database.
`UPLOADED`	The file has been uploaded and is being processed.
`UPLOAD_FAILED`	The file upload has failed.
`PROCESSING`	The file is currently being processed.
`PROCESSED`	The file has been processed and is ready for use.
`FILE_DOWNLOAD_FAILED`	The file download has failed.
`PARSING_FAILED`	Parsing the file has failed.
`SEGMENTATION_FAILED`	File segmentation has failed.
`EMBEDDING_FAILED`	Embedding the file segments has failed.
`VECTORS_DB_UPSERT_FAILED`	Updating the vector database has failed.
`VECTOR_DATA_UPLOAD_TO_STORAGE_FAILED`	Uploading the file’s embedding vectors to the storage has failed.
`SEGMENTS_UPLOAD_TO_DB_FAILED`	Uploading the file’s segments to the database has failed.
`PROCESSING_FAILED`	The processing of the file has failed.
`DELETION_FAILED`	The deletion of the file has failed.
`RE_PROCESS_CANDIDATE`	The file is a candidate for re-processing.
`RE_PROCESSING`	The file is currently being re-processed.
`DELETING`	The file is in the process of being deleted.
`DELETED_FROM_VECTOR_DB`	The file has been deleted from the vector database.
`DELETE_REQUEST_DURING_PROCESSING`	A delete request was received during processing.
`MAX_RETRIES_REACHED`	The maximum number of retries has been reached, the operation has failed.

Filtering file results

When you create or update a file, you can optionally apply path and/or label values to a file. You can later search, query, or list files based on matching paths or labels. This can be useful to query subsets of documents. For example, you can assign a path of either /financial/USA or /financial/UK to certain documents and later query only US financial documents, only UK financial documents, or query all financial documents by specifying /financial/ as the path.

Using the APIs

AI21 Maestro

Foundation Models

File Library Management

Overview

Filtering file results

Using the APIs

AI21 Maestro

Foundation Models

File Library Management

​Overview

​Filtering file results

Overview

Filtering file results