AI21's RAG Engine offers enterprises an all-in-one solution for implementing Retrieval-Augmented Generation. The RAG Engine allows companies to upload their organizational documents (PDF, DOCX, HTML, or TXT), retrieve the most relevant information for a given query, and connect that context to a large language model like Jurassic-2 or a Task-specific Model like Contextual Answers to generate text. The RAG Engine is conveniently packaged and accessible via our SDK.

The RAG Engine addresses implementation challenges, offering a RAG solution built for production use cases in the enterprise. It lets you efficiently integrate advanced retrieval capabilities without having to invest heavily in development and maintenance. The RAG Engine contains built-in mechanisms for every step of the way, including document extraction, chunking, embeddings, vector storage and retrieval.

Features

Seamless integration between retrieval and generation

RAG Engine automatically integrates with many of our task-specific models, so you can surface search results or provide a grounded answer to a query based on your organizational data – all within a single API call. You can also connect RAG Engine with a foundation model like Jurassic – and use Semantic Search results within a prompt.

Supports various file formats including advanced PDF parsing capabilities

Using our document extraction capabilities, ensure high-quality retrieval across file types like PDF, DOCX and HTML files, while adeptly handling complex structures such as tables. Read more about our advanced PDF parser.

Built-in data source integration

You can integrate your organization’s data sources, such as Google Drive, Amazon S3, and others, to automatically sync documents with RAG Engine. To enable data source integration, contact us.

Builds effective contexts for language models

Our Embeddings and Text Segmentation models use advanced semantic and algorithmic logic to create the optimal context from retrieval results, significantly enhancing the accuracy and relevance of generated text.

Easy maintenance

The RAG Engine maintains synchronization between original and indexed documents as they evolve, ensuring up-to-date information retrieval.

Secure and appropriate access to documents

The RAG Engine adheres to organizational user and group permissions with a comprehensive approach that addresses the intricate needs of document management in modern, data-intensive environments.

Using the RAG Engine

• To upload and manage documents within the RAG Engine, see our RAG Engine API reference.
• To retrieve information from the RAG Engine, use our Semantic Search .
• To leverage the RAG Engine with our Contextual Answers model, see our Contextual Answers solution