Overview

Unstructured offers multiple source connectors to connect to your data in its existing location.

Routing determines which strategy Unstructured uses to transform your documents into Unstructured’s canonical JSON schema. Unstructured provides four partitioning strategies for document transformation, as follows.

Unstructured recommends that you choose the Auto partitioning strategy in most cases. With Auto, Unstructured does all the heavy lifting, optimizing at runtime for the highest quality at the lowest cost page-by-page.

You should consider the following additional strategies only if you are absolutely sure that your documents are of the same type. Each of the following strategies are best suited for specific situations. Choosing one of these strategies other than Auto for sets of documents of different types could produce undesirable results, including reduction in transformation quality.

• VLM: For the highest-quality transformation of these file types: .bmp, .gif, .heic, .jpeg, .jpg, .pdf, .png, .tiff, and .webp.
• High Res: For all other supported file types, and for the generation of bounding box coordinates.
• Fast: For text-only documents.

Your source document is transformed into Unstructured’s canonical JSON schema. Regardless of the input document, this JSON schema gives you a standardized output. It contains more than 20 elements, such as Header, Footer, Title, NarrativeText, Table, Image, and many more. Each document is wrapped in extensive metadata so you can understand languages, file types, sources, hierarchies, and much more.

Unstructured provides these chunking strategies:

• Basic combines sequential elements up to specified size limits. Oversized elements are split, while tables are isolated and divided if necessary. Overlap between chunks is optional.
• By Title uses semantic chunking, understands the layout of the document, and makes intelligent splits.
• By Page attempts to preserve page boundaries when determining the chunks’ contents.
• By Similarity uses an embedding model to identify topically similar sequential elements and combines them into chunks.

Images and tables can be optionally summarized. This generates enriched content around the images or tables that were parsed during the transformation process.

Unstructured uses optional third-party embedding providers such as OpenAI.

Unstructured offers multiple destination connectors, including all major vector databases.

Source connectors to ingest your data into Unstructured for transformation.

Destination connectors tell Unstructured where to write your transformed data to.

A workflow connects sources to destinations and provide chunking, embedding, and scheduling options.

Jobs enable you to monitor data transformation progress.