To use the Unstructured Workflow Endpoint to manage workflows, do the following:

  • To get a list of available workflows, use the UnstructuredClient object’s workflows.list_workflows function (for the Python SDK) or the GET method to call the /workflows endpoint (for curl or Postman). Learn more.
  • To get information about a workflow, use the UnstructuredClient object’s workflows.get_workflow function (for the Python SDK) or the GET method to call the /workflows/<workflow-id> endpoint (for curl or Postman)use the GET method to call the /workflows/<workflow-id> endpoint. Learn more.
  • To create a workflow, use the UnstructuredClient object’s workflows.create_workflow function (for the Python SDK) or the POST method to call the /workflows endpoint (for curl or Postman). Learn more.
  • To run a workflow manually, use the UnstructuredClient object’s workflows.run_workflow function (for the Python SDK) or the POST method to call the /workflows/<workflow-id>/run endpoint (for curl or Postman). Learn more.
  • To update a workflow, use the UnstructuredClient object’s workflows.update_workflow function (for the Python SDK) or the PUT method to call the /workflows/<workflow-id> endpoint (for curl or Postman). Learn more.
  • To delete a workflow, use the UnstructuredClient object’s workflows.delete_workflow function (for the Python SDK) or the DELETE method to call the /workflows/<workflow-id> endpoint (for curl or Postman). Learn more.

The following examples assume that you have already met the requirements and understand the basics of working with the Unstructured Workflow Endpoint.

Create a workflow

To create a workflow, use the UnstructuredClient object’s workflows.create_workflow function (for the Python SDK) or the POST method to call the /workflows endpoint (for curl or Postman).

In the CreateWorkflow object (for the Python SDK) or the request body (for curl or Postman), specify the settings for the workflow, as follows:

Replace the preceding placeholders as follows:

  • <name> (required) - A unique name for this workflow.

  • <source-connector-id> (required) - The ID of the target source connector. To get the ID, use the UnstructuredClient object’s sources.list_sources function (for the Python SDK) or the GET method to call the /sources endpoint (for curl or Postman). Learn more.

  • <destination-connector-id> (required) - The ID of the target destination connector. To get the ID, use the UnstructuredClient object’s destinations.list_destinations function (for the Python SDK) or the GET method to call the /destinations endpoint (for curl or Postman). Learn more.

  • <TYPE> (for the Python SDK) or <type> (for curl or Postman) (required) - The workflow type. Available values include CUSTOM (for the Python SDK) and custom (for curl or Postman).

    If <TYPE> is set to CUSTOM (for the Python SDK), or if <type> is set to custom (for curl or Postman), you must add a workflow_nodes array. For instructions, see Custom workflow DAG nodes.

    The previously-available workflow optimization types ADVANCED, BASIC, and PLATINUM (for the Python SDK) and advanced, basic, and platinum (for curl or Postman) are non-operational and planned to be fully removed in a future release.

    The ability to create an automatic workflow type is currently not available but is planned to be added in a future release.

  • <schedule-timeframe> - The repeating automatic run schedule, specified as a predefined phrase. The available predefined phrases are:

    • every 15 minutes (for curl or Postman): Every 15 minutes (cron expression: */15 * * * *).
    • every hour: At the first minute of every hour (cron expression: 0 * * * *).
    • every 2 hours: At the first minute of every second hour (cron expression: 0 */2 * * *).
    • every 4 hours: At the first minute of every fourth hour (cron expression: 0 */4 * * *).
    • every 6 hours: At the first minute of every sixth hour (cron expression: 0 */6 * * *).
    • every 8 hours: At the first minute of every eighth hour (cron expression: 0 */8 * * *).
    • every 10 hours: At the first minute of every tenth hour (cron expression: 0 */10 * * *).
    • every 12 hours: At the first minute of every twelfth hour (cron expression: 0 */12 * * *).
    • daily: At the first minute of every day (cron expression: 0 0 * * *).
    • weekly: At the first minute of every Sunday (cron expression: 0 0 * * 0).
    • monthly: At the first minute of the first day of every month (cron expression: 0 0 1 * *).

    If schedule is not specified, the workflow does not automatically run on a repeating schedule.

Update a workflow

To update information about a workflow, use the UnstructuredClient object’s workflows.update_workflow function (for the Python SDK) or the PUT method to call the /workflows/<workflow-id> endpoint (for curl or Postman), replacing <workflow-id> with the workflow’s unique ID. To get this ID, see List workflows.

In the request body, specify the settings for the workflow. For the specific settings to include, see Create a workflow.

Custom workflow DAG nodes

If WorkflowType is set to CUSTOM (for the Python SDK), or if workflow_type is set to custom (for curl or Postman), you must also specify the settings for the workflow’s directed acyclic graph (DAG) nodes. These nodes’ settings are specified in the workflow_nodes array.

  • A Source node is automatically created when you specify the source_id value outside of the
    workflow_nodes array.
  • A Destination node is automatically created when you specify the destination_id value outside of the workflow_nodes array.
  • You can specify Partitioner, Chunker, Enrichment, and Embedder nodes.
  • The order of the nodes in the workflow_nodes array will be the same order that these nodes appear in the DAG, with the first node in the array added directly after the Source node. The Destination node follows the last node in the array.
  • Be sure to specify nodes in the allowed order. The following DAG placements are all allowed:

Partitioner node

A Partitioner node has a type of WorkflowNodeType.PARTITION (for the Python SDK) or partition (for curl and Postman).

Learn about the available partitioning strategies.

Auto strategy

VLM strategy

Allowed values for provider and model include:

  • "provider": "anthropic"

    • "model": "claude-3-5-sonnet-20241022"
  • "provider": "anthropic""openai"

    • "model": "gpt-4o"
  • "provider": "bedrock"

    • "model": "us.anthropic.claude-3-5-sonnet-20241022-v2:0"
    • "model": "us.anthropic.claude-3-opus-20240229-v1:0"
    • "model": "us.anthropic.claude-3-haiku-20240307-v1:0"
    • "model": "us.anthropic.claude-3-sonnet-20240229-v1:0"
    • "model": "us.amazon.nova-pro-v1:0"
    • "model": "us.amazon.nova-lite-v1:0"
    • "model": "us.meta.llama3-2-90b-instruct-v1:0"
    • "model": "us.meta.llama3-2-11b-instruct-v1:0"
  • "provider": "vertexai"

    • "model": "gemini-2.0-flash-001"

High Res strategy

Fast strategy

Chunker node

A Chunker node has a type of WorkflowNodeType.CHUNK (for the Python SDK) or chunk (for curl and Postman).

Learn about the available chunking strategies.

Chunk by Character strategy

Chunk by Title strategy

Chunk by Page strategy

Chunk by Similarity strategy

Enrichment node

An Enrichment node has a type of WorkflowNodeType.PROMPTER (for the Python SDK) or prompter (for curl and Postman).

Learn about the available enrichments.

Image Description task

Allowed values for <subtype> include:

  • openai_image_description
  • anthropic_image_description
  • bedrock_image_description
  • vertexai_image_description

Table Description task

Allowed values for <subtype> include:

  • openai_table_description
  • anthropic_table_description
  • bedrock_table_description
  • vertexai_table_description

Table to HTML task

Named Entity Recognition (NER) task

Embedder node

An Embedder node has a type of WorkflowNodeType.EMBED (for the Python SDK) or embed (for curl and Postman).

Learn about the available embedding providers and models.

Allowed values for subtype and model_name include:

  • "subtype": "azure_openai"

    • "model_name": "text-embedding-3-small"
    • "model_name": "text-embedding-3-large"
    • "model_name": "text-embedding-ada-002"
  • "subtype": "bedrock"

    • "model_name": "amazon.titan-embed-text-v2:0"
    • "model_name": "amazon.titan-embed-text-v1"
    • "model_name": "amazon.titan-embed-image-v1"
    • "model_name": "cohere.embed-english-v3"
    • "model_name": "cohere.embed-multilingual-v3"
  • "subtype": "togetherai"

    • "model_name": "togethercomputer/m2-bert-80M-2k-retrieval"
    • "model_name": "togethercomputer/m2-bert-80M-8k-retrieval"
    • "model_name": "togethercomputer/m2-bert-80M-32k-retrieval"