MotherDuck

​

Requirements

• A MotherDuck account.
• A MotherDuck access token for the account.
• A database in the account.
  • Create a database.
  • List available databases.
  You can run commands to manage MotherDuck databases, schemas, tables, and more in the MotherDuck UI or for example by connecting to MotherDuck with the DuckDB CLI.
• A schema in the target database.
  • Create a schema.
  • You can list available schemas and their parent catalogs by running the following command in the MotherDuck UI or the DuckDB CLI:
    SELECT * FROM information_schema.schemata;
    
  The MotherDuck connector uses the default schema name of main if not otherwise specified.
• A table in the target schema.
  • Create a table.
  • You can list available tables in a schema by running the following commands in the MotherDuck UI or the DuckDB CLI, replacing the target catalog and schema names:
    USE <catalog_name>.<schema_name>;
    SHOW TABLES;
    
  The MotherDuck connector uses the default table name of elements if not otherwise specified. For maximum compatibility, Unstructured recommends the following table schema:
  CREATE TABLE elements (
      id VARCHAR,
      element_id VARCHAR,
      text TEXT,
      embeddings FLOAT[],
      type VARCHAR,
      system VARCHAR,
      layout_width DECIMAL,
      layout_height DECIMAL,
      points TEXT,
      url TEXT,
      version VARCHAR,
      date_created INTEGER,
      date_modified INTEGER,
      date_processed DOUBLE,
      permissions_data TEXT,
      record_locator TEXT,
      category_depth INTEGER,
      parent_id VARCHAR,
      attached_filename VARCHAR,
      filetype VARCHAR,
      last_modified TIMESTAMP,
      file_directory VARCHAR,
      filename VARCHAR,
      languages VARCHAR[],
      page_number VARCHAR,
      links TEXT,
      page_name VARCHAR,
      link_urls VARCHAR[],
      link_texts VARCHAR[],
      sent_from VARCHAR[],
      sent_to VARCHAR[],
      subject VARCHAR,
      section VARCHAR,
      header_footer_type VARCHAR,
      emphasized_text_contents VARCHAR[],
      emphasized_text_tags VARCHAR[],
      text_as_html TEXT,
      regex_metadata TEXT,
      detection_class_prob DECIMAL,
      partitioner_type VARCHAR
  );
  
  You can list the schema of a table by running the following commands in the MotherDuck UI or the DuckDB CLI, replacing the target catalog, schema, and table names:
  USE <catalog_name>.<schema_name>;
  DESCRIBE TABLE <table_name>;
  

​

Examples

import os

from unstructured_client import UnstructuredClient
from unstructured_client.models.operations import CreateDestinationRequest
from unstructured_client.models.shared import CreateDestinationConnector

with UnstructuredClient(api_key_auth=os.getenv("UNSTRUCTURED_API_KEY")) as client:
    response = client.destinations.create_destination(
        request=CreateDestinationRequest(
            create_destination_connector=CreateDestinationConnector(
                name="<name>",
                type="motherduck",
                config={
                    "database": "<database>",
                    "db_schema": "<db-schema>",
                    "table": "<table>",
                    "md_token": "<md-token>"
                }
            )
        )
    )

    print(response.destination_connector_information)

curl --request 'POST' --location \
"$UNSTRUCTURED_API_URL/destinations" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY" \
--header 'content-type: application/json' \
--data \
'{
    "name": "<name>",
    "type": "motherduck",
    "config": {
        "database": "<database>",
        "db_schema": "<db-schema>",
        "table": "<table>",
        "md_token": "<md-token>"
    }
}'

​

Configuration settings

​

Learn more

•   Unstructured’s New MotherDuck Integration