MotherDuck

Send processed data from Unstructured to MotherDuck.

The requirements are as follows.

• A MotherDuck account.

• A MotherDuck access token for the account.

• A database in the account.

  • Create a database.
  • List available databases.

  You can run commands to manage MotherDuck databases, schemas, tables, and more in the MotherDuck UI or for example by connecting to MotherDuck with the DuckDB CLI.

• A schema in the target database.

  • Create a schema.

  • You can list available schemas and their parent catalogs by running the following command in the MotherDuck UI or the DuckDB CLI:

    SELECT * FROM information_schema.schemata;
    

  The MotherDuck connector uses the default schema name of main if not otherwise specified.

• A table in the target schema.

  • Create a table.

  • You can list available tables in a schema by running the following commands in the MotherDuck UI or the DuckDB CLI, replacing the target catalog and schema names:

    USE <catalog-name>.<schema-name>;
    SHOW TABLES;
    

  The MotherDuck connector uses the default table name of elements if not otherwise specified.

  For maximum compatibility, Unstructured recommends the following table schema:

  CREATE TABLE elements (
      id VARCHAR,
      element_id VARCHAR,
      text TEXT,
      embeddings FLOAT[],
      type VARCHAR,
      system VARCHAR,
      layout_width DECIMAL,
      layout_height DECIMAL,
      points TEXT,
      url TEXT,
      version VARCHAR,
      date_created INTEGER,
      date_modified INTEGER,
      date_processed DOUBLE,
      permissions_data TEXT,
      record_locator TEXT,
      category_depth INTEGER,
      parent_id VARCHAR,
      attached_filename VARCHAR,
      filetype VARCHAR,
      last_modified TIMESTAMP,
      file_directory VARCHAR,
      filename VARCHAR,
      languages VARCHAR[],
      page_number VARCHAR,
      links TEXT,
      page_name VARCHAR,
      link_urls VARCHAR[],
      link_texts VARCHAR[],
      sent_from VARCHAR[],
      sent_to VARCHAR[],
      subject VARCHAR,
      section VARCHAR,
      header_footer_type VARCHAR,
      emphasized_text_contents VARCHAR[],
      emphasized_text_tags VARCHAR[],
      text_as_html TEXT,
      regex_metadata TEXT,
      detection_class_prob DECIMAL
  );
  

  You can list the schema of a table by running the following commands in the MotherDuck UI or the DuckDB CLI, replacing the target catalog, schema, and table names:

  USE <catalog-name>.<schema-name>;
  DESCRIBE TABLE <table-name>;
  

To create or change a MotherDuck destination connector, see the following examples.

curl --request 'POST' --location \
"$UNSTRUCTURED_API_URL/destinations" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY" \
--header 'content-type: application/json' \
--data \
'{
    "name": "<name>",
    "type": "motherduck",
    "config": {
        "database": "<database>",
        "db_schema": "<db-schema>",
        "table": "<table>",
        "md_token": "<md-token>"
    }
}'

Replace the preceding placeholders as follows:

• <name> (required): A unique name for this connector.
• <database> (required): The name of the target MotherDuck database.
• <db-schema> (required): The name of the target schema within the database.
• <table> The name of the target table within the schema. By default, this table is named elements if not otherwise specified.
• <md-token> (required): The access token value within the MotherDuck account that has the appropriate access to the target database, schema, and table.

To change a connector, replace <connector-id> with the destination connector’s unique ID. To get this ID, see List destination connectors.