Neo4j

• A Neo4j deployment.
  • For the Unstructured UI or the Unstructured API, local Neo4j deployments are not supported.
  • For Unstructured Ingest, local and non-local Neo4j deployments are supported.
  The following video shows how to set up a Neo4j Aura deployment:
• The username and password for the user who has access to the Neo4j deployment. The default user is typically neo4j.
  • For a Neo4j Aura instance, the defaut user’s is typically set when the instance is created.
  • For an AWS Marketplace, Microsoft Azure Marketplace, or Google Cloud Marketplace deployment of Neo4j, the default user is typically set during the deployment process.
  • For a local Neo4j deployment, you can set the default user’s initial password or recover an admin user and its password.
• The connection URI for the Neo4j deployment, which starts with neo4j://, neo4j+s://, bolt://, or bolt+s://; followed by localhost or the host name; and sometimes ending with a colon and the port number (such as :7687). For example:
  • For a Neo4j Aura deployment, browse to the target Neo4j instance in the Neo4j Aura account and click Connect > Drivers to get the connection URI, which follows the format neo4j+s://<host-name>. A port number is not used or needed.
  • For an AWS Marketplace, Microsoft Azure Marketplace, or Google Cloud Marketplace deployment of Neo4j, see Neo4j on AWS, Neo4j on Azure, or Neo4j on GCP for details about how to get the connection URI.
  • For a local Neo4j deployment, the URI is typically bolt://localhost:7687
  • For other Neo4j deployment types, see the deployment provider’s documentation.
  Learn more.
• The name of the target database in the Neo4j deployment. A default Neo4j deployment typically contains two standard databases: one named neo4j for user data and another named system for system data and metadata. Some Neo4j deployment types support more than these two databases per deployment; Neo4j Aura instances do not.
  • Create additional databases for a local Neo4j deployment that uses Enterprise Edition; or for Neo4j on AWS, Neo4j on Azure, or Neo4j on GCP deployments.
  • Get a list of additional available databases for a local Neo4j deployment that uses Enterprise Edition; or for Neo4j on AWS, Neo4j on Azure, or Neo4j on GCP deployments.

pip install "unstructured-ingest[neo4j]"

• NEO4J_USERNAME - The name of the target user with access to the target Neo4j deployment, represented by --username (CLI) or username (Python).
• NEO4J_PASSWORD - The user’s password, represented by --password (CLI) or password (Python).
• NEO4J_URI - The connection URI for the deployment, represented by --uri (CLI) or uri (Python).
• NEO4J_DATABASE - The name of the database in the deployment, represented by --database (CLI) or database (Python).

#!/usr/bin/env bash

# Chunking and embedding are optional.

unstructured-ingest \
  local \
    --input-path $LOCAL_FILE_INPUT_DIR \
    --chunking-strategy by_title \
    --embedding-provider huggingface \
    --partition-by-api \
    --api-key $UNSTRUCTURED_API_KEY \
    --partition-endpoint $UNSTRUCTURED_API_URL \
    --strategy hi_res \
    --additional-partition-args="{\"split_pdf_page\":\"true\", \"split_pdf_allow_failed\":\"true\", \"split_pdf_concurrency_level\": 15}" \
  neo4j \
    --username $NEO4J_USERNAME \
    --password $NEO4J_PASSWORD \
    --uri $NEO4J_URI \ # <scheme>://<host>:<port>
    --database $NEO4J_DATABASE \
    --batch-size 100

• To do local file processing, omit --partition-by-api (CLI) or partition_by_api (Python), or explicitly specify partition_by_api=False (Python). Local file processing does not use an Unstructured API key or API URL, so you can also omit the following, if they appear:
  • --api-key $UNSTRUCTURED_API_KEY (CLI) or api_key=os.getenv("UNSTRUCTURED_API_KEY") (Python)
  • --partition-endpoint $UNSTRUCTURED_API_URL (CLI) or partition_endpoint=os.getenv("UNSTRUCTURED_API_URL") (Python)
  • The environment variables UNSTRUCTURED_API_KEY and UNSTRUCTURED_API_URL
• To send files to the Unstructured Partition Endpoint for processing, specify --partition-by-api (CLI) or partition_by_api=True (Python). Unstructured also requires an Unstructured API key and API URL, by adding the following:
  • --api-key $UNSTRUCTURED_API_KEY (CLI) or api_key=os.getenv("UNSTRUCTURED_API_KEY") (Python)
  • --partition-endpoint $UNSTRUCTURED_API_URL (CLI) or partition_endpoint=os.getenv("UNSTRUCTURED_API_URL") (Python)
  • The environment variables UNSTRUCTURED_API_KEY and UNSTRUCTURED_API_URL, representing your API key and API URL, respectively.
  You must specify the API URL only if you are not using the default API URL for Unstructured Ingest, which applies to Starter and Team accounts.The default API URL for Unstructured Ingest is https://api.unstructuredapp.io/general/v0/general, which is the API URL for the Unstructured Partition Endpoint. However, you should always use the URL that was provided to you when your Unstructured account was created. If you do not have this URL, email Unstructured Support at support@unstructured.io.If you do not have an API key, get one now.If you are using an Enterprise account, the process for generating Unstructured API keys, and the Unstructured API URL that you use, are different. For instructions, see your Unstructured account administrator, or email Unstructured Support at support@unstructured.io.

​

Graph Output

• The Document node represents the source file.
• The UnstructuredElement nodes represent the source file’s Unstructured Element objects, before chunking.
• The Chunk nodes represent the source file’s Unstructured Element objects, after chunking.
• Each UnstructuredElement node has a PART_OF_DOCUMENT relationship with the Document node.
• Each Chunk node also has a PART_OF_DOCUMENT relationship with the Document node.
• Each UnstructuredElement node has a PART_OF_CHUNK relationship with a Chunk element.
• Each Chunk node, except for the “last” Chunk node, has a NEXT_CHUNK relationship with its “next” Chunk node.

MATCH path=(source)-[relationship]->(target)
RETURN path

MATCH (chunk:Chunk)-[relationship:PART_OF_DOCUMENT]->(doc:Document)
RETURN chunk, relationship, doc

MATCH (element:UnstructuredElement)-[relationship:PART_OF_DOCUMENT]->(doc:Document)
RETURN element, relationship, doc

MATCH (element:UnstructuredElement)-[relationship:PART_OF_CHUNK]->(chunk:Chunk)
RETURN element, relationship, chunk

MATCH (this:Chunk)-[relationship:NEXT_CHUNK]->(previous:Chunk)
RETURN this, relationship, previous

MATCH (element:UnstructuredElement)-[ecrelationship:PART_OF_CHUNK]-(chunk:Chunk)-[cdrelationship:PART_OF_DOCUMENT]->(doc:Document)
RETURN element, ecrelationship, chunk, cdrelationship, doc

MATCH (element:UnstructuredElement)-[relationship:PART_OF_CHUNK]->(chunk:Chunk)
WHERE element.text =~ '(?i).*jury.*'
RETURN element, relationship, chunk

MATCH (element:UnstructuredElement)-[relationship:PART_OF_CHUNK]->(chunk:Chunk)
WHERE chunk.id = '731508bf53637ce4431fe93f6028ebdf'
RETURN element, relationship, chunk

• The Chunk node represents one of the source file’s Unstructured Element objects, after chunking.
• The Entity node represents a recognized entity.
• A Chunk node can have HAS_ENTITY relationships with Entity nodes.
• An Entity node can have ENTITY_TYPE relationships with other Entity nodes.

MATCH path=(source)-[relationship]->(target)
RETURN path

MATCH (child:Entity)-[relationship:ENTITY_TYPE]->(parent:Entity)
RETURN child, relationship, parent

MATCH (child:Entity)-[relationship:ENTITY_TYPE]->(parent:Entity)
WHERE parent.id = 'PERSON'
RETURN child, relationship, parent

MATCH (element:Chunk)-[relationship:HAS_ENTITY]->(entity:Entity)
WHERE entity.id =~ '(?i).*amendment.*'
RETURN element, relationship, entity

MATCH (chunk:Chunk)-[ccrelationship:HAS_ENTITY]-(child:Entity)-[cprelationship:ENTITY_TYPE]->(parent:Entity)
WHERE parent.id =~ 'PERSON'
RETURN chunk, ccrelationship, child, cprelationship, parent