To load Parquet files containing columns for node IDs, node properties, edge source and target IDs, and edge properties, see: LOAD PARQUET.

To load Parquet files containing a JSON string column, which is then converted into a graph structure, see: turing-parquet.

​

LOAD PARQUET

LOAD PARQUET queries may be used to load a graph into TuringDB. The command is of the following format:

LOAD PARQUET "<directory name> {AS <graph name>}"

where {AS <graph name>} may be omitted, in which case the name of the graph will be that of the directory.

The directory specified need contain nodes.parquet and edges.parquet.

Each row in nodes.parquet defines a node, and the file has a required format of:

• a column of name __id of type INT64 - defines the relative node ID for the row;
• a column of name __labels of type BYTE_ARRAY with exactly 1 level of nesting, forming a byte array of byte arrays - each inner byte array is treated as a string (the name of that label), the path of each element must be __labels.list.element.

Each row in edges.parquet defines an edge, and the file has a required format of:

• a column of name __source of type INT64 - defines the node ID (present in nodes.parquet) of the source node of this edge;
• a column of name __target of type INT64 - defines the node ID (present in nodes.parquet) of the target node of this edge;
• a column of name __type of type BYTE_ARRAY with exactly 0 levels of nesting - interpreted as a string (the name of this edge type).

As well as the above required columns, nodes.parquet and edges.parquet may contain any number of extra columns, all of which are interpreted as a property value for the node/edge of that row. These columns may have any name (other than the names of the columns explicitly required above).

The supported types for property columns are:

• BOOLEAN
• INT64
• DOUBLE
• BYTE_ARRAY with exactly 0 levels of nesting

Note: all property columns of BYTE_ARRAY are treated as strings

​

turing-parquet

turing-parquet is a CLI tool that reads node and edge Parquet files and writes a TuringDB graph to disk. Copy the resulting graph directory into your TuringDB working directory and open it with turingdb.

​

When to use it

• You have one or more Parquet tables of nodes and edges (a common shape for biomedical, financial, or web-scale knowledge graphs).
• Per-row properties are encoded as a JSON string column (typically named properties), possibly containing nested objects and arrays.
• You want nested JSON sub-records to be expanded into their own nodes with HAS_* edges, with automatic deduplication of repeated values.

​

Quick usage

turing-parquet \
    -nodes data/nodes.parquet \
    -edges data/edges.parquet \
    -out  ./turingdb.out \
    -graph mygraph

This writes the graph mygraph under ./turingdb.out/graphs/. Move that subdirectory into your TuringDB working directory (default $HOME/.turing/graphs/) and load it with load graph mygraph.

​

Expected file format

turing-parquet expects a fixed set of top-level columns on each file. Other columns are ignored.

Node files (-nodes):

Edge files (-edges):

​

Import steps

The example below uses the OptimusKG biomedical knowledge graph (190,531 nodes across 10 entity types and ~21.8M edges across 26 relation types), shipped as nodes.parquet and edges.parquet.

turing-parquet \
    -nodes data/nodes.parquet \
    -edges data/edges.parquet \
    -out  ./turingdb.out \
    -graph optimuskg

cp -r ./turingdb.out/graphs/optimuskg ~/.turing/graphs/

load graph optimuskg

cd optimuskg

MATCH (n:GEN) WHERE n.symbol = 'TSPAN6' RETURN n.id, n.symbol, n.biotype

from turingdb import TuringDB

client = TuringDB(host="http://localhost:6666")
client.load_graph("optimuskg")
client.set_graph("optimuskg")

df = client.query(
    "MATCH (n:GEN) WHERE n.symbol = 'TSPAN6' "
    "RETURN n.id, n.symbol, n.biotype"
)
print(df)

​

Sub-record expansion

Because the properties column is JSON, nested structure in the source data is preserved as part of the graph. For OptimusKG, this turns 190,531 source rows into about 5M nodes.

​

Mapping rules

• Each scalar JSON field on a node becomes a property on that node.
• Each nested object becomes a separate node linked by a HAS_<FIELD> edge.
• Each array of objects becomes multiple sub-record nodes via repeated HAS_<FIELD> edges.
• Sub-records with the same inferred label and id/value are deduplicated into a single shared node. A handful of distinct :Tractability values can be referenced by hundreds of thousands of genes.

The mapping is printed during import so you can see the inferred sub-record labels and which properties land where.

​

Example

Take a single row in nodes.parquet for the gene TSPAN6:

The properties JSON, pretty-printed:

{
  "symbol": "TSPAN6",
  "biotype": "protein_coding",
  "tractability": {
    "id": "T1",
    "score": "high",
    "modality": "small molecule"
  },
  "synonyms": [
    {"value": "T245"},
    {"value": "TM4SF6"}
  ]
}

After import, this single row becomes four nodes and three edges:

How each piece of the source row maps:

​

Deduplication in action

If a second gene row also has "tractability": {"id": "T1", ...}, the importer recognizes the matching id and reuses the same :Tractability node instead of creating a duplicate:

In the real OptimusKG dataset, just 19 distinct :Tractability nodes back the 528,500 HAS_TRACTABILITY edges from every gene that references one. The natural key is id when present, falling back to value and then to the full sub-record content.

​

Verify the import

• Cypher
• Python SDK

CALL db.labels()
CALL db.edgeTypes()
CALL db.propertyTypes()

print(client.query("CALL db.labels()"))
print(client.query("CALL db.edgeTypes()"))
print(client.query("CALL db.propertyTypes()"))