Skip to content
No results
We use cookies to ensure that we give you the best experience on our website.

# Visualize your data under Left-Most format with SQLFlow REST API

- leo gu
- April 11, 2023
- Data Governance, SQLFlow

Gudu SQLFlow is an analysis software for analyzing SQL statements and discovering data lineage relationships. It is often used with metadata management tools and is a basic tool for enterprise data governance. Check this introduction blog if you don’t know much about Gudu SQLFlow.

## SQLFlow REST API

Gudu SQLFlow has various functionalities for data lineage generation and it provides you different choices to visualize your data. SQLFlow REST API is one of the most commonly used features. Your data analysis result will be generated and returned from SQLFlow server by sending standard parameterized RESTFul Api request. Apart from report under Json, XML, CSV format, SQLFlow REST API is also able to return result as image(PNG/JPEG) from your SQL.

## The Left-Most format

A common usage scenario is that we would like to have a simple and direct data lineage when retrieving data from database or from multiple SQL statements. One of the features that SQLFlow provides is called the left-most result. The left-most format can simplify the data lineage result.

It generates the selected db elements(such as database/schema/table/column etc…) data lineage without the record set. Given data lineage `a->b->c`, the returned result under the left-most format is `a->c`.

## The API Endpoint

To visualize the left-most data lineage with SQLFlow REST API, we need to invoke

    /sqlflow/generation/sqlflow/leftMostSourceTableGraph/image

This endpoint takes `sessionId` as input. There are two ways to get `sessionId`

### 1. Invoke data lineage API to get sessionId

Sample:

    curl --location 'https://<SQLFlow url>/gspLive_backend/sqlflow/generation/sqlflow/graph' \
    --header 'accept: application/json;charset=utf-8' \
    --form 'sqlfile=@"/home/workspace/test2.sql"' \
    --form 'dbvendor="dbvoracle"' \
    --form 'userId="gudu|0123456789"'

### 2. Create SQLFlow Job

The following endpoint is to create a SQLFlow:

    /sqlflow/job/submitUserJob

Wait few seconds and If we check our job list, we should have this demoJob2 in our job success list. You can verify that ethiter go to the SQLFlow interface or use another SQLFlow REST API `/sqlflow/job/displayUserJobsSummary`.

Great! After checking the SQLFlow web, we have confirmed the SQLFlow Job is succeeded.

Now we have our Job created and we need to select our data now. To do that, invoke:

    /sqlflow/job/displayUserJobGraph

Give the Job Id as the input and in the response of the above API, we will have the sessionId:

## Retrieve left-most data lineage

Now we are good to invoke the left-most SQLFlow Api with the `sessionId` ready. You can use `tableNamePattern` to map the desired table. Full parameter table here:

Field name | Type | Description
dbvendor | string | database vendor, support values: – dbvazuresql – dbvbigquery – dbvcouchbase – dbvdb2 – dbvgreenplum – dbvhana – dbvhive – dbvimpala – dbvinformix – dbvmdx – dbvmysql – dbvnetezza – dbvopenedge – dbvoracle – dbvpostgresql – dbvredshift – dbvsnowflake – dbvmssql – dbvsparksql – dbvsybase – dbvteradata – dbvvertica
hideColumn | boolean | whether hide column
ignoreFunction | boolean | whether ignore function
simpleOutput | boolean | simple output, ignore the intermediate results, defualt is false.
ignoreRecordSet | boolean | same as simpleOutput, but will keep output of the top level select list, default is false.
jobId | string | give the job Id if need to use the job settings and display the job graph
normalizeIdentifier | boolean | whether normalize the Identifier, default is true
showTransform | boolean | whether show transform
token | string | The token is only used when connecting to the SQLFlow Cloud server, It’s not in use when connect to the SQLFlow on-premise version.
treatArgumentsInCountFunctionAsDirectDataflow | boolean | Whether treat the arguments in COUNT function as direct Dataflow
userId | string | the user id of sqlflow web or client
columnLevel | string | whether to show table level or column level data, false or true
isReturnModel | boolean | whether to return model
sessionId | string | the graph session Id
share | boolean | whether shareable
showConstantTable | boolean | whether show constant table
showLinkOnly | boolean | whether show link only
showRelationType | string | show relation type, required false, default value is ‘fdd’, multiple values seperated by comma like fdd,frd,fdr,join. Availables are ‘fdd’ value of target column from source column, ‘frd’ the recordset count of target column which is affected by value of source column, ‘fdr’ value of target column which is affected by the recordset count of source column, ‘join’ combines rows from two or more tables, based on a related column between them
tableColumn | string | target table column
tableNamePattern | string | target table name, supports regex expression

Sample:

    curl --location 'https://<SQLFLOW URL>/gspLive_backend/sqlflow/generation/sqlflow/leftMostSourceTableGraph/image' \
    --header 'accept: image/*' \
    --form 'userId="gudu|0123456789"' \
    --form 'sessionId="85b5119f83fcfefa93136e79dc05f2f61efe3b75e6a2b733568e0879c24f0c08_1680615728918"' \
    --form 'dbvendor="dbvmssql"' \
    --form 'jobId="6def00e8796e4c20ba216826c0ab5b73"' \
    --form 'tableNamePattern="msdb.@event_data.nodes"'

### Newsletter Updates

Enter your email address below to subscribe to our newsletter

I accept the Privacy Policy

### Related Posts

#### Ingest MySQL metadata on Datahub

- July 11, 2023

#### Integrate SQLFlow into Datahub

- July 7, 2023

#### Visualize your upstream/downstream data with SQLFlow REST API

- April 11, 2023