{"id":2839,"date":"2023-07-07T21:14:45","date_gmt":"2023-07-08T02:14:45","guid":{"rendered":"https:\/\/www.dpriver.com\/blog\/?p=2839"},"modified":"2023-07-21T03:48:03","modified_gmt":"2023-07-21T08:48:03","slug":"sqlflow-datahub","status":"publish","type":"post","link":"https:\/\/www.dpriver.com\/blog\/2023\/07\/sqlflow-datahub\/","title":{"rendered":"Integrate SQLFlow into Datahub"},"content":{"rendered":"\n

Datahub<\/a> is an open-source metadata platform for the modern data stack. We have integrated the SQLFlow into Datahub so that the SQLFlow data lineage is enabled in the Datahub UI. This integration is for Datahub v0.10.4<\/strong>. You will need to replace the `datahub-web-react-datahub-web-react-assets.jar<\/code> in datahub with the SQLFlow adapted jar file. The file to replace may change with different Datahub versions and please check this blog regularly to get the corresponding file name for the latest Datahub.<\/p>\n\n\n\n`

Contact james@<\/em>gudusoft.com<\/strong> to get the corresponding SQLFlow adapted jar file.<\/p>\n\n\n\n

<\/p>\n\n\n\n

## 1.Install SQLFlow<\/h2>\n\n\n\n

SQLFlow Regular version<\/strong> is required for the Datahub. You can either directly install the regular version of SQLFlow on your server or you can launch one container with the docker image. <\/p>\n\n\n\n

#### a. Install Directly <\/h4>\n\n\n\n

Check our product documentation<\/a> for the direct Install<\/p>\n\n\n\n

#### b. Using Docker Image<\/h4>\n\n\n\n

Pull the image:<\/p>\n\n\n\n

##

##

##

    docker pull gudusqlflow\/sqlflow-regular-trial:5.7.5<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nLaunch the image with<\/p>\n\n\n\ndocker run -itd -p 7090:80 --name mysqlflow gudusqlflow\/sqlflow-regular-trial:5.7.4<\/code><\/pre>\n\n\n\nThe 7090<\/code> in the above command will be the port to visit SQLFlow UI. You can change the port if 7090 is occupied in your machine.<\/p>\n\n\n\n<\/figure>\n\n\n\nThe mysqlflow<\/code> is the name of the container. For more information of the container creation, you can check the official Docker doc<\/a>.<\/p>\n\n\n\n2. Update the API URL<\/h2>\n\n\n\nThe first thing to do is to update the GuduSqlFlowUrl<\/code> in the config.js<\/code> of datahub-web-react-datahub-web-react-assets.jar<\/code>:<\/p>\n\n\n\nOpen the datahub-web-react-datahub-web-react-assets.jar<\/code> (please do not extract the jar file and re-compress it, it may cause encoding issues)<\/p>\n\n\n\n<\/figure>\n\n\n\nFind the public\/sqlflow\/config.js <\/code>and copy the file.<\/p>\n\n\n\n<\/figure>\n\n\n\n Create a new config.js<\/code>, update the GuduSqlFlowUrl<\/code> to your actual SQLFlow API URL.<\/p>\n\n\n\n<\/figure>\n\n\n\nReplace the old public\/config.js<\/code> in datahub-web-react-datahub-web-react-assets.jar<\/code> with your new config.js<\/code>.<\/p>\n\n\n\nHint<\/strong>: Do not use localhost as the GuduSqlFlowUrl because the localhost IS NOT<\/strong> the server address when browser loads web<\/p>\n\n\n\n3. Replace the jar file<\/h2>\n\n\n\na. Upload the updated datahub-web-react-datahub-web-react-assets.jar<\/code> to your server.<\/p>\n\n\n\n<\/figure>\n\n\n\nb. Stop datahub-frontend-react container<\/p>\n\n\n\n Find the container ID:<\/p>\n\n\n\ndocker ps -a<\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nStop the container:<\/p>\n\n\n\ndocker stop <container ID><\/code><\/pre>\n\n\n\n<\/figure>\n\n\n\nc. Backup the old datahub-web-react-datahub-web-react-assets.jar<\/code><\/p>\n\n\n\ndocker cp datahub-frontend-react:\/datahub-frontend\/lib\/datahub-web-react-datahub-web-react-assets.jar <Backup Address><\/code><\/pre>\n\n\n\nd. Replace the jar file in the datahub-frontend-react<\/p>\n\n\n\ndocker cp <Your_Updated_datahub-web-react-datahub-web-react-assets.jar> datahub-frontend-react:\/datahub-frontend\/lib\/<\/code><\/pre>\n\n\n\nStart the datahub-frontend-react:<\/p>\n\n\n\ndocker start <container ID><\/code><\/pre>\n\n\n\n4. Check your Datahub<\/h2>\n\n\n\nOpen your DataHub UI again and you will find SQLFlow features are enabled for the table level<\/strong> and field level<\/strong> data.<\/p>\n\n\n\na. You can get the upstream and the downstream lineage in the SQLFlow tab of table level view<\/p>\n\n\n\n<\/figure>\n\n\n\nb. Click the Schema tab, you will have a column of Gudu SQLFlow which has been added to the field information list. Click the Lineage corresponding to a field to view the data lineage of the field.<\/p>\n\n\n\n<\/figure>\n\n\n\n<\/figure>\n\n\n\nChecking this document<\/a> to get more details on how to create a Datahub task with the SQLFlow plugin.<\/p>\n","protected":false},"excerpt":{"rendered":"Datahub is an open-source metadata platform for the modern data stack. We have integrated the SQLFlow into Datahub so that the SQLFlow data lineage is enabled in the Datahub UI. This integration is for Datahub v0.10.4. You will need to replace the datahub-web-react-datahub-web-react-assets.jar in datahub with the SQLFlow adapted jar file. The file to replace may change with different Datahub versions and please check this blog regularly to get the corresponding file name for the latest Datahub. Contact james@gudusoft.com to get the corresponding SQLFlow adapted jar file. 1.Install SQLFlow SQLFlow Regular version is required for the Datahub. You can either\u2026<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[66,1],"tags":[],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":5}},"yoast_head":"\nIntegrate SQLFlow into Datahub<\/title>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\t\n\t\n\t\n