Define Data Source and Target

To make use of the data that comes from the Data Connections,_ _users can make use of the Source and Target shapes from the palette, which can be used to configure source and destination data stores.

After dropping any of these two shapes into the canvas and clicking on them, a window will appear at the bottom of the screen. This window will show two main tabs, one named Config, and the other Sample Data. The Config tab will provide four tabs from which the user can choose the type of data input. These options are:

Table Input : This is used to configure a data store which is a from a database in the form of a table.

SQL Input : Custom SQL queries can be defined in this tab to extract data from multiple tables. These queries can be nested queries to chain the SQL statements.

Always prefix schema name before the table name as a good practice.

Example : select * from marketing.users where marketing is the schema name.

File Input : This tab can be used if the data has to be taken from a file which is in one of the following formats: _Parquet, AVRO, JSON, CSV _or Excel.

API Input : This tab can be used to call on an API to extract data. This is a request response pattern where API calls can be scheduled to extract data.