Vexdata.io Documentation
  • Vexdata Overview
  • Installation
    • System Requirements
    • Windows Installation
    • Linux Installation
    • Cloud Installation
      • GCP
      • Azure
      • AWS Cloud Deployment Architecture Diagram
        • Pre-requisites
        • Manual Install on ec2
        • Cloud Formation Scripts (command line)
    • Admin Post Installation Setup
    • User Post Installation Setup
    • Server Improvement (Optional)
  • Getting Started
    • Roles and Permissions
    • Managing Groups and Projects
      • Folders
      • Projects
    • Settings
      • Slack Integration
      • Integrations
      • Rules
      • Properties
      • Integrations
      • Email Template
      • Report Template
      • Create Data Connections
        • Databases/Warehouses
        • Run On
  • Creating and Executing Test Cases
    • Test Case
      • Create Test Cases
        • Define Data Source and Target
          • Configure Table/s from Database
          • Input SQL Data
          • Input Files
            • Excel/CSV
            • XML
            • JSON
            • Parquet/AVRO
          • API Input
          • Table Filter
          • Advance - Input
        • Data Compare
          • ETL Testing/Cell By Cell Compare
            • Data Compare Mapping
            • Data Compare Options
          • Schema Compare
          • Counts Compare
        • Schema Monitoring
        • Data Quality
          • SQL Rules
          • Column Level Rules
          • Duplicates
      • Executing Test Cases
      • Defining Test Case Settings
    • Test Runs
    • Variables
    • Note
    • Label
  • Reports
    • User Reports
    • Admin Report or Manager Reports
  • Current Limitations
Powered by GitBook
On this page
  • Redshift
  • SQL Server
  • Postgres
  • Snowflake
  • Oracle
  • Maria​
  • DB2
  • Bigquery
  1. Getting Started
  2. Settings
  3. Create Data Connections

Databases/Warehouses

PreviousCreate Data ConnectionsNextRun On

Last updated 5 months ago

Some of the databases and warehouse available when clicking on Add New Connection might require more information than the one shown in the previous section, or could have extra options.

Redshift

To add a Redshift connection, you will have to provide:

  • Connection name, which you will assign to this connection for internal use.

  • A JDBC URL. A sample JDBC URL is shown under this text.

jdbc:redshift://redshift-cluster-1.XXXXXXXXXX.us-east-1.redshift.amazonaws.com:5439/dev?ssl=false

  • Credentials, which will be a username and a password that will provide access to this database.

Optionally, you may provide s3TempDir and s3AwsIamRole for better performance, as explained in the following link.

s3TempDir

s3n://<S3_Bucket>/

s3AwsIamRole

arn:aws:iam::XYZ:role/Redshift_Role

It is recommended to create a read only user account in Redshift. The user id needs to have access to the following system tables to read the metadata of all the tables.

  1. svv_table_info

  2. information_schema.views

SQL Server

To add an SQL Server connection, you will have to provide:

  • Connection name, which you will assign to this connection for internal use.

  • A JDBC URL. A sample JDBC URL is shown under this text.

jdbc:sqlserver://XXXXXX:1433;databaseName=XXXXXXX;

  • Credentials, which will be a username and a password that will provide access to this database.

It is recommended to create a read only user in Sql Server. The read only user id should have read only access on the databases/schemas where data resides and also have access to the following system tables to read the metadata of all the tables.

  1. sys.objects

  2. sys.columns

  3. sys.types

  4. sys.indexes

  5. sys.index_columns

  6. sys.columns

  7. sys.key_constraints

  8. sys.tables

  9. sys.all_columns

Postgres

To add a PostgresSQL connection, you will have to provide:

  • Connection name, which you will assign to this connection for internal use.

  • A JDBC URL. A sample JDBC URL is shown under this text.

jdbc:postgresql://<jdbcHost>:XXXX/<jdbcDatabaseName>

  • Credentials, which will be a username and a password that will provide access to this database.

If SSL is enabled, additional parameters will be required for the JDBC URL as shown under this text.

jdbc:postgresql://<jdbcHost>:XXXX/<jdbcDatabaseName>&ssl=true**&sslmode=require&sslfactory=org.postgresql.ssl.NonValidatingFactory**

Create a new read only user for Postgres Testing

The user id should only have read permission on the data. Also, Vexdata needs permission to read metadata. The user id should have read only permissions on the following system tables

pg_class
information_schema.table_constraints
information_schema.columns
pg_namespace
pg_catalog

Snowflake

To add a Snowflake connection, you will have to provide:

  • Connection name, which you will assign to this connection for internal use.

  • A JDBC URL. A sample JDBC URL is shown under this text.

  • Credentials, which will be a username and a password that will provide access to this database.

It is recommended to create a read only user account in Snowflake. The user id needs to have access to the following system tables to read the metadata of all the tables.

  1. information_schema.columns

  2. information_schema.tables

Oracle

It is recommended to create a read only user account in Oracle. The user id needs to have access to the following system tables to read the metadata of all the tables.

  1. sys.dba_tables

  2. sys.dba_constraints

  3. sys.dba_cons_columns

  4. sys.all_users

  5. all_ind_columns

  6. all_indexes

  7. all_cons_columns

  8. all_constraints

To grant read only permission to catalog tables

Maria​

It is recommended to create a read only user account in Maria. The user id needs to have access to the following system tables to read the metadata of all the tables.

  1. information_schema.tables

  2. information_schema.columns

  3. information_schema.table_constraints

  4. information_schema.key_column_usage

  5. information_schema.referential_constraints

  6. information_schema.statistics

DB2

It is recommended to create a read only user account in DB2. The user id needs to have access to the following system tables to read the metadata of all the tables.

  1. syscat.tables

  2. syscat.columns

  3. sysibmadm.admintabinfo

  4. syscat.tabconst

  5. syscat.keycoluse

Bigquery

It is recommended to create a read only user account in Maria. The user id needs to have access to the following system tables to read the metadata of all the tables.

  1. INFORMATION_SCHEMA.SCHEMATA

  2. INFORMATION_SCHEMA.TABLES

  3. INFORMATION_SCHEMA.COLUMNS

jdbc//XXX72530.us-east-1.snowflakecomputing.com/?db=<jdbcDatabaseName>&schema=<schemaName>

❄️
Redshift IAM role for Copy Unload to S3AWS CLOUD DBA
Logo
Redshift Configuration
SQL Server Configuration
Postgres configuration
Snowflake Configuraiotn