Vexdata.io Documentation
  • Vexdata Overview
  • Installation
    • System Requirements
    • Windows Installation
    • Linux Installation
    • Cloud Installation
      • GCP
      • Azure
      • AWS Cloud Deployment Architecture Diagram
        • Pre-requisites
        • Manual Install on ec2
        • Cloud Formation Scripts (command line)
    • Admin Post Installation Setup
    • User Post Installation Setup
    • Server Improvement (Optional)
  • Getting Started
    • Roles and Permissions
    • Managing Groups and Projects
      • Folders
      • Projects
    • Settings
      • Slack Integration
      • Integrations
      • Rules
      • Properties
      • Integrations
      • Email Template
      • Report Template
      • Create Data Connections
        • Databases/Warehouses
        • Run On
  • Creating and Executing Test Cases
    • Test Case
      • Create Test Cases
        • Define Data Source and Target
          • Configure Table/s from Database
          • Input SQL Data
          • Input Files
            • Excel/CSV
            • XML
            • JSON
            • Parquet/AVRO
          • API Input
          • Table Filter
          • Advance - Input
        • Data Compare
          • ETL Testing/Cell By Cell Compare
            • Data Compare Mapping
            • Data Compare Options
          • Schema Compare
          • Counts Compare
        • Schema Monitoring
        • Data Quality
          • SQL Rules
          • Column Level Rules
          • Duplicates
      • Executing Test Cases
      • Defining Test Case Settings
    • Test Runs
    • Variables
    • Note
    • Label
  • Reports
    • User Reports
    • Admin Report or Manager Reports
  • Current Limitations
Powered by GitBook
On this page
  1. Creating and Executing Test Cases
  2. Test Case
  3. Create Test Cases
  4. Define Data Source and Target

API Input

To provide an API as data input:

  • Upload a Python script which can execute and produce output in CSV or JSON format.

  • The output location will be passed as first argument.

  • Provide a sample output JSON for the schema.

Ensure the sample output (few records) represents all of the columns.

Sample Python code snippet for an API_:_

import json
import csv
import requests
import sys
import pandas as pd
def download_json(url):
    response = requests.get(url)
    data = response.json()
    return data['emp_hash']
def write_to_csv(data, output_path):
    df = pd.DataFrame(data)
    df.to_csv(output_path, index=False)
if __name__ == "__main__":
    output_path = sys.argv[1]
    url = "https://dataq-testing-data.s3.amazonaws.com/Input+Files/emp_hash.json"
    data = download_json(url)
    write_to_csv(data, output_path)
PreviousParquet/AVRONextTable Filter

Last updated 5 months ago

427KB
conversation.json
Sample API output file(json)