RSD Generator

Overview

The RSD generator allows users to automatically create their own RSD for a table of their choice such that this RSD can be used with the API.

Query component to load data from a 3rd party into a table. This is all reliant on the 3rd party allowing database access via a RESTful API.

RSD Generation and its configuration is based on the CDATA JDBC REST driver. Column definitions and RepeatElements can also been considered in this same manner.

The RSD generator can be reached through the Matillion ETL client through Project → Manage API Profiles and creating or editing a profile. From there, click 'Generate' to bring up the Generator dialog.

The properties for this configuration are described below. It may also be useful to read our articles on Writing API Profiles and Parameters to better understand the anatomy of RSDs.

 
Property Description
Table Name Any name you want to give to the table.
Description Any description you wish for this table.
XPath to Repeat Element Endpoint that exposes repeating elements to be taken.
Data Format The data format that the repeating elements exist in.
URI URI of the endpoint with file extension.
Properties API parameters that might be needed (such as user and password), separated by semicolons.
 

Static Files

The RSD generator can be pointed at static files and does not necessarily have to go directly through the service. The URI property can be pointed at an XML or JSON sitting on S3 (this file must be public!), for example, instead of expecting one to be returned from an API call.

This method is recommended as it has the advantage of allowing users to use their own preferred API tool to construct their API calls. This can be particularly useful when your API call has many parameters that would be too cumbersome to place together in the 'properties' field or if using a POST method instead of GET.

After using a static file in the generator, the RSD can be altered by hand to change necessary parts such as the uri or to create additional structures.


Example

Table Name: TicketData
Description: Ticket data taken from zendesk
XPath to Repeat Element: /tickets
Data Format: Choose between data being collected in JSON or XML format.
URI: https://matilliondevelopment.zendesk.com/api/v2/tickets.json
Properties: AuthScheme=BASIC;User =user@matillion.com;Password =mypassword;OAuthVersion=;




Clicking 'OK' will save the RSD for use. Now, editing the 'Profile' property of an API Query component will yield this option available to use.