Unite
    • Dark
      Light

    Unite

    • Dark
      Light

    Article Summary

    Unite Component

    Combine all the rows from two or more input flows into a single output flow. The input flows should be very similar for this component to be useful, although slight differences in the input schema can be accommodated by using the 'Cast Types' property.

    Properties

    PropertySettingDescription
    NameStringA human-readable name for the component.
    MethodSelectAll Columns: All columns from all inputs are included in the output. Columns that do not exist in one of the input sources will have the SQL NULL value on any rows that come from that source.
    Overlapping Columns: Only columns that appear in all input sources are included in the output. Columns that do not exist in all of the input sources are dropped.
    Cast TypesSelectYes: If the same-named column from multiple inputs have differing input types, Matillion ETL attempts to cast them to a common type. This is not guaranteed to work, so you should check your data carefully.
    No: If the same-named column from multiple inputs have differing input types, Matillion ETL reports an error and will not continue.
    Add Source Component ColumnSelectYes: Add a column called "source_table" with the value of the input component name that provided each row of output.
    No: Do not add an additional column to identify the input component name.
    Remove DuplicatesSelectYes: Remove (merge) duplicate rows so that only one of the duplicate rows remains and all resulting rows are unique.
    No: Do not remove duplicate rows. Allow duplicate rows to exist in the output table.

    Strategy

    Generates multiple select query separated by UNION ALL.

    Example

    In this example, we have a table of cases from the past month and we want to add it to a master table that is a compiling of all months' cases. The master list, in this way, is a kind of historical backup of all cases. To perform this task, we use the Unite component to append one table to the other. The job is shown below.

    To begin, we note the row counts of the two input tables, 11,210,931 and 3,376.

    Both of these tables feed into the Unite component that is configured as shown below.

    We don't want to remove duplicates since each case has a unique ID to distinguish it and we don't care where the data is from so we don't opt to add a source component column. It is, however, beneficial to cast types in case a mistake has been made with the last month's table and its types are not correct to the master list version.

    After going through the Unite component, the data is appended and the row count reports the sum of both input tables.


    What's Next