Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

File Name and Date Stamp Matillion ETL for Bigquery


I am trying to loop through the CSV files available in the Google Cloud Bucket and was wondering if I can capture the file name and the date stamp available on the file name to store in the staging bigquery table.
I can see the file iterator is available for S3 buckets but nothing is defined for Google Storage Buckets, also I am not sure if there is a way through Matillion Components or Python to capture the file name and the date stamp to be stored in the bigquery tables.

Many thanks


1 Community Answers

Matillion Agent  

Ian Funnell —

Hi Yogesh,

The File Iterator can’t do this yet unfortunately, as you have found. We are looking at adding this functionality but I’m not sure at this point if or when it might be added.

I think the only way to do it currently is in a script: most likely Python or Bash.

Please monitor the release notes for potential future developments in this area.

Best regards,

Post Your Community Answer

To add an answer please login