We are parsing s3 path in snowfalke query to fetch the partition_id for each table.Wrote a query its working fine if the part-*.parquet files present under the partitions.Giving a wrong results if the parquet files having with different names.
Aim is single query should return partition_id=20240101 in both the use cases.
use case1 working fine as expected:
select split(split(split_part('1234/local/table_name1/partition_id=20240101/part-0001.parquet','table_name1'),'part-')[0],'/')coutput : partition_id=20240101use case2 giving output - partition_id and parquet file also
select split(split(split_part('1234/local/table_name2/partition_id=20240101/table.name2.parquet','table_name2'),'part-')[0],'/')coutput: partition_id=20240101table.name2.parquetI have to tweak the query so it should work in both cases and returns only partition_id=20240101. Please share your inputs.
Thanks in advance