Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

S3: Get all files inside folders

I want to get all the files inside of folders in s3, (files doesn’t have extension)

The structure is the following

Folder1 File1 File2 … Folder2 File3 … …

In streamsets documentation I found that using ** I can navigate inside of folders.

I have tried with prefix pattern: */ or * or */ and I am getting infinite loops. Ideas?

S3: Get all files inside folders

I want to get all the files inside of folders in s3, (files doesn’t have extension)

The structure is the following

Folder1 File1 File2 … Folder2 File3 … …

In streamsets documentation I found that using ** I can navigate inside of folders.

I have tried with prefix pattern: */**/* or * ** or */ **/ and I am getting infinite loops. Ideas?