S3 Origin: need help setting up prefixes...

asked 2019-12-17 09:56:08 -0600

daveh gravatar image

I have an S3 origin that I can't get to load s3 objects into simple text records. The structure in S3 bucket is

  • gsodwi/somefile1.txt
    gsodwi/somefile2.txt etc.

The files are csv files. I have the Data Format set to "Text"

My common prefix is "gsodwi/" and my Prefix Pattern is "*.txt".

When I run the pipeline all i get is a "no more data" event.

Any help getting this simple setup to work is greatly appreciated.


edit retag flag offensive close merge delete


Have you tried resetting the origin? Also, you might want to remove the / from the end of the common prefix.

metadaddy gravatar imagemetadaddy ( 2019-12-17 10:29:17 -0600 )edit