Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Processing files and send them to s3

I want to get different avro files from s3, create a file by tag and send it to s3 For example

Origin s3 file1.avro {"tag":"Blue","value"="1"} {"tag":"Red","value"="1"}

file2.avro {"tag":"Blue","value"="2"}

Destination:

Blue.avro {"tag":"Blue","value"="1"} {"tag":"Blue","value"="2"}

Red.avro {"tag":"Red","value"="1"}

I am trying to do it with jython, by record I am creating a dictionary where the key is the tag and adding the record but I am not able to get the whole record. How can I do it? How can I create files to uploaded to s3 or it is direct from the dictionary? Is it something more appropiate for this case than jython? Thank you!

Processing files and send them to s3

I want to get different avro files from s3, create a file by tag and send it to s3 For example

Origin s3 file1.avro file1.avro

{"tag":"Blue","value"="1"} {"tag":"Red","value"="1"}

file2.avro {"tag":"Blue","value"="2"}

Destination:

Blue.avro {"tag":"Blue","value"="1"} {"tag":"Blue","value"="2"}

Red.avro {"tag":"Red","value"="1"}

I am trying to do it with jython, by record I am creating a dictionary where the key is the tag and adding the record but I am not able to get the whole record. How can I do it? How can I create files send the dictionary value to uploaded to s3 or it is direct from the dictionary? s3? Is it something more appropiate for this case than jython? Thank you! you!

Processing files and send them to s3

I want to get different avro files from s3, create a file by tag and send it to s3 For example

Origin s3 file1.avro

file1.avro {"tag":"Blue","value"="1"} {"tag":"Red","value"="1"}

file2.avro {"tag":"Blue","value"="2"}

Destination:

Blue.avro {"tag":"Blue","value"="1"} {"tag":"Blue","value"="2"}

Red.avro {"tag":"Red","value"="1"}

I am trying to do it with jython, by record I am creating a dictionary where the key is the tag and adding the record but I am not able to get the whole record. How can I do it? How can I send the dictionary value to the s3? Is it something more appropiate for this case than jython? Thank you!