Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

S3 Destination WARN "No content length specified..."

I have a pipeline that splits a record into three different types of S3 destinations, one being an AVRO conversion. I am receiving a "WARN" in the logs as follows:

No content length specified for stream data. Stream contents will be buffered in memory and could result in out of memory errors. The logs also say its for the AmazonS3Client.

I cannot find any reference to this for the S3 Destination in the Streamsets documentation or elsewhere on line. I do not see anywhere in the S3 Destinaton configuration to set a content length for writing to S3.

Any insight would be greatly appreciated.