Ask Your Question

Newline characters from source SQL Server

asked 2018-07-23 13:33:29 -0500

jay1988 gravatar image

updated 2018-07-23 15:29:06 -0500

metadaddy gravatar image

While fetching the data from SQL Server certain columns have newline characters and StreamSets is loading the data as it is to Target Hive Locations. Due to a Hive issue the data when queried from Hive is showing as 2 rows.

Is there a way in StreamSets to remove the newline character while fetching from Source itself.This fix shouldn't be column specific as newline can be present for any columns from source.

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-07-23 15:27:39 -0500

metadaddy gravatar image

You can do this with the Field Replacer processor. Set the field to /* (to match every field) and the new value to ${str:replaceAll(f:value(), '\\n', '')} (to replace every newline with an empty string).

image description

You might want to change the empty quotes to a space, like this ' ', if you want to replace the newline with a space.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2018-07-23 13:33:29 -0500

Seen: 120 times

Last updated: Jul 23 '18