Ask Your Question

StreamSet Column Mapping Issue

asked 2017-10-26 01:59:30 -0600

Magesh S gravatar image

updated 2017-10-27 10:57:57 -0600

metadaddy gravatar image

We are using StreamSets for data upload from CSV file to database.

While giving CSV file path, it's not getting read from file path. It has to show number of columns in destination in column mapping dropdown right, thats not showing , else its showing "No Fields Found". Since i'm using 4300 column CSV file, it has any issue on that? Is there any column limit in StreamSet? Having doubt on it.

Note: Already i have tried with up to 100 columns CSV , with this its working fine, for the same Source and Destination DB , different table.

edit retag flag offensive close merge delete


How did you configure 4300 column in your pipeline? Is there any possible way to automate "Field to Column Mapping". I have 4000 column to configure . Need some support. Thank you in advance.

HARSH TG gravatar imageHARSH TG ( 2017-11-28 01:56:59 -0600 )edit

1 Answer

Sort by ยป oldest newest most voted

answered 2017-10-26 14:36:23 -0600

jeff gravatar image

updated 2017-10-27 10:52:32 -0600

This is most likely happening because you need to increase the max allowed record size, with such a large number of columns. Please see documentation here.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-10-26 01:59:30 -0600

Seen: 1,575 times

Last updated: Oct 27 '17