Ask Your Question
1

StreamSet Column Mapping Issue

asked 2017-10-26 01:59:30 -0500

Magesh S gravatar image

updated 2017-10-27 10:57:57 -0500

metadaddy gravatar image

We are using StreamSets for data upload from CSV file to database.

While giving CSV file path, it's not getting read from file path. It has to show number of columns in destination in column mapping dropdown right, thats not showing , else its showing "No Fields Found". Since i'm using 4300 column CSV file, it has any issue on that? Is there any column limit in StreamSet? Having doubt on it.

Note: Already i have tried with up to 100 columns CSV , with this its working fine, for the same Source and Destination DB , different table.

edit retag flag offensive close merge delete

Comments

How did you configure 4300 column in your pipeline? Is there any possible way to automate "Field to Column Mapping". I have 4000 column to configure . Need some support. Thank you in advance.

HARSH TG gravatar imageHARSH TG ( 2017-11-28 01:56:59 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2017-10-26 14:36:23 -0500

jeff gravatar image

updated 2017-10-27 10:52:32 -0500

This is most likely happening because you need to increase the max allowed record size, with such a large number of columns. Please see documentation here.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-10-26 01:59:30 -0500

Seen: 365 times

Last updated: Oct 27 '17