Ask Your Question
0

Why does StreamSets hangs while Loading Data from excel file >10mb into Oracle table

asked 2019-10-04 07:41:31 -0500

jay66online gravatar image

Hi, We have created a pipeline which loads data from an excel file(it has 6 worksheets) kept in a unix directory and loads the same into Oracle Table. The mapping is straightforward without any expressions involved. While we used the source excel file of 1 mb file size. The loading happened within a minute or so. But the moment we keep a 10 mb file in the source directory and run the pipeline, streamsets stops responding and hangs. we tried the threading options and it did not work .

Could someone suggest what parameter needs to be configured to handle a source excel file of the size 10 mb or greater.

Thanks, Jay

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

answered 2019-10-06 05:15:44 -0500

Swayam gravatar image

Hi - as per Streamsets link below - Excel files with higher size should be converted to CSV and then processed as there are some limitations for higher volume of Excel files.

https://streamsets.com/documentation/...

edit flag offensive delete link more

Comments

Thanks Swayam, i'll convert it to csv and try for the same pipeline

jay66online gravatar imagejay66online ( 2019-10-07 03:14:43 -0500 )edit

Thank you Swayam, it actually worked after we converted the excel into a csv.

jay66online gravatar imagejay66online ( 2019-10-14 03:37:11 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-10-04 07:41:31 -0500

Seen: 245 times

Last updated: Oct 06