how much time will take to ingest a 2GB file in stand alone mode and cluster batch mode?

asked 2018-03-06 00:27:42 -0500

nimmy bosco gravatar image

What is the expected time to ingest a 2GB file from directory to HDFS , with below configurations

1.Bach size =1000 2.Batch wait time=10sec 3.Number of threads=7

what is the expected time to ingest a 2GB file from HDFS to HDFS with Bach size =1000.

edit retag flag offensive close merge delete

Comments

There is no way to provide a direct answer to this question. There are too many variables at play (ex: source data format, destination stage/format, physical hardware, network, Hadoop cluster/topology/vendor/version, etc.). Only way to know is to test it.

jeff gravatar imagejeff ( 2018-03-06 11:03:40 -0500 )edit