Ask Your Question

/tmp files that appear to be deleted when there are no pipelines running

asked 2018-06-18 14:47:18 -0600

Speiec75 gravatar image

updated 2018-06-18 14:58:15 -0600

metadaddy gravatar image

We notice that our /tmp directory was filling up and upon further investigation, Data Collector has files in /tmp/xxxx/pipelineid/.... in (deleted) status. In this environment, there are no pipelines running currently.

Are we suppose to be recycling Data Collector on a daily basis?

Is it normal to see /tmp files that appear to be deleted but are still there when you do a lsof -p against the Data Collector pid?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2018-06-18 15:14:26 -0600

metadaddy gravatar image

You should absolutely not need to recycle Data Collector for this.

What sort of files are they? Any clues from the filenames? Is it possible to use the file command to see what sort of data they contain? Failing that, what sort of operations are you doing in your pipelines? I can see in the source that Data Collector will create files in /tmp when working with whole file data format, but it should be cleaning up after itself. If you can pin down enough information for us to reproduce the problem, please go ahead and create an issue.

edit flag offensive delete link more


I didn't screen print them out and recycle SDC for other reason.. It was /tmp/xxxx/pipelineID/yyy/oracle stuff.. CDC and other extentions.. I will ask my testers to startup a pipeline or two to see if they reappear..

Speiec75 gravatar imageSpeiec75 ( 2018-06-18 15:22:59 -0600 )edit

Thanks - that's really helpful. I've alerted the engineer responsible for the Oracle CDC origin.

metadaddy gravatar imagemetadaddy ( 2018-06-18 15:28:15 -0600 )edit

In a different mapr cluster env, sdc filled up /tmp with the following: java 109536 mapr 8393u REG 8,2 134217728 605153378 /tmp/ed0fedbc-4706-11e8-8102-e9d866b6af02/DWSSIDW001incrpub0454f383-64ed-42f7-86c2-e6584e0a1d98/OracleCDCClient_01/7.39.17890/db-7030326246071000779/dbFil

Speiec75 gravatar imageSpeiec75 ( 2018-06-29 07:17:03 -0600 )edit

the process is doing a log miner collection into a stream.

Speiec75 gravatar imageSpeiec75 ( 2018-06-29 08:23:11 -0600 )edit

Hi @metadaddy. Any way to get some feedback on this issue? A user in South Africa ran into the same issue on version 3.4. He also posted the issue on Jira at

Nick gravatar imageNick ( 2019-01-17 08:42:47 -0600 )edit
Login/Signup to Answer

Question Tools



Asked: 2018-06-18 14:47:18 -0600

Seen: 489 times

Last updated: Jun 18 '18