Ask Your Question
1

Directory 1 SPOOLDIR_12 - Directory does not exist : Files Directory

asked 2019-06-12 05:09:44 -0500

akash3247 gravatar image

I am getting the above error while running pipeline using origin as directory am specifying file directory as D:\file. But streamset is unable to read data from origin folder containing CSV files.

edit retag flag offensive close merge delete

Comments

Is this Data Collector Edge? Looks like you're running on Windows, right? How have you configured the origin?

metadaddy gravatar imagemetadaddy ( 2019-06-12 17:10:45 -0500 )edit

it is data collector(not edge). yes i am running on windows in the directory i am specifying my folders path and as it is a csv file i am using pattern as *.csv

akash3247 gravatar imageakash3247 ( 2019-06-13 02:20:22 -0500 )edit

Are you running the Docker image of Data Collector?

metadaddy gravatar imagemetadaddy ( 2019-06-13 19:23:49 -0500 )edit

Yes, I am running docker image of the data collector. All I want to do is to pick up an excel file from D:/file/ directory but when I enter such directory in Files Directory I get the error as no such directory exist.please help.

akash3247 gravatar imageakash3247 ( 2019-06-14 02:01:44 -0500 )edit

Hi Akash347 did you ever get this working on Windows? I'm running into the same issue. Docker image of Data Collector on Windows and it does not see my directory "SPOOLDIR_12 - Directory 'C:\StreamSets\tutorial\orgin' does not exist

lnewportTX gravatar imagelnewportTX ( 2019-06-21 15:14:44 -0500 )edit

1 Answer

Sort by ยป oldest newest most voted
1

answered 2019-06-21 15:49:54 -0500

Tim gravatar image

updated 2019-06-21 15:59:30 -0500

You said in the comments that you are running Data Collector as a container in Windows version of Docker. Note that Docker containers are Linux only. So its execution and function is the same as if you ran the program natively in Linux. Therefore Windows paths will not work. You must specify Linux syntax which do not use drive letters. Even when specifying paths on the Docker command line, you still use Linux's forward slash for paths.

Your steps to access a Windows file is:

  1. Specify the drives that should be shared with the container.
  2. Use Docker's "-v" command line option when starting the Data Collector container. This would mount your local folder into a location within the container.

This is explained in more details and with screenshots in the Docker documentation here.

Our documentation contains an example for how to specify where the /data Linux folder should be mounted on the host system. See the "Detailed Usage" section of the SDC container documentation here.

For your reference, here is an example of mounting c:\my\files folder when you start the container:

docker run -v c:/my/files:/data:rw -p 18630:18630 -d streamsets/datacollector dc

You would then refer to the file using the mounted path "/data". Because from the container's perspective, all the files are under /data and not "c:\my\files". For example, you would specify the file "c:\my\files\a_subfolder\excelfile.csv" as "/data/a_subfolder/excelfile.csv"

As a word of advice, I would try to avoid windows paths with spaces in them. This prevents additional troubleshooting steps while you test the solution. :-)

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-06-12 05:09:44 -0500

Seen: 242 times

Last updated: Jun 21