Ask Your Question
1

Pipeline error - write to File : dynamic directory naming

asked 2017-05-23 05:44:10 -0600

gvd gravatar image

updated 2017-05-23 19:24:15 -0600

LC gravatar image

We would like to setup a dynamic directory naming for our pipeline errors as follow : image description

But seems that the pipeline:title() string is not trimmed.

Tried to call the trim method but it is passed as plain text : image description

Any suggestions ?

Extra question: Is it be possible for StreamSets to make the directory if it does not exist yet ?

edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
2

answered 2017-05-23 09:05:09 -0600

Glad that it's working for you now!

Another thing that I've noticed on the second screenshot is that the EL needs to be enclosed in ${}, so the expression:

/datadrive00/streamsets_error/${cluster_env}/str:trim(${pipeline:title()})

Should probably be:

/datadrive00/streamsets_error/${cluster_env}/${str:trim(pipeline:title())}

(the ${} have moved from wrapping the pipeline:title to wrap the whole str:trim)

edit flag offensive delete link more
1

answered 2017-05-23 05:56:59 -0600

gvd gravatar image

Found out why I had white space in the pipeline title :

I pasted the title from a CSV where it has "tab" at the beginning and I edited the title from the pipeline "header" which didn't take effect.

Modifying the title value from the "General" tab worked.

Issue solved.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-05-23 05:44:10 -0600

Seen: 75 times

Last updated: May 23 '17