Ask Your Question
1

How can I use the time:xxx functions in custom processor configuration fields?

asked 2018-08-08 11:12:35 -0500

dcwatson84 gravatar image

I'm building a custom Java processor and for one of the config parameters I need the user to be able to use the time:xxx functions. However, these do not show up in the list of available functions when filling in my configuration parameter in the UI.

edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted
0

answered 2018-08-08 12:52:38 -0500

metadaddy gravatar image

If you look at the existing stages, for example, the HTTP Client stages, you will see how they use the TimeNowEL class to do this. You need to add elDefs and evaluation attributes to the @ConfigDef annotation on your config parameter, like this:

evaluation = ConfigDef.Evaluation.EXPLICIT,
elDefs = { TimeNowEL.class },

You'll also need to add code to the processor to explicitly evaluate the config parameter:

// Field declarations
private ELVars configVars;
private ELEval configEval;
// Config property field name
private static final String CONFIG_NAME = "myConfig";
...
// in the processor init() method
configVars = context.createELVars();
configEval = context.createELEval(CONFIG_NAME);
...
// Per record, or per batch - however often you want to do this
TimeNowEL.setTimeNowInContext(resourceVars, new Date());
String evaluatedConfig = resourceEval.eval(resourceVars, myConfig, String.class);

Finally, since TimeEL is not a part of the official API for custom stages, you will need to build the stagesupport jar from source in the main datacollector source tree, and include it in your custom processor. See the custom destination tutorial for details - that destination uses commonlib for CSV formatting; you should be able to do something similar with stagesupport.

Be aware that stagesupport is not an official API; it can change without warning from one release to the next.

edit flag offensive delete link more

Comments

So the answer is that there is no *official* way to use the time:xxx functions from a custom processor? I can't risk building pipelines that are going to fail randomly in new releases, so if it's not officially supported then, in my case, it's not supported at all.

dcwatson84 gravatar imagedcwatson84 ( 2018-08-08 18:19:05 -0500 )edit
1

That is the current situation, yes. It may change in the future. Please watch/vote/comment on https://issues.streamsets.com/browse/SDC-8411

metadaddy gravatar imagemetadaddy ( 2018-08-08 18:32:14 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2018-08-08 11:12:35 -0500

Seen: 16 times

Last updated: Aug 08