Streamsets on AWS ECS - unable to use AWS JavaScript SDK

asked 2020-04-02 11:50:04 -0500

Vassil gravatar image

I have installed StreamSets Data Collector on ECS Fargate container. Installed also NodeJS, npm and AWS JavaScript SDK.

I am unable to use the SDK in JavaScript Evaluator. I have tried so far two approaches: load('file:///node_modules/aws-sdk/dist/aws-sdk.js'); var AWS = require('aws-sdk');

I am getting error - com.streamsets.pipeline.api.StageException: SCRIPTING_10 - Script error in user script: javax.script.ScriptException: ReferenceError: "require" is not defined in <eval> at line number XX

Tried also browser version of the SDK library

var window = {};

load('file:///tmp/aws-sdk/aws-sdk-2.650.0.min.js');

var AWS = window.AWS; AWS.config = new AWS.Config(); AWS.config.accessKeyId = "..."; AWS.config.secretAccessKey = "..."; AWS.config.region = '...';

// Create an SSM client

var ssm = new AWS.SSM();

AWS.setTimeout(function(err) {}, 5000);

var params = { Name: '...', WithDecryption: true }; ssm.getParameter(params, function(err, data) { if (err) { cur_batch.addError(record, err.toString()); cur_batch.process(entityName, offset.toString()); hasNext = false; } else { record = sdc.createRecord('generated data'); var value = prefix + entityName + ':' + data.toString(); record.value = value; cur_batch.add(record);

} });

That goes a bit further but am getting error like - com.streamsets.pipeline.api.StageException: SCRIPTING_10 - Script error in user script: javax.script.ScriptException: TypeError: AWS.setTimeout is not a function in <eval> at line number 65

The above script works fine wen executed with the nodejs runtime on the same container

edit retag flag offensive close merge delete

Comments

What exactly are you trying to do in JS evaluator?

iamontheinet gravatar imageiamontheinet ( 2020-04-23 23:51:51 -0500 )edit