Ask Your Question

How to process data from a log file

asked 2017-08-22 04:01:43 -0600

Varun gravatar image

updated 2017-08-29 18:42:20 -0600

metadaddy gravatar image

I have a text file abc.log which have data like this.

dbname|tablename|rowcount abcdb|abctb|234 defdb|deftb|123 ghidb|ghitb|345

I want to use those dbname, tablename and rowcounts using StreamSets Expression Evaluator. Can anyone tell how can i acheive that?

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted

answered 2017-08-28 10:23:24 -0600

When you read the data in, use the Directory origin and in the Data Format tab choose 'delimited' text format, with a custom delimiter. That will allow you to specify the | character as the field delimiter. Choose 'with header line' so that it will pick up the field names from the first line in the file. You can then access the fields for each record in the Expression Evaluator as, for example, ${record:value('/dbname')}.

edit flag offensive delete link more
Login/Signup to Answer

Question Tools

1 follower


Asked: 2017-08-22 04:01:43 -0600

Seen: 66 times

Last updated: Aug 29 '17