Ask Your Question
1

How to convert and initialize numeric fields from delimited record format?

asked 2017-05-22 11:15:37 -0600

jeff gravatar image

When using the delimited record format, what's the best way to initialize/replace empty values with their numeric defaults (ex: 0 for int)? It seems that the Value Replacer, under the Replace Null Values section does not work. When trying to use the Field Type Converter, an error is thrown (Failed to convert field '/field' of type 'STRING' with value '' to type 'DECIMAL').

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2017-05-26 15:40:33 -0600

jwood gravatar image

updated 2017-05-26 15:43:05 -0600

You could use a scripting processor to do this. For example, in Jython:

# Make your list of fields to fix and default values
int_fields = ['age', 'houseNumber']
int_default = 0
string_fields = ['firstName', 'roadName']
string_default = "My default string."

for record in records:

    try:

        # Check and fix int fields. Use the default if field does not exist.
        for int_field in int_fields:
            try:
                record.value[int_field] = int(record.value.get(int_field, int_default))
            except ValueError:
                record.value[int_field] = int_default

        # Check and fix string fields. Use the default if field does not exist.
        for string_field in string_fields:
            try:
                record.value[string_field] = str(record.value.get(string_field, string_default))
            except ValueError:
                record.value[string_field] = string_default

        output.write(record)

    except Exception as e:
        error.write(record, str(e))
edit flag offensive delete link more

Comments

Did a quick edit, as my original string_default was `None` instead of an actual string : )

jwood gravatar imagejwood ( 2017-05-26 15:43:44 -0600 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2017-05-22 11:15:37 -0600

Seen: 391 times

Last updated: May 26 '17