Ask Your Question
1

HTTP Server Will Not Receive Data When Application ID is in URL & x-www-urlencoded

asked 2019-01-31 15:10:20 -0500

tommy_o gravatar image

I have a demo pipeline configured to receive 'Text' data and the application ID in the URL. When the application ID is in the header, this works no problem.

Working curl test:

curl -k 'http://localhost:8000/?sdcApplicationId=asdf' \
    -H 'Content-type: asdfasdf' \
    -d 'one_text=value_one&two_text=value_two'

Curl test that does not work:

curl -k 'http://localhost:8000/?sdcApplicationId=asdf' \
    -H 'Content-type: application/x-www-form-urlencoded' \
    -d 'one_text=value_one&two_text=value_two'

Specifically, I see the record count incrementing up on the HTTP server input, but the counter is not incrementing on the next stage. When I change the Content-type: to anything except application/x-www-form-urlencoded, it works.

Am I missing something obvious here?

image description

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2019-01-31 17:09:02 -0500

metadaddy gravatar image

What seems to be happening is that reading the application ID query parameter is also consuming the POST body, since the default Java servlet implementation checks both the URL and the payload for parameters when the content type is application/x-www-form-urlencoded. This means that there is no data for the HTTP Server origin to read, so it doesn't emit a record.

Please file an issue at https://issues.streamsets.com and post the issue number in a comment here, and we can take a look at how to resolve this.

In the short term, you have two options, as you observe - set the content type to some other value, or send the application id as an HTTP header.

edit flag offensive delete link more

Comments

I've created https://issues.streamsets.com/browse/SDC-10869 . First time using that Jira so please review for errors.

tommy_o gravatar imagetommy_o ( 2019-02-01 11:42:46 -0500 )edit

Looks good to me - many thanks!

metadaddy gravatar imagemetadaddy ( 2019-02-01 11:46:58 -0500 )edit
Login/Signup to Answer

Question Tools

1 follower

Stats

Asked: 2019-01-31 15:10:20 -0500

Seen: 45 times

Last updated: Jan 31