Mongodb destination charset encoding issue

asked 2018-09-17 10:34:26 -0500

casel.chen gravatar image

updated 2018-09-17 23:11:51 -0500

I just migrated pipeline from standalone sdc 3.0.3.0 to dockerized 3.3.0 by export and import. The pipeline is just read kafka data and sink into mongodb. It works well in 3.0.3.0 but after migration I found the data saved in mongodb has charset encoding issue. The user name which saved with Chinese will be displayed as "������" in 3.3.0, why?

edit retag flag offensive close merge delete

Comments

After investigation, I found the root cause is the SDC docker image (https://hub.docker.com/r/streamsets/datacollector/), after switch back to non-docker deployment, the charset encoding issue disappeared. Anyone know how to fix it?

casel.chen gravatar imagecasel.chen ( 2018-09-19 07:42:23 -0500 )edit

Possibly an environment variable is different between the machine where it's working and the Docker image?

metadaddy gravatar imagemetadaddy ( 2018-09-19 09:38:02 -0500 )edit