Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Testing Custom stages/transformers

StreamSets really needs a test library that at the very least exposes the ParserFactorys and ideally a means to emulate a spark job (or any other framework aspects). I spent days building complicated transformers/processors and dealing with the endless deploy/restart/pray-for-a-log cycle. I finally pulled the source and stubbed out a simple test library and was done in hours. The project seems to treat customization as a complete oversight.