788 questions

5

views

1

answer

1

vote

2018-09-21 20:42:57 -0500 metadaddy

Recover a corrupt pipeline post disk space error

How to ensure pipeline is fetching data from all the tables mentioned in JDBC multitable origin?

• jdbcMultitable
• Data
• destination
• preview

7

views

1

answer

1

vote

2018-09-21 08:58:10 -0500 metadaddy

Mongo DB Lookup

Filter File Tail records on the edge

• sdcedge
• sdc
• filter
• tailfile
• tail

How to store status in ETL pipeline?

• ETL
• JDBCQuery

Pipeline Status: RUN_ERROR: com.streamsets.pipeline.api.StageException: JDBC_14 - Error processing batch. SQLState: 50000 Error Code: 1 Message: Failed to map key to node.

• streamsets
• jdbcProducer
• Unique
• database

only have row key inside my mapr-db json table

• mapr

[Data collector 3] JMS to ELK, pipeline transaction

• JMS
• elasticsearch
• datacollector
• errorhandling

Slow DNS lookup!

• logging
• Kudu
• cdh5.15

22

views

1

answer

1

vote

2018-09-19 09:43:09 -0500 metadaddy

Documentation for control hub rest api?

• rest
• api
• controlhub
• documentation

49

views

1

answer

no

votes

2018-09-18 23:10:25 -0500 Maithri

how to create pipeline load all CDC from multiple tables in a MySQL DB

• CDC
• mysql

Oracle CDC client origin with Redo Logs option not able to capture changes

• CDC

Data preview only showing 10 records

Mongodb destination charset encoding issue

• MongoDB

Replace all . with _ in field names

JDBC Stages and On Unknown Type => Convert to String

26

views

1

answer

no

votes

2018-09-17 07:15:28 -0500 Maithri

Is the directory origin able to read new files in a directory (files are written in such directory in a regular time interval) or it reads all files present in the directory every times?

• directory
• origin

Is there a method in streamsets to calculate time difference in seconds between 2 given date/time ?

• datetime

how to do Row to column pivot/transposition

• row
• to
• column
• pivot
• transposition

I am using HTTP server as origin and destination is local FS. My data is originating in "json" and need to in SQL format at destination or any readable So what Processor need to add.

is there any process does kafka producer acknowledge of received messages?

22

views

1

answer

1

vote

2018-09-14 10:07:48 -0500 metadaddy

How to deduplicate kafka messages with SDC?

• kafka

54

views

1

answer

no

votes

2018-09-14 04:05:03 -0500 Shruthi

Streamsets pipeline Error while writing Data in Hadoop

• streamsets
• hadoop

38

views

1

answer

no

votes

2018-09-14 00:02:03 -0500 Maithri

Write XML files

• xml
• Files

21

views

1

answer

no

votes

2018-09-13 23:03:06 -0500 Maithri

How to specify the Table name pattern in the Oracle CDC Client tab

• Oracle
• CDC
• table

ElasticSearch as origin in Incremental mode

• elasticsearch
• origin

18

views

1

answer

1

vote

2018-09-13 14:19:16 -0500 metadaddy

Change S3 prefix based on filename

Why I get the error "[runner:] [thread:Spool Directory Runner - 0] WARN SpoolDirRunnable - Ignoring file <file1> in spool directory as is lesser than offset file <file2>"?

• directory
• origin
• spooling

I am trying to run through the Spark Evaluator tutorial, and get an exception when running preview. Has anyone seen this before?

• SparkTransformer

What are the performance benchmarks of SDC?