Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Social media data extraction using Streamsets

I am trying to pull data from social media for sentimental analysis. For example i am using facebook data. My test case is: i have an url and it results into a list of posts . Now in the next pipeline i want to use this list (iteratively) and extract post_id and on the basis of that post_id need to extract comments on that specific post. Is this can be done iteratively in using streamsets. Because the comment url will be different for different post_id . I want to pass this id as variable in the comment url. code snippet what am trying to achieve using Python. for post in posts_id: comment=[] comment_url="http://api.facebook.com/v2.1.0/post comments=fetch_comment(comment_url) comment.append(comments)

The above code is just for reference what am trying to achieve. Can it be possible with streamsets pipeline?

Social media data extraction using Streamsets

I am trying to pull data from social media for sentimental analysis. For example i am using facebook data. My test case is: i have an url and it results into a list of posts . Now in the next pipeline i want to use this list (iteratively) and extract post_id and on the basis of that post_id need to extract comments on that specific post. Is this can be done iteratively in using streamsets. Because the comment url will be different for different post_id . I want to pass this id as variable in the comment url. code snippet what am trying to achieve using Python. for post in posts_id: comment=[] comment_url="http://api.facebook.com/v2.1.0/post comments=fetch_comment(comment_url) comment.append(comments)Python.

The above below code is just for reference what am trying to achieve. Can it be possible with streamsets pipeline?

for post in posts_id:

comment=[] comment_url="http://api.facebook.com/v2.1.0/post comments=fetch_comment(comment_url) comment.append(comments)re

Social media data extraction using Streamsets

I am trying to pull data from social media for sentimental analysis. For example i am using facebook data. My test case is: i have an url and it results into a list of posts . Now in the next pipeline i want to use this list (iteratively) and extract post_id and on the basis of that post_id need to extract comments on that specific post. Is this can be done iteratively in using streamsets. Because the comment url will be different for different post_id . I want to pass this id as variable in the comment url. code snippet what am trying to achieve using Python.

The below code is just for reference what am trying to achieve. Can it be possible with streamsets pipeline?

for post in posts_id:

comment=[] comment_url="http://api.facebook.com/v2.1.0/post comments=fetch_comment(comment_url) comment.append(comments)re comment.append(comments)

Social media data extraction using Streamsets

I am trying to pull data from social media for sentimental analysis. For example i am using facebook data. My test case is: i have an url and it results into a list of posts . Now in the next pipeline i want to use this list (iteratively) and extract post_id and on the basis of that post_id need to extract comments on that specific post. Is this can be done iteratively in using streamsets. Because the comment url will be different for different post_id . I want to pass this id as variable in the comment url. code snippet what am trying to achieve using Python.

The below code is just for reference what am trying to achieve. Can it be possible with streamsets StreamSets pipeline?

for post in posts_id:
     comment=[]
     comment_url="http://api.facebook.com/v2.1.0/post
     comments=fetch_comment(comment_url)
     comment.append(comments)comment.append(comments)`