talend

Talend - Limit Number of Rows Processed

爱⌒轻易说出口 提交于 2019-12-01 13:32:11
问题 I'm working with Talend ETL to transfer data between two Salesforce Orgs. I'm trying to run preliminary tests to make sure everything is setup properly. Is there a way to limit the number of rows being transferred? The database has over 50,000 rows, and I only want to send over 15 or 20. Thank you. 回答1: On the Talend side, you can use tSampleRow to only process a limited number of rows which were retrieved . For example you can use a line number range to only process rows 1-50. 来源: https:/

Setting a database value to context variable in talend

不羁岁月 提交于 2019-12-01 12:31:35
问题 I have a job that flows like this. tAccessDatabse_1 ---> tFileOutputXML_1. Now, my database has a schema, with usename and userid. My task to create/send data from the database to xml file, file name with username i.e, one file has to create for every user with his/her name. I tried like creating a conetxt varible but how can i set username to that context variable from the database ?? 回答1: Select distinct username from table. Use tFlowToIterate to iterate on each of the usernames. (connect

How can a properties file be created using Talend Open Studio Data Integration?

泄露秘密 提交于 2019-12-01 01:01:29
I've used Talend Open Integration studio for sometime to just create jobs and run it from the IDE or export it as executable jar file. But I haven't used it extensively. Is it possible to create an external configuration file that holds different server names and other variables so that after I create an executable jar file of my Talend job, I can just change configuration text file and select different database servers or port options? I also used MuleSoft for a little bit, so if anyone knows how to do this with MuleSoft any advise would be kindly appreciated. From the content of your

How can a properties file be created using Talend Open Studio Data Integration?

邮差的信 提交于 2019-11-30 19:02:23
问题 I've used Talend Open Integration studio for sometime to just create jobs and run it from the IDE or export it as executable jar file. But I haven't used it extensively. Is it possible to create an external configuration file that holds different server names and other variables so that after I create an executable jar file of my Talend job, I can just change configuration text file and select different database servers or port options? I also used MuleSoft for a little bit, so if anyone

How to add third party libraries to Talend project?

前提是你 提交于 2019-11-30 12:11:04
How to add third party libraries (jar files) to a Talend project ? One more question is, Each Talend component uses LogFactory , but in my case it is throwing java.lang.NoClassDefFoundError: org.apache.commons.logging.LogFactory at ... with tSetGlobalVar component, how can I make use of existing logging library to get rid of this exception? Vikram Takkar You can import the external jars using tLibraryLoad component or Editing Routine Libraries . For more information visit http://vikramtakkar.blogspot.in/2013/03/importing-external-jar-or-library-files.html Just to benefit others, with recent

how to load data faster with talend and sql server

谁说胖子不能爱 提交于 2019-11-30 05:29:57
I use Talend to load data into a sql-server database. It appears that the weakest point of my job is not the dataprocessing, but the effective load in my database, which is not faster than 17 rows/sec. The funny point is that I can launch 5 jobs in the same time, and they'll all load at 17rows/sec . What could explain this slowness and how could I improve the speed? Thanks New informations: The transfer speed between my desktop and the server is about 1MByte My job commits every 10 000 I use sql server 2008 R2 And the schema I use for my jobs is like this: Database INSERT OR UPDATE methods are

Access a “String Array” from tExtractXMLField (XML tag with +1 value)

给你一囗甜甜゛ 提交于 2019-11-29 17:24:25
My Talend Job has the following structure: tREST component that executes a POST HTTP Request, and receives a XML file. tExtractXMLField component that reads a tag from that XML file, like: Now, I want to save all the values from the tagX, and then access them. For each value of the tagX , I will perform a HTTP POST with that value in the body. How can I do that? Thanks You need a component called tHttpRequest . But this component doesn't allow to be connected to row flows. But there's a way to accomplish. Let's say you extracted your tagX field and put the value on a String field called 'tagX'

How to use Java classes in Talend

不羁岁月 提交于 2019-11-29 08:44:04
I have the following three classes : I tried making the routine of 1 & 2 and used tjava to call the main class and the method from 1 & 2 but I am unable to fetch those methods. 1) package page_scraper; import com.gargoylesoftware.htmlunit.Page; import com.gargoylesoftware.htmlunit.WebClient; import com.gargoylesoftware.htmlunit.WebClientOptions; import com.gargoylesoftware.htmlunit.html.FrameWindow; import com.gargoylesoftware.htmlunit.html.HtmlButtonInput; import com.gargoylesoftware.htmlunit.html.HtmlElement; import com.gargoylesoftware.htmlunit.html.HtmlOption; import com.gargoylesoftware

JSON parser for Talend

天涯浪子 提交于 2019-11-28 14:12:26
I need some help devising a strategy to parse JSON docs within a Talend job (Java job, not Perl). I am using Talend Version: 5.0.2 and developing on a Mac, planning to run on a Linux box. Unfortunately, I cannot use the tFileInputJSON component because of the format of my files -- each file contains several hundred JSON docs, with a complete JSON doc taking up one line in the file. I think the right solution is to read the file line by line then pass it into a JSON parser and from there send the results to the rest of the job. As I see it my options are: a) send the line input to some sort of

Access a “String Array” from tExtractXMLField (XML tag with +1 value)

北战南征 提交于 2019-11-28 13:10:01
问题 My Talend Job has the following structure: tREST component that executes a POST HTTP Request, and receives a XML file. tExtractXMLField component that reads a tag from that XML file, like: Now, I want to save all the values from the tagX, and then access them. For each value of the tagX , I will perform a HTTP POST with that value in the body. How can I do that? Thanks 回答1: You need a component called tHttpRequest . But this component doesn't allow to be connected to row flows. But there's a