Featured Post

Applying Email Validation to a JavaFX TextField Using Binding

This example uses the same controller as in a previous post but adds a use case to support email validation.  A Commons Validator object is ...

Tuesday, May 22, 2012

Your Own Exception Handler for Standard Talend Components

Many Talend Open Studio (TOS) components will allow you to specify whether or not processing should stop when an error is encountered.  Some do not, and this can be problematic when an error case is to be expected.  This post describes a technique for adding a custom exception handler to TOS components missing the "Die on error" option.

Talend Open Studio (TOS) components like tMSSqlOutput allow you to specify whether an error is fatal -- causing processing to abort right away -- or non-fatal, allowing processing to continue.  This behavior is specified by the "Die on error" option.
Allow Non-Fatal Errors in a tMSSqlOutput
However, not all TOS components are built this way.  A failure in a tSSH will cause processing to stop, breaking out of any iterations.
Processing Stops During the First Iteration
The tForeach is configured with a list of three values.  The tJava_1 component prints out the current iteration value which is one of the three IPs.

tForeach with List of Three Values
The tSSH stops on the first iteration and does not continue with the remaining input.  This may not be desired.  For example, it may be normal for a host to be unreachable or a login to be bad.  Checking for such conditions may be the whole purpose of the job.  If this is the case, we need a way to continue processing.

Configuration of tSSH - Use Input as Target Host


Code Generation

When TOS generates Java classes that use an iteration connection, exceptions in components like tSSH break out of the iteration.  What we can do is work with the code generation to wrap a few Java keywords around the statements formed by tSSH.  This is done with a tJavaFlex.

Job Supporting Continuing
 In this job, the tJavaFlex provides start and end blocks that wraps around the code generated by the tSSH.  The code added to the tJavaFlex's start and end blocks is a try / catch which will prevent the exception generated by the SSH from leaving the iteration.

The Start and End Blocks Open a try / catch Block
The start and end blocks will add "try {" before the block of code that is generated by tSSH and a "} catch(Exception exc)" after.  This will catch any exceptions thrown by the tSSH (begin, end, main .javajets).

The result is a job that will continue processing over the entire input set.  This screenshot shows three tSSH attempts on the three input IP addresses.  All fail, but processing can continue.

Job That Performs tSSH on All Input (Continuing)
WARNING

If you use this in your jobs, you may find that it is incompatible one day.

It's a bad practice to code TOS jobs that rely on the operation of the code generator.  However, this is a solution that will work across multiple versions of TOS including the latest (5.1.0) and it is a clean way to work with tSSH or any other deficient (no "Die on error") components. 

tJavaFlex prepares start and end blocks that frame the main part of a tJavaFlex as well as downstream components.  This example allows a tSSH to continue processing in the event of an error.


6 comments:

  1. Thank you for this tip!

    i used this with tFileUnarchive when trying to decompress a corrupted file to avoid an exception and end the job.

    ReplyDelete
  2. Thanks for the help. I had a user defined SQL function failing in an iteration.

    ReplyDelete
  3. Yep, helped me with tFileUnarchive as well. Talend definitely needs an improved and consistent way to handle errors.

    ReplyDelete
    Replies
    1. Hi Paul, would you mind sharing what you did with iFileUnarchive exactly? I have the same issue and must continue when decompressing a corrupt file. Thanks!

      Delete
  4. I have implemented it with tFileUnarchive and it works fine. Why in the hell does the tFileUnarchive have an "on component error" option if it is unable to manege errors???

    ReplyDelete
  5. Thanks for this - I used the information to get a list of worksheets in an Excel spreadsheet by looping over a spreadsheet using the loop's current value as the sheet position, breaking when the "no sheet found" error occurred.

    ReplyDelete