Featured Post

Applying Email Validation to a JavaFX TextField Using Binding

This example uses the same controller as in a previous post but adds a use case to support email validation.  A Commons Validator object is ...

Sunday, January 2, 2011

[TUTORIAL] Text File to MySQL Table with Talend

This tutorial loads a MySQL table with contact data from a CSV file using Talend Open Studio.  Talend Open Studio is an Eclipse-based open source integration tool that handles many different formats and provides a rich library of objects for transformation and scripting.


This tutorial performs the load by
  1. Defining a source schema based on a delimited text file (CSV),
  2. Defining a target schema based on a relational table (MySQL), and
  3. Using the tMap component to map the source fields to the target fields.
The definitions are part of a Talend Job which is then run.  The result is a MySQL table containing 29 input records from the 29 record input text file.

NOTE: As I was preparing the demo, I deleted and re-created the source and target metadata several times over.  The initial project I created was scrapped, because I was unable to create the source and target metadata. The files were there, it just wasn't appearing in the IDE.  After deleting the entire workspace and creating a new workspace and project, everything worked well and Open Studio was responsive.

I recommend running the Generation Engine in the foreground to avoid this.  The initialization procedure, which took 4-5 minutes on my laptop, was only run once at the start of the workspace and project creation.  After that, I didn't have to wait.

If you'd like to recreate the demo, here is the data file of sample contacts and here is some MySQL DDL for the table.

3 comments:

  1. If you want to generate JSON using Talend Open Studio, check out Text File to JSON with Talend.

    ReplyDelete
  2. Here's another tutorial that normalizes data with Talend: Turning Columns into Rows.

    ReplyDelete
  3. "and all the records are there" lol

    ReplyDelete