Pentaho data integration tutorial for beginners

Key field,В the field from the source table that containsВ a name of the attribute.

There are three things we need to configure: This component converts EAV data into the fields, one per EAV attribute. To get prepared to the Denormalization, we need to sort rows by rowNumber, that put the the same category related attributesВ together. Both fieldsВ mean the same thing, a field name id. It uses the same component, but with the slightly different configuration: The rules below say: put a value from name tag into a first column (called вЂњcolumnnameвЂќ) and a value of the attribute into the variable вЂњidвЂќ.ДЄs a result, we have a list of categoryВ attributesВ as well as their codes: This job processes the headers, so I filter the header section (see XML above).ДЇields.В This sectionВ defines what XML tag/attribute values should be mapped with what fields. The вЂњLoop XPathвЂќ defines a root node for the processing. You can also use a filename instead of the variable reference.Д¬ontent. XML1 is a constant that is defined in the project configuration. The filename is stored as a project parameter, so I referenced to the variable by its name instead of using the filename directly. ItВ uses DOM parsers that need in memory processing and even the purging of parts of the file is not sufficient when these parts are very big.В If you need to process huge files, Pentaho has another XML parsing component, XML Input Stream (StAX).ДЇile. This step provides the ability to read data from any type of XML file using XPath specifications.

Merge this header into a dataset from the previous step.

ApplyВ a template from the external text file.В Basically, the template defines an Impex Header:.

Merge it with the data from the previous step.

Join them into the dataset from the previous step.

Join it into the dataset retrieved fromВ XML file:.

Merge it with the data from the dataset retrieved from XML file:.According to the task, we add вЂњH_вЂќ to the original ID. Pentaho creates additional columns (PARENT_ID, PRODUCT_CATEGORY_ID, NAME) using the values of the field вЂњvalueвЂќ. Two previous datasets were joined on id=columnNumber. This part of XML doesnвЂ™t contain the column names, only their IDs. The process consists of 19 steps (after each step I put a table with the results of the step): In this tutorial weвЂ™ll come to the following process: LetвЂ™s start at the ends and have a look at the final result. This solution is a configuration forВ Pentaho, a data integration platform. Today I am going to tell you about the alternate solution. In the previous article I showed how it can be implemented without using any out-of-the-shelf software. So our goal is to transform original XML to this ImpEx file. In the target format we will need to put them under one root node (letвЂ™s call it H_ROOT).ДЄfter applying this Impex, hybris creates a category tree: Root nodes are represented by empty values in XML. In the real projects, converting of IDs is also important, but the algorithm may be more complex. It is an optional change, just to demonstrate the Pentaho capabilities of Value Mappers.

Please note that category IDs begin with вЂњH_вЂќ. 'INSERT_UPDATE Category' 'code' 'supercategories(code, $catalogVersion)' 'name' '$catalogVersion' 'allowedPrincipals(uid)' '$catalogVersion=catalogversion(catalog(id),version)' For the XML above, the following result is expected:

Pentaho data integration tutorial for beginners