Importing products from a file

This guide sets up a task that reads your article file and turns it into products, repeatably: the same file read a second time changes nothing, a changed one updates exactly the rows affected. This is the normal route for supplier data, because that arrives as a file, in varying quality, and rarely in the shape one would wish for. The route runs through the custom channel, which reads CSV, XLS, XML and JSON.

It assumes the basics from the concepts: what a task is and what a transfer plan does.

Prerequisites

• A custom channel with the Products Import task.
• A sample file matching the later deliveries: with column names and with one column that identifies every row permanently and uniquely.

Step by step

1. Create the data format and derive the field list. The data format describes the shape of your file, and you do not type it in: open the channel and, in the Data Formats section, the format, and drop the sample file onto the Upload area. TRADElube reads the file and picks up columns and data types. With Columns From Header set, it finds the columns by their names later, and the order in the file no longer matters.

Check the detected data types before you build on. Identifiers belong on text, even when they look like numbers: an EAN or a postal code as an integer loses its leading zero, a value with a unit such as "0.75 l" as a decimal loses its unit. The data type also decides what you can assign later at all. The source list of an assignment only offers columns whose type fits the target field. A column you miss there is almost always one with an unsuitably detected type.

2. Create the plan and set the layout. Open the task and its Transfer Plan section. You create the first plan through Insert. In the Layout section, use Insert to put a record set onto your data format. Of its assignments one is decisive: the Mapping. It is the key under which TRADElube remembers every row, and through it every later execution recognizes the row again.

As the mapping, choose the column that never changes, a record identifier from your supplier or the EAN for instance. The product number is only a good key if it is guaranteed to stay stable. The reason is under channel mappings: if a row's key value changes, the next execution considers it new and creates a second product instead of updating the existing one.

3. Fill the sections. Next to the layout the plan carries one section per kind of data: Common for product number, active flag and the manufacturer reference, Descriptions with one record set per language version, then Prices, Stocks, Categories, Media and Properties. The basic pattern is the same everywhere: tick the assignment's checkbox, choose the source, done. A category path from the file creates the category tree along with the import, and images are fetched through a column carrying the image address. Fill only what your file provides. A section without a source stays empty, and that is the right setting. What the three assignment kinds can do and when a script pays off is under adjusting the transfer plan and scripts in the transfer plan.

   For the manufacturer reference your file decides which of the three ways fits. Where a supplier file concerns a single manufacturer, create it once by hand and assign it As Fixed Value. Where the file carries a manufacturer name per row, an As Script Assignment fetches it with FindOrCreateManufacturerWithName and creates missing manufacturers along with the import. Where the manufacturers arrive as a file of their own anyway, they run ahead through the Manufacturers Import task, and the reference then finds them through their channel mapping.

4. Variants, if your file carries them. The usual case is one row per variant with no row of its own for the parent article. The parent article then comes from the Grouping, and a second plan attaches the variants. The setup is walked through under creating variants and is not repeated here.

5. Start the task. You start the task with Execute. For every data format of the task a dialog asks for the file first. Without a file the task does not start. For this manual mode you need no endpoint. If TRADElube is to fetch the file itself on a schedule instead, from your supplier's FTP server for instance, attach a data source to the task's endpoint and set the Performance field to an interval. The steps are under setting up data sources.

Checking the result

The quickest overview sits in the traces: tick the Created and Modified state filters on and narrow down to the task's name in the Filter field. One transfer is listed per row read, and three checks tell you whether the import holds:

• The numbers add up. The number of transfers matches the file's row count. With grouping, the parent articles come on top.
• Failed rows are explainable. Each is listed with its message, see understanding error messages. The rest of the execution is untouched by them.
• The second execution is silent. The same file read again reports no changes. That is exactly what proves the mapping holds: every row was recognized instead of created anew.

Spot-check on the product itself, including the language tabs, prices and images.

Frequent pitfalls

• The product number as the mapping. If your supplier renumbers, the next execution creates a second product for every changed number. The safe key in step 2 is a column nobody ever touches.
• A renamed column is a deleted one plus a new one. TRADElube finds columns by name. After deriving again in step 1, the dialog shows the changes with one checkbox each, and assignments onto the old column are empty afterwards. Keep column names stable. Add a new column individually in the data format through New instead of deriving the list anew.
• "Successful" with "no changes made". No contradiction: with an unchanged file and an unchanged plan there is nothing for the execution from step 5 to do. After a change to the transfer plan, though, the next execution goes through in full by itself.
• Text in a number column. An "on request" in the price field fails at exactly that row with a conversion error. The type is set by the data format from step 1. The remaining rows go through, and the traces name the failed one.
• The delete checkboxes of the configuration. Disable Unused Items and the delete checkbox next to it refer to everything missing from the file just read. With a partial list that acts like delisting half the range. The details are on the task page Products Import (Custom).

Further reading

• Products Import (Custom): the task in detail, including the delete checkboxes and the media renewal time
• Custom: the channel, its data formats and its remaining tasks
• Creating variants: parent and variant articles from a flat file
• Adjusting the transfer plan: assignment kinds, fixed values and scripts
• Data sources (reference): FTP, SFTP and HTTP as a source
• Creating and maintaining a product: the manual route, where you check the result