Tasks

A task is the unit of execution for a transfer in TRADElube. It connects a channel with exactly one action, for example Products Import, and determines when and how often that happens. The channel says who TRADElube talks to, the task says what is transferred.

Every task runs on its own, at its own cadence, with its own configuration and its own error message. A stock update every ten minutes and a nightly product upload therefore do not disturb each other, and if one transfer fails, the others keep running. The failure also stays with the task it belongs to, instead of marking a whole connection as broken.

What a task is

Every task is one row under the navigation item Tasks. It carries no name of its own that you could assign: it is named after its channel and its action, and those two are the first columns of the list. You therefore meet a task of the channel Supplier Product Data with the action Products Import under that combination everywhere, including in the traces.

Task list with the columns Channel, Action, Performance and the times of the last execution

The remaining columns say what happened last: when the last execution started and ended, which state it left behind, and since when that state has applied. A task that has never run stays empty in those columns.

Task and action

Which actions are available comes with the channel type, not with the task. An online shop offers different ones than a supplier file, and which ones in detail is listed for each system under Channels and tasks.

A task carries exactly one action, and an action has exactly one direction. Uploading products and fetching orders means two tasks, not one with two checkboxes. That is why a single channel quickly reaches eight or ten tasks, and the price for that is small: every transfer gets its own transfer plan, and it applies to that transfer alone.

In an existing task, target and action are only displayed and can no longer be changed. The reason lies in what the task carries with it: it remembers, per record, what was transferred into this one target system. Switching the target would carry that state over to a system that has never seen it, and overwrite records there that nobody touched. If the same transfer is to run through a different channel, that is therefore a new task.

Channels where the other side signs in to TRADElube of its own accord have no tasks at all. What that is about is covered under channels.

Scheduling and manual start

How often a task runs by itself is held in the field Performance. It knows two kinds of value: a fixed interval from 30 seconds up to an hour, and two special cases. Manually means the task only runs when you start it, and it is what a new task is preset to. Nightly means it runs between midnight and five in the morning, the quietest moment for large transfers.

Two peculiarities that might otherwise look like a fault:

Between 0:55 and 2:00 TRADElube starts no scheduled executions. That window is when the nightly maintenance runs, including the backup. A task due in that window catches up afterwards.
A task that has failed is not repeated immediately but after five minutes. If the same failure has persisted for more than a day, that becomes sixty minutes. A broken connection therefore does not hammer a foreign system every minute, producing neither load there nor a thousand identical log entries.

Independently of all this, you start the selected task straight away with the button Execute, including one set to Manually. That is the usual way to check a change without waiting for the next scheduled execution.

Renewal time: when something is transferred again

TRADElube only transfers what has changed. Why that is so is covered under synchronization basics. The field Renewal Time is the exception to it: after the interval set there, a record is transferred again even when nothing about it has changed. That is how a target system finds its way back to TRADElube after someone edited it by hand or it was restored from a backup.

It is therefore a precaution and not a precondition for running. Where nobody makes manual changes in your target system, you can simply leave the field empty.

Common section of a task with target, action, performance and the two renewal times

A scheduled execution takes on only a limited number of due records and works through the rest over the following ones. The reason is everyday operation: a renewal across ten thousand products would otherwise hold up every other transfer for half a day, even though nothing about the data has changed. Spread over many executions it goes unnoticed.

Anyone starting a task with Execute often expects everything to be transferred again now. That is not what happens. A manual start takes on every record that has changed, plus every record that failed last time, without waiting out the delay after an error. Whatever is unchanged it leaves alone, even where the renewal time would have come due. To genuinely repeat a transfer in full, reset it in the traces with Restart or Restart All. Every record concerned then counts as open again.

There is one exception: if the renewal time is set to Nightly, it only applies between midnight and five in the morning. A manual start in the afternoon then renews nothing, and that is not a fault.

Some actions bring more than one such field, for instance a separate Media Renewal Time, because images need renewing less often than prices. Which fields a task shows depends on its action.

Order and dependencies

Tasks do not know about each other. There is no chaining, no predecessor and no condition "start once that one has finished". Each runs at its own cadence.

A sensible order still arises, and it comes from the data: whatever is needed as a reference in the target system has to have arrived there first. A product assigned to a category only finds that category in the shop if it is already there, because the reference runs through the channel mapping. In practice categories, manufacturers and media therefore run before products, stock and order status after them.

This is handled through scheduling, not through a dependency: the upstream tasks run earlier or more often than the ones building on them. When filling a target system for the first time, you start them by hand once in that order. Which order a particular system needs is listed under Channels and tasks.

Where a new task comes from

Creating and deleting tasks is something we take on, together with the channel the task belongs to. The reason is the damage the alternative could do: a task that runs before its first complete configuration writes into a foreign system, and what has landed there is not undone by a correction in TRADElube.

Everything else you set yourself: the cadence, the renewal time, the configuration and the transfer plan. And if you would rather hand that over, we do that too.

The buttons New and Delete are in the toolbar all the same. They do not lead to a creation dialog but show a message with our address, so that you are not left looking for a route that is not there.

A day in operation

Suppose you keep 12,000 articles and sell through an online shop. Your supplier provides a new price list every night, in which a few hundred prices have changed. Four tasks cover that, and their cadences differ because not everything is equally urgent:

Task	Performance	Why this cadence
Products Import, from the supplier	Nightly	The price list is not available until night anyway
Products Upload, into the shop	Nightly	Runs after it, so the shop shows the new prices in the morning
Orders Download	10 minutes	An order left lying overnight costs a day of delivery time
Stocks Upload	10 minutes	What is sold should no longer be orderable in the shop

What actually happens that night: the import reads 12,000 rows and finds that nothing has changed on 11,700 of them. 300 are passed on. The upload into the shop accordingly has 300 products to do and is finished in minutes rather than hours.

Behind this lies the fingerprint per record. For every product TRADElube remembers how it was last transferred and compares the next time, before the record is even loaded. The effort therefore follows what has changed and not the size of your range: a catalogue of 50,000 articles is no different a case from one of 500.

It could be made simpler by processing everything every time. That is considerably less to build and means, in this example, 12,000 transfers every night instead of 300, each with a call to the shop. The price of our decision belongs with it: the first time, a task transfers everything, and once you change the transfer plan, every record counts as new again, because the instruction has changed.

And when something goes wrong at night? This is where the rule behind the delay after an error shows itself: the state of a transfer hangs on the individual record, not on the execution. Suppose the shop is unreachable for ten minutes at three in the morning and 40 of the 300 products do not get through. Then 260 are in the shop, the 40 sit in the traces as failed, and the task is not counted as broken because of it. The next execution picks up exactly those 40. You need not reset anything, need not fill anything in afterwards, and above all need not work out which forty it was.

How it fits together

A task sits between the channel and what is actually transferred:

The channel supplies the connection and the login details. If those fail, all tasks of that channel fail together.
The action determines which objects go in which direction, and with that also which fields there are to fill at all.
The transfer plan hangs off the task and determines which value is written where. Two tasks of the same channel can therefore map things completely differently.
The channel mapping arises during the execution and remembers, per record, the counterpart in the connected system.

What actually ran in the end is in the traces, both on the task itself and on the individual record.